Sharkfest 26EU: HowTo Detect Stealth VPN and AI Inference Traffic

Earlier this week we have presented at Sharkfest Europe 2026 a presentation of our work about traffic detection of Stealth VPNs and AI Inference. The presentation addresses the challenges of detecting modern obfuscated network traffic, specifically from stealth VPNs and custom/local artificial intelligence (AI) services. While traditional signature methods (like JA4) face instability and hash collisions, the presenters introduce a breakthrough data-driven approach using machine learning and AI transformers (specifically a “Context Transformer”) to identify traffic patterns accurately without decrypting payloads.

---

Part I: Introduction to Passive Traffic Fingerprinting

• Network Fingerprinting: Extracting unique, observable characteristics from standard protocols (TCP, IP, TLS) to passively identify a specific device, operating system, or network protocol without altering traffic.
• Active vs. Passive Probing:
  • Passive: Invisible, zero overhead, ideal for fragile environments (OT/medical devices), but dependent on active traffic flow.
  • Active: Triggers packets to force a response (e.g., nmap, JARM), offering high accuracy but risking disruption or firewall detection.
• IP Stack Elements: Security tools look at headers such as Time-To-Live (TTL), initial TCP Window Sizes, and the wire-order of TCP Options to fingerprint operating systems.
• The Encryption Paradox: While encryption hides raw payload data, the unencrypted handshake negotiations (Cipher Suites and TLS Extensions) provide rich criteria to generate application-level signatures like JA4.

---

Part II: Stealth VPN and AI Traffic Analysis

• What is a Stealth VPN?: A service that actively modifies or strips handshake signatures and routes over regular web ports (like Port 443 or Port 80) to disguise VPN usage as ordinary HTTPS web browsing.
• The Limitation of JA4: The presenters discovered that JA4 hashes are unstable on a single device across multiple connections and can experience massive hash collisions with highly popular, benign traffic (e.g., standard Google Android services). This creates massive false-positive risks if used for active traffic blocking.
• The nDPI TLS Fingerprint: As a workaround, they preview a stable, collision-resistant successor to JA4 integrated into ntop’s nDPI library.
• AI Traffic Profiling: Traffic to major AI APIs (like OpenAI or Anthropic) is easily spotted via Server Name Indication (SNI). However, local instances powered by tools like llama.cpp or vllm require deeper flow analytics because their traffic patterns differ significantly based on whether the AI model response is Streamed or Non-Streamed.

---

Part III: From Fingerprints to Transformers

When static signatures fail or are unknown in the wild, Machine Learning offers a data-driven approach by looking at network behavior patterns over time.

• Feature Engineering: The proposed framework transforms network traffic flows into 9 numerical features spanning:
  1. Traffic Volume: Forward/backward bytes and packet counts.
  2. Packet Statistics: Average size and length variations.
  3. Payload Patterns: Entropy of payload lengths.
  4. Timing: Mean and standard deviation of packet inter-arrival times (IAT).
• Windows and Bins: Flows are segmented into overlapping 100-second windows divided into 10-second temporal bins to capture traffic shape trends over time.
• The Context Transformer: Recognizing that an isolated stream might lack clarity, their transformer architecture enriches the target flow by evaluating up to 20 temporally nearby “context elements” (surrounding network activities) and identifying shared IP endpoints.
• Evaluation Metrics:
  • Tested using public datasets (VNAT, ISCXVPN2016) alongside their own smartphone captures of stealth applications like HA Tunnel Plus, XU Tunnel, TLS Tunnel, Psiphon, and Geph.
  • Achieved an impressive 90.44% F1-score at the Window level and 85.71% at the Flow level, catching all 15 targeted VPN flows with only 5 false positives.

The pipeline can be actively deployed by users as a lightweight FastAPI service leveraging TShark and nDPI parser utilities.

Enjoy and contact us for questions !