USDT Probes

nDPI supports USDT (User-level Statically Defined Tracing) probes for zero-overhead dynamic tracing in production environments. Each probe compiles down to a single NOP instruction and incurs no runtime cost when not actively traced. External tools such as bpftrace, perf, and SystemTap can attach to these probes at runtime without restarting or recompiling the application.

Building with USDT Support

Install the required development headers (Linux):

# Debian/Ubuntu
sudo apt-get install systemtap-sdt-dev dwarves

# RHEL/CentOS/Fedora
sudo dnf install systemtap-sdt-devel dwarves

Then configure and build nDPI with USDT support enabled:

./autogen.sh
./configure --enable-usdt-probes --enable-debug-build
make

Note

On macOS, sys/sdt.h is provided by the system. On platforms where it is unavailable, the probes compile to no-ops and have no impact whatsoever.

Note

To allow bpftrace to resolve struct ndpi_flow_struct fields by name without any --include flags, embed BTF into the binaries after building using pahole -J (from the dwarves package). See Struct field access below. Without BTF, the scalar arguments remain fully usable.

Available Probes

Probe Name	Arguments	Description
`flow_classified`	`arg0`: master protocol ID (`u16`) `arg1`: application protocol ID (`u16`) `arg2`: confidence level (`enum`) `arg3`: category (`enum`) `arg4`: flow pointer (`struct ndpi_flow_struct *`)	Fires exactly once per flow when classification is finalised. Covers all exit paths: successful detection, give-up, max-packets reached, nBPF match, and extra-dissector completion. The scalar arguments (`arg0`–`arg3`) support efficient filtering inside bpftrace predicates; `arg4` provides access to all remaining flow fields when deeper inspection is required.
`hostname_set`	`arg0`: hostname string (`char `) `arg1`: flow pointer (`struct ndpi_flow_struct `)	Fires whenever a hostname or SNI is extracted from a flow. Covers all protocols that carry a hostname: TLS (SNI), DNS, HTTP (Host header), QUIC, NetBIOS, DHCP, STUN, and others. The hostname is passed directly as a string for convenience; the flow pointer provides access to all other flow fields.
`fragment_ipv4`	`arg0`: pointer to IPv4 header (`struct ndpi_iphdr *`)	Fires when an IPv4 packet processed by the library is fragmented at the IP layer.
`fragment_ipv6`	`arg0`: pointer to IPv6 header (`struct ndpi_ipv6hdr *`)	Fires when an IPv6 packet processed by the library is fragmented at the IP layer.

bpftrace Notes

Struct field access

To dereference userspace pointers (for example, struct ndpi_flow_struct passed as arg1 in the hostname_set probe) you must embed BTF information into the shared library or executable. The recommended approach is to run pahole -J (from the dwarves package) as a post-build step: it reads the DWARF debug information already present in the binary and inserts a .BTF section with full type information.

# Debian/Ubuntu
sudo apt-get install dwarves

# RHEL/CentOS/Fedora
sudo dnf install dwarves

./configure --enable-usdt-probes --enable-debug-build
make
pahole -J src/lib/libndpi.so
pahole -J example/ndpiReader

Once the .BTF section is present, bpftrace can resolve struct ndpi_flow_struct fields by name — without any --include flags — provided the full binary path is used in the probe specification. Note that the :: shorthand does not trigger BTF lookup:

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:flow_classified {
  $flow = (struct ndpi_flow_struct *)arg4;
  if ($flow->risk != 0) { @risky[arg0] = count(); }
}'

Verify that the section is present with:

readelf -S example/ndpiReader | grep '\.BTF'

BTF Generation — Known Issues and Workarounds

C11 ``_Atomic`` types break pahole BTF encoding

nDPI’s bundled CRoaring library uses _Atomic qualifiers, which the compiler emits as DW_TAG_atomic_type entries in DWARF. All released versions of pahole (including 1.31) abort BTF encoding upon encountering this tag, even when --btf_encode_force is passed:

Unsupported DW_TAG_atomic_type(0x47): type: 0x153c6
Encountered error while encoding BTF.

The workaround used in nDPI’s CI pipeline is to rebuild with CROARING_ATOMIC_IMPL=1, which selects a non-atomic code path and eliminates the offending DWARF entries:

CFLAGS="-DCROARING_ATOMIC_IMPL=1" ./configure \
    --enable-usdt-probes --enable-debug-build
make

Fallback: generate a C header with bpftool

Even with BTF information correctly embedded, pointer dereferences may still fail with:

stdin:1:65-93: ERROR: Cannot resolve unknown type "struct ndpi_flow_struct"

In this case, generate a C header from the BTF information and pass it to bpftrace with --include:

# Generate a C header containing the full layout of the userspace structures
bpftool btf dump file /path/to/ndpiReader format c > ndpi_types.h

# Pass the header to bpftrace
sudo bpftrace --include ndpi_types.h \
  -e 'usdt:/path/to/ndpiReader:ndpi:flow_classified {
    $flow = (struct ndpi_flow_struct *)arg4;
    if ($flow->risk != 0) { @risky[arg0] = count(); }
  }'

bpftrace Map Size Limits

When tracing long or high-throughput captures, bpftrace maps can become full and emit a kernel-level E2BIG warning:

WARNING: Map full; can't update element.
Additional Info - helper: map_update_elem, retcode: -7

Increase the map key limit via the environment variable (supported by all recent bpftrace versions):

sudo BPFTRACE_MAX_MAP_KEYS=100000 bpftrace --include ndpi_types.h \
  -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
    @top[str(arg0)] = count();
  }'

Newer bpftrace builds also accept a config block at the top of the script:

sudo bpftrace --include ndpi_types.h \
  -e 'config = { max_map_keys = 100000 }
usdt:/path/to/ndpiReader:ndpi:hostname_set {
  @top[str(arg0)] = count();
}'

Predicates vs. action blocks

bpftrace predicates (/condition/) work well for filtering on scalar arguments (arg0–arg3 in flow_classified):

bpftrace -e 'usdt::ndpi:flow_classified /arg0 == 91/ { ... }'

Filtering on struct fields via a pointer (e.g., arg4 in flow_classified or arg1 in hostname_set) is not supported in predicates.

Use an if statement inside the action block instead:

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
  $flow = (struct ndpi_flow_struct *)arg1;
  if ($flow->detected_protocol_stack[0] == 5) {
    @dns[str(arg0)] = count();
  }
}'

bpftrace Examples

Note

Examples that dereference a userspace pointer (arg4 in flow_classified, arg1 in hostname_set) require either a .BTF section embedded in the binary via pahole -J or an explicit --include ndpi_types.h header (generated via bpftool btf dump ... format c). See BTF Generation — Known Issues and Workarounds above.

Scalar-only examples (those using only arg0–arg3 in flow_classified or arg0 in hostname_set, without struct dereference) work without BTF or headers and can use the :: shorthand.

List all available probes:

bpftrace -l "usdt:./src/lib/.libs/libndpi.so:ndpi:*"

flow_classified Examples

Real-time protocol classification log:

bpftrace -e 'usdt::ndpi:flow_classified {
  printf("master=%d app=%d confidence=%d category=%d\n",
         arg0, arg1, arg2, arg3);
}'

Protocol distribution histogram:

bpftrace -e 'usdt::ndpi:flow_classified {
  @proto_master[arg0] = count();
}'

Confidence level breakdown:

bpftrace -e 'usdt::ndpi:flow_classified {
  @confidence[arg2] = count();
}'

Category distribution:

bpftrace -e 'usdt::ndpi:flow_classified {
  @category[arg3] = count();
}'

Count unclassified flows (master protocol == 0):

bpftrace -e 'usdt::ndpi:flow_classified /arg0 == 0/ {
  @unknown = count();
}'

Flow classification rate (flows/sec):

bpftrace -e 'usdt::ndpi:flow_classified {
  @ = count();
} interval:s:1 { print(@); clear(@); }'

Filter by a specific protocol (e.g., TLS = 91):

bpftrace -e 'usdt::ndpi:flow_classified /arg0 == 91/ {
  @tls[arg1] = count();
}'

Flows classified under SocialNetwork (category 6):

bpftrace -e 'usdt::ndpi:flow_classified /arg3 == 6/ {
  @social[arg0, arg1] = count();
}'

Flows with a non-zero risk bitmap (requires BTF or --include):

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:flow_classified {
  $flow = (struct ndpi_flow_struct *)arg4;
  if ($flow->risk != 0) {
    @risky[arg0] = count();
  }
}'

hostname_set Examples

Real-time hostname log:

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
  $flow = (struct ndpi_flow_struct *)arg1;
  printf("%s (master=%d app=%d)\n",
         str(arg0),
         $flow->detected_protocol_stack[0],
         $flow->detected_protocol_stack[1]);
}'

Top hostnames by flow count:

bpftrace -e 'usdt::ndpi:hostname_set {
  @top[str(arg0)] = count();
}'

Monitor a specific domain (e.g., all *.google.com traffic):

bpftrace -e 'usdt::ndpi:hostname_set /strcontains(str(arg0), "google.com")/ {
  @google[str(arg0)] = count();
}'

Hostnames resolved via DNS only (DNS = 5):

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
  $flow = (struct ndpi_flow_struct *)arg1;
  if ($flow->detected_protocol_stack[0] == 5) {
    @dns[str(arg0)] = count();
  }
}'

TLS SNI extraction in real time (TLS = 91):

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
  $flow = (struct ndpi_flow_struct *)arg1;
  if ($flow->detected_protocol_stack[0] == 91) {
    printf("TLS SNI: %s\n", str(arg0));
  }
}'

Hostname-to-application-protocol breakdown:

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
  $flow = (struct ndpi_flow_struct *)arg1;
  @host_app[str(arg0), $flow->detected_protocol_stack[1]] = count();
}'

Hostname resolution rate (hostnames/sec):

bpftrace -e 'usdt::ndpi:hostname_set {
  @ = count();
} interval:s:1 { print(@); clear(@); }'

Detect potential DGA activity (count unique DNS hostnames over time):

bpftrace -e 'usdt:/path/to/ndpiReader:ndpi:hostname_set {
  $flow = (struct ndpi_flow_struct *)arg1;
  if ($flow->detected_protocol_stack[0] == 5) {
    @unique_dns = count();
  }
} interval:s:10 {
  printf("Unique DNS hostnames in last 10s: %d\n", @unique_dns);
  clear(@unique_dns);
}'

perf Example

Record probe hits with perf:

perf probe -x ./src/lib/.libs/libndpi.so sdt_ndpi:flow_classified
perf record -e sdt_ndpi:flow_classified -p $(pidof ndpiReader) -- sleep 10
perf report

Overhead

When not tracing: zero overhead. Each probe compiles to a single NOP instruction.
When actively tracing: approximately 2–5 microseconds per probe hit, depending on the tracing tool and the complexity of the attached script.
flow_classified and hostname_set fire at most once per flow (not per packet), so their overhead remains negligible even under active tracing at high traffic volumes. fragment_ipv4 and fragment_ipv6 fire once per fragmented packet.