The current nProbe 8.0 release contains many changes with respect to the 7.x series. We have optimised the code, added the ability to collect non standard fields (e.g. Cisco AVC), improved Kafka export, and reworked many tiny details to make the tool a stable solution for all those looking for a flexible and versatile flow probe and collector.
For all those interested in the whole changelog, below you can find the main changes we have implemented in the past months. In summary we have made nProbe better adding new extensions, opening it to new encapsulations, and extending the collection capabilities.
- Main New Features
- Implemented realtime interface stats via ZMQ to ntopng
- Reworked packet fragmentation support that was not properly rebuilding packet fragments
- Many tiny bugs fixed that increase stability and metrics reliability
- Implemented BPF filtering with PF_PACKET directional sockets
- Added VXLAN support
- Created multiple kafka publishers to enhance performance
- Implemented options template export via Kafka
- Added support for collection of IXIA URI and Host
- Added @SIP@ and @RTP@ plugin shortcuts for VoIP analysis
- Improved SSL dissection
- Added support for GTPv2 PCO
- Added support for IPFIX flowEndMilliSeconds when observationTimeMilliSeconds (often in Cisco ASA)
- Added ability to export sFlow interface counters via ZMQ
- Added drops (export/elk/too many flows) drops
- Added kflow export (kentik.com)
- New Options
- –upscale-traffic to scale sampled sFlow traffic
- –kafka-enable-batch and –kafka-batch-len to batch flow export to kafka
- –load-custom-fields to support custom fields shipped with NetFlow (see https://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/)
- –max-num-untunnels to decapsulate up to 16 tunnelling levels.
- –vlanid-as-iface-idx to use the VLAN tag as the interface index
- –zmq-disable-compression to disable ZMQ data compression
- Extensions
- Implemented min/avg/max throughput with %SRC_TO_DST_MIN_THROUGHPUT %SRC_TO_DST_AVG_THROUGHPUT %SRC_TO_DST_MAX_THROUGHPUT %DST_TO_SRC_MIN_THROUGHPUT %DST_TO_SRC_AVG_THROUGHPUT %DST_TO_SRC_MAX_THROUGHPUT
- Added support in collection of %IN_SRC_MAC %OUT_DST_MAC %FRAGMENTS %CLIENT_NW_LATENCY_MS %SERVER_NW_LATENCY_MS %APPL_LATENCY_MS %RETRANSMITTED_IN_PKTS %RETRANSMITTED_OUT_PKTS %OOORDER_IN_PKTS %OOORDER_OUT_PKTS
- Split %FRAGMENTS IN %SRC_FRAGMENTS and %DST_FRAGMENTS
- Added %NPROBE_IPV4_ADDRESS to export the IP address of the nProbe sensor, whereas %EXPORTER_IPV4_ADDRESS contains the IP address of the flow exporter (e.g. the router that generated the exported flow)
- Implemented %ICMP_IPV4_TYPE, %ICMP_IPV4_CODE, %FLOW_DURATION_MILLISECONDS, %FLOW_DURATION_MICROSECONDS, %FLOW_START_MICROSECONDS, %FLOW_END_MICROSECONDS
- VXLAN VNI exported in %UPSTREAM_TUNNEL_ID and %DOWNSTREAM_TUNNEL_ID