PF_RING FT (Flow Table)

Fast, Assisted Flow Processing and L7 Protocol Detection


Most Network monitoring and security applications are based on flow processing, that includes packet capture, decoding and classification. PF_RING™ is a flexible framework that can be used to accelerate the packet capture, leveraging on PF_RING™ ZC drivers or specialized adapters, and extract packet metadata. This let the application focus on packet processing, rather than dealing with packet capture and packet parsing, while running with the best performance.
PF_RING™ FT is taking one step further, it assists any flow processing application in the packet classification activity. PF_RING™ FT implements a flow table that can be used to keep track of flows and provides many hooks to be able to customize and extend it for building any type of application on top of it, including probes, IDSs, IPSs, L7 firewalls.
Although PF_RING™ FT is distributed with PF_RING™, it is possible to use the library with any third-party packet capture framework (including Libpcap and DPDK), as its data-ingestion API is capture-agnostic.

Clean and Simple API

Designing and implementing a flow processing application on top of PF_RING™ FT is quite straightforward as it provides a clean API that can be used to do complex things in a few lines of code. The following code snippet shows how it is easy to capture traffic and export flow informations with PF_RING™ FT. For a full code example have a look at the demo applications available in PF_RING™. For more information please refer to the guide and API documentation.

ft = pfring_ft_create_table(0);
pfring_ft_set_flow_export_callback(ft, processFlow, NULL);

while (1) {
if (pfring_recv(pd, &packet, 0, &header, 0) > 0)
action = pfring_ft_process(ft, packet, &header);
}

void processFlow(pfring_ft_flow *flow, void *user){
pfring_ft_flow_key *k = pfring_ft_flow_get_key(flow);
pfring_ft_flow_value *v = pfring_ft_flow_get_value(flow);
/* flow export here with metadata in k and v */
}

Event Hooks

The flow of an application designed on top of PF_RING™ FT is event-driven. Through a few hooks it is possible to register to events like “new flow”, “flow expired”, “packet classified”, and access the flow informations in order to compute actions based on the flow status. Flow informations can be extended with custom metadata defined by the application.

nDPI Integration

PF_RING™ FT is natively integrated with nDPI for providing L7 protocol informations out of the box. The application itself does not need to deal with the nDPI library directly as everything happens behind the scenes, getting the L7 protocol is just as easy as enabling L7 detection through the API and reading the L7 protocol from the flow metadata. In addition to protocol detection, PF_RING™ FT also categorizes the traffic leveraging the nDPI categories and Intel Hyperscan support for pattern matching.

L7 Filtering and Shunting

PF_RING™ FT features a L7 filtering engine that can be used by inline applications for filtering flows based on the application protocol. In addition to the built-in filtering engine, the application can mark flows for filtering or shunting them based on custom policies.

IDS Acceleration

The PF_RING™ FT L7 filtering engine can also be used for accelerating CPU-bound applications, such as IDS/IPSs including Suricata, Bro and Snort, shunting flows based on the application protocol. Discarding elephant flows is becoming a common yet effective practice for reducing the amount of traffic an IDS/IPS need to inspect (typically multimedia traffic), dramatically reducing packet loss and improving the system performance.
Suricata has native support (bypass) for shunting elephant flows using eBPF, this means that the application is injecting filtering rules (5-tuples) in kernel space as soon as an elephant flow is detected. This approach has come limitations: it requires a ruleset in Suricata able to detect all multimedia protocols, packet parsing is not flexible as eBPF programs cannot loop (it does not work with encapsulations, including vlan and QinQ), it cannot keep flow state (making it complicated to handle flows expiration).
Leveraging on PF_RING™ FT, a PF_RING-based or Libpcap-based application can take advantage of L7 shunting without changing a single line of code. Since PF_RING™ FT is based on nDPI for protocols detection, filtering multimedia traffic mens just listing the protocol names in a configuration file. More information for accelerating Suricata or other IDSs can be found in the user’s guide.

Performance

PF_RING™ FT is highly optimized and it is capable of processing 10 Gbit line-rate using a single CPU core on a low-end Xeon E3, and scale up to 100 Gbit on many-core systems. You can test this yourself using the demo applications available on github.

Traffic Capture Card Number of Cores Per Core Performance All Cores Performance
10 Gbit / 1-kbyte packets Intel 10G 1 1.2 Mpps / 10 Gbps 1.2 Mpps / 10 Gbps
10 Gbit / 64-byte packets Intel 10G 1 14.8 Mpps / 10 Gbps 14.8 Mpps / 10 Gbps
100 Gbit / 1-kbyte packets FPGA 100G 1 10.8 Mpps / 90 Gbps 10.8 Mpps / 90 Gbps
100 Gbit / 1-kbyte packets FPGA 100G 4 2.8 Mpps / 24 Gbps 11.5 Mpps / 96 Gbps
100 Gbit / 1-kbyte packets FPGA 100G 6 + 6 (2 CPUs) 0.95 Mpps / 8 Gbps 11.5 Mpps / 96 Gbps
100 Gbit / 64-byte packets FPGA 100G 1 11.3 Mpps / 7.6 Gbps 11.3 Mpps / 7.6 Gbps
100 Gbit / 64-byte packets FPGA 100G 4 11.2 Mpps / 7.6 Gbps 45.2 Mpps / 30.4 Gbps
100 Gbit / 64-byte packets FPGA 100G 6 + 6 (2 CPUs) 10.8 Mpps / 7.3 Gbps 130 Mpps / 87.6 Gbps

The table above shows the results of performance tests using:

  • PF_RING FT 7.1
  • Ubuntu Linux 16.04
  • CPU Intel E3-1230 v5 3.4GHz DDR4 2133 (10 Gbit Tests)
  • CPU 2x Intel E5-2630 v2 2.6GHz DDR3 1600 (100 Gbit Tests)

Operating Systems

License

PF_RING™ FT is distributed under the EULA and requires a license per system.

Get It

If after testing the PF_RING™ FT you decide to use it permanently, you need a license. The PF_RING™ FT is available from the ntop web site. If you are interested in large quantities or if you need a volume discount please contact us. Instead if you are looking for the software, you can download it here.