This tiny device that can fit on your hand, has been integrated with ntop for the purpose of traffic monitoring as you can read on this technical note Enabling ntop Packet Monitoring with Cisco OnPlus Service.

Taps are important as they allow network probes to operate passively without interfering with network operations. The two traffic directions (A to B and B to A) are plugged into two network ports of the [...]]]> In network monitoring it is very common to use taps for duplicating network traffic (RX and TX directions).

Taps are important as they allow network probes to operate passively without interfering with network operations. The two traffic directions (A to B and B to A) are plugged into two network ports of the probe. Having the two directions separated has advantages (e.g. packets are not mixed across directions) and disadvantages. The main disadvantage is that when reading packets from the two interfaces, it is not possible to know which packet out of the two directions arrived first. For many applications (e.g. NetFlow traffic monitoring) this is not a problem, but for some others it is necessary to know the exact order of the packets. This is because if two hosts, one on the A side and the other on the B side, communicate over  a protocol that has some kind of state, the probe must see messages in the exact order they hit the wire. If this does not happen, it might be confused and not operate as expected. We call this precise interface merging (PIM).

One of the myths about PIM is that it is necessary to use costly hardware timestamp-based NICs that are able to know the exact time (usually with a precision of 50 or 100 nsec) when incoming packets hit the probe interface. While hardware timestamps can definitively be used for PIM, we claim that they in practice are not necessary as commodity adapters with DNA and without hardware timestamps are enough for this task. We’re now going to explain why this statement holds.

With the advent of PF_RING DNA, it is possible to capture packets on multiple interfaces at line rate 1/10 Gbit with no loss. At 10 Gbit line rate, the adapter receives a packet every 67.2 nsec (1 sec /14.880.000 pps). Modern adapters as those based on the Intel 82599 chip can buffer up to 4096 packets in memory that is 275.2 usec (4096 * 67.2). As DNA operates at line rate, it means that it is able to consume packets as fast as the network (otherwise we wouldn’t be able to operate at line rate). With DNA we have written an example application named pfcount_aggregator that is able to read packets from two interfaces simulatenously. Using this application it is possible to read packets in order as two peers (one on the A and the other on the B side) that exchange data over a state-machine based protocol (i.e. A sends a request, and B replies with a response to the previous request) such as GTP can safely assume that packets are read in order by the aggregator. This is because the time it takes for a request (as seen on the first tap-ed interface) to go from A to B, and the response from B to A (as seen on the second tap-ed interface) is definitively more (in particular for a WAN) than 275.2 usec. This means that by the time the aggregator has read the request, it has also read all the following packets (up to the buffer length) and thus it’s impossible to read the response packet before the request. Basically it’s impossible for the probe to see the response before the request when an application such as pfcount_aggregator is used. Note that this is as such, because of DNA as with the NAPI-based polling model where interfaces can be polled sequentially, this property might not be true.

On a nutshell hardware timestamps and costly NICs are not necessary for PIM. This contrary to the common belief that without them PIM wouldn’t happen. Juvenal said 2000 years ago: Quis custodiet ipsos custodes? Have fun with DNA and commodity adapters.

In the past months I have received a few emails about how to position DNA with respect to netmap. To many people they look like two competing solutions, but in reality they are just two solutions to the same problem. Yesterday I had a nice meeting with Luigi Rizzo, the author of netmap.

I personally know Luigi since almost 15 years as we both live pretty close. The first time I saw him (1999 or so) he was hacking a driver for a CD-ROM drive on FreeBSD while speaking with me. Impressive. He was telling me about projects he previously did, including a PC-based bridge able to run a few university departments. I have to admit that if I have started to code in the kernel, it’s also because he has inspired me showing that the kernel was a radical new (for me) way of looking at things. I decided to go for Linux, he’s a recognized FreeBSD contributor, but our view are not that far.

Even if DNA and netmap have been developed in totally independent ways, they solve the same problem: how to move packets back/forth a network adapter without using too many CPU cycles. And I believe it’s no surprise that the performance of DNA and netmap is basically the same. So yesterday we’ve not discussed about who’s faster, but we talked about where we wanted to drive our technologies.

As you can see from the backboard (pictures are courtesy of our common friend Gianmarco), we have some ideas on the next thing to do. First of all, both DNA and netmap are not the ultimate goal, but instead some enabling technologies for improving networking on operating systems. They are not revolutionary in the sense that the idea of memory mapping is out for years, but companies like Endace and Napatech (that used it for so long time) have been unable to use it properly and thus drive innovation anywhere. If you look at at their technology 15 years ago and today, you will see that nothing changed: just pass me the packet and I will do the rest. In my view (and I think I can also mostly speak for Luigi), we need to go beyond simple packet juggling. Modern network and applications are significantly different from similar applications developed in the last decade. Operating systems changed too, and thus it’s time to rethink a few things, and not just to wait until somebody else would do that. Stay tuned!

For 2012 we have [...]]]> From time to time the kernel folks are sick and tired of people saying PF_RING is better than what we have upstream, it really isn’t. Fortunately (for PF_RING) the story is a bit different not to mention that some of PF_RING features such as clustering have probably inspired AF_PACKET too.

For 2012 we have planned to make some enhancements to PF_RING and (we’ll be doing much more but this is just the next thing that will see the light) add one of the last missing features you can find on costly FPGA-based NICs. No, I am not talking about multi-10G packet capture/transmission speed, low latency, IDS/IPS acceleration, virtual machine support, or hardware packet filtering: all this is already part of PF_RING. What we are currently implementing is the ability to support TX and RX hardware timestamps as those offered by Intel 82580 NICs. Some of you might wonder why it’s so important. The answer is manyfold:

• RX and TX timestamps are the recipe for computing one way delay.
• The use of splitter or network partitions prevents receiving all packets from a single NIC. As network monitors might need to know which packet came first out of N interfaces, the only way of doing that is via packet timestamps that can enable us comparing packet arrival timestamps.

PF_RING filters have been designed to be efficient and versatile. PF_RING-based applications can use them for both reducing the amount of packets they need to process, and passing incoming packets to kernel plugins for further processing.

Years ago, hardware packet filtering was limited to costly FPGA-based NICs, whereas today it is available also on [...]]]> Introduction

PF_RING filters have been designed to be efficient and versatile. PF_RING-based applications can use them for both reducing the amount of packets they need to process, and passing incoming packets to kernel plugins for further processing.

Years ago, hardware packet filtering was limited to costly FPGA-based NICs, whereas today it is available also on commodity adapters such as Silicom 1/10 Gbit Director card, and Intel 82599-based 10 Gbit network adapters.

Filtering in PF_RING 5.2

Although past PF_RING versions supported limited hardware filtering, in PF_RING 5.2 we have completely redesigned packet filtering so that applications can use a single and consistent packet filtering API, letting PF_RING do the magic based on the hardware NIC being used. This means that you need to code the application once, regardless of the type of NIC being used, and let PF_RING set the native filter type. As you can see from the figure below there are many combinations we support in PF_RING, and using a single API is pre-requisite for avoid wasting time supporting all of them.

When you set a filter, you can either specify it as NIC native filter (e.g. silicom_redirector_hw_rule) or as PF_RING generic filter (e.g. filtering_rule/hash_filtering_rule). In the former case you have full control over the filter at the cost of being unable to seamlessly run the code over various adapter types. In the latter case, you code once and let PF_RING convert the filter format into native, hardware filter, based on the NIC type you use.

Although all hardware filter types are very efficient being them executed in hardware, they are not all alike. As of today we support:

• 82599-based NICs (see the Intel 82599 datasheet for more details)
• Up to 32k FlowDirector filters (IPv4/v6 hash_filtering_rule in PF_RING parlance)
• Up to 128 5-tuple filters (IPv4/v6 filtering_rule in PF_RING parlance)
• Silicom Director
• Up to 1’000 filters per port (IPv4-only flexible filters)

In the case of Silicom Director, it is also possible to execute in hardware actions across ports (e.g. mirror packets matching a given filters across ports); on non-Director NICs you can do that  too at the cost of doing it in the CPU.

Using Hardware Filters

In order to demonstrate the power and usefulness of PF_RING filters, we have enhanced the pfcount demo application so that it adds a filter (-u command line option) for dropping incoming packets. The pfcount_82599 demo application shows how to set native filters.

Hardware Filtering in Snort

For all snort users, we have enhanced our PF_RING DAQ module, so that snort can transparently set “drop filters” whenever bad packets are detected. This is a fantastic way to address DDoS attacks and drop in hardware nasty packets that would waste too many CPU cycles.As snort PF_RING DAQ can work in both IDS (Intrusion Detection System) and IPS (Intrusion Prevention System) mode, so hardware filtering in IPS mode is a very nice thing to have.

If You Don’t Own a Hardware-Filtering NIC…

Software PF_RING filters are still enabled when hardware filtering is used. This because it is not always possible to filter all the traffic you would like in hardware. So you can use PF_RING filtering as you have done so far, although you have to be aware that switching to a hardware filtering NIC allows your CPU to offload filters to the NIC and thus preserve precious CPU cycles for packet processing.

Notes on 82599 filters

• In order to enable hardware filters you need to load the ixgbe module with specific parameters. Example: insmod ixgbe.ko  RSS=0,0,0,0 FdirMode=2,2,2,2 FdirPballoc=3,3,3,3
• Filters are not used just for pass/drop actions, but also for steering packets to specific cores/RX queues. For instance you can say: all incoming TCP packets on port 80 must be diverted to RX queue 3. So you can bind you HTTP analysis application to queue ethX@3 and forget all other queues.
• You can simultaneously set both 5-tuple and FlowDirector filters. However there are some limitations in the silicon, so please refer to the 82599 datasheet for more information about this subject.

In this release we introduce (PF_RING 5.0 was lacking TNAPI as we were busy coding [...]]]> PF_RING 5.1 is a maintenance release that addresses some issues we identified in 5.0 that we released early this month. We have listen to your comments and tried to improve our software applications both in terms of stability and speed.

In this release we introduce (PF_RING 5.0 was lacking TNAPI as we were busy coding this new TNAPIv2) a new version of TNAPI (v2) that has been completely rewritten based on the lessons learnt with DNA. The performance improvement with respect to v1 has been major. Just to give you an idea with a low end Core2Duo and a single RX queue, you can achieve wire-rate 1 Gbit packet capture (with v1 we stopped at 750 Kpps) and over 2.8 Mpps at 10 Gbit (with v1 we could’t go above 1 Mpps). TNAPI is now able to both receive and send packets; unfortunately we need to work a bit more on the PF_RING side in order to exploit TX, thus even if TNAPIv2 features packet transmission, we have not yet enabled until we complete developments on the PF_RING side.

We remind you that beside speed, the fundamental difference between DNA and TNAPI is that on DNA the interface is piloted by a single (i.e. quality one) user-space application, whereas TNAPI can be used to capture packets that will be dispatched to multiple PF_RING-based applications. The cost of dispatching is paid on performance as with DNA you can have wire speed with any packet size, whereas TNAPI features wire-rate at 1 Gbit and over 5 Mpps packet capture at 10 Gbit on low-end Xeon boxes. Please note that in order to use TNAPIv2 you must use PF_RING 5.1.

Finally, in order to figure out immediately what interface type we’re using, we have renamed the interfaces. The new standard for network interfaces is:

• ethX: vanilla or PF_RING-aware driver
• tnapiX: TNAPI driver
• dnaX: DNA driver

For instance:

tnapi0    Link encap:Ethernet  HWaddr 00:1b:21:a1:cc:20
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:e0320000-e0340000 

tnapi1    Link encap:Ethernet  HWaddr 00:1b:21:a1:cc:21
          inet6 addr: fe80::21b:21ff:fea1:cc21/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:e0340000-e0360000