Zero copy technologies such as PF_RING ZC allow applications to read packets in memory without any actor involved, being it the kernel or a memory copy. This is the reason why using ZC you can easily fill up a 10 Gbit line using a single thread and a single network card queue. The drawback of zero copy is that applications must be well behaved as the same packet is shared across multiple applications and thus if one application pollutes the packet memory, this problem affects all the consumers. The same problem happens in case an application crashes and writes in (unexpected) packet memory locations, corrupting other processe’s memory.
You might say “ok but apps should not crash nor misbehave”. This is a fair statement but sometimes this might not be the case. For instance many Bro users use PF_RING to accelerate packet capture by leveraging on PF_RING’s speed and ability to distribute the traffic across multiple consumer queues using the zbalance_ipc clustering application. Unfortunately sometimes Bro workers die unexpectedly and this requires zbalance_ipc to be restarted as a worker might have corrupted the memory of other workers. Even though in practice our “zbalance_ipc restart required” policy is over-safe as Bro is unlikely to corrupt the memory, we cannot risk that a fault on one worker creates problems to all the others.
In order to overcome this problem, there is a simple solution you can use to isolate the various packet consumers so that they can be restarted independently in case of crash. Load the dummy network driver specifying the number of ‘fake’ interfaces, matching the number of application instances you need for load balancing the traffic.
Example for 4 consumers, do:
modprobe dummy numdummies=4 ifconfig dummy0 up ifconfig dummy1 up ifconfig dummy2 up ifconfig dummy3 up
Run zbalance_ipc (make sure you use the latest PF_RING code), specifying the number of consumers application, remapping each queue to a ‘safe’ dummy interface using -r <queue id>:<dummy interface> for each queue. In the example below zbalance_ipc is load balancing the traffic from p2p2 (-i zc:p2p2) across 4 queues/consumers (-n 4) using an IP-based hash function (-m 1) remapping queue 0 to the dummy interface dummy0 and so on.
zbalance_ipc -i zc:p2p2 -n 4 -m 1 -c 2 -r 0:dummy0 -r 1:dummy1 -r 2:dummy2 -r 3:dummy3
You can now run an application on each ‘dummy’ interface.
# ./pfcount -i dummy0 Using PF_RING v.6.3.0 Capturing from dummy0 [86:1E:71:67:FA:40][ifIndex: 56] # Device RX channels: 1 # Polling threads: 1 Dumping statistics on /proc/net/pf_ring/stats/8712-dummy0.387 ========================= Absolute Stats: [1472571 pkts rcvd][0 pkts dropped] Total Pkts=1472571/Dropped=0.0 % 1'472'571 pkts - 123'695'964 bytes ========================= ========================= Absolute Stats: [2947157 pkts rcvd][0 pkts dropped] Total Pkts=2947157/Dropped=0.0 % 2'947'157 pkts - 247'561'188 bytes [2'946'938.92 pkt/sec - 1'980.34 Mbit/sec] ========================= Actual Stats: 1474586 pkts [1'000.07 ms][1'474'476.88 pps/0.99 Gbps] =========================
Now all you have to do is to configure Bro to capture packets from the dummy interfaces instead of reading them directly from zbalance_ipc.