We discussed many times about the large quantity of work IDSs have to carry on, and the high CPU load they require, this is the case of Suricata due to the thousands of rules that need to be evaluated for every single packet, but this is also the case of the Bro Network Security Monitor.
In a previous post we’ve seen How to accelerate Suricata with PF_RING FT in a few steps. In that guide we leveraged on the flow classification and L7 protocol detection provided by PF_RING FT, to filter and shunt meaningless traffic, reducing this way most of the (unneeded) load on the IDS.
In fact, as we’ve already seen, most of the Internet traffic is multimedia traffic (or encrypted traffic) that the IDS does not care about, and this traffic mainly consists of elephant flows.
In this post we will see how to enable PF_RING FT to accelerate Bro, with no change or recompilation to the application (PF_RING FT is already part of PF_RING, if you are using PF_RING or Libpcap-over-PF_RING as capture library you are ready to go).
Running FT with Bro
First of all we need to compile Bro on top of PF_RING. PF_RING should be installed first:
cd ~ git clone https://github.com/ntop/PF_RING.git cd ~/PF_RING/kernel make && sudo make install cd ~/PF_RING/userland/lib ./configure && make && sudo make install
Then we can compile Bro linking it to the PF_RING-aware libpcap:
cd ~ wget https://www.bro.org/downloads/release/bro-X.X.X.tar.gz tar xvzf bro-X.X.X.tar.gz cd bro-X.X.X ./configure --with-pcap=/usr/local/lib make && make install
We can make sure that Bro is linked to the PF_RING-aware libpcap with the command below:
ldd /usr/local/bro/bin/bro | grep pcap
libpcap.so.1 => /usr/local/lib/libpcap.so.1 (0x00007fa371e33000)
At this point we can set pf_ring as capture method and the interface that we want to use in /usr/local/bro/etc/node.cfg. We can also set the number of threads and the cpu affinity in the same file. In this example we will use a basic configuration with just one thread, please read the PF_RING documentation for more details about the Bro configuration and loading PF_RING and ZC drivers for maximum capture performance.
[worker-1] type=worker host=localhost interface=eth1 lb_method=pf_ring lb_procs=1 pin_cpus=2
As we said PF_RING FT is already part of PF_RING, however in order to be able to detect and filter L7 protocols, we need to install nDPI:
cd ~ git clone https://github.com/ntop/nDPI.git ./autogen.sh make && make install
In order to filter or shunt out multimedia or other meaningless traffic, we just need to create a simple configuration file with the list of application/protocol names: PF_RING FT is based on the nDPI Deep Packet Inspection library for protocols detection, you can specify all the protocols detected by nDPI.
In the example below, we are listing a few L7 protocols that we want to discard, and we are telling the FT library that we want to shunt SSL, specifying the number of packets to forward for each session before discarding them:
cat /etc/pf_ring/ids-rules.conf [filter] YouTube = discard Netflix = discard Spotify = discard [shunt] SSL = 10
In order to enable L7 filtering all we need to do is setting the PF_RING_FT_CONF environment variable with the path of the configuration file with the filtering rules. This can be done in Bro using the env_vars setting in /usr/local/bro/etc/node.cfg.
[worker-1] type=worker host=localhost interface=eth1 lb_method=pf_ring lb_procs=1 pin_cpus=2 env_vars=PF_RING_FT_CONF=/etc/pf_ring/ft-rules.conf
At this point we are able to run Bro.
/usr/local/bro/bin/broctl [BroControl] > start
Validation
For validating this work we used a huge PCAP file with mixed internet traffic, and we replayed the PCAP file using disk2n at 4 Gigabit / ~500 Kpps. We have created a simple Bro configuration, with a single worker in order to check the limit of a single processing unit, using PF_RING ZC drivers for reducing the packet capture overhead and remove any noise. We ran a few tests with and without PF_RING FT filtering out all multimedia/meaningless data (NetFlix, Youtube, Spotify, Google, Facebook, SSL) and we compared the results.
Without PF_RING FT Bro dropped 3.1 Mpkts out of 7.1 Mpkts transmitted, 44% of packet loss. After enabling PF_RING FT, Bro was able to process the same traffic dropping just 0.8 Mpkts, 12% of packet loss, out of 7.1 Mpkts transmitted, as a huge amount of data has been filtered out by FT. After the amazing results with Suricata, we’ve seen that PF_RING FT can be used to provide a huge performance improvement also with Bro, and all this with minimal effort.
Have fun!