2. Accolade Support

2.1. Prerequisite

PF_RING has native support for Accolade adapters, the Accolade SDK (v.1.2.26 or later) needs to be installed (under /opt/accolade) in order to enable the Accolade module at runtime.

2.2. Installation

In order to install the Accolade SDK and drivers run the following commands:

tar xvzf SDK_*.tgz
cd SDK_*
cd drv; sudo make install; cd ..
cd lib; sudo make install; cd ..
sudo sh -c "echo /opt/accolade/lib/ > /etc/ld.so.conf.d/accolade.conf"

Before running any application please load the driver with:

sudo sh -c "echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages"
sudo insmod /opt/accolade/driver/anic_mem.ko mbytes_per_device=64
sudo insmod /opt/accolade/driver/anic.ko

Note: with the SDK version <=1.2.26.20180510 the anic_load utility should be used to load the ‘anic’ module:

sudo sh -c "echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages"
sudo insmod /opt/accolade/driver/anic_mem.ko mbytes_per_device=64
sudo /opt/accolade/bin/anic_load

Please note that FEC (Forward Error Correction) is disabled by default on Accolade adapters. If you experience issues bringing up the link when connecting Accolade to other adapters, you probably need to enable it (e.g. FM10K on SR4 media run with FEC enabled). In order to enable it please run anic_load as follows:

/opt/accolade/bin/anic_load fec_enabled=1,1

If you are running an old firmware and you need to update it, you should run the following commands:

cd SDK_*/tools/; make
gunzip fpga_*.rbt.gz
sudo ./anic_fwupdate --id 0 --script fpga_*.rbt
sudo reboot

Note: if you get a “Device or resource busy” failure message running anic_fwupdate, please try killing the mm_tmon process.

Now you are ready to compile and run PF_RING with Accolade support. Note that if you are installing pfring from packages, Accolade support is already enabled.

cd PF_RING/kernel; make
sudo insmod pf_ring.ko; cd ..
cd userland; ./configure
cd lib; make; cd ..
cd libpcap; ./configure; make; cd ..
cd examples; make
sudo ./pfcount -i anic:0

2.3. Packet Capture

In order to open port 0 from adapter 0 you should specify anic:DEV:PORT, example:

pfcount -i anic:0:0

or just anic:PORT when using the default adapter 0, example:

pfcount -i anic:0

Opening a port this way, the full card is initialised, causing issues when opening other ports later (previous ports may stop working), this can be avoided using the port-to-ring binding as explained later on.

2.4. Load-Balancing

The anic_mfl_config tool included in the Accolade SDK can be used to aggregate traffic from multiple ports and setup the card for load-balancing (similar to RSS) and multi-process applications.

Example of ports aggregation and load-balancing to 2 rings:

anic_mfl_config -i 0 --mode=2

Example of port-to-ring (ring 0 is port 0) binding:

anic_mfl_config -i 0 --mode=port

In order to open ring 0 from adapter 0 you should specify anic:DEV@RING, example:

pfcount -i anic:0@0

Note: on SDK version >1.2.26.20180510 the default Accolade ‘blocks’ setting has been changed and you might get errors like “ANIC_block_add(ring:0 buf:16) failed, oversubscribed?” There are two options for setting the number of blocks and solve this error:

1. run anic_mfl_config with –blocks=16 (new pf_ring default)
2. set the ANIC_RING_BLOCKS env var to the number set in anic_mfl_config when running pf_ring as anic_mfl_config consumer

Example:

anic_mfl_config -i 0 --mode=port --blocks=64

ANIC_RING_BLOCKS=64 pfcount -i anic:0@0

2.5. Accolade and Packet Copy

If you use the PF_RING (non-ZC) API packets are read in zero-copy. Instead if you use PF_RING ZC API, a per-packet copy takes place, which is required to move payload data from Accolade memory to ZC memory. Keep this in mind!

2.6. Hw Filtering

Accolade adapters support packet filtering in hw. In order to set an hw filter there are two options:

• Using the standard BPF filter: PF_RING thanks to the nBPF library automatically translates BPF filters into hw filters
• Using the pfring_add_hw_rule() API.

Sample code for filtering traffic with Accolade (as well as with other adapters) is available in the pfcount.c sample application (look for sample_filtering_rules).

When using the pfring_add_hw_rule() API, as first action we need to set the default behaviour for packets, this can be ‘pass’ or ‘drop’. Example:

hw_filtering_rule r;
r.rule_family_type = accolade_default;
r.rule_family.accolade_rule.action = accolade_pass;
pfring_add_hw_rule(pd, &r);

In order to set a filtering rule, we need to create a rule and assign a rule ID, which is a unique identifier for the rule. A standard Accolade firmware supports up to 32 rules (called ‘legacy mode’ or ‘mode 1’), with IDs from 0 to 31. Enhanced Accolade firmwares for 100 Gbit adapters can support up to 1000 rules, with IDs from 0 to 999 (called ‘mode 2’). PF_RING automatically select ‘mode 2’ when available, and ‘mode 1’ as fallback. It is possible to use the FILTERING_RULE_AUTO_RULE_ID macro as rule.rule_id in order to automatically assign the next available rule ID. Example of setting a filtering rule with ‘drop’ action for an IPv4 packet:

hw_filtering_rule r = { 0 };
r.rule_id = rule_id++;
r.rule_family_type = accolade_rule;
r.rule_family.accolade_rule.action = accolade_drop;
r.rule_family.accolade_rule.ip_version = h->extended_hdr.parsed_pkt.ip_version;
r.rule_family.accolade_rule.src_addr_bits = 32;
r.rule_family.accolade_rule.src_addr.v4 = h->extended_hdr.parsed_pkt.ipv4_src;
r.rule_family.accolade_rule.dst_addr_bits = 32;
r.rule_family.accolade_rule.dst_addr.v4 = h->extended_hdr.parsed_pkt.ipv4_dst;
r.rule_family.accolade_rule.protocol = h->extended_hdr.parsed_pkt.l3_proto;
r.rule_family.accolade_rule.src_port_low = h->extended_hdr.parsed_pkt.l4_src_port;
r.rule_family.accolade_rule.dst_port_low = h->extended_hdr.parsed_pkt.l4_dst_port;
pfring_add_hw_rule(pd, &r);

Please note that all fields are in host byte order.

For a full list of supported fields please take a look at the hw_filtering_rule struct. Please also note that mode 1 and 2 support different fields, please refer to the fields description to check what is supported in each mode.

Example of removing a filtering rule by id:

pfring_remove_hw_rule(pd, rule_id);

2.6.1. Steering

Packet steering across rings is also supported in order to forward selected traffic to a specific queue for processing. In order to create a steering rule, the steer_to_ring field of the accolade_hw_rule structure should be set to 1, and the destination ring ID should be set in the ring_id field of the same struct. Example:

hw_filtering_rule r = { 0 };
u_int8_t ring_id = 1; /* use any ID in the range configured in anic_mfl_config */
r.rule_id = rule_id++;
r.rule_family_type = accolade_rule;
r.rule_family.accolade_rule.action = accolade_drop;
r.rule_family.accolade_rule.ip_version = h->extended_hdr.parsed_pkt.ip_version;
r.rule_family.accolade_rule.src_addr_bits = 32;
r.rule_family.accolade_rule.src_addr.v4 = h->extended_hdr.parsed_pkt.ipv4_src;
r.rule_family.accolade_rule.dst_addr_bits = 32;
r.rule_family.accolade_rule.dst_addr.v4 = h->extended_hdr.parsed_pkt.ipv4_dst;
r.rule_family.accolade_rule.protocol = h->extended_hdr.parsed_pkt.l3_proto;
r.rule_family.accolade_rule.src_port_low = h->extended_hdr.parsed_pkt.l4_src_port;
r.rule_family.accolade_rule.dst_port_low = h->extended_hdr.parsed_pkt.l4_dst_port;
r.rule_family.accolade_rule.steer_to_ring = 1;
r.rule_family.accolade_rule.ring_id = ring_id;
pfring_add_hw_rule(pd, &r);

2.7. TX DMA

Please note that packet transmission is currently not implemented using the PF_RING API.

If you have an Accolade adapter (e.g. 200Ku-Flex) with a firmware supporting TX DMA for replaying PCAP files at line-rate up to 100G, you need to enable both 2MB and 1G hugepages.

First of all you should check that your CPU supports 1G hugepages, running the command below you should get some output:

grep pdpe1gb /proc/cpuinfo

In order to make sure that the system is able to reserve 1G pages from physical memory, it is recommended to add the boot parameters below to GRUB_CMDLINE_LINUX in /etc/default/grub:

default_hugepagesz=2MB hugepagesz=1GB hugepages=1 hugepagesz=2M hugepages=1024

In order to apply the changes, update grub and reboot the system:

sudo update-grub
sudo reboot

After rebooting the system, you should mount the hugepages and reload the Accolade driver:

sudo sh -c "echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages"
sudo mkdir /mnt/hugetlbfs1G
sudo mount -t hugetlbfs none -opagesize=1G /mnt/hugetlbfs1G
sudo insmod /opt/accolade/driver/anic_mem.ko mbytes_per_device=64
sudo /opt/accolade/bin/anic_load

At this point you should be finally able to run the TX tool provided by Accolade, example:

cd SDK_*/examples/; make
sudo ./anic_200k_tx -i 0 -p 0 -r 1000000 mixed.pcap