Earlier in 2014 we advised how to build a continuous packet recorder using n2disk and PF_RING. Since that time computing architectures have progressed, we have added support for new ethernet controllers, and so it’s now time to refresh that post for all those willing to build a box themselves. The specs below are for 2 x 10 Gbit; for 1 x 10G you can use half of the components in most cases.
- CPU: we advise an Intel E5 with at least 3 GHz and 8 cores for all options (indexing and compression). Options include E5 2667 v3 or E5-2687W v4. If you do not need anything but the pcap (i.e. no index or on-the-fly pcap compression) you can use an E3 processor (e.g. E3 1271v3) that are significantly cheaper. The CPU type depends on the network adapter you plan to use (see below in this post).
- RAM: fill up all the available memory channels so that all the available memory bandwidth is used. This means that based on the CPU you select you need to use at least 4 memory bars. If unsure check the CPU specs, or in the very worst case fill up all the available memory slots.
- RAID controller: we suggest an LSI or Adaptec controller with at least 1 GB of onboard memory cache. Note that CPUs often do not have enough slots for the network adapter and the RAID controller (in many servers the first CPU has often only one slot available). This means that if you need extra slots you need to buy two CPUs and in this case have one RAID controller and network adapter per node (so 2 CPUs, 2 RAID controllers, 2 network adapters with one ethernet port each).
- Disks: we suggest 24 x 10k RPM SAS drives (the minimum is 16, but we advise not less than 20). if you only have 10 Gbit, you can use 8 (minimum) or 10 (suggested) disks. While SSDs and NVMe disks are faster (and thus you need fewer disks) than SAS drives, we do not advise you using them as:
- You need space so you will need many disks anyway.
- Flash memories are guaranteed up to a specified number of write cycles per day (write endurance) that do not make them suitable for permanent write 24 x 7.
- Network adapters (A-Z) supported by PF_RING ZC. Both Myricom and Napatech support hardware timestamps and GPS synchronisation, whereas Intel do not support it.
- Intel
- If you need to merge 2 ports into one, usually you cannot go above 18 Mpps. Hence we advise you to use two NUMA nodes, where each node captures one traffic direction and merging happens at runtime during packet extraction.
- Myricom
- 10 Gbit Myricom NICs can merge packets in hardware up to 21 Mpps. In this case you can use only one NUMA node and save money on RAID controller and CPU/RAM. if you need line rate instead, you need two controllers etc. such as in the Intel case.
- Napatech
- Napatech NICs can merge 2×10 in hardware with no packet drops, any packet size. Although these NICs are more expensive than others listed above, one NUMA node is enough so you can save money on the server and thus the price gap decreases.
- Intel
In conclusion:
- We have provided a list of components you can use for building your own traffic recorder: they depend on the network adapter you plan to use and on the requested performance.
- If you need the cheapest recorder and you do not care of hw timestamps or indexing, an E3 box with an Intel NIC is a good start.
- For most users a single NUMA node E5 box with a Myricom adapter, is the best compromise in price, performance and features.
- For high-end users Napatech is definitively the best choice you can find on the market.