Using Hardware Timestamps with PF_RING

Posted · Add Comment

Up to some years ago, hardware timestamps were available only on costly FPGA-based NICs. Slowly, NIC manufactures started to consider hw timestamps as an important feature, and they started to introduce them in new cards. As of today Silicom PE2Gi80, Intel 1 Gbit Ethernet Server Adapter i340 (1 Gbit) and Neterion X3110/X3120 (10 Gbit) offer off-the-shelf hardware timestamps. These cards do not feature a GPS connector, but support IEEE 1588 for clock synchronization. The accuracy of the hw timestamps of these cards ranges from 3 to 7 ns.

PF_RING has been enhanced to support hw timestamps, whatever is their source. PF_RING 4.6.1 introduced hw timestamp support in both the PF_RING kernel module, applications (e.g. pfcount), and even libpcap. The user-space pfring library via ioctl()’s call with SIOCSHWTSTAMP, informs the NIC that received packets must be timestamped. The new PF_RING module does the magic by exploiting and passing to userland all the hw timestamps present in packets. Using them is pretty simple

  • Intel 1Gbit
    • cd PF_RING
    • insmod ./drivers/intel/igb/igb-2.4.12/src/igb.ko
  • Neterion X3110/X3210
    • cd PF_RING
    • insmod ./drivers/neterion/vxge.ko pf_ring_en=1 lro=0 func_mode=0

Supposing to access you card as ethX, you can access hw timestamps using pfcount.

Example:

pfcount -i ethX
00:07:14.230020938 [00:1B:ED:26:A9:4F -> 00:16:9C:6E:FD:80][eth_type=0x0800] [77.95.141.6:1637 -> 62.149.128.203:25] [tos=0][tcp_seq_num=2571580375][caplen=128][len=1514][parsed_header_len=0][eth_offset=0][l3_offset=14][l4_offset=34][payload_offset=54]

generated packet with ns timestamp accuracy.

The libpcap library present under PF_RING/userland and it companion product tcpdump, demonstrate hw timestamps

root@ubuntu:/home/deri/PF_RING/userland/tcpdump-4.1.1# ./tcpdump -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 8192 bytes
23:49:37.155010242 IP 10.211.55.10.ssh > 10.211.55.2.63850: Flags [P.], seq 3954409477:3954409669, ack 1206887258, win 239, options [nop,nop,TS val 34171550 ecr 663605968], length 192
23:49:37.155272286 IP 10.211.55.2.63850 > 10.211.55.10.ssh: Flags [.], ack 192, win 65535, options [nop,nop,TS val 663605974 ecr 34171550], length 0
23:49:37.202867182 ARP, Request who-has 10.211.55.1 tell 10.211.55.10, length 28

Applications based on hw timestamps can be manyfold. You can use it to measure network delay or reorder packets hitting different adapter ports.
The author would like to than Silicom, Exar, and Intel for their support during this project.