The need to perform on-time and per-second traffic measurements clashes with protocols such as NetFlow where all counters are cumulative with respect to the flow lifetime. So if you have a flow that lasted 2 minutes and moved X bytes, you have no clue what was the throughput of this flow across the 2 minutes. For this reason people started to shorten flow duration with the drawback of putting a lot of pressure on probes as well to increase the disk space and flow records cardinality on collectors. In essence it was not a solution nor a workaround.
In order to address requests coming from our user community, nProbe (Pro) has been extended to provide per-second byte flow counters using two new information elements
[NFv9 57944][IPFIX 35632.472] %SRC_TO_DST_SECOND_BYTES Bytes/sec (src->dst) [NFv9 57945][IPFIX 35632.473] %DST_TO_SRC_SECOND_BYTES Bytes/sec2 (dst->src)
As flow can potentially last a while, whereas IPFIX/NetFlow space is limited in packets, we have decided to export per-second counters only on disk/JSON while exporting only the numeric byte counters on the wire.
Example:
nprobe -P /data -T "%IPV4_SRC_ADDR %IPV4_DST_ADDR %INPUT_SNMP %OUTPUT_SNMP %IN_PKTS %IN_BYTES %FIRST_SWITCHED %LAST_SWITCHED %L4_SRC_PORT %L4_DST_PORT %TCP_FLAGS %PROTOCOL %SRC_TO_DST_SECOND_BYTES %DST_TO_SRC_SECOND_BYTES" -i eth0
will export text files containing lines like
131.114.21.22|114.79.1.15|0|0|3|156|1380114078|1380114095|80|18151|17|6|52,,,,,,52,,,,,,,,,,,52|,,,,,,,,,,,,,,,,,,|HTTP 131.114.21.22|114.79.1.15|0|0|3|156|1380114078|1380114095|80|18144|17|6|52,,,,,,52,,,,,,,,,,,52|,,,,,,,,,,,,,,,,,,|HTTP 131.114.21.22|114.79.1.15|0|0|3|156|1380114079|1380114100|80|18156|17|6|52,,,,,,,52,,,,,,,,,,,,,,52|,,,,,,,,,,,,,,,,,,,,,,|HTTP 213.121.168.130|131.114.21.22|0|0|5|224|1380114081|1380114099|54306|80|19|6|92,,52,,,,,,,,,,,,,,,,80|52,52,,,,,,,,,,,,,,,,,40|HTTP
Let’s consider the first flow. You need to read values as follows:
The element “52,,,,,,52,,,,,,,,,,,52” means that during the first flow second (so that started at 1380114078) 52 bytes have been sent, on the following second no traffic (to shorten flow format nProbe omits the 0), …
Per-second counters are useful to troubleshoot multimedia applications such as VoIP where the codec is expecting to send the same amount of data every second. See for instance this call below to see how it works:
IPV4_SRC_ADDR|IPV4_DST_ADDR|INPUT_SNMP|OUTPUT_SNMP|IN_PKTS|IN_BYTES|FIRST_SWITCHED|LAST_SWITCHED|L4_SRC_PORT|L4_DST_PORT|TCP_FLAGS|PROTOCOL|SRC_TO_DST_SECOND_BYTES|DST_TO_SRC_SECOND_BYTES|L7_PROTO_NAME 212.97.59.76|10.6.4.71|0|0|5|3466|1187006259|1187006295|5061|5060|0|17|1055,,,,,,,,582,,,,,,,,,,,,,,603,,,,,,,,,,,,,,1226|1006,,,,,,,,876,,,,,,,,,,,,,,395,,,,,,,,,,,,,,940|SIP 10.6.4.71|212.97.59.80|0|0|933|261240|1187006267|1187006295|16418|52542|0|17|3920,9520,9240,9240,9520,9240,9240,9520,9240,9240,9520,9240,9240,9520,9240,9240,9520,9240,9240,9520,9240,9240,9520,9240,9240,9520,9240,9240,5320|4200,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,10000,6400|RTP
This will enable you to understand if from the network standpoint everything worked as planned.
In addition to this if you use “-b 1” nProbe will print also application protocol counters at every interval by dumping the total and partial layer-7 protocol counter.
03/Aug/2018 11:11:10 [nprobe.c:3228] L7 Proto Diff Total 03/Aug/2018 11:11:10 [nprobe.c:3242] Unknown/0 164.12 KB 164.12 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] POP3/2 30.36 KB 30.36 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] SMTP/3 2.29 KB 2.29 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] DNS/5 2.95 KB 2.95 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] HTTP/7 912.20 KB 912.20 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] ICMP/81 964 B 964 B 03/Aug/2018 11:11:10 [nprobe.c:3242] RTP/87 424.13 KB 424.13 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] SIP/100 77.95 KB 77.95 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] Google/126 85.39 KB 85.39 KB 03/Aug/2018 11:11:10 [nprobe.c:3242] Radius/146 622 B 622 B 03/Aug/2018 11:11:10 [nprobe.c:3242] H323/158 160 B 160 B
This feature is currently implemented in nProbe 8.5.x and it will be included in the next stable release due late this summer.
Enjoy!