HowTo Use nProbe To Create Traffic Timeseries in InfluxDB

Posted · Add Comment

One of the latest additions in nProbe, is the ability to create network traffic timeseries that will be stored in the popular InfluxDB database. This features allows nProbe users to create timeseries that can be depicted and integrated in Grafana dashboard for instance.

Timeseries are dumped by means of two new nProbe command line options:

  • --influxdb-dump-dir <dir>

    This allows timeseries to be stored in Line protocol format into the specified directory. A new file is created every minute.

  • --influxdb-exec-cmd <cmd>

    This option allows to process an timeseries file as soon as the dump is completed. We provide a companion tool that allows to import this timeseries file in InfluxDB.

Currently nProbe allows you to create timeseries from:

  • nProbe internals. With “-b 1”, nProbe allows you to create timeseries of internal metrics such as number of processed packets, drops, exports. Below you can find a sample line: 
    nprobe,name=192.168.2.178 rx_pkts=5380,rx_bytes=525620,zmq_flow_exports=0,flow_exports=50,flow_drops=0,rx_pkts_drops=0,export_queue_len=0,num_collected_pkts=27,tot_processed_flows=0,tot_collected_flow_drops=0,tot_flows_dumped_to_file=0,tot_exported_flows=50,max_bucket_search=3 1656403368000000000
  • sFlow counter samples. When you use nProbe in collector mode (-3 <port>) with sFlow, nProbe uses flow samples to generate flow information, and counter samples to generate timeseries similar to what you could achieve using SNMP and polling data from the exporter. Below you can find a sample line: 
    sflow,deviceIP=192.168.2.1,ifIndex=9 ifInOctets=0,ifInPackets=0,ifInErrors=0,ifOutOctets=714615609544,ifOutPackets=591251324,ifOutErrors=0 1656403323000000000
  • Calix is a network equipment vendor that exports via proprietary IPFIX traffic counters. Thanks to timeseries support you can turn this data into timeseries and depict them. All you need to do is to start nprobe as follows:

    nprobe -n none -i none -3 2055 –load-custom-fields olt_custom_fields.txt –influxdb-dump-dir /tmp/timeseries –influxdb-exec-cmd influx_upload.sh -F 30

You can check the timeseries export as follows:

$ influx
Connected to http://localhost:8086 version 1.8.10
InfluxDB shell version: 1.8.10
> use calix;
Using database calix
> select * from calix_ipfix;
name: calix_ipfix
time                down_bytes  hostname ont_id    pon_ont_util_aid up_bytes
----                ----------  -------- ------    ---------------- --------
1638909102000000000 13980732227 OLT10    olt10test OLT10/1/1/gp8    44795119
1638909132000000000 13980733994 OLT10    olt10test OLT10/1/1/gp8    44798055
1638909162000000000 13980735371 OLT10    olt10test OLT10/1/1/gp8    44799898
1638909192000000000 13980737382 OLT10    olt10test OLT10/1/1/gp8    44803008
1638909222000000000 13980738425 OLT10    olt10test OLT10/1/1/gp8    44804390
1638909252000000000 13980742574 OLT10    olt10test OLT10/1/1/gp8    44810872
......

or depict them graphically.

In conclusion thanks this new feature, nProbe can now create timeseries, collect sFlow counters samples, and convert proprietary IPFIX fields into timeseries you can use to create dashboards.

 

Enjoy !