1.2. Custom Timeseries

ntopng supports the creation of custom timeseries for:

  • Local hosts
  • Interfaces

Neither remote hosts nor flows can be used when creating custom timeseries.

Custom timeseries can be created out of metrics.

1.2.1. Interface Metrics

Supported metrics for the creation of interface timeseries are:

  • Layer-7 applications bytes sent and received
  • Layer-4 TCP, UDP, ICMP bytes sent and received
  • Total bytes sent and received
  • Total alerts
  • etc.

An always-updated list of metrics can be determined by inspecting method NetworkInterface::lua: https://github.com/ntop/ntopng/blob/dev/src/NetworkInterface.cpp

Interface metrics are available as a lua table. An excerpt of such table is shown below:

speed number 1000
id number 1
stats table
stats.http_hosts number 0
stats.drops number 0
stats.devices number 2
stats.current_macs number 6
stats.hosts number 23
stats.num_live_captures number 0
stats.bytes number 50559082
stats.flows number 30
stats.local_hosts number 3
stats.packets number 64984

1.2.2. Host Metrics

Supported metrics for the creation of host timeseries are:

  • Layer-7 applications bytes sent and received
  • Layer-4 TCP, UDP, ICMP bytes sent and received
  • Total bytes sent and received
  • Active flows as client and as server
  • Anomalous flows as client and as server
  • Total alerts
  • Number of hosts contacted as client
  • Number of hosts contacts as server

An always-updated list of host metrics can be determined by inspecting this file: https://github.com/ntop/ntopng/blob/dev/src/HostTimeseriesPoint.cpp

Host metrics are available in an handy lua table such as the one exemplified below:

ndpi_categories table
ndpi_categories.Cloud number 2880
anomalous_flows.as_server number 0
active_flows.as_client number 0
bytes.rcvd number 2880
icmp.bytes.rcvd number 0
tcp.bytes.rcvd number 0
total_alerts number 0
udp.bytes.rcvd number 2880
icmp.bytes.sent number 0
other_ip.bytes.rcvd number 0
other_ip.bytes.sent number 0
anomalous_flows.as_client number 0
contacts.as_server number 1
bytes.sent number 0
instant number 1550836500
tcp.bytes.sent number 0
udp.bytes.sent number 0
contacts.as_client number 0
ndpi table
ndpi.Dropbox string 0|2880
active_flows.as_server number 1

Specifically, Layer-7 application protocols are pushed in a table ndpi, whose keys are the application names such as Dropbox. For every application there are two values separated by a pipe, namely, bytes sent and bytes received. For example, in the excerpt above, Dropbox application had received 0 bytes and had sent 2880 bytes at the time the excerpt was generated.

The table also contain a field instant that represents the time at which metrics have been sampled.

The table above can be accessed and its contents can be read/modified to prepare timeseries points.

1.2.3. Metric Types

ntopng provides metrics of two types, namely gauges and counters. Timeseries can be created out of gauges and counters, transparently. The only thing that is necessary is to tell the timeseries engine the actual type, then the rest will be handled automatically.

1.2.3.1. Gauges

Gauges are metrics such as the number of active flows (e.g., active_flows.as_client, active_flows.as_server) or active hosts at a certain point in time.

1.2.3.2. Counters

Counters are for continuous incrementing metrics such as the total number of bytes (e.g., bytes.sent, bytes.rcvd).

1.2.4. Adding Custom Timeseries

ntopng handles custom timeseries with updates every:

  • 1 minute for interfaces
  • 5 minutes for local hosts

This means that custom timeseries with a point every minute and a point every 5 minutes can be generated for interfaces and local hosts, respectively.

ntopng looks for custom timeseries in the following lua files under scripts/lua/modules/timeseries/custom/:

  • ts_minute_custom.lua for local hosts timeseries with 5-minute updates
  • ts_5min_custom.lua for interface timeseries with 1-minute updates

If file ts_5min_custom.lua does not exist, ntopng will skip the creation of custom timeseries with 5-minute updates. Similarly, if file ts_minute_custom.lua does not exist, ntopng will skip the creation of custom timeseries with 1-minute updates.

Sample files ts_5min_custom.lua.sample and ts_minute_custom.lua.sample are created automatically upon ntopng installation with some example contents. Those files are ignored by ntopng. However, it is safe to copy them to ts_5min_custom.lua and ts_minute_custom.lua and modify the copies when it is necessary to add custom timeseries.

1.2.4.1. Structure of Custom Timeseries Files

Every custom file must contain a method setup which defines one or more schemas. Every custom timeseries needs a schema to function. A schema defines the timeseries in terms of tags and metrics. The documentation describes what is a schema in detail. Later in this section an example schema will be shown.

File ts_5min_custom.lua must contain a callback ts_custom.host_update_stats which is called by ntopng every 5 minutes for every active local host. This callback accepts the following arguments:

  • when The time (expressed as a Unix Epoch) of the call
  • hostname The IP address of the host, possibly followed by a VLAN tag
  • host The host metrics in a lua table
  • ifstats The interface stats of the host interface
  • verbose and extra flag passed when ntopng is working in verbose mode

File ts_minute_custom.lua must contain a callback ts_custom.iface_update_stats which is called by ntopng every minute for every monitored interface. This callback accepts the following arguments:

  • when The time (expressed as a Unix Epoch) of the call
  • _ifname The name of the monitored interface
  • ifstats The interface stats of the monitored interface
  • verbose and extra flag passed when ntopng is working in verbose mode

Callbacks can be used to append points to the timeseries. Indeed, once the schema is defined, it is necessary to append points to the timeseries. The function used to append points to the timeseries is the ts_utils.append documented later in this section of the documentation.

1.2.4.2. Example

Let’s see how to add an interface timseries that counts the number of issues detected when analyzing sequence numbers. The total issues detected when analyzing sequence numbers is considered as the sum of TCP retransmitted, out-of-order and lost packets.

The first thing to do is to add a schema to the setup function of ts_minute_custom.lua. The schema is created as as follows:

schema = ts_utils.newSchema("iface:tcp_seq_errors", {step = 60})
schema:addTag("ifid")
schema:addMetric("packets")

The first argument of newSchema specifies the timeseries name "iface:tcp_seq_errors". Timeseries interfaces must start with prefix iface:. The second argument is a table that must contain argument step which tells how frequently the timeseries will be updated. As we are in the 1-minute local hosts timeseries lua script, a value of 60 must be specified here.

Then, function addTag is used to indicate an interface id ifid that will be used to uniquely identify the timeseries when multiple interfaces are monitored. Finally, addMetric is called with an argument packets to indicate the metric name. Note that both the ifid and packets are just plain strings here, their actual values will be set in the ts_custom.iface_update_stats when updating the timeseries with new points.

The number of issues detected when analyzing sequence numbers is a counter, that is, is an always-increasing function of time. By default, schemas consider metrics as counters so there is no need to specify this type upon schema addition. For gauges, one has to indicate an extra metrics_type in the table containing the step. So for example, to create a 1-minute timeseries for the number of active flows of a given host, one can use the following syntax ts_utils.newSchema("host:flows", {step=60, metrics_type=ts_utils.metrics.gauge}).

To update the timeseries with new points, callback ts_custom.iface_update_stats is extended with a ts_utils.append call as follows.

ts_utils.append("iface:tcp_seq_errors",
{ifid = ifstats.id,
packets = ifstats.tcpPacketStats.retransmissions
             + ifstats.tcpPacketStats.out_of_order
             + ifstats.tcpPacketStats.lost},
when, verbose)

The first argument of ts_utils.append is the timeseries name and must be equal to the one specified when defining the schema. The second argument is a table which must contain the tag (ifid) and the metric (packets) which must be set to their actual values. As it can be seen from the example above, the field id of table ifstats is used to set tag ifid, whereas the sum of ifstats.tcpPacketStats table fields retransmissions, out_of_order and lost are used as value for the metric packets.

Finally, the third argument when is the time of the call, and the latest argument verbose indicates whether ntopng is operating in verbose mode.

From that point on, the timeseries will be consistently updated by ntopng.

Multiple schemas and multiple ts_utils.append can be added in the same file.

The full example can be seen at: https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/timeseries/custom/ts_minute_custom.lua.sample

Another example that creates 5-minute timeseries of local hosts total bytes can be seen at https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/timeseries/custom/ts_5min_custom.lua.sample

1.2.5. Locating Stored Custom Timeseries

TODO

1.2.6. Charting Custom Timeseries

TODO