11. High-Resolution Timeseries

11.1. Overview

Standard flow records summarise all traffic for the lifetime of a connection into a pair of byte and packet counters. This is sufficient for long-term trend analysis, but it hides short-lived spikes and makes it impossible to plot throughput at sub-minute granularity.

High-Resolution (HR) timeseries solve this by recording byte counts in 15-second slots for every active flow. Each completed flow stored in ClickHouse therefore carries an embedded timeline of its own throughput history, enabling charts at 15-second granularity without a separate timeseries database.

Note

High-Resolution timeseries require:

An ntopng Enterprise M (or better) license.
ClickHouse configured as the flow dump backend (-F clickhouse). See ClickHouse (Flow Dump).
nProbe with HR counters support.

11.2. Data Flow

Physical/virtual interface
       │
       ▼
   nProbe (live capture)
       │  exports @NTOPNG@ template
       │  + HR_SRC_TO_DST_BYTES (array of 15s byte buckets)
       │  + HR_DST_TO_SRC_BYTES (array of 15s byte buckets)
       │  via ZMQ
       ▼
   ntopng (ZMQ collector)
       │  auto-detects HR fields
       │  writes consolidated flow record
       ▼
   ClickHouse flows table
       HR_SRC2DST_BYTES  Array(UInt64)
       HR_DST2SRC_BYTES  Array(UInt64)
       │
       ▼
   Grafana dashboard
       per-flow bidirectional throughput at 15-second resolution

11.3. nProbe Configuration

To enable HR timeseries, add the two HR information elements to the nProbe export template. No other nProbe option is required:

nprobe -i enp1s0 -n none --zmq "tcp://*:5556" -T "@NTOPNG@ %HR_DST_TO_SRC_BYTES %HR_SRC_TO_DST_BYTES"

11.4. ntopng Configuration

No ntopng configuration change is required. ntopng automatically detects the presence of HR counters in the incoming ZMQ flow records and writes consolidated data to the corresponding ClickHouse columns.

ntopng must be configured to use ClickHouse as the flow dump backend:

ntopng -i "tcp://127.0.0.1:5556" -F clickhouse

See ClickHouse (Flow Dump) for full ClickHouse setup instructions.

11.5. ClickHouse Schema

When HR data is present, ntopng stores it in additional columns of the flows table. Example:

Column	Type	Description
`HR_SRC2DST_BYTES`	`Array(UInt64)`	15-second delta byte counters, source-to-destination direction. Element i covers the interval `[FIRST_SEEN + (i·15s), FIRST_SEEN + ((i+1)·15s))`.
`HR_DST2SRC_BYTES`	`Array(UInt64)`	15-second delta byte counters, destination-to-source direction.

Those columns are empty arrays ([]) for flows that were not captured by nProbe with HR counters enabled. Existing deployments without HR support are therefore unaffected.

Sample ClickHouse query to read the per-slot throughput of a specific flow:

SELECT
    FLOW_ID,
    arrayEnumerate(HR_SRC2DST_BYTES) AS slot,
    arrayElement(HR_SRC2DST_BYTES, slot) AS src2dst_bytes,
    arrayElement(HR_DST2SRC_BYTES, slot) AS dst2src_bytes
FROM flows
ARRAY JOIN HR_SRC2DST_BYTES
WHERE FLOW_ID = <flow_id>
ORDER BY slot;

11.6. Grafana Dashboard

Sample Grafana dashboards are available at:

https://github.com/ntop/ntopng/tree/dev/httpdocs/misc/grafana

The ntopng High-Resolution Charts sample dashboard (hr-flow-throughput-dashboard.json) is a simple example demonstrating how to use High-Resolution timeseries in Grafana.

The dashboard provides two panels:

Service Flow Throughput — bidirectional throughput for a specific flow, filtered by source IP, destination IP, and destination port. Each data point represents one 15-second slot.
Application Protocol Throughput (All Traffic) — aggregate throughput broken down by application protocol across all flows in the selected time window.

To import the dashboard into Grafana:

In Grafana, go to Dashboards → Import.
Upload hr-flow-throughput-dashboard.json or paste its contents.
When prompted, select the ClickHouse datasource that points to the ntopng database.

The dashboard requires the Grafana ClickHouse plugin. See Grafana for general Grafana integration instructions.