Data Aggregation in ntopng: Host Pools vs Observation Points

Posted · Add Comment

ntopng allows users to aggregate data according to various criteria. In networking, IP addressing (network and mask/CIDR) and VLANs are typical solutions to the problem of aggregating homogeneous hosts (e.g. when hosts carry on similar tasks). Sometimes these aggregation facilities are not flexible enough to cluster hosts that have the same operating system, or flows originated by the same router/switch.

In addition to typical network-based criteria such as IP, VLAN, ntopng implements two more data aggregation facilities.

Hosts Aggregation: Host Pools

A host pool is a logical aggregation of hosts, networks and MAC addresses (this facility is available only if L2 information is available). Pools are used to group host that have a common property. For instance in ntopng there is a “Jailed Hosts” pool, that contains hosts that are considered dangerous (e.g. when their score is too high for a long time). Pools are a host aggregation facility.

Flows Aggregation: Observation Points

In flow-based analysis (e.g. when ntopng collects flows created/collected by nProbe), in addition to pools, it is often required to identify flows (not hosts) based on additional criteria. All flows, in addition to properties such as IP/port/bytes/packets, are also marked based on the IP address of the flow device exporter that has created the flow. However the exporter IP might be too granular as a single company location (e.g. site A) can have multiple probes (hence with different IPs) that need to be aggregated. In this case, the nProbe/ntopng implement the observation point concept that is a numerical identifier used to mark flows coming from various exporters that need to be logically aggregated.

In conclusion, the observation point is a way to logically aggregate flows whereas pool are used to aggregate hosts. For this reason they can be used simultaneously.

Instead, if you need yo do the opposite, i.e. divide data into homogeneous groups, ntopng offers a disaggregation facility that can implement this per-interface.

 

Enjoy !