Traditionally nDPI was used by ntopng to detect flows L7 protocol. With the advent of more and more protocols, speaking about single protocols is often too difficult. Users usually are not interested in the specific protocol but rathen on a whole group of protocols. For example, it’s easier to reason about VPN traffic as a whole rather than a particular VPN implementation.
For these reasons, nDPI (and ntopng) has been extended to provide a logical grouping of protocols, called Categories. With Categories it’s possible, for example, to get an idea of the network traffic of a host:
Some use cases solved by the Categories include:
- Block all advertisement sites (nEdge)
- Trigger an alert whenever my employees access a malware site (ntopng, whereas in nEdge there is the ability to block this traffic)
- Prevent clients from accessing the WiFi sites of competitors as they are using them for comparing prices (nEdge)
The picture above shows the Collaborative category being reported on the flow details of a Github/DNS flow.
6.1. Protocol Category¶
The flow Category is usually determined based on the flow protocol. It is possible to review and modify the category associated to each protocol through the Protocols tab in the Categories page:
Please note that in addition to the built-in protocols, it is possible to define custom protocols providing a nDPI protocol file to ntopng through the –ndpi-protocols|-p <file> option, with the following format:
# host:"<value>",host:"<value>",.....@<subproto> host:"googlesyndacation.com"@Google host:"venere.com"@Venere
An example for this configuration file is available here.
After providing the protocols file to ntopng, the Protocols page will show the new protocols, and it will we possible to associate them to categories.
6.2. Custom Category Hosts¶
As shown above, ntopng already assigns a default category to the known L7 protocols. Nevertheless, it’s also possible for the user to specify a list of additional hosts to be included into a particular category. ntopng provides 5 empty “custom categories” dedicated to this task, but users are also free to modify the other categories.
The custom category hosts can be specified via some host-based rules. The host-based rules will be used to perform substring matching on some of the flow information:
- Client/Server IP
- DNS query
- Host SNI
- HTTP Host
If a match is found, the flow category will be set to the corresponding matching category. These rules can be configured from the Categories page.
By clicking “Edit Hosts” it’s possible to define some hosts which will be considered as part of the category.
The picture above shows some custom hosts defined for the Advertisement category. For example, the .ads. host rule will match any host containing .ads. . It is important to play with the dots to avoid excessive matching (e.g. a simple ads rule would also match mads.com).
Note: host matching based on IP addresses is currently limited to IPv4 flows.
ntopng also supports external lists to define custom categories, loaded from text file (local) or online services (e.g. emergingthreats for the Malware category). Since lists are also used to raise alerts (e.g. for hosts in the Malware or Mining category), you may need to add exceptions to those lists, whitelisting selected hosts. This is possible adding an host to the list, prepending “!” to the IP/hostname (e.g. !126.96.36.199).
6.3. Flow Shortcut¶
From the flow details view there is a convenient way to add the flow SNI/HTTP host to a customized category.