In a previous post, What’s In The (Alert) Inbox?, we’ve discussed how alerts are organised in the Alerts Explorer. The new “inbox” design allows us to cluster alerts into separate folders high-priority events, that require attention and needs to be addresses as soon as possible, from other minor events. This solves one issue: having all critical alerts under control, while still tracking and archiving all minor Network issues (that contribute to the hosts score, and may be still of interest when drilling down during our analysis).
In a system which analyses traffic of a large network, the amount of alerts (in particular those which are about Network issues) can be high, also depending on the number of Behavioural Checks we enable, and how we tune them. Checking the health of the Network, by looking at those alerts one by one, may definitely be a challenging and time-consuming task. For this reason some time ago we introduced in the Alerts Explorer the computation of the Top Alert Types, Clients, Servers, etc, to summarize what are the main issues affecting our networks, and who are the main actors.
This was really useful in understanding at a glance what’s going on. However this is somehow limited, as the information they provide are in many cases not enough for a real analysis and we still need to go through every single alert. For this reason we decided to add support for something more flexible: the ability to define custom queries.
With custom queries, the Alerts Explorer provides the ability to define additional views on top of the full list of alerts, which allows you to aggregate alerts based on arbitrary criteria for example. They are defined as “SQL Queries” using a simple JSON syntax, placed as .json files on the filesystem (this to promote easy extensibility from end-users), and automatically appear in a dropdown under the Queries section in the Alerts Explorer. It is possible to build a query which groups alerts based on Client, Server and Alert Type for instance, and list all alerts matching a specific 3-tuple from the table Actions.
ntopng already provides a few built-in custom queries for Flow and Host alerts. The corresponding definition is available on the filesystem as JSON files under /usr/share/ntopng/scripts/historical/alerts/{flow, host}/*.json
Adding a new custom query is as simple as placing one more JSON file under the same folder. Let’s see as an example the definition of a query grouping flow alerts on Client IP, Server IP and Alert Type:
{
"name" : "Client / Server / Alert Type",
"i18n_name" : "contacts_by_type",
"select" : {
"items" : [
{
"name" : "cli_ip"
},
{
"name" : "srv_ip"
},
{
"name" : "alert_id"
},
{
"name" : "count",
"func" : "COUNT",
"param" : "*",
"value_type" : "number"
}
]
},
"groupby" : {
"items" : [
{
"name" : "cli_ip"
},
{
"name" : "srv_ip"
},
{
"name" : "alert_id"
}
]
},
"sortby" : {
"items" : [
{
"name" : "count",
"order" : "DESC"
}
]
}
}
The JSON format is self-explanatory: you can define the columns to show under the select tree, the columns on which the group by is applied under the groupby tree, and the default column on which sorting is applies under the sortby tree. Aggregation functions can also be defined as you can see for the count item, which is used in the example to display the number of alerts for each 3-tuple. For more complicated examples we recommend to take a look at the built-in query definitions available in the same folders.
For a full list of columns, please check the database schema under /usr/share/ntopng/httpdocs/misc/alert_store_schema.sql
Enjoy!