Using Python (including Jupyter Notebook) with ntopng

Posted · Add Comment

Most programmers and network/security administrators are familiar with the Python language. As from time to time we receive requests from our users for creating custom reports, or extracting other type of data (e.g. alerts or timeseries) from ntopng, we have decided to create a Python API for ntopng. Such API allows developers to extract data from ntopng similar to what other Python APIs do (e.g. pyshark for Wireshark).

Using this API you can

  • Read host statistics
  • Get the active flows list
  • Query network interface stats
  • Search historical flows

Those familiar with Jupyter Notebooks can also use them to interact with ntopng. You can see this example as starting point fo your experiments.

Using the python API is simple. What you need is the latest ntopng dev version (any version will work) and the python APi that you can easily install with

  • pip3 install ntopng

Done that you can write your first application. The API is basically a wrapper around the ntopng REST API so in essence the first thing you need to do is to connect to a remote ntopng instance (using login and password, or the authentication token) and issue queries via the python API.

try:
    my_ntopng = Ntopng(username, password, auth_token, ntopng_url)
    my_historical = Historical(my_ntopng)

    epoch_end   = int(time.time())
    epoch_begin = epoch_end - 3600
    host = "28:37:37:00:6D:C8"
    ifid = 0

    print(my_historical.get_timeseries("mac:traffic", "ifid:"+str(ifid)+",mac:"+host, epoch_begin, epoch_end))

except ValueError as e:
    print(e)

Above you can find a simple example for extracting a time series of the last hour traffic of a specific MAC address. In the example below we extract historical flows from the ClickHouse database.

try:
    my_ntopng = Ntopng(username, password, auth_token, ntopng_url)
    my_historical = Historical(my_ntopng)

    epoch_end   = int(time.time())
    epoch_begin = epoch_end - 3600
    host = "192.168.1.1"
    ifid = 0
    
    select_clause = "IPV4_SRC_ADDR,IPV4_DST_ADDR,PROTOCOL,IP_SRC_PORT,IP_DST_PORT,L7_PROTO,L7_PROTO_MASTER"
    where_clause  = "(PROTOCOL=6) AND IPV4_SRC_ADDR=(\""+host+"\")"
    maxhits       = 10 # 10 records max
    print(my_historical.get_flows(ifid, epoch_begin, epoch_end, select_clause, where_clause, maxhits, None, None))
except ValueError as e:
    print(e)

You can find the python API on github as well as simple examples. Soon we will produce additional code examples and documentation that shows you how to interact with ntopng. Please let us know your feedback on the community channels and feel free to contribute to the API with a pull request.

 

Enjoy !