ntopng

Announce

Welcome to ntopng 3.8 with continuous drill down: packets, flows, activities

We are happy to announce ntopng stable 3.8. The is the core of the next 4.0 release as it integrates new features that will be consolidated in the next release scheduled for spring. The main features include: SQL database-free high-speed traffic indexing based on a new home-grown technology. As explained in this post, we managed to store compressed flow information on disk combined with high-speed retrieval. Just add “-F nindex” to ntopng to start using this new feature, currently available in the ntopng enterprise edition. You can read more here. …
n2disk

Drill Down Deeper: Using ntopng to Zoom In, Filter Out and Go Straight to the Packets

ntopng has grown significantly over the past years, providing an increasingly-interesting set of features to support network analysts and troubleshooters in their decisions. Among the most relevant features, it is worth mentioning that timeseries inspection pages have been redesigned and reworked profoundly to facilitate the drill-down of historical data. Similarly, a home-grown high-speed special-purpose flow database has been seamlessly integrated in ntopng to ease the storage and retrieval of historical flows. However, the circle was not really closed. A piece was missing. Something that could take us down to the …
ntopng

Say hello to nIndex: Personal Big Data System for Network Flows

Being able to store network flows is a very challenging task using generic databases. Networks are becoming faster and faster and, nowadays, flow-based analysis tools should store tens, or even hundreds, of thousands of flows per second, to keep up with SME and enterprise demands. Existing tools, such as relational databases, fail to accomplish this task. Unless you have unlimited resources available, tons of RAM and clusters of machines, chances are your database will choke, quickly becoming too slow to enable queries from being performed in a reasonable time. It was incredible …
nProbe

Measuring ntopng+nProbe Flow Processing Performance

NOTE: this post is outdated. Latest versions of ntopng and nProbe improve performance significantly. New figures are given in this post. In this post we try to analyze the performance of nProbe and ntopng for the collection of NetFlow. ntopng and nProbe will be broken down into smaller functional units and such units will be analyzed to understand the maximum performance of every single task as well as of the overall collection architecture. The machine used for the analysis is equipped with an 4-core Intel(R) Xeon(R) CPU E3-1230 v5 @ 3.40GHz …
ntopng

ntopng Disk Requirements for Timeseries and Flows

Being able to do a priori estimations of the space that ntopng is going to use in a production environment is fundamental for the provisioning of the storage. In this post we try to estimate the space used by ntopng to store timeseries and flows. Timeseries The number of timeseries generated by ntopng depends almost exclusively on the number of local hosts. Other timeseries generated, including those for the interfaces or SNMP devices, are generally orders of magnitude less than those generated for local hosts. For this reason, it is …
ntopng

Advanced SNMP Monitoring with ntopng

It has been a while since we have added SNMP support to ntopng. The first release, presented in this blog post, implemented basic SNMP support. Since then we have code various improvements and new feature, with the aim of turning ntopng in an advanced SNMP monitor. Among the extensions we have implemented are the following: A cache to decouple the polling of devices from the browsing of polled data Devices are polled periodically by ntopng with a background task that cycles them at 5-minute intervals and sends polled data to …
Components

Remote ntopng Authentication with RADIUS and LDAP

In large organizations, it is common to have a centralised authentication system usually named AAA (Authentication, Authorization and Accounting). Managing users typically involves the definition and enforcement of the rights to do some operations or to access certain resources in a network. Being able to grant (or deny) such rights using a centralized authentication system is the only viable solution when it comes to dealing with large organizations with hundreds, or even thousands, of users that periodically join and leave. AAA protocols include Remote Authentication Dial-In User Service (RADIUS) and …
n2n

Use Remote Assistance to Connect to ntopng Instances

A problem same ntop users how to face with, is the ability to remote access a ntopng instance running behind a firewall. This can be solved using a VPN or other means that often require to deploy an additional network service. Some of our ntop users are familiar with n2n, an open source peer-to-peer VPN ntop developed and maintains. With n2n in essence is possible to create a network overlay that allows you to access your assets in a secure way, this regardless of your network configuration. For this reason …
nProbe

sFlow Collection and Analysis with nProbe and ntopng

sFlow, short for sampled Flow, is a sampling technology designed to export network devices information, namely: Interface counters (à la SNMP MIB-II); Traffic packets (à la ERSPAN). sFlow agents run on switches, routers, firewalls and other devices, and periodically export interface counters and traffic packets via UDP towards one or more sFlow collectors. sFlow, relying on sampling processes to periodically counters and packets, is scalable and ultra-lightweight and has been embedded into network devices by tens of vendors and manufacturers. Contrary to NetFlow (please note that in sFlow parlance the …
nDPI

Promoting Traffic Visibility: from Application Protocols to Traffic Categories in nDPI and ntopng

Often we receive emails asking question like: “how many protocols nDPI supports?”, “how do you position nDPI against commercial DPI toolkit A, B, C?”. Although these questions are reasonable, they do not grasp the significance of DPI. For years commercial toolkits have run the race for protocols: I have 200 protocols, I have 1000 protocols, I have 500. Then asking that is the meaning with the term “protocol” people list traffic from to sites like cnn.com or bbc.co.uk. But BBC is not a protocol but rather some traffic (for instance …
ntopng

Securing ntopng with SSL and Let’s Encrypt

As you know ntopng web interface supports both HTTP (default) and HTTPS. The reason why ntopng does not default to HTTPS is because we provide self-signed certificates that web browsers dislike. Fortunately today you can create a free SSL certificate recognised by all browsers by using Let’s Encrypt open certificate authority (CA). This article describes how you can do this in a few simple steps: for simplicity we limit our scope to Ubuntu/Debian but on other distro’s the procedure is similar. Install certbot as described in this article Suppose that you …
nProbe

Using nProbe and ntopng for Collecting and Visualizing Sonicwall Flows

nProbe is both a probe and a NetFlow/sFlow collector. Recently, we’ve also added added the ability to collect flows with proprietary information elements. This greatly improves nProbe flexibility as any custon, vendor-proprietary information element can be understood, correctly parsed, and exported downstream. Adding proprietary information elements to nProbe is a breeze. Indeed, it suffices to use a plain-text file with the elements description. That’s all. Once the fields have been loaded from the plain-text file, they can be treated as if they were regular fields. So for example they can …