nProbe

nProbe

Best Practices for High Speed Flow Collection

Most people use nProbe and ntopng to collect flows using an architecture similar to the one below where nprobe and ntopng are started as follows: nprobe -3 <collector port> -i none -n none —zmq "tcp://*:1234" --zmq-encryption-key <pub key> ntopng -i tcp://nprobe_host:1234 --zmq-encryption-key <pub key> In this case ntopng communicates with nProbe over an encrypted channel and flows are sent in a compact binary format for maximum performance. If you do not need nProbe to cache and aggregate flows, you can also add --collector-passthrough on the nProbe side to further increase …
nProbe

How To Monitor Traffic Behind a Firewall (During and Post Pandemic)

Due to pandemic, many people are now working in a delocalised world: some work from home, others from the office. To make things even more complicated, in the past remote workers used to connect to the company network via a VPN. While this option is still possible, many resources are now available from the cloud thus making VPNs obsolete in some environments, in particular for mobile workforce that connects to the Internet by means of a cellular network. In the past months, some people contact us to ask how they …
Announce

Bringing Network Visibility, Cybersecurity and Encrypted Traffic Analysis to OPNsense, pfSense and FreeBSD

This is to announce the immediate availability of both ntopng and nProbe for OPNsense, pfSense and FreeBSD, directly supported by ntop, with nightly builds and all the features present on all other supported platforms such as Linux, Windows and MacOS. You can now Monitor network traffic based on nDPI. Encrypted traffic analysis (ETA) that enables you to have visibility of encrypted traffic and answer to questions such as: what portion of my available bandwidth is used by Netflix? Cyber threats analysis: ntopng con be used to effectively detect attacks, anomalies …
nProbe

Introducing nProbe 9.4: New Platforms Support and Product Editions

This is to announce nProbe 9.4 stable that is an incremental update of 9.2 released last fall. The goal of this maintenance release is to pave the way to pervasive embedded systems support as we now support OPNsense/pfSense/FreeBSD Soon we’ll make a separate announcement as soon as more ntop packages will be available for these platforms. Ubiquity EdgeRouter X Read this blog post for learning more about sub 100$ Ubiquity-based hardware probes. OpenWRT In addition we have decided to simplify the nProbe versions that were hard to understand for most …
nProbe

Introducing nProbe 9.2: Collection Pass-Through and Reforge, OpenWRT support, Flexible JSON-export

This is to announce the release of nProbe 9.2. The main new features of this release are focused on flow collection speed and flexibility in particular for modern JSON-based flow consumers. This is to enable applications relying on nProbe, e.g. ntopng, to scale up when collecting flows: The new –collector-passthrough option allows the flow cache to be bypassed when flows are collected. This mean that flows are forwarded to remote collectors unmodified (i.e. -T is not used) without placing them into the flow cache (i.e. flows are not merged by …
nProbe

Introducing nProbe 9.0: Traffic Behaviour Analysis and High Speed Flow Collection (Even Behind a Firewall)

This is to introduce nProbe 9.0 stable release whose the two main features are traffic behaviour analysis and high speed flow collection. Traffic Behaviour Analysis When in 2002 nProbe™ development started, the idea was to create a drop-in replacement for physical probes present in routers. Later the advent of IPFIX pushed the monitoring community towards standardisation of flow exports, and promoted interoperability across probes and collectors. Then the market started to ask solutions for visibility (and not just traffic accounting), and we developed nDPI™ for going beyond port and protocols …
nProbe

Packets vs eBPF/System Events: Positioning nProbe vs nProbe Agent

nProbe (and ntopng) is a traditional packet-based application, whose lifecycle is Capture a packet and dissect/decode it Update the representation in memory of the network traffic (e.g. the flow table) Export the information Using packets for traffic analysis has several positive things including: Ability to analyse traffic using a port mirror/TAP without installing and agent on every monitored host, thing that might be a nightmare if your network is heterogeneous. Scalability issues have been solved (e.g. see PF_RING ZC) years ago, so monitoring a 40/100G network is no longer a …
nProbe

Containers and Networks Visibility with ntopng and InfluxDB

For a while we have investigated how to combine system and network monitoring in a simple and effective way. In 2014 we have done a few experiments with Sysdig, and recently thanks to eBPF we have revamped our work to exploit this technology as well to be able to monitoring containerised environments. Months ago we have shown how to detect, count and measure the network activity which is taking place at a certain host just by leveraging certain functionalities of the linux operating system, without even looking at the traffic …
nProbe

Measuring nProbe ElasticSearch Flow Export Performance

nProbe (via its export plugin) supports ElasticSearch flows export. Setting up nProbe for the ElasticSearch export is a breeze, it just boils down to specifying option --elastic. For example, to export NetFlow flows collected on port 2058 (--collector-port 2058)  to an ElasticSearch cluster running on localhost port 9200, one can use the following nprobe -i none -n none --collector-port 2058 --elastic "flows;nprobe-%Y.%m.%d;http://localhost:9200/_bulk" nProbe will take care of pushing a template to ElasticSearch to have IP fields properly indexed, and will also POST flows in bulk to maximize the performance. Recently …
nProbe

Packets vs Flows: Which Option is the Best?

One of the most difficult steps on a monitoring deployment scenario is to choose where is the best point where traffic has to be monitored, and what is the best strategy to observe this traffic. The main options are basically: Port Mirroring/Network Tap NetFlow/sFlow Flow Collector Port Mirroring/Network Tap Port mirroring (often called span port) and network tap have already been covered on a previous post. They are two techniques used to provide packet access that often are the best way to troubleshoot network issues as packets are often perceived as the …
Announce

Cento 1.6 Stable Just Released

After more than one year since the latest stable release, we are glad to announce cento 1.6-stable. This new release brings stability, fixes and several new features. Among the new features, it is worth mentioning that: Flows can be exported in a standardized JSON to text files. By default, a user cento runs and owns both the process and process files. This makes running cento more secure than using root. In addition, any user in the system can be used to run cento. A capture direction can be indicated so …
nProbe

How to export BGP routing information (AS Path) in network flows

Tools like traceroute have been used for a long time to track the forward path of packets, i.e. the journey of our packets to a remote destination. Unfortunately with traceroute nothing can be said about the path of ingress packets, it not assuming that routing is symmetrical, fact that is often not correct. For this reason we have designed a solution that allows path information to be report in emitted flows. As the most popular exterior gateway protocol used on the internet is BGP, we have designed a tool that …