nProbe

nProbe

Measuring nProbe ElasticSearch Flow Export Performance

nProbe (via its export plugin) supports ElasticSearch flows export. Setting up nProbe for the ElasticSearch export is a breeze, it just boils down to specifying option --elastic. For example, to export NetFlow flows collected on port 2058 (--collector-port 2058)  to an ElasticSearch cluster running on localhost port 9200, one can use the following nprobe -i none -n none --collector-port 2058 --elastic "flows;nprobe-%Y.%m.%d;http://localhost:9200/_bulk" nProbe will take care of pushing a template to ElasticSearch to have IP fields properly indexed, and will also POST flows in bulk to maximize the performance. Recently …
nProbe

Packets vs Flows: Which Option is the Best?

One of the most difficult steps on a monitoring deployment scenario is to choose where is the best point where traffic has to be monitored, and what is the best strategy to observe this traffic. The main options are basically: Port Mirroring/Network Tap NetFlow/sFlow Flow Collector Port Mirroring/Network Tap Port mirroring (often called span port) and network tap have already been covered on a previous post. They are two techniques used to provide packet access that often are the best way to troubleshoot network issues as packets are often perceived as the …
Announce

Cento 1.6 Stable Just Released

After more than one year since the latest stable release, we are glad to announce cento 1.6-stable. This new release brings stability, fixes and several new features. Among the new features, it is worth mentioning that: Flows can be exported in a standardized JSON to text files. By default, a user cento runs and owns both the process and process files. This makes running cento more secure than using root. In addition, any user in the system can be used to run cento. A capture direction can be indicated so …
nProbe

How to export BGP routing information (AS Path) in network flows

Tools like traceroute have been used for a long time to track the forward path of packets, i.e. the journey of our packets to a remote destination. Unfortunately with traceroute nothing can be said about the path of ingress packets, it not assuming that routing is symmetrical, fact that is often not correct. For this reason we have designed a solution that allows path information to be report in emitted flows. As the most popular exterior gateway protocol used on the internet is BGP, we have designed a tool that …
nProbe

Measuring ntopng+nProbe Flow Processing Performance

NOTE: this post is outdated. Latest versions of ntopng and nProbe improve performance significantly. New figures are given in this post. In this post we try to analyze the performance of nProbe and ntopng for the collection of NetFlow. ntopng and nProbe will be broken down into smaller functional units and such units will be analyzed to understand the maximum performance of every single task as well as of the overall collection architecture. The machine used for the analysis is equipped with an 4-core Intel(R) Xeon(R) CPU E3-1230 v5 @ 3.40GHz …
nProbe

sFlow Collection and Analysis with nProbe and ntopng

sFlow, short for sampled Flow, is a sampling technology designed to export network devices information, namely: Interface counters (à la SNMP MIB-II); Traffic packets (à la ERSPAN). sFlow agents run on switches, routers, firewalls and other devices, and periodically export interface counters and traffic packets via UDP towards one or more sFlow collectors. sFlow, relying on sampling processes to periodically counters and packets, is scalable and ultra-lightweight and has been embedded into network devices by tens of vendors and manufacturers. Contrary to NetFlow (please note that in sFlow parlance the …
nProbe

Using nProbe for Collecting Ixia IPFIX with IxFlow extensions

Ixia allows to enrich IPFIX records with value-add extensions. Additional information that can be exported, along with standard fields such as source and destination IP addresses, include: Geographical information such as region IP, latitude and city name Application ID or name, device, browser and even SSL cipher used Detail on application and handset (device) type for mobile users HTTP URL and hostname for web activity tracking HTTP and DNS metadata for rapid breach detection Transaction Latency for application performance tracking The latest version of nProbe provides full support for Ixia …
nProbe

Using nProbe and ntopng for Collecting and Visualizing Sonicwall Flows

nProbe is both a probe and a NetFlow/sFlow collector. Recently, we’ve also added added the ability to collect flows with proprietary information elements. This greatly improves nProbe flexibility as any custon, vendor-proprietary information element can be understood, correctly parsed, and exported downstream. Adding proprietary information elements to nProbe is a breeze. Indeed, it suffices to use a plain-text file with the elements description. That’s all. Once the fields have been loaded from the plain-text file, they can be treated as if they were regular fields. So for example they can …
nProbe

Introducing nProbe 8.6: Per-Second Measurements and Collection of Proprietary Flows

We are glad to announce the release of nProbe 8.6 stable release. Among the main new features, this release brings: Per-second measurements of flows traffic Ability to collect proprietary (i.e. using non standard information elements) flows These new features come along with a wide range of new extensions and improvements to the currently existing features and, least but not last, security and stability fixes. Let’s have a brief look at the two main new features mentioned above. Per-second Traffic Measurements Getting cumulative measurements with respect to the flow lifetime not …
Guides

Best Practices for the Collection of Flows with ntopng and nProbe

ntopng can be used to visualize traffic data that has been generated or collected by nProbe. Using ntopng with nProbe is convenient in several scenarios, including: The visualization of NetFlow/sFlow data originated by routers, switches, and network devices in general. In this scenario, nProbe collects and parse NetFlow/sFlow traffic from the devices, and send the resulting flows to ntopng for the visualization. The monitoring of physical network interfaces that are attached to remote systems. In this scenario, ntopng cannot directly monitor network interfaces nor it can see their packets. One …
nProbe

Introducing per-Second Measurements in nProbe Flow Exports

The need to perform on-time and per-second traffic measurements clashes with protocols such as NetFlow where all counters are cumulative with respect to the flow lifetime. So if you have a flow that lasted 2 minutes and moved X bytes, you have no clue what was the throughput of this flow across the 2 minutes. For this reason people started to shorten flow duration with the drawback of putting a lot of pressure on probes as well to increase the disk space and flow records cardinality on collectors. In essence …
nProbe

Using nProbe for Collecting Palo Alto Flows

NOTE: This article is outdated. Please see “Collecting Proprietary Flows with nProbe” for learning  how to collect proprietary Palo Alto flows. nProbe is both a probe and a NetFlow/sFlow collector. As you all know, we have recently added the ability to collect flows with proprietary information elements. However we natively support in nProbe popular flow exporter devices such as Cisco NBAR and Palo Alto security devices. In this article we show you how to collect the latter flows in nProbe. A typical Palo Alto flow is depicted below. As explained …