ntopng

Clustering Network Devices using ntopng Host Pools

In computer networks, devices are identified by an IP and a MAC. The IP can be dynamically assigned (so it might not be persistent), whereas the MAC is (in theory) unique and persistent for identifying a device. Non-technical users, do not know these low-level details, and in general it makes sense to cluster devices using other criteria. VLANs are a way to logically group devices belonging to the same administrative domain, but this is still a low-level network-level properly. When administering a network, we have have realised that we need …
nProbe

Flow-Based Monitoring, Troubleshooting and Security using nProbe

nProbe is a tool developed over the last 10 years, and thus it has been extended and improved year by year. However many users, even those who are using it since a long time, might not know all its features. Next week at Flocon 2017, I will give a talk about nProbe. The idea is to position nProbe (e.g. against the popular YAF tool), highlight what people can do with it (in addition to traffic monitoring and troubleshooting) and learn that nProbe is much more than a network sensor. I invite …
nProbe

Monitoring VoIP Traffic with nProbe and ntopng

VoIP applications usually limit theirs monitoring capabilities to the generation of CDR (Call Data Records) that are used for the generation of billing/consumption data. In essence you know how many calls a certain user/number has made, the duration etc. While this information can be enough for basic monitoring, it is not enough for guaranteeing reliable call quality as these systems are essentially blind with respect to call quality. Wireshark can analyse both call signalling and voice, but it is a troubleshooting tool meaning that it cannot be used for permanent …
cento

Stream That Flow: How to Publish nProbe/Cento Flows in a Kafka Cluster

Apache Kafka can be used across an organization to collect data from multiple sources and make them available in standard format to multiple consumers, including Hadoop, Apache HBase, and Apache Solr. nProbe — and it’s ultra-high-speed sibling nProbe cento — integration with the Kafka messaging system makes them good candidates source of network data. The delivery of network data to a redundant, scalable, and fault-tolerant messaging system such as Kafka enables companies to protect their data even in-flight, that is, when the consolidation in a database has still to occur. An impatient reader who is eager to use Cento for …
ntopng

ntopng MySQL Flow Export: Increase the Maximum Number of Open Files

ntopng uses partitioned MySQL tables when storing flows. As MySQL needs a file handle for each partition and its index, it is important to make sure that the open_files_limit is large enough to allow the process to keep all these files open. Typically, open_files_limit  default value works out-of-the-box but there are some packages/distributions that keeps this number pretty low. When the current value is too low, ntopng can show errors such as [MySQLDB.cpp:55] ERROR: MySQL error: Out of resources when opening file './ntopng/flowsv6#P#p23.MYD' (Errcode: 24 - Too many open files) [rc=-1] …
News

ntop Users Meeting 2016 Retrospective

Earlier this week we have organised a ntop user’s workshop hosted at Sharkfest EU 2016. For those who have not been able to attend this session, below you can find the slides we have used for presentation. Introduction ntopng: Web-based Traffic Analysis nDPI: Open Source Deep Packet Inspection PF_RING: High-Speed Traffic Processing Hands-On Session Product Roadmap Feel free to contact us if you have any questions. …
n2disk

Filtering Terabytes of pcaps using nBPF and Wireshark

In a previous post we introduced our new nBPF library that able to convert a BPF filter to hardware rules for offloading traffic filtering to the network card. We did not mention that the same engine can be used for accelerating traffic extraction from an indexed dump set produced by n2disk. n2disk is a traffic recording application able to produce multiple PCAP files (a per-file limit in duration or size can be used to control the file size) together with an index (for accelerating extraction) and a timeline (for keeping all the files in …
PF_RING

Introducing nBPF: line-rate hardware packet filtering (yes Wireshark at 100G is possible)

Modern network adapters such as Exablaze, Napatech and Silicom’s Intel FM10K, support hardware filters. Unfortunately every company has its own way to set filters, no unified API, and no support of any BPF-like filters. Most of the network monitoring community instead is used to set filters using BPF and thus powerful hardware filtering is present but unused. This has been the driving force for developing nBPF (ntop BPF). We have realized that most of the times filters include IP, port and protocol, that are exactly the features that hardware-based filters …
nProbe

ntop and Kentik bring nProbe to the Cloud

Traditionally nProbe is used as a host-based network monitoring probe able to produce “augmented” flow records including performance monitoring, security and visibility information. We have a common vision with Kentik of how network instrumentation needs to evolve beyond “just” bytes and packets-based NetFlow, and of how that can enable users to understand network performance and security challenges. This year, we entered a partnership with Kentik to leverage nProbe to export rich network metrics to the Kentik Detect big data network analytics cloud platform, and we’re proud to announce the first …
Announce

You’re Invited to the ntop Users Meeting and (free) Tutorial

Earlier this year we have held a ntop meetup in USA. Now we want to invite you to attend the ntop users meeting that will take place on October 17th (2 PM-5 PM), during the SharkFest Europe 2016 conference. The idea is to meet the ntop community, present our tools, highlight future work items and teach you how to master our tools. The ntop core team will be present at the event, and we would like to meet our users in person as we need to learn what are the things we …
nProbe

Flow-based Monitoring: nProbe Cento vs Standard/Pro

Since the introduction of nProbe Cento, we receive periodically emails of users wondering what are the differences between these two applications. This post is to clarify the differences, and better position them. The nProbe family is a set of flow-oriented applications, meaning that each packet is not handled individually but as part of a flow (e.g. a TCP connection or a UDP communication such as a VoIP call). This task is significantly more expensive than handling packets individually because we need both to keep the flow state and process packets in …