ntopng

ntopng

How Flow Deduplication Works in nProbe and ntopng

Flow deduplication is the process of identifying and removing duplicate flow records that appear in NetFlow/IPFIX data when the same traffic is monitored and exported by multiple observation points (typically network devices) in the network. Without deduplication there are various issues that can arise including: Common flows duplication scenarios include: Said that flow duplication needs to be avoided, ntopng (Enterprise XL and superior) and nProbe (Enterprise L and superior) implement flow deduplication. In ntopng you can enable it from preferences and it works only with flow collection (i.e. ZMQ) and …
ntopng

ntopng Direct Dump Mode for High-Speed Flow Collection

When ntopng receives flows from nProbe (NetFlow collector) or nProbe Cento (100 Gbit probe) over ZMQ or Kafka, each flow must go through several processing stages before it is finally stored in the database. These stages include metadata enrichment, classification, analytics, behavioural checks, and additional internal operations. While this processing pipeline is essential for ntopng’s real-time monitoring, it naturally adds latency between the moment a flow arrives and when it becomes queryable in the (ClickHouse) storage backend. In large deployments ingesting thousands or hundreds of thousands of flows per second, …
ntopng

ntopng 6.6: IXP/Telco Traffic Observability, Faster Flow Collection

We’re excited to announce the release of ntopng 6.6, available today! This release focuses on Autonomous Systems (AS) analytics, a major rework of the flow collection engine to provide better correlations and improve performance, and a native ClickHouse Cloud integration. But, as usual, there are many other improvements. Key Breakthroughs Autonomous Systems Intelligence ntopng 6.6 introduces brand new Autonomous Systems dashboards, Sankey visualizations, and comprehensive AS statistics.You can now easily understand traffic relationships between transit and origin ASes, track top contributors, and visualize AS-level traffic flows in real time. The release also brings: These …
cento

HowTo Measure the Status and Performance of Network Flows

NetFlow has been originally designed to monitor network traffic using simple bytes/packets metrics. For TCP, it is also possible to know what TCP flags (that indicate the connection state) have been used on a flow, as NetFlow/IPFIX exports them as a cumulative OR of all TCP flags of the flow. This allows you to know if a SYN flag has been observed on a flow but not the number of SYN flags that have been reported for a flow. No other information elements have been implemented to report detailed TCP flow …
ntopng

Flow Direction Swapping Explained

A flow is a set of traffic packets sharing the same tuple (IP src, IP dst, port src, port dst, protocol, VLAN, …). When a flow is observed from the beginning, the first packet is sent by the client towards the server. Unfortunately, sometimes the flow was already in place when monitoring tools (e.g. ntopng or nProbe) started, and thus there is a chance that the flow direction is wrong simply because the first observed packet was from server to client. In this case, the flow is reported as if …
ntopng

AS Traffic Observability using ntopng

Since the first version of our tools, we have focused on packets. Having access to packets is a privilege that is not always possible; observing packets provides high-detailed information. At the edge of the Internet, traffic received/sent by hosts can be captured and observed, but in the case of network operators that act as a transit from the customers to the Internet, observing packets is not a good practice. This is because network operators need to make sure the service is available, but without going too deep. For this reason, network operators usually leverage NetFlow/IPFIX, sometimes …
ntopng

Introducing ntopng Alerts Graph: Visualize Security Events Like Never Before

Network security analysts often struggle to understand how alerts are connected across different hosts. Traditionally, ntopng displayed flow alerts in a table format, perfect for listing issues, but limited when it comes to spotting patterns or identifying which host is the real problem or victim. Additionally, tabular visualization does not let security analysts or network managers quickly determine which problem to tackle first, causes alert fatigue what are the main network issues, such as brute force attempts, obsolete TLS or SSH version connections, periodic flows etc. These issues are now …
nProbe

Best Practices for nProbe and ntopng Deployment

We often receive inquiries about the best practices for deploying nProbe and ntopng. This post will try to shed some light on this subject. The first thing to know is how many flows/second in total the nProbe instances will deliver to ntopng.  nProbe Flow CollectionEach nProbe instance can collect a high number of flows (in the 50/100k flows/sec range depending on hardware and flow types), but we typically suggest loading balance flows across multiple instances. Ideally, each nProbe instance should handle no more than 25k flow/sec. As ntop licenses are …
ntop

New, Fast, Scalable ClickHouse Integration for High-Volume Networks

When it comes to monitoring very large networks and the flows’ cardinality reaches into the billions, the performance of historical data storage and query systems becomes a critical bottleneck. Network operators, analysts, and engineers need to access flow records quickly and reliably, whether for traffic analysis, security investigations, or compliance reporting. When faced with massive datasets, even small inefficiencies in the data pipeline can result in slow queries, high CPU and disk usage, and poor responsiveness. At ntop, our mission is to help users gain visibility into their networks with …
ntop

Network Visibility and Observability: ntopng vs SNMP+

Recently, we’ve encountered users with high monitoring requirements. Some users need to monitor 1,000 routers and want to know who are the top talkers or top protocols. Others have a network with 200 branches, each with a NetFlow-enabled router. They need to know from a central location who are the top bandwidth users and ports on selected branches. Essentially, these users don’t need fine-grained network traffic monitoring. They just need a rough idea of who the top network users are (IP and ports). Often, users who ask us these questions …
ntop

HowTo Monitor+nDPI Traffic on Mikrotik Devices Using TZSP

Mikrotik devices are very popular in the ntop community. The simplest way to monitor traffic of these devices is using flows as described in this blog post. However sometimes flows might not be the best choice for various reasons including the inability to perform DPI on the captured traffic.  For full visibility you can use a different option offered by Mikrotik devices. Under Tools -> Packet Sniffer  you can export packets over the TZSP protocol (it is a sort of remote span protocol): just specify the IP of the remote …
ntopng

ntopng and nDPI Technical Webinars

One of the feedbacks we have collected at the PacketFest conference is to schedule periodic webinars about popular ntop tools we develop. For this reason, we have decided to start with ntopng and nDPI: Below you can find the video of the webinars that took plance on May 27th and June 10th.     Enjoy ! …