Technologies and Trends

Data Privacy

Export and Archive ClickHouse Flows in ntopng for Regulatory Compliance

Most ntopng users make extensive use of ClickHouse support for storing historical flow data and running analysis on it. ClickHouse is highly optimized and offers a high compression rate (estimated at an average of 60 bytes per flow), allowing for long data retention even with limited storage. However, to comply with regulations such as GDPR, SOX, HIPAA, and PCI DSS, it is often necessary to retain data for extended periods. This is manageable when flow rates are low to moderate, but can require significant disk space when flow rates are …
ntop

New, Fast, Scalable ClickHouse Integration for High-Volume Networks

When it comes to monitoring very large networks and the flows’ cardinality reaches into the billions, the performance of historical data storage and query systems becomes a critical bottleneck. Network operators, analysts, and engineers need to access flow records quickly and reliably, whether for traffic analysis, security investigations, or compliance reporting. When faced with massive datasets, even small inefficiencies in the data pipeline can result in slow queries, high CPU and disk usage, and poor responsiveness. At ntop, our mission is to help users gain visibility into their networks with …