Using ntop in Education: South Panola School District

South Panola School District’s (SPSD) network continues to evolve to better serve the needs of its students and staff. Upon employment at SPSD, the district had less than 1gbps to the internet and now boasts 3gpbs. With more and more traffic flowing through our network, SPSD has a need to better monitor the traffic to determine more soundly the direction of our network’s evolution.

Throughout SPSD’s evolution multiple tools have been employed for this effort. Ultimately, the tools employed throughout the years have been found deficient or lacking in some regard, be it flexibility, retention, capability, or depth. Sometimes software was too complex or expensive and failed to survive beyond its trial use. Over time, SPSD concluded it needed to take a great amount of time and effort to determine key points of data that was wanted, and ways to extrapolate and aggregate it into visualizations, as well as retain it for deep dives. Therein lies the issue, where would time afford such efforts? After leaving such goals on the backburner, SPSD discovered ntopng completely by accident.

SPSD’s discovery tied into an event wherein software and hardware costs were escalating with regards to firewalling and it needed a solution that was price efficient and capable. SPSD swapped its firewalls to an open-source firewall capable of meeting its needs and from there learned of ntopng’s existence. At this point, we decided to run ntopng on the firewalls themselves and quickly determined that ntopng was the answer to our needs. Soon, however, we realized that we were unable to take advantage of the full capabilities of ntopng due to hardware design.

Our firewalls were targeted at their task, firewalling, and ntopng, while light on resource usage for the task it was ultimately accomplishing, was in need of more resources than our firewalls could afford. It also didn’t help that it was in essence running on more than one firewall, making the data collected split between them. SPSD wanted the data all in one place. There were multiple ways to accomplish such, but ultimately SPSD had to come up with a sound plan to employ the advantages that ntopng could afford.

After the realization, and over no short amount of time, SPSD finally decided to buy the hardware necessary, from NICs to a rack-mount server with plenty of drive slots, and started to execute its plan. Overall the key areas we wanted to employ ntopng were in monitoring traffic entering and exiting the firewall, the network edge. SPSD also wanted to keep a few ports available for monitoring devices we might plug directly into the box running ntopng so as to determine issues with them or if they were inadvertent participants in malicious activity.

With the purchase of the new hardware, port mirrors set up on switches, 10gbps links established … since most of the edge is a bunch of 10gbps links to and from our network, firewall and ISP, we began to determine the software necessary and potential capabilities ntop.org’s software could offer us. The most important capability, by and far, in our opinion, was that ntop’s products could, when afforded sufficient hardware, monitor line rate 100gbps. SPSD is hard pressed to find any contender capable of the same without having to break the bank to

attain the capability. Ntopng’s capabilities don’t end there and it is not the only ntop.org software employed by SPSD either.

To monitor at line rate above 1 Gbps PF_RING ZC was determined necessary, for ntopng, nProbe Cento, and n2disk. N2disk facilitates packet captures (pcap files basically) at line rate provided your disk bandwidth is sufficient. nProbe Cento turns a stream of packets into flows in the same vein as NetFlow/IPFIX. Ntopng combines with other ntop software seamlessly, allowing us to create pcap files for whatever time period selected so long as n2disk was recording at the time. Ntopng also receives the flows from nProbe Cento allowing us to use ntopng’s capabilities to visualize and extrapolate the data into easier to consume visualizations and aggregations. Then there is nProbe, which can be used to collect flows from devices generating IPFIX/sFlow/NetFlow data. Again, ntopng eventually receives the data from nProbe so as to facilitate easier consumption. nProbe and nProbe Cento aren’t limited to just the uses we have employed either, being capable of forwarding the flows to any flow collector. nProbe Cento again has yet more capabilities that we are not currently employing due to lack of need.

At the end of all this SPSD has to consider how the data is retained, and for how long, and ntopng doesn’t disappoint here, exceeding our expectations in both regards. Ntopng has a way to limit the retention to a time period, as well as store the data such that the resolution is retained. We have ntopng export time series to influxdb and flows to ClickHouse, which ultimately store their data on a 12 disk RAID 10 of SATA SSDs. This allows SPSD to maintain high resolution historical data points for exploration upon necessity or to learn more about how the network’s usage is shifting over time, all of which facilitate future network design decisions. N2disk is now being run in such a way as to constantly capture data. Its design allows us to decide what percentage of a disk it can/should use and therefore we never overrun disk capacity. This does mean we are limited in how much pcap type data we retain but it is physically impossible to store all data sent and received over the network over a lengthy time period without major investments that are outside the reach of a school district. Besides that, the idea isn’t to have pcap’s from forever ago, but to be able to export the data for more in depth analysis as necessary. Again, ntopng facilitates such with a way to determine what time period of data is wanted. In general, SPSD believes pcaps are more an on demand thing for us, than something we want to retain for ages to come. Just as ntopng can export pcaps, it can export flows to time series as well. This means if we notice a set of flows are due to roll off but are necessary for some purpose, we can retain them manually. Finally, for graphs and visualizations, ntopng can take what amounts to some degree as a screenshot of the graph you currently have presented and store it indefinitely. All of these things are features we are using, or intend to use as time progresses and needs arise.

With all the talk of retention one might wonder as to what’s involved with the live data, derived from recent packets and flows captured. Ntopng remains just as capable with live data as it does with historical data and provides many different ways to visualize and consume the data available. Extrapolating the dearth of data provided by ntop’s products in any other fashion would be an unwieldy and time consuming task, likely resulting in poor assumptions being made with regards to SPSD’s evolving network direction. Ntopng has enough features and capabilities

to it that we may find ourselves discovering uses for years to come, especially considering that it will expand in capability and features itself as time progresses. Ntopng’s reporting capabilities are still being explored and will definitely be of use to SPSD as we move forward. Ntopng’s alerting capabilities remain to be explored to a greater depth and we find it necessary to customize in this regard to ultimately arrive at a set of alerts that are important to SPSD. Ntopng is facilitating insight into network applications crossing from our local network over to the internet. It provides us traffic graphs as well. With this list I’ve barely scratched the surface of what it can and does do for us. I know we’ll make use of its maps and ASN correlation features over time for instance, in addition to all that which has so far been listed. We’re even using its SNMP monitoring for a low resolution graph of traffic on certain interfaces within our network.

Using ntop.org’s products is by necessity a journey, as more uses continue to be discovered, and SPSD’s needs evolve. As networking evolves, traffic usage increases and changes, so does SPSD itself evolve. Ultimately, ntop.org’s products empower SPSD to monitor its network for critical information as well as malicious activity if such were to occur. SPSD intends to use ntop.org’s products to help us derive future directions to service the needs of our students and staff more efficiently as the network direction evolves.

The only question before us is can we even take advantage of all that ntop.org’s products offer us? Time will tell and every effort will be made, but with our evolving direction, ntop.org’s products evolving, and yet more capabilities being discovered within their products, it feels that SPSD will likely find that it’s still just at the tip of the iceberg and that more advantages could be provided should we just seek a bit further.