Using ntop Applications with Docker and OpenStack

Posted · Add Comment

In order to ease the deployment of our applications, in addition to source code distribution, we have released binary packages (x64 and ARM) for CentOS/RedHat and Ubuntu/Debian. For PF_RING, that requires to be compiled against the installed kernel version, we have moved to DKMS so that you are no longer required to use the same kernel version we use for packaging it.

However the current trend is going towards virtualised environments (not just VMs such as VMware) and IaaS (Infrastructure as a Service) and thus we need to support them.

 

Docker


In essence there are two types of virtualisation:

  • Virtual Machine: emulation of a particular computer system, including its devices (network, storage, USB etc).
  • Operating-system level virtualisation: run multiple isolated user-space instances (often called containers) that look like a real server.

Docker is an open-source software that automates the deployment of applications inside software containers. Each container runs within a single Linux instance without the overhead of starting VMs. We have created a Docker container for ntopng (but others can be created for the other ntop apps) that allows you to run ntopng on a clean and isolated environment. We have built a dock on hub.docker.com

DockerHub

so that you can go to docker.com and search for ntopng Screen Shot 2014-11-04 at 13.26.03install it,

root@ubuntu:/home/deri# docker pull lucaderi/ntopng-docker
Pulling repository lucaderi/ntopng-docker
8077c18a90a8: Download complete
511136ea3c5a: Download complete
d497ad3926c8: Download complete
ccb62158e970: Download complete
e791be0477f2: Download complete
…
e072f31bb2a5: Download complete
9e52f4c92f80: Download complete
ecc46895937f: Download complete
3a3f2545e225: Download complete
4f1229fadea7: Download complete
5b5364929cbf: Download complete
Status: Downloaded newer image for lucaderi/ntopng-docker:latest

then run it

root@ubuntu:/home/deri# docker run --net=host --name ntopng -t -i lucaderi/ntopng-docker ntopng -v
….
02/Nov/2014 12:55:20 [main.cpp:183] PID stored in file /var/tmp/ntopng.pid
02/Nov/2014 12:55:20 [HTTPserver.cpp:374] HTTPS Disabled: missing SSL certificate /usr/share/ntopng/httpdocs/ssl/ntopng-cert.pem
02/Nov/2014 12:55:20 [HTTPserver.cpp:376] Please read https://svn.ntop.org/svn/ntop/trunk/ntopng/README.SSL if you want to enable SSL.
02/Nov/2014 12:55:20 [HTTPserver.cpp:420] Web server dirs [/usr/share/ntopng/httpdocs][/usr/share/ntopng/scripts]
02/Nov/2014 12:55:20 [HTTPserver.cpp:423] HTTP server listening on port 3000
02/Nov/2014 12:55:20 [main.cpp:231] Working directory: /var/tmp/ntopng
02/Nov/2014 12:55:20 [main.cpp:233] Scripts/HTML pages directory: /usr/share/ntopng
02/Nov/2014 12:55:20 [Ntop.cpp:218] Welcome to ntopng x86_64 v.1.2.2 (r8539) - (C) 1998-14 ntop.org

The –net directive allows you to instruct ntopng to monitor all the host traffic and not just the container running ntopng.

 

OpenStack


OpenStack is a technology that allows to
deploy and control resources on a data
center (VMs, storage, networking). Our interest in OpenStack is manyfold:

  • Create an OpenStack VM image for enabling people to easily deploy ntop monitoring apps on datacenter.
  • Exploit ntop’s PF_RING open-source packet processing technology for bringing packets in 0-copy at 10 Gbit on a VM managed by OpenStack. This is to enable efficient traffic monitoring on a data center.

Through OpenStack we want to be able to deploy VMs with ntopng and attach them to virtual controllers (Open vSwitch) or 0-copy PF_RING ZC-based packet sources. With ZC, packets are captured in 0-copy from network adapters and deployed in 0-copy to VMs. ZC packets are deployed on the VM using virtual adapters attached dynamically to the VM though a ntop-developed kernel module based on PCI hotplug as described in this document. We have no interest, as many companies did, to accelerate Open vSwitch as for us:

  • This is just a way to communicate with the VM: nice if it’s faster, but the current Open vSwitch is good enough for carrying on activities such as flow-export or connect to the ntopng GUI via https.
  • We need to focus on what a VM can do in OpenStack, so that we can provide 10G line rate to the VM in RX and TX, with minor performance degradation with respect to the performance you can achieve on bare metal.

The good news is that we have prepared all you need to be productive immediately. If you’re an OpenStack user, we have create a VM image you can use for deploying our apps in minutes. You just need to download the OpenStack VM image, and place it onto your datacenter and create in minutes simple or complex topologies such as those depicted below.

 

OpenStack

OpenStackVMs

 

This slideshow requires JavaScript.

 

Final Remarks


Either you run ntop apps or PF_RING ZC on a physical machine, a container, or a OpenStack VM, we have created for you all the basic pieces you need. If you are running on a pure virtual environment, we also have the ability to monitor both your processes and your network using nProbe with Sysdig. In essence we have pre-built all you need for processing packets at high speed on both physical and virtual environments.