Since its introduction in 1998, ntop(ng) has been a pure (well beside DNS address resolution if enabled) passive network monitoring tool. Recently we have complemented it with active device discovery in order to find out if there are silent devices in our network, and what services/OS our devices are featuring. In this article we will analyze how active discovery works, leaving to a future article the analysis of passive discovery.
Active discovery can be started on demand from the menu
or from the network preferences to enable periodic discovery
The result of network discovery is stored in redis and kept for future use. It allows to figure out the device type (is it a printer, a router or a tablet?) and their OS/features and advertised network services.
Above you can see a scan of a small home network. As you can see ntopng has recognised the router, and for my iMac detected the network services as well the exact computer model and type.
If you are wondering how active discovery works in ntopng, unless you want to read the its source code, below you can have a brief summary.
- The first thing ntopng does is a “ARP ping” to all local subnet hosts to identify the active assets. ntopng reads the IP subnet/mask from the network internet interface (if the interface has no subnet, for instance when you have a span port, active discovery won’t work) and starts pinging all devices. We do this via ARP as it’s a reliable method contrary to ICMP ping that might be disabled for some hosts. At the end of this process we have a list of active hosts that is compared against the list of devices that you see under Devices -> Layer 2. If there is a device that answers ARP ping that was not listed under L2 we mark it as ghost device (i.e. a device that is active on our local subnet but that is silent/ghost)
- We do a SSDP discovery so that devices (not all devices will answer to SSDP) can tell us the services they advertise. This works by sending a message to a multicast address and waiting for responses. Note that we might receive answers from hosts that do NOT belong to our subnet as they have seen the SSDP request: this is a good way to figure out it our local LAN is serving multiple networks or if somebody has created a private network (without permission?). So if you do that, make sure your devices are silent (note that ARP messages are sent in broadcast so even without SSDP you will be discovered by the network administrators), otherwise your secret network overlay will be detected. Also note that via SSDP is possible to learn the device capabilities as well the icon that is eventually stored inside the network device and that is shown by ntopng next to the IP address.
- In the meantime while SSDP responses are being received, for all active hosts discovered via ARP, ntopng sends a SNMP request to learn more about the device. As the standard “public” community is used, we might discover only a part of the devices. SNMP helps detecting features of devices such as printers or access points/routers.
- As small networks do not usually have a DNS, we use MDNS to resolve names (see for instance fritx.box) for local hosts when DNS is not available. Via MDNS it is also possible to know (in particular for Apple devices/phones) the advertised services and the device model/type. Recent OSX versions are much more verbose than old releases.
- Done all this we merge the information so far collected and produce discovery information as you can see on the above screen. The discovery information is kept on redis so it survives across ntopng restarts.
- If you have “hidden” devices on your network using a different subnet other than the one that you are supposed to use, make sure that devices are really hidden. But in all cases ARP will be broadcasted, so you will be caught soon or later.
- If you have installed in your network a NAT router that hides your private devices, ntopng will figure this out. For instance looking at User-Agent in HTTP headers that report OS and not just browser. Beware of this or always use SSL that is unlikely as your OS is sending many spontaneous requests so you will be caught too.
- SSDP/MDNS are very verbose protocols and advertise more information that you might think. Consider this when planning network security.
Finally we need your help! In our networks there are not so many different device types. Please send us reports form discovery in your network so we can improve the discovery process, or if you can send us a pull request with enhancements. Thank you!