For years, applications have used static ports so that port 80 means HTTP, and port 25 SMTP. Unfortunately this 1:1 mapping has been relaxed years ago with dynamic ports so that a given service could use a range of ports (e.g. for circumventing security policies) or even a fully dynamic port (e.g. see portmap). The opposite is also true, namely HTTP can run on ports other than 80, so that you can see it for instance on port 3000 that is the default HTTP port in ntop. HTTP is also a good example of a popular protocol that is used also for non-hypertext based applications, including DropBox, Twitter and even Skype. The conclusion is that unless we inspect the data payload, we cannot decide which protocol is used by a given protocol.
Until release 4.1, ntop allowed users to identify IP-based protocol by specifying with -p <list> the list of ports. Today this association port/protocol is no longer true as we stated earlier in this post. On the other hand people want to know whether most of the traffic on their LAN is produced by FaceBook or BitTorrent, or if some “smart” user is using HTTP on port 1234 to circumvent the company HTTP filtering proxy. For this reason as of ntop 4.1.x and nProbe 6.7.x we have decided to give people visibility of application protocols so that their users can be aware of what’s flowing on the network.
After some internal libraries, tests with third party signatures (e.g. l7-filter), and public signatures found on the Internet, we have decided to use an open source library named OpenDPI as starting point. This library contains many protocols, but it lacks popular ones (e.g. Skype) or business protocols (e.g. Microsoft Exchange or Oracle). I have contacted the authors asking whether in case of extensions, they would have accepted the patches for inclusion. Unfortunately nobody answer, and then we forked the library and created an ntop-based version of OpenDPI you can find here, that already includes many new protocols including Skype, FaceBook and Twitter. Note that technically speaking FaceBook is not a protocol (it’s just HTTP/HTTPS) but we consider it as an application protocol (or a HTTP sub-protocol if you wish) because we want to distinguish it from other HTTP-based applications.
The above picture shows the current development version of ntop how detects and depicts application protocol. In nProbe (as of 6.7 version) we have also added application visibility in both NetFlow v9 and IPFIX by adding two new information elements:
[NFv9 57590][IPFIX 35632.118] %L7_PROTO Layer 7 protocol (numeric) [NFv9 57591][IPFIX 35632.119] %L7_PROTO_NAME Layer 7 protocol name
This means that with nProbe you can detect application protocol and send it to a V9/IPFIX compliant flow collector (including ntop).
So far we support over 125 application protocols, including Gmail, Google Maps, BitTorrent, Skype, VNC and VoIP protocols. We’re adding new protocols quite often, so you can be sure that your favorite protocol will be soon detected.
Credits: many thanks to a french company (that doesn’t like to be mentioned publicly) for funding this work.