HowTo Implement Flow Relay, Replication and Fanout with nProbe

Posted · Add Comment

Sometimes flow (sFlow/NetFlow/IPFIX) collection can become a complicated activity when you need to:

  • Collect, on your private network, flows originated by devices with a public IP.
  • Migrate your infrastructure to nProbe/ntopng while sending flows to both nProbe and your legacy collector.

Implementing all this is often an expensive exercise with non-ntop solutions, therefore in order to ease migration to ntop tools, we made available in the nProbe package a couple of tools that can implement typical activities such as flow relay, replication and fanout easily.

Below you can learn how to do all this. Instead if you need to explore all nProbe features we suggest you to read the user’s guide.

Enjoy !

 

Flow Relay

Sometimes you need to collect (using a host in the private network) flows (over UDP) from devices located on the Internet/DMZ, and you want to avoid making a hole in your firewall for security reasons. In this case you need a flow relay that is basically an application deployed on the public Internet that acts as a rendez-vous point:

  • your Internet devices will send flows (sFlow/NetFlow/IPFIX) to the relay
  • your nProbe, deployed on the private LAN, will connect to the relay via (encrypted) ZMQ and receive the flows

This way you can collect flows from the private network without the need to create security weaknesses. The flowRelay application is part of the nProbe package and it works as follows

Welcome to flowRelay v1.0: sFlow/NetFlow/IPFIX flow relay
Copyright 2019-22 ntop.org
flowRelay [-v] [-h] -z <ZMQ enpoint>] -c <port>
   -z <ZMQ enpoint> | Where to connect to or accept connections from.
                    | Examples:
                    |   -z zmq://*:5556c        [collector mode]
   -c <port>        | Flow collection port
   -k <ZMQ key>     | ZMQ encryption public key
   -v               | Verbose
   -h               | Help

Example:
 - flowRelay -c 2055 -z "zmq://*:1234c" [running on host 192.168.1.2]
 - nprobe -i none -n none --collector-port zmq://192.168.1.2:1234

Suppose to have deploy the flowRelay on host with public IP a.b.c.d listening for incoming flows on port 2055, and nProbe on host 192.168.2.23. All you need to do is:

  • [host a.b.c.d] flowRelay -c 2055 -z "zmq://a.b.c.d:1234c"
  • [host 192.168.2.23] nprobe -i none -n none --collector-port zmq://a.b.c.d:1234

Flow Replication/Fanout

Sometimes (e.g. when you migrate to nProbe but you need to keep running your legacy monitoring system) you need to collect flows and send them to multiple (two or more) collectors. Without using expensive software solutions, nProbe users have a turn-key solution available out of the box named nfFanout. This application allows you to collect flows (sFlow/NetFlow/IPFIX) over UDP and send them to multiple collectors simultaneously.

 

19/Aug/2022 16:42:54 [nfFanout.c:99] Copyright (C) 2010-21 ntop.org
 Usage: nfFanout -c <port> -a <collector IPv4:port> [-a <collector IPv4:port>]*
                 [-v] [-V] [-r] [-h]
 -c <port>              | UDP port where incoming flows are received
 -a <collector IP:port> | Address:port where to send collected flows to
 -r                     | Use round-robin instead of fan-out
 -v                     | Enable verbose logging
 -V                     | Show application version
 -h                     | Print this help

Using it is pretty straightforward. Suppose you need to collect flows on port 2055 and send them to two collectors 192.168.0.1:1234 and 192.168.0.2:1234. All you need to do is to start the following command nfFanout -c 2055 -a 192.168.0.1:1234 -a 192.168.0.2:1234.