.. _CliOptions: Command Line Options -------------------- nProbe supports a large number of command line parameters. To see what they are, simply enter the command :code:`nprobe -h` and the help information should be printed. The most important parameters are briefly discussed here. :code:`-n: collector addresses` This specifies the NetFlow collectors addresses to which nProbe will send the flows. If more than one is specified, they need to be separated with a comma or the --n flag can be repeated several times (e.g. -n 172.22.3.4:33,172.22.3.4:34 and -n 172.22.3.4:33 --n 172.22.3.4:34 are equivalent). When multiple collectors are defined, you can control the way flows are exported using the --a option (see below); if on a collector address the destination port is omitted, flows are sent to 2055 port and whereas if all the option is not specified, by default, flows are sent to the loop back interface (127.0.0.1) on port 2055. If this parameter is used, nProbe exports flows towards collector running at 127.0.0.1:2055. By default the UDP protocol is used but also TCP and SCTP (Linux only when nProbe is compiled with SCTP support and the kernel supports it). In this case you can specify the collector address as udp://:, tcp://:, and sctp://:, :code:`-i: interface name` It specifies the interface from which packets are captured. If -i is not used, nProbe will use the default interface (if any). In case a user needs to activate nProbe on two different interfaces, then he/she needs to activate multiple nProbe instances once per interface. For debugging purposes it is possible to pass nProbe a .pcap file from which packets will be read. If nProbe is compiled and activated with PF_RING support, you can specify multiple interfaces from which packets are captured. For example, “-i eth0,eth1”, will merge packets received on eth0 and eth1 into a single traffic stream. This configuration is particularly useful when merging the two directions (TX and RX) of a network TAP. :code:`-t: maximum flow lifetime` Regardless of the flow duration, a flow that has been active for more that the specified maximum lifetime is considered expired and it will be emitted. Further packets belonging to the same flow will be accounted on a new flow. :code:`-d: maximum flow idle lifetime` A flow is over when the last packet received is older that the maximum flow idle lifetime. This means that whenever applicable, (e.g. SNMP walk) UDP flows will not be accounted on 1 packet/1 flow basis, but on one global flow that accounts all the traffic. This has a benefit on the total number of generated flows and on the overall collector performance. :code:`-l: maximum queue timeout` It specifies the maximum amount of time that a flow can be queued waiting to be exported. Use this option in order to try to pack several flows into fewer packets, but at the same time have an upper bound timeout for queuing flows into the probe. :code:`-s: snaplen` This flag specifies the portion of the packet (also called snaplen) that will be captured by nProbe. By default nprobe sets the snaplen automatically according to its configuration, but you can override its value using thia flag. :code:`-p: //////` Flows can be aggregated both at collector and probe side. However probe allocation is much more effective as it reduces significantly the number of emitted flows hence the work that the collector has to carry on. nProbe supports various aggregation levels that can be selected specifying with the -p flag. The aggregation format is ////// where each option can be set to 0 (ignore) or 1 (take care). Ignored fields are set to a null value when doing the aggregation as well as when doing the export. For example setting the to 0 (ignore) will consider all the incoming flows as if they were coming from the same null exporter that will be output in %EXPORTER_IPV4_ADDRESS as 0.0.0.0. By default no aggregation is performed. For the sake of example, the value 0/0/1/0/0/0/0 can be used to create a map of who's talking to who (network conversation matrix). :code:`-f: packet capture filter` This BPF filter (see the appendix for further information about BPF filters) allows nProbe to take into account only those packets that match the filter (if specified). :code:`-a: select flow export policy` When multiple collectors are defined (see --n option), nProbe sends them flows in round robin. However it is possible to send the same flow to all collectors as a flow redirector does if the --a option is used. :code:`-b: enable verbose logging` Using this flag, nProbe generates verbose output that can be used to tune its performance (see chapter 2.4). Zero is the lowest level (little information is printed), 1 displays traffic statistics, 2 is really verbose. Example of traffic statistics: .. code:: bash 30/Oct/2020 16:10:00 Average traffic: [1.7 pkt/sec][1 Kb/sec] 30/Oct/2020 16:10:00 Current traffic: [1.9 pkt/sec][1 Kb/sec] 30/Oct/2020 16:10:00 Current flow export rate: [0.9 flows/sec] 30/Oct/2020 16:10:00 [Buckets: [active=13][allocated=21][free=8][toBeExported=0][frags=0] 30/Oct/2020 16:10:00 Fragment queue: [len=0] 30/Oct/2020 16:10:00 Num Packets: 111 (max bucket search: 0) 30/Oct/2020 16:10:00 115 pkts rcvd/0 pkts dropped :code:`-G: start nprobe as a daemon.` Useful when starting nprobe as daemon. :code:`--ndpi-protocols|-O: export only flows whose L7 protocol is in the specified list.` This option can be used to filter exported flows based on the application protocol. Example: -O "DNS,HTTP,BitTorrent". :code:`-P: dump flows` This path specifies the directory where flows will be dumped. The dump format is text and it depends on the nProbe template specified with -T. :code:`-F` It specifies the frequency at which files are dumped on disk :code:`-D: dump flows format` Flows stored on disks can be stored in multiple formats: text (default), binary (as they are exported), JSON, or gzip-compressed text flows. Note that this flag has no effect unless -P is used. :code:`-u: input device index` The NetFlow specification contains a numeric index in order to identify flows coming from different interfaces of the same probe. As multiple nProbe instances can be started on the same host but on different devices, the collector can use this flag to divide flows according to the interface number. If --u is not used, then nprobe will use 0 as interface index. Alternatively, if -1 is used then the last two bytes of the mac address of the flow sender are used as index. :code:`-Q: output device index` Similar to --u but for the output interface. :code:`--vlanid-as-iface-idx ` nProbe can use the VLAN tag as interface identifier. Using this flag you enable this feature. As VLAN tags can be stacked you need to specify if the inner or outer tag will be used for the interface identifier. :code:`--discard-unknown-flows ` nProbe includes nDPI support for analyzing packet contents in order to detect application protocol. The mode value can be used to: - 0: Export all know (i.e. those whose application protocol has been detected) and unknown (i.e. the application protocol is unknown) - 1: Export only know flows, discarding unknown flows. - 2: Export only unknown flows, discarding known flows. :code:`-v: print version` This flag is used to print the nProbe version number and date. :code:`-C: flow export lock` This is a simple way to implement high-availability. Start two probes capturing the same data. The master probe emit flows, the slave probe is started with --C . As long as exists, the slave works but no flow is emitted. If the file is deleted (e.g. using an external program for controlling the master/slave such as heartbeat) the slave starts emitting flows. If the file is restored, the slave is silent again. :code:`-h: print help` Prints the nProbe help. :code:`--dont-nest-dump-dirs` nProbe dumps data on disk (e.g. with -P) using a nested directory. In essence the base directory will be partitioned in sub-directories with //// structure. use this option is you want nProbe to dump all data in the base directory without creating this nested directory tree. :code:`-I: log to syslog ` nProbe logs on stdout unless the --g flag (see above) is used. If the syslog needs to be used instead of a file, this flag instruments nProbe to log on it using the specified name (this is useful when multiple nProbe instances are active on the same host). Please note that --g is ignored if --I is used, and this option is not available on nProbe for Win32. :code:`-w: size of the hash that stores the flows` The default size is 131072 and it should be enough for most of networks. In case flows are not emitted often and with strong traffic conditions it would be necessary to increase the hash. See later in this manual for knowing more about nProbe tuning. :code:`-W: Discard IPv6 traffic` Use this flag if you want nProbe not to account IPv6 traffic. :code:`-e: flow export delay` Some collectors cannot keep up with nProbe export speed. This flag allows flows to be slow down by adding a short delay (specified in ms) between two consecutive exports. The maximum allowed delay is 1000 ms. :code:`-B: packet count delay` It specified how many flow packets need to be sent before --e is applied, :code:`-z: ` Peer-to-peer applications, attacks or misconfigured applications often generate a lot of tiny flows that can cause significant load on the collector side. As most collector setups often discarded those flows, it is possible to instrument nProbe via the --z flag not to emit such flows. :code:`-M: maximum number of active flows` It is used to limit the maximum number of concurrent flows that the probe can sustain. This is useful for preventing the probe from creating as many flows as needed and hence to take over all the available resources. :code:`-E: netflow engine` Specify the netflow engineType:engineId into the generated flows. :code:`-m: minimum number of flows per packet` In order to minimize the number of emitted packets containing flows, it is possible to specify the minimum number of flows that necessarily need to be contained in a packet. This means that the packet is not emitted until the specified number of flows is reached. :code:`-q: :[] flow sender address and port` This option is used to specify the address and, optionally, the port that will be used by nProbe to emit the flows towards the destination indicated with -n. In practice, nProbe will create a socket and bind it to :code:`:[port]`, thus allowing the user to choose the interface taken by the emitted flows when leaving the host. :code:`-S ::` Three different rates can be specified with this option: - Packet capture sampling rate . This rate is effective for interfaces specified with -i and allows to control the sampling rate of incoming packets. For example, a sampling rate of 100 will instruct nprobe to actually process one packet out of 100, discarding all the others. All the statistics, including total bytes and packets, will be automatically up-scaled by nprobe to reflect the sample rate. In the previous example, the size of the sampled packet will be multiplied by 100. can be prepended with a '@' to instruct nprobe to only use the sampling rate for the up-scaling, without performing any actual sampling. This is particularly useful when incoming packets are already sampled on the capture device connected to nprobe but it is still meaningful to have up-scaled statistics. - Flow collection sampling rate . This rate works when nprobe is in collector mode, that is, when option --collector-port is used and specifies the flow rate at which flows being collected have been sampled. In this case, no actual sampling is performed on the incoming flows. The specified rate is only used to perform the upscaling. For example, a flow with 250 IN_BYTES will be up-scaled by a factor equal to the sampling rate. If the sampling rate is 100, a total of 2500 IN_BYTES will be accounted for that flow. - Flow export rate . This rate is effective when nprobe exports NetFlow towards a downstream collector, that is, when option -n is used. It controls the output sampling. For example, a of 100 will cause nprobe to only export 1 flow out of 100 towards the downstream collector. :code:`-A: AS file` Network probes are usually installed on systems where the routing information is available (e.g. via BGP) in order to specify the AS (Autonomous System) id of the flow peer. As nProbe has no access to BGP information unless you enable the BGP plugin, users need to provide this information by means of a static file whose format is :. The file can be stored in both plain text and gzip format. :code:`--city-list: City List` With this option you can enable geolocation of IP addresses at city/country detail level. Here you need to specify the GeoIP city database (e.g. GeoLiteCity.dat) :code:`-g` It specifies the path where nProbe will save the process PID. :code:`-T: flow template definition` Contrary to NetFlow v5 where the flow format is fixed, NetFlow V9 and IPFIX flows have a custom format that can be specified at runtime using this option as specified in appendix. :code:`-U: flow template id` NetFlow v9 and IPFIX flows format is specified in a template whose definition is sent by nProbe before to start sending flows. The flow format is defined by --T, where --U is used to set the template identifier. This option should not be used unless the default template value (257) needs to be changed. As based on -T nProbe can define several templates, this value is the one used for the first defined template. :code:`-V: flow export version` It is used to specify the flow version for exported flows. Supported versions are 5 (v5), 9 (v9) and 10 (IPFIX). :code:`-o: intra templates packet export.` It specifies the number of flow packets that are exported between two templates export. :code:`--aggregate-gtp-tunnels` Aggregates traffic flowing in each GTP tunnel based in tunnel id. :code:`-L: local networks` Use this flag to specify (format network/mask, e.g. 192.168.0.10/24) the list of networks that are considered local (see --c). :code:`-c: track local hosts only` It allows nProbe to set to 0.0.0.0 all those hosts that are considered non-local (see --L). This is useful when it is necessary to restrict the traffic analysis only to local hosts. :code:`-r: set traffic direction` When this option is used (-L must be specified before --r), all the traffic that goes towards the local networks is considered incoming, all the rest is outgoing. This has effect on the --u/-Q that are then forced with --r. :code:`--if-networks` Flags -u and -Q are used to specify the SNMP interface identifiers for emitted flows. In mirrored environments, it is possible to simulate a switched environment by playing with MAC addresses. This option allows users to bind a MAC or IP address to a specified interfaceId.. The syntax of --if-networks is @ where multiple entries can be separated by a comma (,). Example: --if-networks "AA:BB:CC:DD:EE:FF@3,192.168.0.0/24@2" or --if-networks @ where is a file path containing the networks specified using the above format. :code:`--count: debug only` Let the probe capture only up to the specified number of packets. :code:`--collector-port: specifies the NetFlow collector port` Use nProbe to collect NetFlow/jFlow/IPFIX/sFlow packets. Use option :code:`--collector-port` to specify on which on which ports such packets should be collected. nProbe is able to ingest and convert flows from various versions. For instance :code:`nprobe --collector-port 2055 --i 192.168.0.1:2056 --V 10` converts each flow received on port 2055 to IPFIX and sends them to 192.168.0.1:2056. By default nProbe binds the collection port to all available interfaces. If you want you can bind the port only to one interface. This can accomplshed specifying an optional local (to the host where nprobe is running) IP address. Exampple -3 192.168.1.23:2055. Option :code:`--collector-port` can also be used to receive NetFlow/jFlow/IPFIX/sFlow packets through a ZMQ relay. In this case one should specify a ZMQ endpoint. An implementation of a ZMQ relay is available in executable :code:`flowRelay`. Run :code:`flowRelay -h` to see how to use it. :code:`--collector-passthrough` Export flows to the configured ZMQ endpoints as-is, ignoring the :code:`-T`. Using :code:`--collector-passthrough` gives the highest collection throughput. ZMQ/Syslog/Kafka exports are supported. See :ref:`UnderstandingFlowCollection` for a detailed discussion. Note that -T is ignore when passthrough is used. This is a nProbe Pro-only feature. :code:`--collector-nf-reforge ` Flow collection-only feature. It allows users to configure NetFlow collection filtering and reforge by specifying a configuration file and passing it as argument. The file format is the one shown in this example (columns are tab separated). In this example, flows sent by NeFflow probe active at IP address 192.168.1.1 are collected by nProbe and exported (e.g. via ZMQ to ntopng or to a remote collector via -n) as if they were sent by host 192.168.1.1: only flows from Netflow interfaceId 1,2,3,4 are handled, all other interfaces are discarded. For collecting all interfaces and just reforginf the probe IP address use * in the interface list. Probes not listed in the file are handled as-is without any reforging or template filtering. .. code:: bash # CollectorIP ReforgedIP ListOfAllowedInterfaces # Example: 127.0.0.1 10.0.24.25 12 192.168.1.1 192.168.1.1 1,2,3,4 :code:`--tunnel` Let the probe decode tunneled traffic (e.g. GTP or GRE traffic) and thus extract traffic information from such traffic rather than from the external envelope. :code:`--no-promisc` With this option nProbe does not use promiscuous mode to capture packets. :code:`--smart-udp-frags:` Ignore UDP fragmented packets with fragment offset greater than zero, and compute the fragmented packet length on the initial fragment header. This flag might lead to inaccuracy in measurement but it speeds us operations with fragmented traffic. :code:`--ipsec-auth-data-len` Length of the authentication data of IPSec in tunnel mode. If not set, IPSec will not be decoded but just accounted. :code:`--dump-stats: dump some flow statistics on file` Periodically dump NetFlow statistics on the specified file. Note that when using nProbe over PF_RING, nProbe dumps statistics on /proc/net/pf_ring/stats/. :code:`--black-list` With this option you can specify a list of networks or hosts from which all the incoming packets will be discarded by the probe. The accepted notation can be CIDR format or the classical network/netmask format. :code:`--pcap-file-list ` The specified file path contains a list of pcap files to be read in sequence by nProbe. Use this option when you want nProbe to read a list of pcap files (e.g. when generated using tcpdump). :code:`--biflows-export-policy ` Bi-directional flows are such when there is traffic in both direction of the flow (i.e. source->dest and dest->source). As mono-directional flows might indicate suspicious activities, this flag is used to determine the export policy: - 0: Export all know (i.e. mono and bi-directional flows) - 1: Export only bi-directional flows, discarding mono-directional flows. - 2: Export only mono-directional flows, discarding bi-directional flows. :code:`--csv-separator ` Override the default ‘|’ separator in dumps with the specified one. :code:`--dont-drop-privileges` Do not drop root privileges to user ‘nobody’ when this option is specified. See al --unprivileged-user later int this manual. :code:`--account-l2` NetFlow accounts IP traffic only, not counting layer 2 headers. Using this option the layer 2 headers are also accounted in flow traffic statistics. :code:`--dump-metadata ` Dump metadata information into the specified file and quit. This option is useful when users want to know the type of each information element exported by nProbe so that (for instance) they can properly import into a database. :code:`--ntopng