Collecting Proprietary Flow Fields¶
nProbe is not only able to collect standard flow fields, that is, those defined in the NetFlow/IPFIX RFCs, it is also able to collect custom and proprietary flow fields. Proprietary fields are ofen used by vendors to export data using standard NetFlow v9 or IPFIX for the transport. Cisco Application Visibility and Control (AVC), Cisco Medianet, IXIA IxFlow, and SonicWall AppFlow are just a few examples.
Proprietary fields can be declared using a plain text file that has to
be input to nProbe and they will be treated by nProbe as if they were
regular, standard fields. This means that those fields can be exported
simply by specifying them in the template
The plain text file has one or more lines in the following format:
Name STANDARD_ALIAS PEN FieldId Len Format
Nameis just a mnemonic name assigned to the proprietary field that is used to identify the field within nProbe, for example when indicating the field in the template.
STANDARD_ALIASis a mapping to a standard field.
NONEcan be specified when no mapping to a standard field is available. Typically, it is useful to specify this mapping to convert proprietary fields into standard, well-known ones.
PENis the Private Enterprise Number of the proprietary field. If the field has no PEN it is safe to specify
FieldIdis the integer number that, along with the PEN, uniquely identifies the proprietary field.
Lenis the length of the field, expressed in bytes.
Formatspecifies how to interpret the bytes of the field:
dump_as_uintinterprets the field as an unsigned integer
dump_as_ipv4_addressinterprets the field as an IPv4 address
dump_as_mac_addressinterprets the field as a mac address
dump_as_hexinterprets the field as an hexadecimal value
dump_as_asciias a text string in ASCII format.
Available Custom Flow Fields Definitions¶
nProbe comes with preconfigured vendor flow definitions, so users can immediately collect popular flow fields without having to create a custom flow configuration. The list of available configurations is available at https://github.com/ntop/nProbe/tree/master/custom_fields
A snippet of a text file with the declaration of custom fields for a few vendors is available below:
# # Name STANDARD_ALIAS PEN FieldId Len Format # # Notes # - STANDARD_EQUIVALENT # If this field is equivalenet/alias of a standard field, put its name here # or put NONE if there is no standard equivalent element # # Notes # - Use 0 for no PEN # # # Available formats # - dump_as_uint # - dump_as_ipv4_address # - dump_as_mac_address # - dump_as_hex # - dump_as_ascii # # Gigamon HttpReqUrl NONE 26866 1 64 dump_as_ascii DnsQueryName NONE 26866 204 64 dump_as_ascii DnsResponseIpv4Address NONE 26866 207 4 dump_as_ipv4_address
Let’s assume proprietary fields
DnsQueryName have to be exported. one can put the
declarations in a text file
custom-fields.txt and start nProbe
./nprobe --load-custom-fields "custom-fields.txt" -T "@NTOPNG@ %HttpReqUrl %DnsQueryName" ...
This is an nProbe Professional feature only.