Howto use Kafka (instead of ZMQ) For Reliable Flow Collection and IPC
Historically, we have used ZMQ for interconnecting nProbe to ntopng, as this is a fast and simple messaging system. However one of they key advantage of ZMQ of being broker-less is sometime a problem. In case of maintenance, traffic peaks, or unreliable communications, the ZMQ communication between nProbe and ntopng will drop flows with the result that some data will never reach ntopng. As Apache Kafka is the de-facto standard for messaging communications, we have decided to extend its support from flow egress in nProbe / nProbe Cento, to communication …