ntopng Integration

nEdge Lite is designed to work seamlessly with ntopng, providing centralized monitoring, visualization, and policy management. This section explains how to set up and optimize the integration.

Overview

The nEdge Lite-ntopng integration provides:

  • Real-Time Flow Monitoring: View all flows inspected by nEdge Lite in ntopng’s web interface

  • Protocol Analytics: See detailed statistics on detected protocols and applications

  • Dynamic Policy Management: Configure and update policies through ntopng’s GUI

  • Historical Data: Store and query flow data over time

  • Alerting: Set up alerts for policy violations or suspicious traffic

  • Multi-Instance Support: Multiple nEdge Lite instances can report to a single ntopng

Integration Architecture

Communication Channels

nEdge Lite and ntopng communicate via two ZeroMQ channels:

  1. Flow Export Channel: nEdge Lite sends flow data to ntopng (ZMQ PUB/SUB)

  2. Policy Update Channel: ntopng sends policy updates to nEdge Lite (ZMQ PUB/SUB)

┌─────────┐                    ┌─────────┐
│   nEdge Lite   │──── Flows ────────>│  ntopng │
│         │<─── Policies ──────│         │
└─────────┘                    └─────────┘

ZeroMQ Endpoints

  • ntopng as Collector: Endpoint URL must end with c (e.g., tcp://127.0.0.1:1234c)

  • nEdge Lite as Publisher: Endpoint URL without c (e.g., tcp://127.0.0.1:1234)

Basic Setup

Same-Host Deployment

When nEdge Lite and ntopng run on the same machine:

Start ntopng:

sudo ntopng -i tcp://127.0.0.1:1234c

Start nEdge Lite:

sudo nedgelite -q 0 -z tcp://127.0.0.1:1234

Access ntopng Web Interface:

Open http://localhost:3000 in your browser. You should see flows from nEdge Lite appearing in real-time.

Remote Deployment

When nEdge Lite and ntopng run on different machines:

On ntopng host (192.168.1.10):

sudo ntopng -i tcp://0.0.0.0:1234c

On nEdge Lite host:

sudo nedgelite -q 0 -z tcp://192.168.1.10:1234

Firewall Configuration:

Ensure TCP port 1234 is accessible from the nEdge Lite host to ntopng host.

Dynamic Policy Management

To enable bidirectional communication (flow export + policy updates):

Setup with Policy Updates

On ntopng host:

sudo ntopng -i tcp://0.0.0.0:5556c --zmq-publish-events tcp://0.0.0.0:5557

On nEdge Lite host:

sudo nedgelite -q 0 -z tcp://ntopng-host:5556 -p tcp://ntopng-host:5557

Configure Policies in ntopng:

  1. Open ntopng web interface

  2. Navigate to Settings → Policies

  3. Create or modify policies

  4. Changes are automatically pushed to nEdge Lite

Managing Policies via ntopng

Creating a Pool:

  1. Go to Pools → Host Pools

  2. Click Add Pool

  3. Define IP ranges and/or MAC addresses

  4. Assign a policy to the pool

Creating a Policy:

  1. Go to Policies → Traffic Policies

  2. Click Add Policy

  3. Configure protocol/category filters

  4. Set default action (Pass/Drop)

  5. Click Save

Viewing Applied Policies:

  1. Go to Flows → All Flows

  2. Look for the Policy column

  3. Flows show which policy was applied

Multiple nEdge Lite Instances

Single ntopng, Multiple nEdge Lite Deployments

Deploy nEdge Lite on multiple hosts, all reporting to one ntopng:

On ntopng host:

sudo ntopng -i tcp://0.0.0.0:5556c --zmq-publish-events tcp://0.0.0.0:5557

On nEdge Lite host 1:

sudo nedgelite -q 0 -z tcp://ntopng-host:5556 -p tcp://ntopng-host:5557

On nEdge Lite host 2:

sudo nedgelite -q 0 -z tcp://ntopng-host:5556 -p tcp://ntopng-host:5557

On nEdge Lite host 3:

sudo nedgelite -q 0 -z tcp://ntopng-host:5556 -p tcp://ntopng-host:5557

All nEdge Lite instances will:

  • Send flows to the same ntopng

  • Receive policy updates from the same ntopng

  • Appear as separate interfaces/collectors in ntopng

High Availability

For redundancy, send flows to multiple ntopng instances:

sudo nedgelite -q 0 \
  -z tcp://ntopng1:5556 \
  -z tcp://ntopng2:5556 \
  -p tcp://ntopng1:5557

Encrypted Communication

Secure flow export with ZeroMQ CURVE encryption.

Generate CURVE Keys

On ntopng host, generate a key pair:

# Generate keys using zmq tools
curve_keygen

This outputs:

public-key: Yne@$w-vo<fVvi]a<NY6T1ed:M$fCG*[IaLV{hID
secret-key: D:)Q[IlAW!ahhC2ac:9*A}h:p?([4%wOTJ%JR%cs

Configure ntopng with Encryption

sudo ntopng -i tcp://0.0.0.0:5556c \
  --zmq-encryption-key "Yne@$w-vo<fVvi]a<NY6T1ed:M$fCG*[IaLV{hID:D:)Q[IlAW!ahhC2ac:9*A}h:p?([4%wOTJ%JR%cs"

Configure nEdge Lite with Encryption

sudo nedgelite -q 0 \
  -z tcp://ntopng-host:5556 \
  -y "Yne@$w-vo<fVvi]a<NY6T1ed:M$fCG*[IaLV{hID"

Note: nEdge Lite only needs the public key, ntopng needs both.

Flow Export Formats

TLV Format (Default)

Type-Length-Value format is compact and efficient:

sudo nedgelite -q 0 -z tcp://127.0.0.1:1234

Advantages:

  • Compact binary format

  • Lower bandwidth usage

  • Faster serialization

JSON Format

Human-readable JSON format for debugging:

sudo nedgelite -q 0 -z tcp://127.0.0.1:1234 -j

Advantages:

  • Human-readable

  • Easy to debug

  • Compatible with custom integrations

Disadvantages:

  • Higher bandwidth usage

  • Slower serialization

Optimizing Flow Export

Flow Update Interval

Adjust how often flows are updated:

# Update every 10 seconds (more real-time)
sudo nedgelite -q 0 -z tcp://127.0.0.1:1234 -u 10

# Update every 60 seconds (less overhead)
sudo nedgelite -q 0 -z tcp://127.0.0.1:1234 -u 60

Recommendations:

  • Real-time monitoring: 5-15 seconds

  • Historical analysis: 30-60 seconds

  • High-volume environments: 60-120 seconds

Immediate Flush

Disable batching for lowest latency:

sudo nedgelite -q 0 -z tcp://127.0.0.1:1234 -f

Use Cases:

  • Debugging and troubleshooting

  • Low-traffic environments

  • When latency is critical

Trade-offs:

  • Higher CPU usage

  • More ZMQ messages

  • Increased bandwidth

Monitoring the Integration

Verify Flow Export

In ntopng:

  1. Go to Interfaces

  2. Look for the ZMQ collector interface

  3. Check packet/byte counters

Check ZMQ Connection:

# On ntopng host
sudo netstat -tnlp | grep 1234

# On nEdge Lite host
sudo netstat -tn | grep 1234

View Flows in ntopng

  1. All Flows: Go to Flows → All Flows

  2. Top Protocols: Go to Dashboard → Top Protocols

  3. Top Applications: Go to Dashboard → Top Applications

  4. Alerts: Go to Alerts → Detected Alerts

Troubleshooting Integration

No Flows Appearing in ntopng

  1. Verify ntopng is listening:

    sudo netstat -tnlp | grep ntopng

  2. Check nEdge Lite is running:

    ps aux | grep nedgelite

  3. Verify ZMQ endpoint configuration:

    • ntopng endpoint must end with c

    • nEdge Lite endpoint must not have c

  4. Check firewall rules:

    sudo iptables -L -n -v | grep 1234

  5. Enable verbose logging:

    sudo nedgelite -q 0 -z tcp://127.0.0.1:1234 -v

Policy Updates Not Working

  1. Verify policy endpoint:

    sudo nedgelite -q 0 -z tcp://127.0.0.1:5556 -p tcp://127.0.0.1:5557 -v

  2. Check ntopng is publishing:

    sudo netstat -tnlp | grep 5557

  3. Restart nEdge Lite after policy changes:

    If using static files, send SIGHUP:

    sudo kill -HUP $(pidof nedgelite)

High Latency or Packet Loss

  1. Reduce flow update interval:

    sudo nedgelite -q 0 -z tcp://127.0.0.1:1234 -u 60

  2. Use multiple queues:

    sudo nedgelite -q 0:4 -z tcp://127.0.0.1:1234

  3. Check network bandwidth:

    iftop -i eth0

Integration Best Practices

  1. Co-locate when possible: Run nEdge Lite and ntopng on the same host for lowest latency

  2. Use TLV format: More efficient than JSON for production

  3. Tune update interval: Balance real-time visibility with performance

  4. Enable encryption: For remote deployments or sensitive environments

  5. Monitor ZMQ queue: Watch for backlog indicating performance issues

  6. Use multiple collectors: For high-availability deployments

  7. Regular backups: Export ntopng configuration and historical data

Next Steps