Submitting a Bug Report
For open-source applications such as ntopng, if you want to report a bug, please open an issue on our official Github page. For commercial applications such as nProbe you can contact us privately or fill this form.
Submitting a Code Patch
If you want to submit a patch please send us a merge request via Github. We will be happy to review it and possibly include in one official branch.
What To Do In Case Of Crash
In the (unlikely) case of crash, it is fundamental to understand the root cause of the crash. Finding the cause of a crash boils down to debugging the application, but don’t worry, you do not have to be a programmer to do that! In the remainder of this page we will show some simple steps that that will allow you to debug any product and report us the resulting information.
Debugging Applications Built from Source
When debugging applications built from source, you have to make sure the compiler has not performed compile-time code optimizations. This kind of optimizations usually make the code (marginally) faster but prevent the application from being debugged easily.
Typically, when using the GNU toolchain, debugging symbols are configured in the Makefile. Therefore, you have to make sure no option such as
-Os is used there. Another option that you should enable when using the GNU toolchain is
Once all the optimizations have been turned off, you are ready to execute the application but, to record a meaningful trace of the crash, you have to enable the core dump generation. Enabling core dump generation is something that varies slightly across operating systems and typically requires you to set
ulimit -c unlimited (in bash) or
limit coredumpsize unlimited (in tcsh).
A core dump is a file that contains the status of the program at the exact moment of the crash. For this reason this file is fundamental to understand what has caused the crash. To obtain this file you have to wait until the application crashes and this can take some time. Thus you have to be patient, and use the application as usual. When the crash happens a core dump should be generated. Depending on the platform the core is saved into the same directory where you have started the application, or on MacOS X it can be found in the
/cores directory. Usually the core file is named
core, sometimes it has a PID (process ID) appended to it
Once you’ve the core you can analyze it. In order to analyze the core file you need to install the
gdb debugger. Done that you can debug the core as follow:
- Run the gdb:
gdb ./<application executable name> -c <path to the core>. At this point the debugger will open a shell where you can run commands.
(gdb) bt #0 0x00007fff96a6ee42 in __semwait_signal () #1 0x00007fff92d55dea in nanosleep () #2 0x00000001060d7601 in _ntopSleepMSWhileSameState (file=0x105fa8d82 "main.c", line=678, ulDelay=5000) at util.c:4103 #3 0x00000001060d77e9 in _ntopSleepWhileSameState (file=0x105fa8d82 "main.c", line=678, secs=5) at util.c:4131 #4 0x0000000105f9eb09 in main (argc=6, argv=0x7fff65b99ae0) at main.c:678
- Show threads information
(gdb) info threads 6 0x00007fff96a6faf2 in read () 5 0x00007fff96a6edf2 in select$DARWIN_EXTSN () 4 0x00007fff96a6ebca in __psynch_cvwait () 3 0x00007fff96a6ee42 in __semwait_signal () 2 0x00007fff96a6ee42 in __semwait_signal () * 1 0x00007fff96a6ee42 in __semwait_signal ()
If you can’t figure out what is causing the application to crash, you can send us the generated core dump along with the ntopng binary for inspection.
Debugging Packaged Applications
ntop also distributes binary applications that can easily be debugged even when they run as services. On operating systems with
systemd, generating cores for applications running as services require
systemd-coredump executable. On ubuntu the executable and its ancillary services can be installed with a single package installation:
apt-get install systemd-coredump
Other distributions may have a slightly different name for the package. Once the package is installed, core dumps will automatically be generated for all the running services that undergo a crash. It may also be necessary to add
LimitCORE=infinity to the service unit file’s
[Service] section to prevent core dumps from being limited in size.
systemd writes core dumps to the journal. In order to see generated core dumps, run:
coredumpctl list TIME PID UID GID SIG PRESENT EXE Wed 2018-05-30 15:02:14 BST 14595 0 0 11 * /bin/bash Wed 2018-05-30 15:10:20 BST 22290 0 0 11 * /usr/bin/myapp Wed 2018-05-30 15:50:38 BST 22739 0 0 11 * /usr/bin/myapp
To get the most recent
myapp coredump and export it to a file called
coredumpctl -o myapp.core dump /usr/bin/myapp
Similarly, core dumps can also be extracted using the pid. For example, the following command can be run to extract core dump related to process with pid 22739 into file
coredumpctl -o myapp.core dump 22739
Once the core file has been extracted from the journal, send it to us along with the application executable so we can fix the crash and distribute a fix.
In the remainder of this page we will show examples of ntopng core dump generation on ubuntu16, one of the most common distributions used with our products.
Debugging ntopng on Ubuntu16
Install the systemd coredump utility
sudo apt-get install systemd-coredump
Edit the ntopng service file and add
LimitCORE=infinity to the
$ cat /etc/systemd/system/ntopng.service [Unit] Description=ntopng high-speed web-based traffic monitoring and analysis tool After=network.target syslog.target redis.service pf_ring.service cluster.service mysql.service Requires= Wants=pf_ring.service cluster.service PartOf=pf_ring.service cluster.service [Service] LimitCORE=infinity Type=simple <file continues...>
Tell systemd to reload the service files
$ sudo systemctl daemon-reload
At this point you can restart ntopng and wait for the crash.
$ sudo systemctl restart ntopng
To make sure the core dump is generated, you can artifically send a “crash” signal to the ntopng process. To do this, get the PID of the process and use the kill command as follows
$ ps aux | grep ntopng nobody 26108 6.7 2.6 3034576 436848 ? Ssl 18:21 0:26 /usr/local/bin/ntopng /run/ntopng.conf simone 32642 0.0 0.0 11280 964 pts/1 S+ 18:28 0:00 grep --color=auto ntopng $ sudo kill -SEGV 26108
Once the process has crashed, the coredumpctl utility will show it in the list of core dumps
$ sudo coredumpctl list TIME PID UID GID SIG PRESENT EXE Mon 2018-09-10 18:19:11 CEST 21398 0 0 11 * /usr/bin/ntopng Mon 2018-09-10 18:19:26 CEST 22092 0 0 11 * /usr/bin/ntopng Mon 2018-09-10 18:21:47 CEST 23132 0 0 11 * /usr/bin/ntopng
Core dump extraction and analysis can be performed as described above.