Active vs Passive Polling in Packet Processing

Posted · Add Comment

From time to time, PF_RING users ask us whether they should use passive polling techniques (i.e. call pfring_poll()) or use active polling that basically means to implement an active loop until the next packet to process becomes available. All those who have read a programming book or attended university classes, might answer that polling is the answer. This also for various other reasons including energy saving in CPUs.  Unfortunately in practice the story is a bit different.

If you want to avoid wasting CPU cycles, when you have nothing to do (i.e. no packet is waiting to be processed), you should either call pfring_poll(), poll/select and ask the system to wake up the program when there is a packet to process. If you create an active polling loop you might want to do something like

while(<no packet available>) { usleep(1); }

that reduces the CPU loop taking (in theory) a short (one microsecond) nap. This is a good practice if usleep() (or nanosleep() if you prefer to use it instead of usleep()) last the time you specify. Unfortunately this is not always the case. These functions make a system call to implement the sleep. The cost of a simple system call is pretty low (e.g. you can test it using this test program) and is usually less than 100 nsec/call that is much less than 1 usec sleep we want to have. Let’s now measure the accuracy of usleep() and nanosleep() using this simple program (sleep.c)

#include <string.h>
#include <sys/time.h>
#include <stdio.h>

double delta_time_usec(struct timeval *now, struct timeval *before) {
  time_t delta_seconds;
  time_t delta_microseconds;

  delta_seconds      = now->tv_sec  - before->tv_sec;

  if(now->tv_usec > before->tv_usec)
     delta_microseconds = now->tv_usec - before->tv_usec;
  else
    delta_microseconds = now->tv_usec - before->tv_usec + 1000000;  /* 1e6 */

  return((double)(delta_seconds * 1000000) + (double)delta_microseconds);
}

int main(int argc, char* argv[]) {
  int i, n = argc > 1 ? atoi(argv[1]) : 100000;
  static struct timeval start, end;
  struct timespec req, rem;
  int how_many = 1;

  gettimeofday(&start, NULL);
  for (i = 0; i < n; i++)
    usleep(how_many);
  gettimeofday(&end, NULL);

  printf("usleep(1) took %f usecs\n", delta_time_usec(&end, &start) / n);

  gettimeofday(&start, NULL);
  for (i = 0; i < n; i++) {
    req.tv_sec = 0, req.tv_nsec = how_many;
    nanosleep(&req, &rem);
  }
  gettimeofday(&end, NULL);

  printf("nanosleep(1) took %f usecs\n", delta_time_usec(&end, &start) / n);
}

The results change slightly from machine to machine but they are around 60 usec.

# ./sleep
usleep(1) took 56.248760 usecs
nanosleep(1) took 65.165280 usecs

This means that both usleep and nanosleep() when are used to sleep for 1 microsecond in practice they sleep for about 60 microseconds. This result is not surprising and it has been also observed by others [1] [2].

What does this mean in practice? Knowing that at 10G line rate you receive a packet every 67 nsec, sleeping for 60 usec means that you will receive about 895 packets during this time and that you must have good buffers to handle this situation. It also means that you cannot do any polling but use pure active polling (i.e. do not call any usleep/nanosleep in your active packet poll) in critical situations such as when you are time-merging packets from two network adapters.

Conclusion. Active polling is not elegant but when processing packets at high rate it might be compulsory in order to achieve great/accurate results. PF_RING applications support both passive and active polling. For instance in pfcount you can use the -a flag to force active packet polling and thus (in some cases) increase the packet capture performance at a cost of loading your CPU core at 100% (i.e. when you use active polling make sure you also bind your application to a core; in pfcount you can do it with -g).