High CPU usage due to millisecond polling about monitor-core HOT 9 CLOSED

It looks as though it only hits the apr_sleep(wait) call if it's "deaf". If so, it would probably be better done as:

debug_msg("Starting TCP listener thread...");
if(!deaf)
  {
    for(;!done;)
      {
        now = apr_time_now();
        /* Pull in incoming data */
        poll_tcp_listen_channels(wait, now);
        apr_sleep( wait );
      }
  }
 else
  {
      /* Do nothing or something else here */
  }

from monitor-core.

poll_tcp_listen_channels(wait, now) actually times out after wait microseconds, does it not? Adding an apr_sleep(wait) call would just double the interval, which would cut the CPU usage in half. I'd have expected the timeout to just be set to something more sensible, like 1 second.

from monitor-core.

I think this was meant to be a 1 second timeout but I might have misread the API docs for apr_interval_time_t as @psobot suggested. Would 100ms be a more sensible value or should we go for the full 1 second?

Alternatively, the TCP listener thread could not be started at all if the gmond is configured to be deaf.

  /* Create TCP listener thread */
  if(!deaf)
    {
      apr_thread_t *thread;
      if (apr_thread_create(&thread, NULL, tcp_listener, NULL, global_context) != APR_SUCCESS)
        {
          err_msg("Failed to create TCP listener thread. Exiting.\n");
          exit(1);
        }
    }

from monitor-core.

Forgive my ignorance, but why is this function polling in the first place? It's waiting on a TCP connection and re-checking every n µs to see if one has arrived... but if it's in a separate thread, couldn't that thread just block until connection arrives? (The apr docs suggest that a negative timeout will block indefinitely, which would work in this case.)

from monitor-core.

If the TCP thread blocks until a connection arrives then the TCP thread will prevent the gmond from reloading its config due to a SIGHUP or exiting due to any other signal interrupt. I think we should increase the timeout to 500ms or so and not even start the TCP thread if the gmond is in deaf mode.

from monitor-core.

I agree on not starting the TCP thread if gmond is in deaf mode. 500ms may be too large. I'd go for say 100ms. @psobot can you test with value of 100ms and see if CPU utilization is greatly diminished. Thank you.

from monitor-core.

@satterly - Good point.

@vvuksan - Quick testing results on my laptop, as reported by ps:

Timeout: 1000    (1ms)    => 1.2% CPU
Timeout: 100000  (100ms)  => 0.1% CPU
Timeout: 500000  (500ms)  => 0.0% CPU

from monitor-core.

Is there consensus here?

from monitor-core.

I'd go for 100ms.

from monitor-core.