'Lookup timed out' about dnspython HOT 8 CLOSED

I don't know of anything in dnspython specifically that would cause this. I think you have to look into which bit of code is using dnspython directly, and see what timeouts its using. Perhaps you can adjust them.

/Bob

from dnspython.

We're having the same issue, it seems that eventlet is setting a 10.0 seconds DNS Timeout. Changing the timeout setting doesn't seem to fix this behavior.

from dnspython.

Yes, it may be the problem with eventlet. Specifically, it may be that due to cooperative scheduling, CPU intensive parsing does not yield to IO tasks.

The other problem I faced with a similar environment/task is that your DNS resolver considers you are making too many requests and stops responding either due to network buffer overflows or because of rate limits. If that's the case, a sleep(5) before each HTTP query would help.

Please, measure responsiveness of your code. Does it use a lot of CPU or does it spend all time in IO waits?

For example, spawn a greenthread with roughly this code:

def heartbeat():
  while True:
    t1 = time.time()
    eventlet.sleep(0.1)
    td = time.time() - t1
    print(td)

and graph it to show median and spikes. If graphing is too hard, just attach raw values.

from dnspython.

The same question was asked here http://stackoverflow.com/questions/15706173/dnspython-and-lookup-timed-out
linking for the history.

from dnspython.

I'm having similar issues with Gevent and dnspython.
I'll post if I find out more.

from dnspython.

I found out that dnspython is using non-blocking IO with busy waiting. Honestly, I do not see a reason for that but it doesn't work well with gevent and probably other frameworks. I didn't find any explanation why this approach has been used. Could you please clarify?

from dnspython.

@henadzit could you link the relevant code?

If waiting is implemented properly, there could be no problem with gevent/eventlet even with monkey patching. Explicitly using non-blocking IO effectively bypasses gevent/eventlet magic, it works like regular event driven programming.

from dnspython.

On May 28, 2014, at 3:31, henadzit [email protected] wrote:

I found out that dnspython is using non-blocking IO with busy waiting. Honestly, I do not see a reason for that but it doesn't work well with gevent and probably other frameworks. I didn't find any explanation why this approach has been used. Could you please clarify?

It does not busy wait; it uses poll() or select(). It works this way because otherwise it wouldn't have fine-grained control over I/O timeouts.

You might be able to hook it into other frameworks by making a polling backend.

/Bob

from dnspython.