re-using dns resolvers about lua-resty-dns HOT 7 CLOSED

@Tieske What real problem are you trying to solve here? Optimizations by saving a Lua table's creation and collection? Your use case limits the concurrency level of DNS resolving to 1 per nginx worker. Are you aware of that? If that's indeed what you're trying to do, then you can just create a background light thead (via ngx.timer.at in init_worker_by_lua*, for example) that is delegated by all the requests to do the DNS resolving work (serially).

from lua-resty-dns.

@Tieske I suggest you only do optimizations based on real profiling results :)

from lua-resty-dns.

@Tieske If you are worried about the extra socket and close syscalls upon udp cosocket creation and collection, then we should implement the socket pool instead. Will you be interested in contributing a setkeepalive method just like that of the tcp cosockets? Thanks!

from lua-resty-dns.

Your use case limits the concurrency level of DNS resolving to 1 per nginx worker. Are you aware of that?

I'm sorry, but I don't see what you mean. The intent is to create a module level table (in the init_worker phase), holding multiple resolver objects. Upon resolving, grab one from the table (remove it) resolve the query, and put it back.
I don't see how this would limit concurrency? I think it would actually improve on it.
And instead of creating the resolver object on each request (not just the tables, but the input validation, creating and setting up sockets, etc) it can be reused.

What am I missing here?

you can just create a background light thread (via ngx.timer.at in init_worker_by_lua*, for example) that is delegated by all the requests to do the DNS resolving work (serially)

How would such delegation work? There is no way to yield the light thread (the incoming request) to wait for the resolving results (of the timer thread) other than using sleep right? or am I missing something obvious here as well?

from lua-resty-dns.

The intent is to create a module level table (in the init_worker phase), holding multiple resolver objects. Upon resolving, grab one from the table (remove it) resolve the query, and put it back.

Seems like you are trying to emulate a socket pool on the Lua land yourself. No, cosockets' lifetime and scope are bound to the Lua handler context that creates them. This is an important design decision made for OpenResty to ensure

1. Socket leaks can never happen.
2. Bad simultaneous uses of the same cosocket object can never happen (unless by different light threads created in the same Lua handler context).

As I've already suggested above:

If you are worried about the extra socket and close syscalls upon udp cosocket creation and collection, then we should implement the socket pool instead. Will you be interested in contributing a setkeepalive method just like that of the tcp cosockets?

This is the right way to go.

How would such delegation work? There is no way to yield the light thread (the incoming request) to wait for the resolving results (of the timer thread) other than using sleep right? or am I missing something obvious here as well?

Seems like you haven't looked at the ngx.semaphore module?

https://github.com/openresty/lua-resty-core/blob/master/lib/ngx/semaphore.md

It's way more efficient than manual polling via ngx.sleep.

from lua-resty-dns.

Seems like you are trying to emulate a socket pool on the Lua land yourself.

yes (due to absence of a pool)

cosockets' lifetime and scope are bound to the Lua handler context that creates them

I understand the decision, but think it's a pity, it's easy to work around the problems (if you know what you're doing)

If you are worried about the extra socket and close syscalls upon udp cosocket creation and collection, then we should implement the socket pool instead. Will you be interested in contributing a setkeepalive method just like that of the tcp cosockets?

This is the right way to go.

yes (but unfortunately not in my skill set)

Seems like you haven't looked at the ngx.semaphore module?

thx for pointing that out

from lua-resty-dns.

@Tieske

I understand the decision, but think it's a pity, it's easy to work around the problems (if you know what you're doing)

Not really. Such things are notoriously hard to debug so we decide to provide the one true way of doing things here.

from lua-resty-dns.