Multi-thread socket listening (aka SO_REUSEPORT(_LB)

[Ondřej Surý]

Hey all,

so I have this a bit out-of-ordinary question - as far as I understand it, on systems where SO_REUSEPORT_LB (or SO_REUSEPORT on Linux) is not available, is there any method how to let multiple UDP and/or TCP handles to listen and compete over a single socket?

So, I tried few obvious things:

- Passing same fd to uv_udp_open() doesn't work because uv__fd_exists check

- Calling dup() on the socket and then doing uv_udp_open(..., fd) doesn't work for me on Linux (if I fake the non-existence of the socket option) and I get ENOMEM (what?); maybe it's just a problem doing this on Linux and it will work on freebsd11 / openbsd / netbsd

Anybody has some other idea I missed?

Ondrej

[Ondřej Surý]

I'll answer myself :-)

For UDP the second uv_udp_bind on dup()ed uv_udp_t->fd will always fail (because the socket is already bound), so it's just a matter of setting the correct internal flags on the second and further handle and then starting the uv_udp_start_read() works.

For TCP it's a similar situation with bind, but the epoll will wake up only one thread (probably the first numerically), so there must exist a mechanism to wake up the other threads to call uv_accept() on the shared listening socket.

I have yet to write complete proof of concept, but this is the basics.

Ondrej

[Ondřej Surý]

Again replying to myself...

The TCP is also ok with just sharing the socket and nothing else has to be done for the accept. If anybody is interested in looking at the implementation, here's the relevant commit in BIND 9: https://gitlab.isc.org/isc-projects/bind9/-/merge_requests/4459/diffs?commit_id=1d066e4bc5dfcd72a8d82c5318247efb9ed86aae

I think it might be useful to somehow indicate to uv_udp_open and uv_tcp_open that the socket is already bound to refrain from touching the internal flags field.

Ondrej