Need help to close/shutdown http server when underlying socket fd explicitly bind used

[Bing]

Hi,

I'm trying to implement a http server and I need to bind the underlying socket to a specific VRF instance so that the server only listens on that VRF instance. When some specific event happens, the program needs to shutdown the http server. I'm using syscall.SetsockoptString() and option "SO_BINDTODEVICE".

This is what I observe:

1. if there is no SetsockoptString() to bind the underlying socket fd call, the http server close call would succeed.

2. if there is a SetsockoptString() to bind the underlying socket fd call, then the http server close won't complete. In fact, it stuck in close() and won't return.

Seems in the path of close(), decref() does call fd.destroy(), and later on the program stuck in 

func poll_runtime_Semacquire(addr *uint32) {

     semacquire1(addr, false, semaBlockProfile, 0, waitReasonSemacquire) >>>>>>>>>>>>>> stuck here}

comparing #1 and #2, the state number for mu is different, so that fd.fdmu.decref() returns false and fd.destroy not getting called.

if fd.fdmu.decref() {return fd.destroy()}

I have my program attached to this email.

The same goes for http server Shutdown(context) call. My question, does golang net/http packet support this scenario, shutdown() or close a http server where the server has some explicit underlying socket fd binding? If so, what code change should I make?

Thank you very much.

Bing

[Reto]

On Thu, Feb 15, 2024 at 11:30:53AM -0800, Bing wrote:
> I'm trying to implement a http server and I need to bind the underlying
> socket to a specific VRF instance so that the server only listens on that
> VRF instance. When some specific event happens, the program needs to
> shutdown the http server. I'm using syscall.SetsockoptString() and option
> "SO_BINDTODEVICE".

Pretty sure what you are doing doesn't make sense... aren't you hooking into
the wrong timing?

You need to set the socket option prior to binding, not after.
There's a more convenient interface with a syscall.RawConn than what you are
using.
The attached mod seems to work, maybe try that?

Cheers,
Reto

[binbinschen00]

Thanks Reto, that works!

If I want to bind the socket fd to another interface for the same http server (meaning same address:port), do I need to close/shutdown the existing server first and start a new one? Is there a way to just modify the socket bind without close/shutdown the socket?

Thanks,

Bing

[Reto]

On Sat, Feb 17, 2024 at 10:23:40AM -0800, binbinschen00 wrote:
> If I want to bind the socket fd to another interface for the same http
> server (meaning same address:port), do I need to close/shutdown the
> existing server first and start a new one? Is there a way to just modify
> the socket bind without close/shutdown the socket?

Turns out you can apparently do it post bind, just played around with it.
Question is, should you? It smells like an XY problem to me.
What are you actually trying to do?

But sure, just use the same pattern with the syscall con in whatever logic
you use to switch.
`listener.(*net.TCPListener).SyscallConn()` will give you the interface
needed to do so.