Do you have to read/write until EGAIN in RawConn.Read/Write?

[Daniel Eloff]

Digging a little, it seems Go net poller uses edge triggered epoll on Linux. One of the harder considerations about using edge-triggered epoll is you have to read()/write() until EGAIN, otherwise you will not receive another read/write event for the socket when you call epoll_wait.

My question is, how does this work if you use the new(ish) RawConn interface? It's not stated that there are any requirements to read/write until EGAIN, just that it's expected that the programmer calls read/write (presumably through syscall.Read/Write.) What happens if you return e.g. done=true without calling read/write at all? This may be useful if you just want to wait until data is available for reading without tying up a buffer in a read call, e.g. if you have a long-lived but idle connection like a websocket, or you're waiting for follow-up requests on an http keep-alive connection, or http2 connection as in grpc.

I went through the poller code, but did not manage to figure out the answer.

[Ian Lance Taylor]

As far as I can see there is no problem if the function returns true
without doing a read. In the normal file I/O case, a Read will never
wait for the poller unless the system call returns EAGAIN. That is, a
call to the Read method does not start by waiting for the poller to
report that the descriptor is ready. It always calls the read system
call, and only if read returns EAGAIN does it wait for the poller.

However, I think there might be a potential problem if the RawConn
Read method is called with a function that returns false after making
a read call that does not return EAGAIN. In that case the net package
will wait for the poller, but there might be nothing that triggers it.

Come to think of it, you said return done=true but you probably meant
return done=false. You're right: I don't think that will work. I
think that as the docs say "f is expected to try to read from the file
descriptor."

Ian

[Daniel Eloff]

On Saturday, December 21, 2019 at 11:33:15 AM UTC-8, Ian Lance Taylor wrote:

On Sat, Dec 21, 2019 at 7:09 AM Daniel Eloff <dan....@gmail.com> wrote:
>
> Digging a little, it seems Go net poller uses edge triggered epoll on Linux. One of the harder considerations about using edge-triggered epoll is you have to read()/write() until EGAIN, otherwise you will not receive another read/write event for the socket when you call epoll_wait.
>
> My question is, how does this work if you use the new(ish) RawConn interface? It's not stated that there are any requirements to read/write until EGAIN, just that it's expected that the programmer calls read/write (presumably through syscall.Read/Write.) What happens if you return e.g. done=true without calling read/write at all? This may be useful if you just want to wait until data is available for reading without tying up a buffer in a read call, e.g. if you have a long-lived but idle connection like a websocket, or you're waiting for follow-up requests on an http keep-alive connection, or http2 connection as in grpc.
>
> I went through the poller code, but did not manage to figure out the answer.

As far as I can see there is no problem if the function returns true
without doing a read.  In the normal file I/O case, a Read will never
wait for the poller unless the system call returns EAGAIN.  That is, a
call to the Read method does not start by waiting for the poller to
report that the descriptor is ready.  It always calls the read system
call, and only if read returns EAGAIN does it wait for the poller.

I think that's the missing piece for me to understand what the poller is doing here.

 

However, I think there might be a potential problem if the RawConn
Read method is called with a function that returns false after making
a read call that does not return EAGAIN.  In that case the net package
will wait for the poller, but there might be nothing that triggers it.

Yes, I think that should be added to the docs.

 

Come to think of it, you said return done=true but you probably meant
return done=false.  You're right: I don't think that will work.  I
think that as the docs say "f is expected to try to read from the file
descriptor."

Yeah you're right, I meant return done=false without calling read. This will cause the goroutine to be suspended until more data comes in for the fd. But, if there was data waiting already then I agree this code is broken in my new understanding, as the edge triggered event will not fire for it. So really one can only ever return done=false if you read until EAGAIN, which should be clarified in the docs. I could switch the fd to level-triggered, which is not portable and could interact badly with the netpoller, so I think that's a bad idea. Better would be call read with a small stack buffer, and if it succeeds, allocate a real buffer and copy the data to there and read until EAGAIN or sufficient data has been read. That doesn't cost anything more (either way it's still one syscall.)