Proper 'go' way of listening to multiple simultaneous sockets

[Viktor Kojouharov]

Hello,

I'm curious as to what the proper way of listening multiple simultaneous sockets is?

Please consider the two implementations I currently have: http://play.golang.org/p/LOd7q3aawd

Receive1 is a rather simple implementation, spawning a goroutine per connection, and simply blocking on each one via the ReadFromUDP method of the connection.

Receive2 is a more C-style approach, using the real select to block until either one of the sockets actually receives something, and then actually trying to read.

Now, Receive2 is _supposedly_ better, since we rely on the kernel to notify the select if one of the file descriptors is ready for reading. That would, under ideal conditions (and maybe less goroutines), allow the process itself to fall asleep until data starts pouring into the socket.

Receive1 is relying on the I/O read blocking until something comes along, and is doing so for each connection. Or at least that's what my understanding is, I'm not sure whether internally, the connection is still using select. Though even if it was, I don't thing the go scheduler is smart enough to put the whole process to sleep if two separate goroutines are waiting for I/O. That being said, Receive1 looks so much better than its counterpart.

If there is also a better way than either of these, please share it with me.

[Matt Harden]

Recieve1 is better. Go will use asynchronous I/O (equivalent to select) under the covers for you. The go scheduler is smart enough to "put the whole process to sleep if [all] goroutines are waiting for I/O". Don't worry about it. Just use idiomatic Go.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Uli Kunitz]

Receive1 is certainly the Go way. I wonder however why you need to read from two UDP ports. UDP is connectionless, so you can support multiple clients with one open UDP port.

That being said you should know that any goroutine blocking in a system call consumes one kernel thread. This will not be a problem until you need to support thousands of connections. But at this scale the file descriptor bitmaps used by select become a performance bottleneck as well. In this situation you might want to look at epoll on Linux. On other systems poll might be an alternative. If you are in this territory I strongly recommend to have a look into Michael Kerrisk's excellent reference "The LINUX Programming Interface".

[Matt Harden]

It's true that a goroutine blocking in a syscall consumes a kernel thread. However, Receive1 will *not* use any kernel threads while waiting in conn.ReadFromUDP, because under the covers, the Go runtime uses nonblocking I/O for all network activity. It's much better just to rely on the runtime implementation of network I/O rather than trying to roll your own. If you don't believe me, try doing syscall traces or profiling to prove it out.

On Wed, Jul 8, 2015 at 2:07 PM Uli Kunitz <uli.k...@gmail.com> wrote:

Receive1 is certainly the Go way. I wonder however why you need to read from two UDP ports. UDP is connectionless, so you can support multiple clients with one open UDP port.

That being said you should know that any goroutine blocking in a system call consumes one kernel thread. This will not be a problem until you need to support thousands of connections. But at this scale the file descriptor bitmaps used by select become a performance bottleneck as well. In this situation you might want to look at epoll on Linux. On other systems poll might be an alternative. If you are in this territory I strongly recommend to have a look into Michael Kerrisk's excellent reference "The LINUX Programming Interface".

[Skip Tavakkolian]

receive2 approach is not portable (due to syscall), and is more complex. also, unless profiling can prove it, efficiency of the approach is a speculation.

-Skip

--

[Uli Kunitz]

Matt, thank you I learned something. During network access the Goroutine is not blocked in a syscall and Go is already using epoll internally. So unless you know what you are doing the Goroutine approach will work best.