What is the best way to broadcast a single message to 20K socket descriptors concurrently in golang?

[kant kodali]

Hi All,

I am wondering how to broadcast a stream of messages I get (one by one...each message is of size 5KB) to 20K websocket clients to achieve the lowest latency and max throughput possible on a 10Gbps network? I tried creating a go routine for each socket and calling socket.write but this approach had a median latency of 75ms. Goal is to be under 5ms so I wonder what is the best way to dispatch to 20K concurrent clients? my machine got about 4 cores so I also tried splitting 20K socket descriptors into groups of 16 such that each go routine takes care of 16 sockets instead of 1.  This approach also brought down the median latency to 50ms but not 5ms. Any suggestions will be appreciated!

Thanks  

[Robert Engels]

A kernel call is about 6 usec. So 20k calls is 120ms. Divide by the number of cores. That is you absolute best case. And even that you won’t achieve because they need to be sent serially by the network card. 

Your best solution is to use multicast.  Or you need hardware support. 

On Jul 18, 2022, at 7:11 AM, kant kodali <kant...@gmail.com> wrote:



Hi All,

I am wondering how to broadcast a stream of messages I get (one by one...each message is of size 5KB) to 20K websocket clients to achieve the lowest latency and max throughput possible on a 10Gbps network? I tried creating a go routine for each socket and calling socket.write but this approach had a median latency of 75ms. Goal is to be under 5ms so I wonder what is the best way to dispatch to 20K concurrent clients? my machine got about 4 cores so I also tried splitting 20K socket descriptors into groups of 16 such that each go routine takes care of 16 sockets instead of 1.  This approach also brought down the median latency to 50ms but not 5ms. Any suggestions will be appreciated!

Thanks  

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CA%2BiiNx80YON9zwUMgsLBpTU6kv0FPnHQgtmox3ULjCOLrgMd0g%40mail.gmail.com.

[kant kodali]

Hi Robert,

Thanks a lot for that response. I see, so it's all syscall overhead. Based on your equation I can probably launch a 16 or 32 core machine for now.

when you say use multicast. How to do that in Go? Is there any library you are referring to? 

Any thoughts on io_uring or mtcp?

[Robert Engels]

You can look at github.com/robaho/go-trader - it has a multicast market data module. 

Multicast is udp so you need your own error control. 

You might want to look at the Go implementation of lrmp in github.com/robaho/lrmp if that’s the case. 

On Jul 18, 2022, at 10:42 AM, kant kodali <kant...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CA%2BiiNx9hMCzHHf_yXRGb0BdSeAGEkQ-%3DnY-Luz9VP0S%3DVw1juw%40mail.gmail.com.

[Robert Engels]

Also, simply adding more cores may not solve your problem due to contention and hardware limitations on how fast the network card can submit packets to the network. 

Also, you will probably saturate them network causes lots of errors.

Multicast is really the only viable solution for those timing requirements. 

On Jul 18, 2022, at 11:27 AM, Robert Engels <ren...@ix.netcom.com> wrote:



[Florian Weimer]

* Robert Engels:

> A kernel call is about 6 usec. So 20k calls is 120ms. Divide by the
> number of cores. That is you absolute best case. And even that you
> won’t achieve because they need to be sent serially by the network
> card.

Linux supports sendmmsg, which can be used to cut down context switches.
The speedup may not be sufficiently large for this application, though.

Thanks,
Florian

[Robert Engels]

sendmmsg won’t help. That is for sending multiple messages on a single socket. He needs to the send the same message on multiple sockets.

With multicast you only need to send a single message on a single socket.

> On Jul 19, 2022, at 4:23 AM, Florian Weimer <fwe...@redhat.com> wrote:
>
> * Robert Engels:

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/874jzda234.fsf%40oldenburg.str.redhat.com.

[Robert Engels]

Also, rather than lrmp a kernel based reliable protocol like PGM is easier if you have it available 

See https://events.static.linuxfound.org/slides/2010/linuxcon2010_lameter.pdf

On Jul 19, 2022, at 6:14 AM, Robert Engels <ren...@ix.netcom.com> wrote:

sendmmsg won’t help. That is for sending multiple messages on a single socket. He needs to the send the same message on multiple sockets.

[Robert Engels]

Finally, looks like ZeroMQ implements PGM (or similar) so that is an option. 

On Jul 19, 2022, at 6:36 AM, Robert Engels <ren...@ix.netcom.com> wrote:



[Florian Weimer]

* Robert Engels:

> sendmmsg won’t help. That is for sending multiple messages on a single
> socket. He needs to the send the same message on multiple sockets.
>
> With multicast you only need to send a single message on a single socket.

Oh, I didn't realize this was a connected socket. Yes, sendmmsg will
not work in this case. It can however handle the case of an unconnected
socket with many target addresses.

People have tried using userspace protocol implementations to get around
the context switch overhead. io_uring might also be an option if you
can control the kernel.

Thanks,
Florian

[kant kodali]

To be clear, I am trying to have a server send a message to 20K clients.and it's all one way that is from server to 20K clients(one to many). There is no need for a client sending a message to the server or one client receiving a message from another client.

[Robert Engels]

The lrmp will do that. What you just said doesn’t match the comments you added in the lrmp issue - you specifically asked about having the client send a message back to the server. 

On Jul 19, 2022, at 11:52 AM, kant kodali <kant...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CA%2BiiNx-%3D%2BnizZO0oi9NgETrv8VGd9x9ve7TtcHPsFbc6f74_vg%40mail.gmail.com.

[kant kodali]

Hi Robert,

Sorry if my message did not come across correctly but I believe I said from the server I want to send a message from the server to 20K clients(pasted my text from the issue) and it's in that direction only. I just tried the lrmp example. It looks like anyone( a peer) can initiate the message and it will be multicasted to the rest of them but this is not what I am looking for. I want only servers to multicast to 20K clients (don't want to receive any message from clients to server or from one client to another). If this is possible in lrmp can you please share a hello world example?

"so in my case it has to be all one way meaning from the server I want to send notifications to bunch of clients/receivers/peers in a multicast fashion"

[Robert Engels]

You simply write the code so that the clients don’t try to send??? Or you comment out the public sending. If you read the lrmp spec a peer can send in order to efficiently broadcast retries across muktiple networks - it is handled automatically. 

If you are looking for something where clients can never send - even outside of your code control - you would need to enable security/certificates. 

On Jul 19, 2022, at 12:27 PM, kant kodali <kant...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-dev/CA%2BiiNx-dWDxqhkgkFQ%3Deh6wfQs_MpjV7E3ZkLUht%3Db9Dj7eRcg%40mail.gmail.com.