Writing a non-blocking thread-safe buffer

[Eno Compton]

Hi All,

I'm trying to write a non-blocking thread-safe buffer for use in a high throughput system. In short, I want to create a buffer that decouples the speed of writes from that of reads. 

For a first attempt, using channels in the implementation seems best. Here is a link to the current implementation. I have a write channel and a buffered read channel. As an intermediary between the two channels, I spin off a goroutine on initialization of the buffer which constantly pulls values off the write channel and attempts to put them on to the read channel. If the read channel is full, I discard a value from the read channel before inserting the new value.

This implementation works. What I seek to do now is improve the throughput of the buffer. I understand doing so requires proper benchmarking and measuring. What I'm curious about though is the experience of others on this list. In systems which require high throughput, am I best suited sticking with channels? Would atomics be an appropriate design choice? What about mutexes?

Forgive me if this question seems naive. I'm new to Go and am still developing a sense for the language.

Thanks,

Eno

[Michael Jones]

I get about 3 million channel transfers per second on my computer. 

If each transfer is a single byte, that's 3 MB/sec. If transfers are pointers to buffers of a kilobyte, then that's 3 GB/sec...which is about as many bytes/sec as a CPU could possibly touch. If those buffers are 25kb then you're at the intel CPU memory rate limit of even moving the bytes those buffers contain. 

One approach is to have a pool of reused data buffers, fill them in the reader, pass them through a buffered channel to the writer, and send the processed data buffer pointer back on another buffered channel to the reader (or use a Go Pool for them). The size of the buffers is a tunable parameter: small means low latency, large means high throughput.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Michael T. Jones
michae...@gmail.com

[Albert Tedja]

I don't think using separate write/read channels do anything much here.  If you are concerned goroutines reading the channel somehow slowing down the writes, you already have a goroutine that's doing the transfer on the other end.

[John Souvestre]

A few ideas…

 

Instead of using two channels, use just one.  Let the writer(s) use a “select” to do the write, else default to doing a read (then loop to try the write again).  I believe that would accomplish what you are doing now but with less overhead.

 

I think that it’s great that you implemented this with channels first.  If you need more throughput then I’d think mutexes next, and atomics as a last resort.

 

Perhaps using an array as a circular buffer with read and write accesses synchronized by mutexes would be a good next step.

 

John

    John Souvestre - New Orleans LA

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Eno Compton]

Thanks for all your responses, folks. I'm curious about swapping to a single channel and will try that.