How to pass a value to multiple readers of a shared channel

[st ov]

A value sent through a channel can be read by only one reader, so once its read its no longer available to other readers is that right?

In what ways can I pass/communicate/distribute a value through a channel and have it shared across an unknown number of readers?

[dja...@gmail.com]

This post might help:

https://rogpeppe.wordpress.com/2009/12/01/concurrent-idioms-1-broadcasting-values-in-go-with-linked-channels/

Djadala

[dja...@gmail.com]

And here is some working code:
https://play.golang.org/p/1Qt-LqKiph

Djadala

On Sunday, October 15, 2017 at 11:36:36 PM UTC+3, st ov wrote:

[wolf...@gmail.com]

My feeling is that a variable paired with a sync.Cond might be the right choice if you are broadcasting a single value change.

I can't think of an readily available lib / language feature to support broadcasting multiple value change though.

[rene.rei...@zalando.de]

https://golang.org/pkg/io/#TeeReader 

should help you out

best,

Rene

[roger peppe]

The answer depends on whether you have just one value or many. For a single value (sometimes known as a "future"), I tend to pair a chan of struct{} with a value to be set. To make the value available, set it, then close the channel. Readers wait on the channel, then read the value.

For a succession of values, there are lots of possible choices, depending on what semantics you require. If you can discard intermediate values when the producer is faster than the consumer, then this package might fit the bill. https://godoc.org/github.com/juju/utils/voyeur

Watch out though: broadcast values are easy to get wrong as ownership is not clear. This is one case where better support for immutable data structures in Go might be useful.

Hope this helps, 

  rog.

On 15 Oct 2017 21:36, "st ov" <so.q...@gmail.com> wrote:

A value sent through a channel can be read by only one reader, so once its read its no longer available to other readers is that right?

In what ways can I pass/communicate/distribute a value through a channel and have it shared across an unknown number of readers?

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[st ov]

that's an great article thank you!

[Jesper Louis Andersen]

If you have serious performance needs, then something like the "Disruptor" pattern is useful. There are ports of that from Java to Go out there, but I don't know how stable or useful they are.

The Disruptor pattern is to keep a circular ring-buffer of elements. The writer and each reader keeps track of where they are in the ring by means of atomic counters. By keeping track of where the slowest reader is, you can track if everyone has read the value in the ring. The atomic counters acts like barriers which make sure when you can trust a slot in the ring to be correctly updated.

The solution is very efficient if you have a single-writer, many reader broadcast where speed is of essence. But I'd definitely not go down this path unless you hit a wall and need to squeeze out more performance. These systems tend to need an ample amount of testing before they can be deemed stable and data-races tend to lurk around every corner you look.

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Juliusz Chroboczek]

> The answer depends on whether you have just one value or many. For a single
> value (sometimes known as a "future"), I tend to pair a chan of struct{}
> with a value to be set. To make the value available, set it, then close the
> channel. Readers wait on the channel, then read the value.

That's an interesting idea. Surprisingly enough, it turns out to be
only slightly slower than the obvious implementation with a mutex, and
(even more suprisignly) actually faster when using hardware threading.
This is on a 2-core hyperthreaded SandyBridge (4 hardware threads):

BenchmarkMutexFuture 100000000 22.2 ns/op
BenchmarkMutexFuture-2 50000000 39.9 ns/op
BenchmarkMutexFuture-4 20000000 86.0 ns/op
BenchmarkAtomicFuture 200000000 7.01 ns/op
BenchmarkAtomicFuture-2 500000000 3.69 ns/op
BenchmarkAtomicFuture-4 1000000000 2.69 ns/op
BenchmarkChannelFuture 50000000 29.7 ns/op
BenchmarkChannelFuture-2 50000000 54.6 ns/op
BenchmarkChannelFuture-4 20000000 64.4 ns/op

Code here: https://play.golang.org/p/Wm71o50HJ8

-- Juliusz

[Sokolov Yura]

Following is a dangerous error-prone technique. If you use it in you program, most likely you have a hidden bug, that will appear in high concurrency situation.

```
func wait(c chan T) T {
v := <-c
c <- v;
return v;
}
```

I had proposal for future primitive: https://github.com/golang/go/issues/17466 , but looks like community doesn't agree with my intention.

[Axel Wagner]

What you are doing is what sync.Once was invented for: https://play.golang.org/p/erZqwFYgIR

-- Juliusz

[roger peppe]

On 18 October 2017 at 06:05, Sokolov Yura <funny....@gmail.com> wrote:
> Following is a dangerous error-prone technique. If you use it in you program, most likely you have a hidden bug, that will appear in high concurrency situation.
>
> ```
> func wait(c chan T) T {
> v := <-c
> c <- v;
> return v;
> }
> ```

Assuming c has a capacity of 1, what's dangerous about the above?

[Юрий Соколов]

Between those two lines (fetching value from channel and sending it back to channel) channel is empty.

This leads to two possible errors:

- less dangerous is if some checks channel in non-blocking mode (ie select with default). Then it sees empty channel and thinks value is not set yet.

- more dangerous is that other goroutine may send other value to the same channel at this moment. Then a) you have two different values that are randomly sent to different waiters, b) some of waiters will block trying to send value to already full channel (cause previous waiter has already sent other value back to channel).

18 окт. 2017 г. 10:31 AM пользователь "roger peppe" <rogp...@gmail.com> написал:

[Juliusz Chroboczek]

> What you are doing is what sync.Once was invented for:

Uh-huh. Once is similar but slightly more primitive than a future,
since the thunk is passed explicitly to the Do() method instead of being
stored in the future itself.

> https://play.golang.org/p/erZqwFYgIR

I think there's a bug in your code: you retain f.thunk, hence any values
the thunk closes over won't be garbage collected as long as the future
itself is retained. (Which might be an argument in favour of having
futures in a library, they're not completely trivial to implement correctly.)

Your idea turns out to be very slightly slower than my AtomicFuture,
probably due to the unconditional call to Once.Do() in the fast path.
But the code is much cleaner -- the ugly double checking is encapsulated
within the Once.

BenchmarkMutexFuture 50000000 22.1 ns/op
BenchmarkMutexFuture-2 30000000 40.1 ns/op
BenchmarkMutexFuture-4 20000000 86.2 ns/op
BenchmarkAtomicFuture 200000000 6.92 ns/op
BenchmarkAtomicFuture-2 500000000 3.68 ns/op
BenchmarkAtomicFuture-4 1000000000 2.82 ns/op
BenchmarkChannelFuture 50000000 29.5 ns/op
BenchmarkChannelFuture-2 30000000 54.5 ns/op
BenchmarkChannelFuture-4 20000000 64.6 ns/op
BenchmarkOnceFuture 200000000 8.54 ns/op
BenchmarkOnceFuture-2 300000000 4.52 ns/op
BenchmarkOnceFuture-4 300000000 4.04 ns/op

-- Juliusz