A barrier primitive for signalling shutdown

[Peter Waller]

Apologies if this has been discussed before, I had a brief search but didn't find anything.

The code is up on github: https://github.com/pwaller/barrier

I find myself reimplementing this type of barrier fairly frequently at the moment, so I've made it a package.

The idea is that one can have `defer b.Fall()` at the beginning of many go routines and `<-b.Barrier()` at the end. If one returns, the barrier falls and they all continue. It can be used to implement "signal shutdown if any of these points is reached", among other things.

`b.Barrier()` returns a channel so that it can be used in a select{}.

I've implemented it with an `init` rather than a `NewBarrier() *Barrier` because I want it to look more like the sync primitives `Once` and `WaitGroup`, where the zero of the type is a ready-to-use value.

It took me a while to think of using a `sync.Once` to protect against a channel being closed repeatedly from different places, but once I did it seems very obvious.

Question barrage:

Is the code correct? Is the fact I'm using `sync.Once` like this to protect channel close a code smell? Is there a better way to achieve the use case of "If one exits they all exit"?

Does such a barrier primitive already exist somewhere? Does it have a name in Go or otherwise? I'm surprised to need to invent it for myself.

Would such a primitive be a candidate for living in the sync package?

Thanks,

- Peter

Approximate code follows.

type Barrier struct {
    channel chan struct{}
    o       sync.Once
    init    sync.Once
}

func (b *Barrier) Init() {
    b.init.Do(func() {
        b.channel = make(chan struct{})
    })
}

func (b *Barrier) Fall() {
    b.Init()
    b.o.Do(func() { close(b.channel) })
}

func (b *Barrier) Barrier() <-chan struct{} {
    b.Init()
    return b.channel
}

func main() {

    // Simple case showing example use

    var shutdown Barrier
    go func() {
        defer shutdown.Fall()
        println("GO!")

        // returns.

    }()
    <-shutdown.Barrier()
}

[Tamás Gulácsi]

Have you considered using sync.WaitGroup instead?
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
Do sth...
}
wg.Wait()

[Nick Craig-Wood]

On 22/05/14 08:59, Peter Waller wrote:
> Apologies if this has been discussed before, I had a brief search but
> didn't find anything.
>
> The code is up on github: https://github.com/pwaller/barrier

I think you need a better example - preferably one which doesn't have
sync.WaitGroup in as well!

> I find myself reimplementing this type of barrier fairly frequently at
> the moment, so I've made it a package.
>
> The idea is that one can have `defer b.Fall()` at the beginning of many
> go routines and `<-b.Barrier()` at the end. If one returns, the barrier
> falls and they all continue. It can be used to implement "signal
> shutdown if any of these points is reached", among other things.
>
> `b.Barrier()` returns a channel so that it can be used in a select{}.

I've used exactly this technique before - signalling to multiple
goroutines by the action of closing a channel.

I didn't feel it needed any more abstraction than that, but it does rely
on you knowing exactly what a closed channel does in select and <-.

> I've implemented it with an `init` rather than a `NewBarrier() *Barrier`
> because I want it to look more like the sync primitives `Once` and
> `WaitGroup`, where the zero of the type is a ready-to-use value.
>
> It took me a while to think of using a `sync.Once` to protect against a
> channel being closed repeatedly from different places, but once I did it
> seems very obvious.
>
> Question barrage:
>
> Is the code correct? Is the fact I'm using `sync.Once` like this to
> protect channel close a code smell?

`sync.Once` looks reasonable to me.

> Is there a better way to achieve the
> use case of "If one exits they all exit"?

I think signalling by closing a channel is the best way of doing this
which you've encapsulated nicely.

> Does such a barrier primitive already exist somewhere? Does it have a
> name in Go or otherwise? I'm surprised to need to invent it for myself.

You could do this with a `sync.Cond` lock too I think

http://golang.org/pkg/sync/#Cond


--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick

[Peter Waller]

The reason I don't use a `WaitGroup` is that you can only call `Done()` on a `WaitGroup` once, and I want to wait on the barrier from a select {} statement.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Waller]

On 22 May 2014 14:37, Nick Craig-Wood <ni...@craig-wood.com> wrote:

On 22/05/14 08:59, Peter Waller wrote:
> Apologies if this has been discussed before, I had a brief search but
> didn't find anything.
>
> The code is up on github: https://github.com/pwaller/barrier

I think you need a better example - preferably one which doesn't have
sync.WaitGroup in as well!

Hmm, not sure how to achieve that, given that `WaitGroup` is necessary wait for goroutines to complete. The barrier isn't intended to help with that, it's just a signalling mechanism.

[Nick Craig-Wood]

On 22/05/14 15:06, Peter Waller wrote:
> On 22 May 2014 14:37, Nick Craig-Wood <ni...@craig-wood.com

Just use a time.Sleep() or something - the eye is immediately distracted
by the WaitGroup and the similar methods which makes the example much
more confusing that it really is.

[Peter Waller]

On 22 May 2014 15:57, Nick Craig-Wood <ni...@craig-wood.com> wrote:

Just use a time.Sleep() or something - the eye is immediately distracted
by the WaitGroup and the similar methods which makes the example much
more confusing that it really is.

Hmm, difficult. I wrote it this way to show a correct way to do this in a non-racy manner, because someone will copy-paste the code.

[roger peppe]

Hi Peter,

The package we tend to use for things like this is launchpad.net/tomb,
which provides a somewhat more structured approach to closing things down
(it keeps track of an error value too, and has two phases in the
shutdown process).
I quite like it - the semantics often seem to map well to the problem at hand.

As for your Barrier implementation, it looks reasonable - if you were using
it a lot and space were an issue, you might want to replace the two sync.Once's
with a single sync.Mutex
(you can close the channel inside the mutex, first checking that it hasn't
been already closed by attempting a read from it)

cheers,
rog.

[Peter Waller]

Thanks rog!

I did look at tomb before writing this, but didn't consider it further because it says "A Tomb tracks the lifecycle of a goroutine as alive, dying or dead, and the reason for its death." i.e, a single goroutine rather than many. I didn't want to have to track each goroutine individually.

Also, in my particular case Tomb and death invoked the wrong image for me, since it is a graceful shutdown signal. Also "Done flags the goroutine as dead, and should be called a single time", where I want many goroutines to be able to signal.

It does seem as though I might be able to use Tomb in my case, but I can't tell easily from the documentation, and it looks more complicated than what I was after.

[roger peppe]

On 22 May 2014 18:08, Peter Waller <pe...@scraperwiki.com> wrote:
> Thanks rog!
>
> I did look at tomb before writing this, but didn't consider it further
> because it says "A Tomb tracks the lifecycle of a goroutine as alive, dying
> or dead, and the reason for its death." i.e, a single goroutine rather than
> many. I didn't want to have to track each goroutine individually.

I think that sentence in the tomb documentation
is a bit misleading, as the first phase of shutting
down, at least, can involve any number of goroutines.

Specifically, any number of callers can call Kill, and any number of
listeners can wait on Dying.

> Also, in my particular case Tomb and death invoked the wrong image for me,
> since it is a graceful shutdown signal. Also "Done flags the goroutine as
> dead, and should be called a single time", where I want many goroutines to
> be able to signal.

The structure we usually end up having is that there's one goroutine
that acts as a kind of overseer for the rest - it may use a WaitGroup
to wait for the others to die before calling Done. There's no *need*
to use Done or Dead though (see below), although they're convenient
when there are resources to be cleaned up.

> It does seem as though I might be able to use Tomb in my case, but I can't
> tell easily from the documentation, and it looks more complicated than what
> I was after.

FWIW, I believe that the below code implements exactly your
Barrier semantics.

type Barrier struct {
tomb tomb.Tomb
}

func (b *Barrier) Fall() {
b.tomb.Kill(nil)

}

func (b *Barrier) Barrier() <-chan struct{} {

return b.tomb.Dying()
}

cheers,
rog.