Channel patterns?

[Starfish]

I've looked through Rob's excellent 'Go Concurrency Patterns', and started experimenting with channels.

It appears that channels enable many new concepts, but those concepts are hard to find out about. Where can I find out more?

Also, I have a few other questions, numbered for convenience. I would sincerely appreciate any input on any of those:

1. What is the appropriate naming convention for channels?

2. Is it a good idea to create 'kill channels' for go routines?

3. Is it better to return channels, rather than sending them as arguments?

4. Does it make sense to always start a project of non-trivial size with a 'for select' loop?

5. Should one use channels rather than callbacks?              

[roger peppe]

On 26 February 2013 09:26, Starfish <ahn...@rocketmail.com> wrote:
> I've looked through Rob's excellent 'Go Concurrency Patterns', and started
> experimenting with channels.
>
> It appears that channels enable many new concepts, but those concepts are
> hard to find out about. Where can I find out more?
>
> Also, I have a few other questions, numbered for convenience. I would
> sincerely appreciate any input on any of those:

These are all excellent questions. The answers to most of them
are subjective, not universal IMHO. I'll reply with my own
take on them, which might or might not intersect with anyone else's ideas.

> 1. What is the appropriate naming convention for channels?

For a simple channel in an obvious context, "c" often suffices.
I often name a channel after the thing that's sent on it.
For instance "work" for a channel of work requests;
"done" for a channel that goroutines send on when they're done, etc.
If I need to disambiguate, I'll sometimes add a trailing "c";
for instance donec, workc. Some people prefer a trailing "Ch";
for instance doneCh, workCh.

> 2. Is it a good idea to create 'kill channels' for go routines?

If you want to kill goroutines, this can be a good idea, yes.
See also launchpad.net/tomb, which can be useful too.

> 3. Is it better to return channels, rather than sending them as arguments?

Both ways have advantages, and both techniques are used in the
Go standard library. The place that creates the channel controls the
channel's use, so by returning a (read-only) channel you can make sure that
it's not sent to by any other goroutine. See time.After for an example
of that. If you send a channel as an argument, the caller can cause
values from several different origins to be sent to the same channel,
which can also be good, as it makes it trivial to listen for a value
from any of those origins without starting a goroutine per channel.
http://golang.org/pkg/os/signal/#Notify is one example.

> 4. Does it make sense to always start a project of non-trivial size with a
> 'for select' loop?

I don't think so. It depends very much on the project. Any given
project will probably have several such loops. That's the beauty of
Go's concurrency model: there's not necessarily a single "central loop",
just a collection of concurrently running peer goroutines that communicate
with one another. Each for-select loop is master of its own domain :-)

> 5. Should one use channels rather than callbacks?

The answer depends very much on the problem. Both are appropriate
in different places. In a sense, any place where you pass in an interface
value you're using callbacks. I tend to reserve the use of channels
for places where there's something asynchronous going on (a timer
firing, a message being received on a network connection).
In general, the decision isn't irreversible - a channel can be turned
into a callback by creating a goroutine to read from the channel
and make the call; a callback can be turned into a channel send
by making the callback send on the channel.

cheers,
rog.

[Kyle Lemons]

To add my 2c:

On Tue, Feb 26, 2013 at 1:26 AM, Starfish <ahn...@rocketmail.com> wrote:

I've looked through Rob's excellent 'Go Concurrency Patterns', and started experimenting with channels.

It appears that channels enable many new concepts, but those concepts are hard to find out about. Where can I find out more?

Channels are so simple that their use is limited only by your imagination.  There are many imaginative gophers out there, so there are often approaches to a problem which have already been shared, but I wouldn't say there is some finite set of concepts of which a list could be made.

 

Also, I have a few other questions, numbered for convenience. I would sincerely appreciate any input on any of those:

1. What is the appropriate naming convention for channels?

The shortest name that conveys what the variable is.  If the channel is a receiver type, c usually suffices.  The "ch" suffix seems most commonly used when the non-"ch" name is already used, as variables are not usually named after their type.  Don't name it "stopBoolChan" for instance.  Names like "stop" and "done" and "requests" and such are common.

 

2. Is it a good idea to create 'kill channels' for go routines?

Create one if you need one. Otherwise, don't.  If you think you might need one, just make sure that it wouldn't be hard to add with your design (typically it's pretty straightforward).

3. Is it better to return channels, rather than sending them as arguments?

I don't usually like returning channels, because then the caller has no choice about what buffering to use, and an appropriate choice could be dependent on the calling code.  Things which do know their required buffer size (e.g. time.After, to prevent stalling the goroutine forever) can return them.  In general, I tend to think that if you want to wire two things together, you should create the channel and give it to the pieces in question.

 

4. Does it make sense to always start a project of non-trivial size with a 'for select' loop?

That sounds too much like an event loop, which I would not suggest.  Start a project with focused code and unit tests for it.  Once you have the smaller pieces working solidly, it's often easy to wire them together.  Whether this requires a for/select or a straight-line main() depends dramatically on the situation.

5. Should one use channels rather than callbacks?

Callbacks have their place, but channels and blocking APIs tend to be more idiomatic. 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Tony Worm]

I had the privilege of giving this presentation as a guest lecturer last night.

The most confusing example, in terms of behavior was the example on slide 30, restoring sequencing of messages on fanIn with a wait channel

( sequenceboring.go code posted below for convenience)

We had difficulty understanding why, and if, it guarentees lockstep sequencing.

or that the output cannot be

Joe

Ann

Joe

Ann

Ann

Joe

The argument was that both Joe and Ann receive the msg#.wait <- true

and that the next time around there could be a race to the channel 'c' returned from the fanIn function

Can anyone elaborate on this further?

Does a race exist?

--------------------------------------------------

type Message struct {

str string

wait chan bool // HL

}

func main() {

c := fanIn(boring("Joe"), boring("Ann")) // HL

for i := 0; i < 5; i++ {

msg1 := <-c; fmt.Println(msg1.str)

msg2 := <-c; fmt.Println(msg2.str)

msg1.wait <- true

msg2.wait <- true

}

fmt.Println("You're all boring; I'm leaving.")

}

func boring(msg string) <-chan Message { // Returns receive-only channel of strings. // HL

c := make(chan Message)

waitForIt := make(chan bool) // Shared between all messages.

go func() { // We launch the goroutine from inside the function. // HL

for i := 0; ; i++ {

c <- Message{ fmt.Sprintf("%s: %d", msg, i), waitForIt }

time.Sleep(time.Duration(rand.Intn(2e3)) * time.Millisecond)

<-waitForIt

}

}()

return c // Return the channel to the caller. // HL

}

func fanIn(inputs ... <-chan Message) <-chan Message { // HL

c := make(chan Message)

for i := range inputs {

input := inputs[i] // New instance of 'input' for each loop.

go func() { for { c <- <-input } }()

}

return c

[Kevin Gillette]

5. Callbacks are idiomatically used in situations like strings.SplitFunc, when the underlying process lends itself to sequential processing (and when the callback returns something meaningful, rather than being ”fire and forget”). Callbacks are generally not idiomatic to use for event notification, and even code that interfaces with a c-based callback event dispatch system often is just wrapped to send the event down a channel (this is what os/signal does, even though the kernel interface takes a function pointer).

There are pragmatic exceptions to this, however, such as net/http, which is technically a callback model; providing a func is more convenient in this case than pulling requests from a channel and also better reflects the underlying operations compared to a channel, though there's nothing to stop someone from wrapping http request handlers with a channel based work queue, for example.

[Tony Worm]

In general, with multiplexed channels, if more than

one goroutine is waiting to read/write on the multiplexed end,

are the 'requests' queued in some manner?

[Kyle Lemons]

On Tue, Feb 26, 2013 at 9:50 AM, Tony Worm <verd...@gmail.com> wrote:

I had the privilege of giving this presentation as a guest lecturer last night.

The most confusing example, in terms of behavior was the example on slide 30, restoring sequencing of messages on fanIn with a wait channel

( sequenceboring.go code posted below for convenience)

We had difficulty understanding why, and if, it guarentees lockstep sequencing.

or that the output cannot be

Joe

Ann

Joe

Ann

Ann

Joe

The argument was that both Joe and Ann receive the msg#.wait <- true

and that the next time around there could be a race to the channel 'c' returned from the fanIn function

Can anyone elaborate on this further?

Does a race exist?

--------------------------------------------------

type Message struct {

str string

wait chan bool // HL

}

func main() {

c := fanIn(boring("Joe"), boring("Ann")) // HL

There is no guarantee that boring(joe)s goroutine will run before boring(ann)s.  If you need that, you'd have to explicitly teach one to wait on the other.

 

--

[Kevin Gillette]

No guarantees in the spec that I'm aware of, but I believe the current implementation does have fifo for data already in buffered channels--but when the channel is empty or unbuffered, there is no defined order for servicing waiting goroutines (so a goroutine that just started blocking could get serviced before a goroutine that's been blocked for a longer time).

[Andrew Gerrand]

On 26 February 2013 20:26, Starfish <ahn...@rocketmail.com> wrote:

Where can I find out more?

You might be interested in this talk:

  http://talks.golang.org/2012/chat.slide

Andrew

[Bryan Mills]

On Tuesday, February 26, 2013 4:26:33 AM UTC-5, Starfish wrote:

I've looked through Rob's excellent 'Go Concurrency Patterns', and started experimenting with channels.

It appears that channels enable many new concepts, but those concepts are hard to find out about. Where can I find out more?

Also, I have a few other questions, numbered for convenience. I would sincerely appreciate any input on any of those:

1. What is the appropriate naming convention for channels?

2. Is it a good idea to create 'kill channels' for go routines?

Any time you spawn a goroutine, you should be able to answer the question "when will this goroutine exit?" and prove it - goroutines that don't exit won't be garbage-collected.  If you can't answer that question satisfactorily, adjust the API until you can.

If you add a "kill channel", be sure to document whether the kill signal consists of closing the channel, sending to it, or receiving from it.  Sending or receiving on an unbuffered channel have better synchronization properties, but closing the channel allows multiple goroutines to watch it concurrently.

3. Is it better to return channels, rather than sending them as arguments?

Doesn't matter, but you should keep careful track of some very important details, such as:

* Will the channel be closed?  If so, when, and by which goroutine?  (Are there other goroutines that will still be sending to it?)

* Must the channel be drained in order for the goroutine to exit?  (What happens if the caller stops receiving from the channel before it is closed?)

* Must/can the channel be buffered?  If so, what should happen when the buffer is full?  (In many cases, APIs rely on channel events for synchronization, and using a buffered channel in those cases can lead to synchronization-related bugs.)

 

4. Does it make sense to always start a project of non-trivial size with a 'for select' loop?

No, but if you do have a "for select" loop, you should be careful not to introduce races; channels are still subject to lock-ordering, just like mutexes.  (You can think of "sending on a channel" as being akin to "acquiring a lock on the send-side of the channel".)  You can usually avoid lock-ordering bugs by always selecting on the group of channels you're interested in within the same select-statement; try not to perform other channel operations within the bodies of the cases.

 

5. Should one use channels rather than callbacks?

For exported APIs, try not to use either: instead, use synchronous, blocking calls and return values with errors.  It's trivial for a caller to transform a synchronous call into either a channel or a callback as needed, and that way you won't have to document details like "who closes the channel?" or "does the callback need to handle concurrent calls?".

For internal APIs, use whichever is more natural: channels are amenable to for-loops and select-statements, while callbacks are better for purely-asynchronous events (and sometimes preferable for sequential events as well - filepath.Walk is a great example, because it allows custom error-handling).