Channels of multiple values

[Joe Farro]

I find myself creating structs with the primary purpose of passing multiple values through a channel. I'm conflicted about this due to:

Pros of using structs: 

• Multiple values via a struct have multiple levels of labels (type name and names of values) therefore easier to understand
  
• Encourages consistency and logical grouping
  
• If a situation lends itself to a struct, a struct will almost surely be used (vs an interface)
  

Cons:

• Hinders adoption of the (value, err) and (value, ok) idiom with channels 
• Increases exposure to import loops or the overuse of interfaces
• Increases exposure to proliferation of unnecessary structs
  
• Creates a temptation to use existing structs to group values even if there is a conceptual mismatch with the originally intended purpose of the struct (maybe my OOP background is getting the best of me, here)
  

Seems like channels of multiple values would have most or all of the same benefits as functions with multiple return values. The format could be consistent with multiple return values, including optionally naming the values, while also preventing breaking changes:

// single value would be the same

var updates chan State

// multiple value requires parens

var updates chan (url, status string)

updates = make(chan (string, string))

// buffered, named values, created with :=

updates := make(chan (url, status string), 15)

// value, err idiom

filesRead := make(chan (lines []string, err error))

I looked around but didn't see anything on this topic. Just sharing some thoughts. I'm very new to concurrency and channels, so maybe these ramblings are mostly related to inexperience. If that's the case, suggestions for reference reading would be greatly appreciated.

Thanks,

Joe

[⚛]

You can already write:

  var updates chan struct { X, Y string }

It is impossible to deconstruct a received value in one line of code:

  X, Y := <-updates     // impossible

because it interferes with the optional status returned by the receive operator (http://golang.org/ref/spec#Receive_operator). This also applies to your proposal.

[Kevin Gillette]

On Monday, October 1, 2012 12:22:25 AM UTC-6, ⚛ wrote:

It is impossible to deconstruct a received value in one line of code:

  X, Y := <-updates     // impossible

because it interferes with the optional status returned by the receive operator (http://golang.org/ref/spec#Receive_operator). This also applies to your proposal.

The spec could be tweaked so that the "optional status value" is not specifically the 2nd parameter, but merely the "last" (so if you have 4 values bundled in a single channel operation, the status boolean would be the 5th).

For a number of reasons, I think that even such a backwards compatible spec tweak as this isn't a good idea at this time.

Joe Farro: I think multi-valued constructs may be useful/expressive, but we'd probably need to come up with several more use cases besides just channels before the idea could gain much momentum, and even if it's accepted, I wouldn't expect the any actual implementation work on it before Go 2, if and when that happens.

[⚛]

Doesn't "to come up with several more use cases besides just channels" mean the inclusion of tuples into the language as a 1st class value? If it does, it is impossible to decide what "X, Y := <-updates" means because X could be a tuple or the 1st element of the tuple.

[Kevin Gillette]

On Monday, October 1, 2012 1:21:56 AM UTC-6, ⚛ wrote:

Doesn't "to come up with several more use cases besides just channels" mean the inclusion of tuples into the language as a 1st class value? If it does, it is impossible to decide what "X, Y := <-updates" means because X could be a tuple or the 1st element of the tuple.

I was worried someone would mention that. First class tuples would also mean you'd have to deal with nested tuples, and for consistency, all functions should take and return statically typed tuples (but then, why not have nested return values?) I think all of that is pretty bad for Go, and that the use cases I was talking about would be those where flat (non-nested) "multi-valued constructs" would be applicable (places where their use could be implicitly treated as an anonymous struct literal type [but with unnamed fields?!], for example). The whole thing is problematic, and should be given a lot of thought if people do want something like this, _before_ anything is decided upon.

[Joe Farro]

Thanks for your replies.

pts := make(chan struct { X, Y int })

go func() { pts <- struct { X, Y int } {0, 0} }()

if pt, ok := <- pts; ok {

fmt.Println(pt.X, pt.Y)

}

vs 

pts := make(chan (x, y int))

go func() { pts <- 0, 0 }()

if x, y, ok := <- pts; ok {

fmt.Println(x, y)

}

I do know about the anonymous struct syntax and almost included an example using it but thought it would obscure the general idea.

Is the use of anonymous structs generally advisable? It does seem a lot better than an import loop, mis-using existing types or something along those lines, but I'm reluctant to use them because I think they're hard to read and cumbersome to work with. For instance, anonymous structs aren't generally preferred over multiple return values for functions. 

Kevin, I was just going to make the same point about the "optional status value" simply being an additional, optional last value returned by the receive operator. But, with that, things are starting to get dicey.

I don't know, seems to me it would be a natural and consistent extension, the mechanism of which is already described in the language spec here http://golang.org/ref/spec#Assignments (as far as I can tell) and seems to be referred to as a "multi-valued expression." Looks like the makings of both defining the structure of the expression and working with it are already present:

func test() (a, b string) {

return "one", "two"

}

I am not advocating first-class tuples (to my knowledge, anyway), but am suggesting that expanding the capabilities of channels to work with multi-valued expressions is worth considering.

Also, I *definitely* am not advocating making a hasty decision. I urge against that, actually. I think Go is a masterpiece, as is. I have no idea if anyone else would find this practical and I think it's entirely possible my suggestion may be deeply flawed in ways I'm not aware of. 

[Martin Hurton]

This was discussed some time ago on the ML:

https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/K11-UgsHQQs

- Martin

> --
>
>

[Kevin Gillette]

On Monday, October 1, 2012 6:05:57 AM UTC-6, Martin Hurton wrote:

This was discussed some time ago on the ML: 

https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/K11-UgsHQQs 

I think the last three posts in that thread (Steven Blenkinsop and Paul Borman) make a lot of good arguments both for and against. I personally agree that use of structs just for cheap packing/unpacking in channels is awkward, but that may also hint that the solution is to make _struct_ handling in those cases less awkward. I also agree that the human readability of multivalued channel receives would go way down if you had a two-valued channel and happened to assign the second value into a variable called ok -- it would look like the current go idiom, but would have unexpected semantics.

I also have found anonymous nested structs (a struct type literal within another struct type literal) are awkward any time you have to handle the result yourself -- when passing them to something else (which typically uses reflect, like the encoding packages), it's alright. In many other cases, it's just "awkward" at the moment, but there's at least room for improvement (we're not locked into the awkward parts).

For example, I'd expect this to work: <http://play.golang.org/p/L89P1eego3>

To get it to work right now with anonymous structs, you have to do something like: <http://play.golang.org/p/VsyfBSQYGj>

What this means right now is that unless you're washing your hands of the data as soon as you've constructed it (e.g. encoding/json), or you only need to deal with the data using type inference (rather than value literals as shown in the above links), then it's currently pretty much worthless to use struct type literals past the first level deep.

[Jeremy Wall]

I primarily see two use cases for multiple value channels which I
think go already covers well.

Mutually exclusive multiple values as in the common val, err := f()
construct in go. In which case multiple channels with a select is more
than adequate and in some cases superior to a multiple value channel.
Mutually inclusive multiple values. The struct case seems a natural
fit for this as well since the values are usually meant to be bundled
anyway.

Is there a use case other than these that you had in mind?

> --
>
>

[Larry Clapp]

Thinking out loud ...

How about (untested):

var c chan func()(int, int, string)

c <- func()(int, int, string){return 1, 2, "three"}

i1, i2, s1 := (<-c)()

That's kind of yucky and you could probably make a constructor function:

func makeTuple(i1, i2 int, s string) func()(int, int, string) {

   return func()(int, int, string) { return i1, i2, s }

}

Then we have

var c chan func()(int, int, string)

c <- makeTuple(1, 2, "three")

i1, i2, s1 := (<-c)()

Is this any better than an anonymous struct?  I'm not sure.

But that leads me to some sort of unpack function:

type SomeStruct struct { i1, i2 int, s string }

func Unpack(ss SomeStruct) (int, int, string) {

    return ss.i1, ss.i2, ss.s

}

var c chan SomeStruct

c <- SomeStruct{1, 2, "three"}

i1, i2, s := Unpack(<-c)

Probably other refinements are possible.

In short it seems possible to abuse(?) closures and multiple-return-value functions to make tuples by hand.  Some sort of generic "unpack()" function would be nice, but at the cost of some boilerplate per struct, isn't really necessary.  Of the three, I think I like the 2nd version with makeTuple() the best, but it seems kind of inefficient, with a closure per send and a function call per receive.  But on the other hand I'm not sure if that's all that much worse than creating a structure per send and then unpacking it manually per receive.

Hmm, and I just now noticed that closing the channel gets messy, so you'd have to say

var i1, i2 int; var s string

if f, ok := <-c; ok {

  i1, i2, s := f()

}

Which, again, isn't much better than destructuring a structure by hand.  Well, maybe it would be for longer structures, I dunno.

Anyway, like I said, just thinking out loud.  But you can go a long way with closures.  :)

-- Larry

[minux]

i'm think about something like this (post on phone, so
there might be errors):

type tuple struct { a, b, c int }
func (t *tuple) get() (a, b, c int) {
   if t != nil {
      return t.a, t.b, t.c
   }
   return
}

ch := make(chan *tuple)

// to send
ch <- &tuple{1, 2, 3}

// receive
a, b, c := (<-ch).get()

in this way, closed channel is automatically translated
to receiving all zeros.

On Oct 2, 2012 1:36 AM, "Larry Clapp" <la...@theclapp.org> wrote:

[Larry Clapp]

Sorry to reply to myself, but I also thought of this: http://play.golang.org/p/n9iYer0glo

type Tuple []interface{}

func main() {

c := make(chan Tuple, 1)

c <- Tuple{1, 2, "three"}

a := <-c

i1 := a[0].(int)

i2 := a[1].(int)

s3 := a[2].(string)

fmt.Println(i1, i2, s3)

c <- Tuple{4, "five", 6, 7.0}

a = <-c

i4 := a[0].(int)

s5 := a[1].(string)

i6 := a[2].(int)

f7 := a[3].(float64)

fmt.Println(i4, s5, i6, f7)

}

Ugly?  Uglier than a struct?  I dunno, maybe.  If you're not using the value of i4 or s5, then you could omit the type assertion, and just pass it around as-is.  Of course go through too many levels of that and you get into the problem of "what type is this thing supposed to be?" and whether, depending on execution path, it could differ from call to call.  Ew.

It seems like one way or the other you have some annoying code (though in some cases only a little annoying) to disambiguate the types.

It does sort of point the way to some interesting syntax:

   i4, s5, i6, f7 := a[...].(int, string, int, float64) // where "..." are literally 3 dots

Just a thought.

Or, again, an unpack primitive, which for any struct{x T1, y T2, z T3} returns 3 values of type T1, T2, T3:

  st1 := struct{1, "two"}

  i1, s2 := unpack(st1)

  st2 := struct{3.0, "four", 5}

  f3, s4, i5 := unpack(st2)

Though on the third hand, writing Unpack() for any given struct type is just a few lines of code.  You could probably even automate it, if you did it enough.

-- L

[Kevin Gillette]

I think this is a pretty clean solution. I can see it being inadequate for anyone with dozens of type-combinations in channels, but I don't expect that happens too often.

[Joe Farro]

Martin, thanks for pointing out that thread.

The last three posts do sum things up quite nicely and the 3rd to last post, in particular, pretty much says it all.

The ok-idiom is the biggest sticking point. One possibility is to mandate the final bool value:

// compile error

ch := make(chan (string, int))

s1, i1 := <-ch

// compiles

ch := make(chan (string, int))

s1, i1, _ := <-ch

But, I don't know that I would go so far as to advocate for this approach.

[Joe Farro]

On Monday, October 1, 2012 11:53:32 AM UTC-4, Jeremy Wall wrote:

[...]

Mutually exclusive multiple values as in the common val, err := f()
construct in go. In which case multiple channels with a select is more
than adequate and in some cases superior to a multiple value channel.

Is this the sort of scenario you mean by this? 

http://play.golang.org/p/b2UCKsHbzh

I think this is easier to read:

http://play.golang.org/p/ByHHC84rLB

In a more complex context (a third return value): 

http://play.golang.org/p/jIs0RwPw0F

Maybe not everyone would agree, but I think it expresses the intent more clearly. I speculate that multiple valued channels would make things easier to read, potentially by quite a bit, in many cases.

Mutually inclusive multiple values. The struct case seems a natural
fit for this as well since the values are usually meant to be bundled
anyway.

That's a rather blanket statement. There are many cases in the standard library (example) where mutually inclusive values are returned from a function without a struct to bundle them. The third link, above, applies to this. For something like a find and read operation, I don't feel the need for the results to be bundled in a struct. This may generally apply in cases with only a few relevant pieces of information.

I again want to note that I don't consider myself an expert. More experience with the select statement and channels may change my take on things.

Is there a use case other than these that you had in mind?

Well, you pretty much covered everything with "cases that are like so-and-so and cases that are not." But, I'll describe what I'm doing to add some context to my stubbornness. If I'm banging my head against the wall it would be nice to know :)

The errors of my ways are probably abundant, but here goes: I have a situation similar to Mark Summerfield's approach to a thread-safe map (link?) where he is, as I understand it, governing access to a resource by having API methods pass operation parameters through channels to a single go routine that performs the operations and returns the results when finished. Basically, I'm doing the same thing although it's governing access to another process, not a map. I'd like to get rid of the commandData struct. It has fields that aren't always relevant and generally prevents the scaling of anything employing this approach (it seems to me). I think it obscures things, in general. If I want diverse and expressive parameters for API methods and their return values it looks like I will either have to revert to more flexible type specifications (like interface{} and I would likely revamp the whole thing to avoid this) or I will need to have a variety of commandData variants and corresponding channels to pass them through. I think the use of the commandData struct, interface{}, or a set of commandData variants feels wrong on a variety of levels. This use-case is why I keep going back to multiple return values as an analogous scenario. It must be that my inexperience with concurrency and channels is why I'm struggling with this.  

[si guy]

So.. You're looking for a polymorphic channel type?
I can think of a few ways to do it in go.

The simplest is still a command struct with a message code and an interface, which is switch type-asserted on the receiving side. (Or a type switch, but this involves reflection overhead)

Slightly more complicated would be a serializer/deserializer combination like gob(or a custom). This has the advantage of being extendable over a network later.

Slightly more flexible (and less thread safe) would be a function pointer channel, over which you pass closures or methods. This is my favourite as it can be highly modular and extensible, but you have to watch out for shared memory access (try to close over channels). It gets fun when the worker passes function pointers back.

There are many other ways, but it really depends on what the performance constraints/demands of your program are.

-Simon Watt

[Kevin Gillette]

The approach of using a separate channel for errors from the channel used for normal data -- the out-of-band method -- is great for stopping a whole series of computations as soon as an error is encountered, especially if that error can come from a lot of places. If the error is isolated to just one item on the 'normal' channel and doesn't propagate to the larger task, then it's much better to use a chan struct or chan interface approach (depending on mutual inclusivity or exclusivity) -- this is the in-band method -- since using a separate channel means that you lose coupling of the error to the work-unit, unless you've been synchronizing the channels by sending nil errors (in which case, you might as well do in-band anyway).

[Glenn Brown]

Larry, your example can be simplified as http://play.golang.org/p/COL68pDwOE :

package main

import "fmt"

type Tuple []interface{}

func main() {
c := make(chan Tuple, 1)
c <- Tuple{1, 2, "three"}

fmt.Println(<-c...)

c <- Tuple{4, "five", 6, 7.0}

fmt.Println(<-c...)
}

--Glenn

[Larry Clapp]

Well, true, but it doesn't really illustrate the manual destructuring you have to do in real usage.  That's a nice trick, though, I'd forgotten about that.

-- Larry

[slav...@gmail.com]

I know this is an old post, just thought to share possible work around. Idea is basically return a func that will return multiple values. I'm not sure how much performance overhead this will create and it's a little bit ugly, but works as intended.

func f(c chan func() (int, string)) {

c <- (func() (int, string) { return 0, "s" })

}

func main() {

c := make(chan func() (int, string))

go f(c)

y, z := (<-c)()

fmt.Println(y)

fmt.Println(z)

[Florin Pățan]

An even better solution is to return a struct with the fields that you need.