Check if the output channel is closed
Vladimir Sibirov
receiver before sending the value?
I know that on receiver side I can use
val, ok := <-ch
if !ok {
// we are closed
}
But what about the send operation:
// var ch chan int<-
ch <- val // this would hang if nobody's listening or panic if nil
roger peppe
this has been discussed at length on this group, but in short:
senders close; receivers check for closed.
John Beisley
that there are arbitrary numbers of senders. Is this a bad pattern to
use to begin with?
Vladimir Sibirov
type Generator struct {
term <-chan bool
out chan<- int
wg *sync.WaitGroup
}
type Multiplier struct {
arg1, arg2 <-chan int
out chan<- int
wg *sync.WaitGroup
}
func main() {
// Making channels
tg1 := make(chan bool)
tg2 := make(chan bool)
g1m := make(chan int)
g2m := make(chan int)
wg := new(sync.WaitGroup)
// Making and connecting components
g1 := new(Generator)
g1.Connect(wg, tg1, g1m)
g2 := new(Generator)
g2.Connect(wg, tg1, g2m)
m := new(Multiplier)
m.Connect(wg, g1m, g2m)
// Running the gorotines
go g1.Run()
go g2.Run()
go m.Run()
// Terminate the network
tg1 <- true // g1 terminates fine, closes out and so terminates m
tg2 <- true // OOPS, g2 will continue generating into nowhere until
this if m doesn't inform it that it has shut down e.g. by closing g2m
wg.Wait()
}
2011/5/10 roger peppe <rogp...@gmail.com>:
André Moraes
for close could cause a race condition.
They proposed other alternatives to close on that thread.
Maybe the release history for the project may help you.
http://golang.org/doc/devel/weekly.html#2011-03-07.
Hope it helps
--
André Moraes
http://andredevchannel.blogspot.com/
roger peppe
> OK, but consider this example:
>
> type Generator struct {
> term <-chan bool
> out chan<- int
> wg *sync.WaitGroup
> }
>
> type Multiplier struct {
> arg1, arg2 <-chan int
> out chan<- int
> wg *sync.WaitGroup
> }
>
> func main() {
> // Making channels
> tg1 := make(chan bool)
> tg2 := make(chan bool)
> g1m := make(chan int)
> g2m := make(chan int)
>
> wg := new(sync.WaitGroup)
>
> // Making and connecting components
> g1 := new(Generator)
> g1.Connect(wg, tg1, g1m)
> g2 := new(Generator)
> g2.Connect(wg, tg1, g2m)
i assume you mean to pass tg2 here.
> m := new(Multiplier)
> m.Connect(wg, g1m, g2m)
>
> // Running the gorotines
> go g1.Run()
> go g2.Run()
> go m.Run()
>
> // Terminate the network
> tg1 <- true // g1 terminates fine, closes out and so terminates m
> tg2 <- true // OOPS, g2 will continue generating into nowhere until
> this if m doesn't inform it that it has shut down e.g. by closing g2m
at least two possible answers:
1) why should it matter if g2 is generating into nowhere?
if it is also waiting on tg2, then it will correctly exit anyway.
2) g2 should only terminate when all its inputs have been
closed. this is a better solution IMHO, otherwise you
can get components that will never exit (given that you cannot
close a channel on the receiving side).
Vladimir Sibirov
>
> i assume you mean to pass tg2 here.
Right, I meant tg2, I was in a hurry, sorry.
> at least two possible answers:
>
> 1) why should it matter if g2 is generating into nowhere?
> if it is also waiting on tg2, then it will correctly exit anyway.
Because send is a blocking operation and it blocks g2 while nobody can
receive what it sends if m has already terminated. This results into a
deadlock in Go's scheduler mind and the program panics instead of
terminating gracefully.
> 2) g2 should only terminate when all its inputs have been
> closed. this is a better solution IMHO, otherwise you
> can get components that will never exit (given that you cannot
> close a channel on the receiving side).
Good point, I will close them instead then.
roger peppe
>>> g2.Connect(wg, tg1, g2m)
>>
>> i assume you mean to pass tg2 here.
>
> Right, I meant tg2, I was in a hurry, sorry.
>
>> at least two possible answers:
>>
>> 1) why should it matter if g2 is generating into nowhere?
>> if it is also waiting on tg2, then it will correctly exit anyway.
>
> Because send is a blocking operation and it blocks g2 while nobody can
> receive what it sends if m has already terminated. This results into a
> deadlock in Go's scheduler mind and the program panics instead of
> terminating gracefully.
send is blocking, but you can block on other channels at the
same time with select.
e.g.
func g2(...) {
for {
select {
case g1m <- someValue:
case <-tg1:
return
}
}
}
Vladimir Sibirov
pattern and implement one more mechanism to detect common events like
"group of necessary inputs received", "control received", "output
sent", "any of inputs is closed"?
2011/5/10 roger peppe <rogp...@gmail.com>:
André Moraes
No, you can do it by the hard way, by using mutex and locks
> pattern and implement one more mechanism to detect common events like
> "group of necessary inputs received", "control received", "output
> sent", "any of inputs is closed"?
In a sync environ that will be, more or less, easy to do.
But since the idea is to be async, I don't see a more easy approach
other than select with multiple chanels input.
I think using select with default just to make a async send that you
don't care if happens or not is too much, but by doing that you start
to care about what happens and as a result the code gets better.
roger peppe
> Does it mean that I am enforced to write components with "select"
> pattern and implement one more mechanism to detect common events like
> "group of necessary inputs received", "control received", "output
> sent", "any of inputs is closed"?
if you want a goroutne for more than one channel at the same time,
you must use select, yes. that shouldn't be too much hardship - there's
usually some code associated with the arrival of each input,
and the syntax is not unduly heavyweight.
if you can cope with additional buffering, it's always possible
to add extra goroutines to do the waiting for you, multiplexing
down to one channel, so with your generalised system
it should be possible to define operations to perform those
actions.
without knowing what kind of problem you're ultimately trying
to solve, i couldn't say whether that would be appropriate
for you or not.
Ian Lance Taylor
> I was asking a similar question in IRC recently, but with the problem
> that there are arbitrary numbers of senders. Is this a bad pattern to
> use to begin with?
There is nothing wrong with having arbitrary numbers of senders, but if
you do then it doesn't work to close the channel. You need some other
way to indicate EOF.
The close function basically exists so that range works on a channel.
If you aren't using range you don't have to use close. You can just
invent your own EOF marker appropriate to whatever data you are sending.
Ian
Vladimir Sibirov
workaround and see if it works or not.
Another question then: what about CPU load? In a system with a hundred
of goroutines doing for { select { } }, won't that mean that CPU will
be 100% busy all the time, most of which is simply multiplexing
between channels while no input has arrived?
roger peppe
> Thank you guys for explaining the Go ways. I'll try to implement some
> workaround and see if it works or not.
>
> Another question then: what about CPU load? In a system with a hundred
> of goroutines doing for { select { } }, won't that mean that CPU will
> be 100% busy all the time, most of which is simply multiplexing
> between channels while no input has arrived?
select blocks until any of its channels are ready.
it uses no CPU time while it is blocking.
André Moraes
> of goroutines doing for { select { } }, won't that mean that CPU will
> be 100% busy all the time, most of which is simply multiplexing
> between channels while no input has arrived?
If you have many producers sending too much data on a channel and you
worker you don't make blocking I/O.
The worker will take all the CPU that it can.
But most of the time the goroutines don't take too much processor time
and memory. They are very lightweigth.
I wrote a message queue server application, It handled 20 concurrent
clients sending messages to the server, and the server handled 7.000
requests per second and just a few MB and under 60% processor usage.
William Waites
] if you want a goroutne for more than one channel at the same time,
] you must use select, yes. that shouldn't be too much hardship - there's
] usually some code associated with the arrival of each input,
] and the syntax is not unduly heavyweight.
select on a set of channels, the size of which is not known at
compile time?
-w
--
William Waites <mailto:w...@styx.org>
http://river.styx.org/ww/ <sip:w...@styx.org>
F4B3 39BF E775 CF42 0BAB 3DF0 BE40 A6DF B06F FD45
roger peppe
> * [2011-05-10 15:02:32 +0100] roger peppe <rogp...@gmail.com> écrit:
>
> ] if you want a goroutne for more than one channel at the same time,
> ] you must use select, yes. that shouldn't be too much hardship - there's
> ] usually some code associated with the arrival of each input,
> ] and the syntax is not unduly heavyweight.
>
> select on a set of channels, the size of which is not known at
> compile time?
if you don't mind introducing additional buffering, you can
use additional goroutines to do this, as i said.
Steven
Yes, you can use goroutines to mux an arbitrary number of channels
onto one channel. Quite often, however, you can restructure your
program to use only one channel rather than several channels which
will then have to be muxed together. Or you can have goroutines
servicing different sets of channels concurrently. It all depends on
what your goal is and the communication dynamics of your use case.
Vladimir Sibirov
channels? I mean channels coming from different goroutines, so it
isn't likely it is fine to use just one simple channel of struct
instead.
2011/5/10 Steven <stev...@gmail.com>:
roger peppe
> Can somebody show an example how to mux on arbitrary number of
> channels? I mean channels coming from different goroutines, so it
> isn't likely it is fine to use just one simple channel of struct
> instead.
something like this, perhaps (untested).
(BTW you can arrange that output from different goroutines can
be sent to the same channel, which would avoid the need).
type muxed struct {
which int
value string
ok bool
}
func mux(in []<-chan string) chan<- muxed {
out := make(chan muxed)
for i, c := range in {
i, c := i, c
go func() {
for v := range c {
out <- muxed{i, v, true}
}
out <- muxed{i, "", false}
}()
}
return out
Nic Dade
Can somebody show an example how to mux on arbitrary number of
channels? I mean channels coming from different goroutines, so it
isn't likely it is fine to use just one simple channel of struct
instead.