Design Question: Channel Closing
Dave Grijalva
or close on a closed channel cause a panic?
I feel like this is almost never the behavior the developer wants and
the workarounds cause a lot of clutter. It also seems contrary to
common pattern of returning errors. I would like to understand why
the design is necessary or better.
Sam Fredrickson
statements, not expressions, so they can't "return" an error. The
reason why these are statements is probably the same why (in)
(de)crement are, and the lack of a ternary operator: to promote
readability and discourage clever, hard-to-comprehend code.
Ian Lance Taylor
The close function should only be used on a channel used to send data,
after all data has been sent. It does not make sense to send more data
after the channel has been closed; if all data has not been sent, the
channel should not be closed. Similarly, it does not make sense to
close the channel twice.
Note that it is only necessary to close a channel if the receiver is
looking for a close. Closing the channel is a control signal on the
channel indicating that no more data follows.
Ian
roger peppe
channel was ok. it's not unsafe and i've had a few
occasions where i've needed a strictly-unnecessary
interlock and extra boolean to avoid this.
Dave Cheney
(and managing it's visibility across multiple goroutines) to avoid
closing a channel twice. I freely admin this is a side effect of a
deficiency in my design, but if it were easy to relax the restriction
on closing an already closed channel, I would be a nice to have.
Cheers
Dave
Han-Wen Nienhuys
func (r *R) Close() {
if r.output != nil {
close(r.output)
r.output = nil
}
}
(of course, this assumes a single thread does the sending.)
--
Han-Wen Nienhuys
Google Engineering Belo Horizonte
han...@google.com
David Symonds
> Couldn't you use a != nil check to avoid the boolean?
>
> func (r *R) Close() {
> if r.output != nil {
> close(r.output)
> r.output = nil
> }
> }
>
> (of course, this assumes a single thread does the sending.)
That also assumes that only a single goroutine accesses r.output;
assignment to it requires synchronisation for it to be observed by
other goroutines.
Dave.
Han-Wen Nienhuys
>> func (r *R) Close() {
>> if r.output != nil {
>> close(r.output)
>> r.output = nil
>> }
>> }
>>
>> (of course, this assumes a single thread does the sending.)
>
> That also assumes that only a single goroutine accesses r.output;
> assignment to it requires synchronisation for it to be observed by
> other goroutines.
Yes, the channel receivers would receive a copy of the channel pointer.
Dave Cheney
boolean achieve the same thing, and also suffer from the same
visibility problem. Referring to my earlier assertion that this was a
design defect, I'm currently replacing the chan with a linked queue
with it's own close semantics, protected by a mutex.
Dave Grijalva
-dave
Jesse McNelis
> There are cases where the receiver needs to indicate it cannot accept
> any more data. In these cases, it would be useful to be able to close
> the channel from the receiver end without the application crashing. The
> workaround that's been proposed on this list is to use a second channel
> that does nothing but close use it as follows:
The problem with a receiver being able to close a channel is that a
sender makes something it wants to send before it knows it can send it
so by having the ability for a receiver to close a channel you also then
require that every producer can handle the rollback if it's unable to
send what it produced. Rollback is often impossible.
In your particular case, your sender is happy to just throw away what it
produced if it can't send it, but this isn't the general case and by
making your case a bit easier the language would make the general case
impossible.
- jessta
Dave Grijalva
-dave
Jesse McNelis
> I don't believe that's true. Having a send on a closed channel return an
> error instead of panicking would not break exiting behavior. It's up to
> the developer whether they would close a channel on the receiver end. It
> is, in fact, possible to do it the same thing now. The only difference
> is, if you do it today, you have to make sure to catch the panic when
> you try to send.
The reason it panics is because it's a programmer mistake, like
dereferencing a nil pointer or doing a type assertion to the wrong type.
There is nothing the program can do what will be correct, even if it had
a return value there is still nothing that can be done in the general
case to recover from that situation.
Catching that panic and continuing on is possible in your particular
problem but not generally.
The panic is there to discourage you from doing that because it's the
wrong way to go about it and it gets tricky.
> In cases where the receiver is incapable of processing new items,
> allowing the sender to keep sending items seems like the wrong thing.
A receiver that wants to stop processing can signal the producer to stop
producing, but it still has to continue to read from the channel until
the channel is actually closed by the producer so that it doesn't miss
anything.
In the case that a receiver could close a channel you'd end up with code
that looked like:
{
var ch chan bool // ch is a chan
close(ch) // close the chan
for b := range ch {} // read from the chan that we just closed???
}
which is semantically silly.
Dave Cheney
Sent from my iPhone
Dave Grijalva
Robert Johnstone
I'd like to second the suggestion that it should be possible to close
an already closed channel. I find as a general rule that resource
clean-up operations that are idempotent greatly simplify coding,
especially in the presence of errors. While I've found that this has
been less of an issue in Go, I still believe that this is a useful
principle.
Robert
On Nov 2, 11:23 pm, Ian Lance Taylor <i...@google.com> wrote:
> The close function should only be used on a channel used to send data,
> after all data has been sent. It does not make sense to send more data
> after the channel has been closed; if all data has not been sent, the
> channel should not be closed. Similarly, it does not make sense to
> close the channel twice.
>
Matthew Sackman
> Either assigning the channel value to nil, or having a separate closed
> boolean achieve the same thing, and also suffer from the same
> visibility problem. Referring to my earlier assertion that this was a
> design defect, I'm currently replacing the chan with a linked queue
> with it's own close semantics, protected by a mutex.
Yes, and when you've added programmer-visible mutexes to work around
deficiencies of queues, you've probably lost much of the benefit of
queues.
If a queue should not have multiple senders then it should explode when
multiple senders try to use it. Having it explode only when multiple
queues try to close it is just bizarre. That said, the alternatives are
rather more complex - essentially you'd end up having close be a join
pattern and thus the "real" close happens only after all go routines
that use the channel for sending have stopped sending.
It thus seems risky to use "close" at all except for very simple
point-to-point patterns.
Matthew
Matthew Sackman
> If a queue should not have multiple senders then it should explode when
> multiple senders try to use it. Having it explode only when multiple
> queues try to close it is just bizarre. That said, the alternatives are
Matthew
Ian Lance Taylor
> There are cases where the receiver needs to indicate it cannot accept
> any more data. In these cases, it would be useful to be able to close
> the channel from the receiver end without the application crashing.
Having the receiver close a channel is a race condition. There is no
way that the receiver can know whether there are more values in flight
on the channel. Closing a channel as a signal that there is no more
data can only work from the sending side.
Your program evidently needs some sort of signal from the receiver to
the sender indicating that the receiver does not need any more data.
There are certainly programs that work that way. You are also arguing
that you should be able to use close to send that signal. But that's
not how close works. If you think of close as sending a special value
on the channel, which is essentially what it does, then it may be
clearer that you can't have the receiver call close, any more than you
can have the receiver send a value on the channel to tell the sender to
stop sending. The receiver's close will in effect just pop back up at
the receiver's side.
> This makes for very messy code in my opinion. First of all, we're using a synchronization primitive to protect a synchronization primitive. Secondly, it could just look like this:
>
> if ok := sendChan <- myThing {
> // it worked!
> } else {
> // we're done here
> }
But at that point you have no idea how many values were sent that the
receiver did not see. I understand that in your program it does not
matter, but that is not the general case. You are accepting and
ignoring the race condition, but that doesn't mean that the language
should encourage that approach in general.
> We now have an extra closeChan to tote around, but it's not even that simple. If it's possible for multiple different goroutines to close the signal channel, you have to add a third synchronization primitive to synchronize around the closeChan (I've been using sync.Once for this). You end up with something like this:
>
> func (t *myType) close() {
> t.closeOnce.Do(func(){
> close(t.closeChan)
> }
> }
The close function is normally used to indicate that the end of a stream
of data of indeterminate length. There's no need to use close here, so
there is no need to use closeOnce. Just send a value on the channel.
That is enough for the sender's select to see that the receivers are
done. It doesn't matter if more than "done" message is sent.
Perhaps it would help to consider that the close function was introduced
for one purpose: to permit range over a channel to work. Without a
general signaling mechanism to indicate that there is no more data,
there would be no way to range over a channel. If you need some other
communication approach, then you shouldn't be using close.
Ian
Jan Mercl
Yes, and when you've added programmer-visible mutexes to work around
deficiencies of queues, you've probably lost much of the benefit of
queues.If a queue should not have multiple senders then it should explode when
multiple senders try to use it. Having it explode only when multiple
queues try to close it is just bizarre. That said, the alternatives are
rather more complex - essentially you'd end up having close be a join
pattern and thus the "real" close happens only after all go routines
that use the channel for sending have stopped sending.It thus seems risky to use "close" at all except for very simple
point-to-point patterns.
Matthew Sackman
> But at that point you have no idea how many values were sent that the
> receiver did not see. I understand that in your program it does not
> matter, but that is not the general case. You are accepting and
> ignoring the race condition, but that doesn't mean that the language
> should encourage that approach in general.
Right, but some languages do. For example Erlang. Once a process
receives an EXIT signal (which you could argue is _similar_ to closing
its mailbox), the process will terminate. Depending on some settings of
the process itself, the termination may occur as soon as the EXIT signal
is sent (i.e. there could be lots of msgs in the process's mailbox in
front of the EXIT signal that never get looked at), or one the EXIT
signal makes it to the front of the mailbox (i.e. there could be lots of
msgs in the process's mailbox behind the EXIT signal that never get
looked at).
Obviously, no one solution is "right", but it might be worthwhile
looking at some of the documentation to ensure people don't expect
semantics they're used to from elsewhere to be supported by Go.
> The close function is normally used to indicate that the end of a stream
> of data of indeterminate length. There's no need to use close here, so
> there is no need to use closeOnce. Just send a value on the channel.
> That is enough for the sender's select to see that the receivers are
> done. It doesn't matter if more than "done" message is sent.
Which is unfortunate as you then have to ensure the "done" message is
not in the domain of expected non-done values.
> Perhaps it would help to consider that the close function was introduced
> for one purpose: to permit range over a channel to work. Without a
> general signaling mechanism to indicate that there is no more data,
> there would be no way to range over a channel. If you need some other
> communication approach, then you shouldn't be using close.
In light of the above point about using some alternative "done" message,
it would be very useful if range could be parameterised by what it
should consider the "close" message to be.
Matthew
Matthew Sackman
> In such scenarios it's not risky, it's probably a design error of the
> program. If there are multiple senders to s single channel *and* closing
> the channel should signal an event/condition to the receiving side, then
> those senders must coordinate between themselves (otherwise some of them
> might try to send to a closed channel, which probably doesn't makes sense).
> If they coordinate then there is a place/time where a single close of the
> channel can/should happen.
If, as I suggested, close was actually a join pattern, then this would
be safe. But I'm not suggesting I think that should happen - the
additional complexity smells of a kludge solution to the problem to me.
Matthew
Russ Cox
> Could someone explain to me the design considerations of having a send
> or close on a closed channel cause a panic?
If you close the channel, you are asserting that no more
goroutines are going to send on it. If that assertion is false,
there is usually a serious logic error in your program,
as serious as indexing an array outside its bounds.
Note that you don't have to close channels at all.
Russ
Erwin
Which is unfortunate as you then have to ensure the "done" message is
> The close function is normally used to indicate that the end of a stream
> of data of indeterminate length. There's no need to use close here, so
> there is no need to use closeOnce. Just send a value on the channel.
> That is enough for the sender's select to see that the receivers are
> done. It doesn't matter if more than "done" message is sent.
not in the domain of expected non-done values.
In light of the above point about using some alternative "done" message,
> Perhaps it would help to consider that the close function was introduced
> for one purpose: to permit range over a channel to work. Without a
> general signaling mechanism to indicate that there is no more data,
> there would be no way to range over a channel. If you need some other
> communication approach, then you shouldn't be using close.
it would be very useful if range could be parameterised by what it
should consider the "close" message to be.
Uriel
<r.w.jo...@gmail.com> wrote:
> I'd like to second the suggestion that it should be possible to close
> an already closed channel. I find as a general rule that resource
> clean-up operations that are idempotent greatly simplify coding,
But close() on a channel is **not** a "resource clean-up operation"!
Calling close() on a channel is better understood as sending a special
value that says "nothing more is coming down this channel".
This is one of the several fundamental misunderstandings about close()
that cause all this trouble and confusion.
Renaming close() to something else might help avoid this.
uriel
Dave Cheney
I don't think it would improve the situation, and just raise
additional questions on why the method isn't called close anymore.
Robert Johnstone
While close is not necessary to release the memory associated with its
internal structures, close is definitely a clean-up operation. There
will be a go routine on the receiving end of the channel. Do you let
them that routine hang? If there is another communication channel to
coordinate shutting down both routines, then close is unnecessary.
However, where the call to close is required, it will be part of the
clean-up operations of the routine.
Memory is not the only resource.
Robert
On Nov 3, 7:21 pm, Uriel <ur...@berlinblue.org> wrote:
> On Thu, Nov 3, 2011 at 4:24 PM, Robert Johnstone
Jan Mercl
Hello,
While close is not necessary to release the memory associated with its
internal structures, close is definitely a clean-up operation.
There
will be a go routine on the receiving end of the channel. Do you let
them that routine hang?
If there is another communication channel to
coordinate shutting down both routines, then close is unnecessary.
However, where the call to close is required, it will be part of the
clean-up operations of the routine.
Uriel
<r.w.jo...@gmail.com> wrote:
> Hello,
>
> While close is not necessary to release the memory associated with its
> internal structures, close is definitely a clean-up operation.
No it is not. And that despite having this explained over and over
again, people still don't get it, shows the name is deeply confusing
for many.
Remember, as far as I know the only reason close() was added to the
language is to make range loops work on channels.
Limbo and other CSP-based languages with channels do not have close()
and work just fine.
uriel
Russ Cox
> There will be a go routine on the receiving end of the channel.
Only if your communication protocol is for the receiver
to wait for a close. It is equally common to have protocols
where the receiver reads a fixed number of messages
(most often 1) or reads until it receives a sentinel like a nil.
As long as receiver and sender agree that they're done,
close is unnecessary.
It is definitely true that if you are using a range loop and
the loop is depending on the channel being closed to
know to break the loop, then yes, you should call close.
But that is not always the situation.
Russ
Robert Johnstone
It may not always be the situation, but *it is* sometimes the
situation. This is an idiom supported in Go. Use a range expression
in the receiver and use close in the sender. In this particular (I
thought idiomatic) case, the call to close is part up of the necessary
operations to terminate the channel.
Perhaps the real confusion is that channels are used in different
scenarios. It seams strange to me to argue that close has nothing to
do with clean up by referring to a scenario where the call to close is
unnecessary because the usage pattern is different.
Robert
Rob 'Commander' Pike
recovering the resources; its purpose is to provide the receiver with
the information that no more elements will be sent. If there's no need
to tell the receiver that, and there usually isn't, there's no need to
call close.
-rob
Robert Johnstone
Thank-you for replying. I expect that you are very busy with all the
traffic in this forum. However, the point I was trying to make (and
failing) had absolutely nothing to do with releasing memory. I won't
bother reiterating, since you can easily read my earlier comments if
interested. Unfortunately, those comments generated significantly
more heat than light.
Robert
Kyle Lemons
Thank-you for replying. I expect that you are very busy with all the
traffic in this forum. However, the point I was trying to make (and
failing) had absolutely nothing to do with releasing memory. I won't
bother reiterating, since you can easily read my earlier comments if
interested. Unfortunately, those comments generated significantly
more heat than light.
Uriel
<r.w.jo...@gmail.com> wrote:
> Hello,
>
> Thank-you for replying. I expect that you are very busy with all the
> traffic in this forum. However, the point I was trying to make (and
> failing) had absolutely nothing to do with releasing memory. I won't
> bother reiterating, since you can easily read my earlier comments if
> interested. Unfortunately, those comments generated significantly
> more heat than light.
Sorry, but comments like: "close is definitely a cleanup operation"
generate neither heat or light, but do generate needles confusion and
frustration for people who are already confused enough by the name of
close().
Such a comment is not just wrong, is worse than wrong, it reinforces a
dangerous and insidious but sadly common misunderstanding.
uriel
