Shutting down goroutines, or, cooperating through communicating doesn't solve anything?

[JT Olds]

Preface: I'm a Go newb, but need help.

Okay, so, I'm writing a little Go server application. Initially I just
had a few
different goroutines sending some data over a TCP connection. I had
some
initial stupidity.

* First, I realized that (net.Conn) Write does not guarantee your
entire buffer
got flushed to the stream. So I added a Write wrapper method that
makes sure
the entire buffer is sent.
* Next, I realized that this is no longer thread/goroutine safe, and
there are
race conditions. Fix: cooperate by communicating! I'll make a
goroutine that
is responsible for sending data. It will listen on a channel that
receives
request objects that contain both buffer data and a response channel
for
success/failure notification.
* Then, I realized I wanted to find out whether or not the connection
I'm
sending on is still alive and safely tear down everything related to
that
connection if so. So, I wrote another goroutine that occasionally
puts keep
alives on the send channel, and if a specific keep alive's response
channel
returns with a failure, closes the connection.

So, this is almost perfect, I have a bunch of goroutines that are all
now
attempting to send things on the connection by sending requests over
the send
request channel.

However, Let's say they all have nothing to do for a while, and in the
mean
time, the user kills the connection. The keepalive goroutine notices
and causes
a connection close. Then let's say the other goroutines have work
again and
attempt to send, so they send requests on the sending channel. The
sending
goroutine will send back failures, and the worker goroutines will all
stop running
due to the failures.

What is the standard approach to getting the sending goroutine to die?

Possible solutions:
* If I have the keepalive goroutine kill the sending channel, then
the sending
goroutine can notice and die, but other still running worker
goroutines might
try a send on a closed channel and panic (btw, wtf? the docs say a
send on a
closed channel will just call panic and be over).
* If I don't have the keepalive goroutine kill the sending channel,
then I
could possibly have the sending goroutine do reference counting on
how many
other goroutines still expect to be able to send to it, and shut
down if no
remaining worker goroutine wants to send? Boy that sounds
complicated.
* I could assume boolean variable mutation is atomic, and try and
implement
mutexes around some flag of whether or not I've closed the
connection in the
senders, so the senders don't even try to send on a closed
connection? That
seems like not the right Go approach.

I feel like by making a process responsible for a piece of ephemeral
data, I
have moved the problem of garbage collection to process management.
That sucks.
So, now I have this sending goroutine that I can't figure out how to
tell it to
stop.

Go seems great, as long as your process is long running and you never
care about
tearing anything down.

I don't see how cooperating with communicating fixes anything here. I
still want
a mutex.

[Gustavo Niemeyer]

> What is the standard approach to getting the sending goroutine to die?

There's no right or wrong in this case. Specific cases work best in
one way or the other, and people tend to look at the context to create
a simple solution that solves specific problem at hand.

That said, I wrote this while facing a problem related to the one you
seem to describe:

http://blog.labix.org/2011/10/09/death-of-goroutines-under-control

--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog

-- I'm not absolutely sure of anything.

[Kyle Lemons]

One possible solution would be to have a channel which is closed when the TCP send routine exits, and select on this when a sender tries to send something to be written.  The TCP send need not know all clients.

[JT Olds]

First off, whoa, formatting fail. Gmail hates the 80 char wide text, eh? I've modified this email to include a better formatted message in the reply text at the bottom.

Thanks Kyle, that sounds like the best solution that I've heard of so far. So, you're suggesting something like

func (this *Client) Send(data []byte) os.Error {

  request := new(Request)

  request.data = data

  request.response_channel = make(chan os.Error)

  select {

    case v, ok <- this.closing_channel:

      if !ok { return os.NewError("unable to send") }

      return os.NewError("unexpected closing value")

    case this.send_channel <- request:

      return <-request.response_channel

  }

  return os.NewError("unreachable")

}

and this works okay (read: doesn't panic) in all four cases where the sending_channel and the closing_channel are each either open or closed?

Thanks

-JT

[Kyle Lemons]

Comments inline

On Tue, Jan 17, 2012 at 5:40 PM, JT Olds <jto...@gmail.com> wrote:

First off, whoa, formatting fail. Gmail hates the 80 char wide text, eh? I've modified this email to include a better formatted message in the reply text at the bottom.

Thanks Kyle, that sounds like the best solution that I've heard of so far. So, you're suggesting something like

func (this *Client) Send(data []byte) os.Error {

You don't specify what a Client is, and don't call it "this" ("c" would be idiomatic).

 

  request := new(Request)

  request.data = data

  request.response_channel = make(chan os.Error)

req := &Request{data: data, response: make(chan os.Error)}

response_channel would be better named simply "response" (and also, camelCase is idiomatic).

 

  select {

    case v, ok <- this.closing_channel:

The "closing_channel" would be better named "done" or "closed" and you simply need:

select {

case <-c.closed:

  return os.EOF

case this.send <- request:

}

return <-req.response:

      if !ok { return os.NewError("unable to send") }

      return os.NewError("unexpected closing value")

    case this.send_channel <- request:

      return <-request.response_channel

  }

  return os.NewError("unreachable")

}

and this works okay (read: doesn't panic) in all four cases where the sending_channel and the closing_channel are each either open or closed?

There are only really two cases: the send completes (and the other goroutine should always send an error back) or the channel is closed (whether before or after the send is attempted). 

[JT Olds]

Thanks Kyle,

I guess I assumed that which case in a select statement executed was nondeterministic, or at least not guaranteed to be one or another, and I figured that if a send on a closed channel caused a panic, then a send inside a select on a closed channel would be no different albeit more intermittent.

-JT

[roger peppe]

the answer is that you should not close a channel if things
might still write to it.

you can arrange that whenever something tries to send some
data down a channel to the writer, it also reads on a channel
that is closed when the writer quits.

then there's no chance of a panic.

this kind of thing (written in haste!):

type writer struct {
f io.Writer
data chan<- []byte
done chan bool
}

func (w *writer) writerProc() {
for{
data := <-w.data
n, err := w.f.Write(data)
if err != nil {
close(f.done)
break
}
}
}

func (w *writer) Write(buf []byte) {
select {
case w.data <- buf:
case <-done: