Unifying exceptions and goroutine termination

[Jonathan Amsterdam]

Here is a simple idea for non-local error propagation, inspired by
Erlang's concept of monitors and by the simulation of exceptions in
Go's exception package (not yet in the golang.org release, but visible
here:
http://code.google.com/p/go/source/browse/src/pkg/exp/exception/exception.go?spec=svnaff697587839774c069bbf35bc120257d4a83865&r=aff697587839774c069bbf35bc120257d4a83865).

The basic idea is that you can associate a channel with a goroutine,
and a message will be sent to the channel when the goroutine
terminates.

--------
Any goroutine can optionally be associated with an exit channel, which
is a normal channel of *ExitInfos. ExitInfo is a struct whose contents
are to be specified, but would include at the least a reason for the
goroutine's exit, and possibly stack information in the case of an
error. To start a goroutine with an exit channel C, write

go f(), C;

We add a function runtime.GoexitWith(interface{}), so the programmer
can specify arbitrary information to be included in the ExitInfo.

If a goroutine does not have an exit channel, then the existing
semantics of the language is unchanged, and runtime.GoexitWith causes
the program to terminate. (Yes, the whole program--think of it as an
unhandled exception.)

If a goroutine does have an exit channel, then built-in errors (e.g.
array out of bounds), runtime.Goexit and runtime.GoexitWith all unwind
the stack, executing defer functions. Then an attempt is made to write
on the exit channel. If the exit channel is no longer referenced by
any other part of the program, or if it is closed, then the program
terminates. Otherwise, an *ExitInfo appropriate to the type of exit
(or nil, if the goroutine terminates normally) is sent on the channel.
If the channel send blocks, the zombie goroutine blocks. If at any
time before the value has been received, the channel becomes
unreferenced or closed, then the program terminates.
--------

Advantages of this idea:

- The existing semantics is unchanged.

- The additional syntax is small, and the additional semantics is
simple and intuitive.

- One mechanism unifies non-local propagation of user errors, handling
of built-in exceptions, and notification of goroutine termination in
the presence of runtime.Goexit (and, when distributed, in the presence
of network or remote node failure).

- Improves the simulation of exceptions in package exceptions
mentioned above: now the handler can be a global function instead of
having to be passed along, and certain errors, like holding the
handler after it has outlived its Try or having a sub-goroutine call
it, are no longer possible.

- There is a wealth of experience with this mechanism in the Erlang
community. In fact, Erlang folks will tell you that the real power of
Erlang lies in the supervisor patterns they have built that rely on
this ability to cleanly and reliably detect process (= goroutine)
termination.

The main disadvantage is that, despite my efforts to legislate
otherwise, it is easy to ignore errors. You can't create an exit
channel and throw it on the floor -- my spec handles that case -- but
you can create an exit channel and never receive from it, and you will
accumulate plenty of zombie goroutines and as many unhandled errors.
And unfortunately, it is harder to see that problem by inspection of
the source than it is to see try { ... } catch () {}.

[smosher]

While I like the idea of taking cues from Erlang here, I'm still
divided on the idea of using channels. There are two reasons. One is
the fact that existing semantics cannot go unchanged. IIRC you cannot
select on a collection (array/slice, map) of channels so it is
difficult to supervise an arbitrary number of children without
actually polling the whole collection, busily. (Hopefully this will
change of course, it's needed for other things.)

The other is that I'm not convinced that channels are the correct way
to handle it. When the goroutine goes abnormal, the supervisor will
have to explicitly check the channel before the fact will be observed.
If the supervisor has other things to do, it could clutter up the code
at the very least. Essentially, I'm concerned that a lot of the
benefit of Erlang's approach goes missing.

What I do like cannot be understated: the model here is the goroutine
as the fallible entity, not an arbitrary code block. The handler is
part of a separate entity, and this is a step in the right direction.

On Nov 23, 5:23 pm, Jonathan Amsterdam <jbamster...@gmail.com> wrote:
> Here is a simple idea for non-local error propagation, inspired by
> Erlang's concept of monitors and by the simulation of exceptions in
> Go's exception package (not yet in the golang.org release, but visible

> here:http://code.google.com/p/go/source/browse/src/pkg/exp/exception/excep...).

[Ben Tilly]

On Mon, Nov 23, 2009 at 3:48 PM, smosher <dark.n...@gmail.com> wrote:
> While I like the idea of taking cues from Erlang here, I'm still
> divided on the idea of using channels. There are two reasons. One is
> the fact that existing semantics cannot go unchanged. IIRC you cannot
> select on a collection (array/slice, map) of channels so it is
> difficult to supervise an arbitrary number of children without
> actually polling the whole collection, busily. (Hopefully this will
> change of course, it's needed for other things.)

[...]

No need for change. You can already achieve the same effect as
selecting on a collection.

What you do is take a collection of channels, and spawn a collection
of goroutines, one per channel, that each reads from one channel and
echos it to another. The trick is that all echo to the SAME channel.

So if the original channels all have type (chan T), the combined
channel might have a type like (struct {id int, value T}), and then
every time you read from it you'll know which channel the message came
from, and what the message is. And now every time you read from the
combined channel you get the same effect as selecting from the whole
list.

(I only know this because I didn't see how to do it and one of the
core developers told me the trick.)

Cheers,
Ben

[smosher]

I am well aware of that solution, but it's a tradeoff between two
terrible situations. It is unacceptable to have to spawn additional
processes to handle a language-supported feature (the proposed
exception model) just because the semantics of select still falls just
a tad short. I had presented the other solution because it would make
sense in the context of a supervisor that had it's own work-loop or
what have you (which is presented for the purposes of illustrating the
clumsiness of using channels for this purpose.)

On Nov 23, 7:01 pm, Ben Tilly <bti...@gmail.com> wrote:

[Ben Tilly]

Please don't top post. It makes the flow of conversation much harder
to read. (Let me quickly rearrange.)

On Mon, Nov 23, 2009 at 4:16 PM, smosher <dark.n...@gmail.com> wrote:
> On Nov 23, 7:01 pm, Ben Tilly <bti...@gmail.com> wrote:
>> On Mon, Nov 23, 2009 at 3:48 PM, smosher <dark.nowh...@gmail.com> wrote:
[...]
>> No need for change.  You can already achieve the same effect as
>> selecting on a collection.
>>
>> What you do is take a collection of channels, and spawn a collection
>> of goroutines, one per channel, that each reads from one channel and
>> echos it to another.  The trick is that all echo to the SAME channel.

>> [...]

>
> I am well aware of that solution, but it's a tradeoff between two
> terrible situations. It is unacceptable to have to spawn additional
> processes to handle a language-supported feature (the proposed
> exception model) just because the semantics of select still falls just
> a tad short. I had presented the other solution because it would make
> sense in the context of a supervisor that had it's own work-loop or
> what have you (which is presented for the purposes of illustrating the
> clumsiness of using channels for this purpose.)

Would you prefer that select become a O(n) operation? Because that's
what the language designers claimed that the trade-off was.
Personally I think goroutines are cheap enough that I prefer having to
spawn extra goroutines rather than making select slower.

Cheers,
Ben

[smosher]

> Would you prefer that select become a O(n) operation?  Because that's
> what the language designers claimed that the trade-off was.
> Personally I think goroutines are cheap enough that I prefer having to
> spawn extra goroutines rather than making select slower.

I would prefer to have the option to perform a kind of select that
does that, yes. If I can't do it optionally, i.e. all or nothing, I'll
take all--so still yes. (Otherwise I'll rarely if ever write select at
all.) Those goroutines are certainly not free, and for something that
targets ARM, wasting 3k or whatever it is per call is hilarious. I
wonder what's cheaper, spawning n goroutines plus a select, or an O(n)
select?

The alternatives to a convenient means of selecting over a collection
are: to clutter up code, add overhead and, should exceptions have
anything to do with channels, invite programmers to ignore them. (This
last one doesn't bother me that much; letting the whole ship sink is
better than standardizing on bad practice you know.)

[Ryanne Dolan]

The 3k stack is a fairly arbitrary implementation detail I think, not so much the language semantics. It is very likely that a better compiler could eliminate the current goroutine overhead in the case where goroutines are used just to block and don't need a stack.  That doesn't mean a new language primitive is needed.

- from my phone -

On Nov 23, 2009 7:02 PM, "smosher" <dark.n...@gmail.com> wrote:

> Would you prefer that select become a O(n) operation?  Because that's > what the language designer...

[Russ Cox]

> The alternatives to a convenient means of selecting over a collection
> are: to clutter up code, add overhead and, should exceptions have
> anything to do with channels, invite programmers to ignore them. (This
> last one doesn't bother me that much; letting the whole ship sink is
> better than standardizing on bad practice you know.)

This isn't fair. You've set up a false choice between an O(n)-select
and n goroutines, but the right way is not to create those n channels
in the first place. Instead, arrange for those n writers to be writing
on a single channel. If you look at the Go method in the rpc package
(http://golang.org/pkg/rpc/#Client.Go) it takes a channel as an argument
for precisely this reason: if you want to wait for n RPCs, you just kick
them all off with the same done channel and use an ordinary receive
instead of an n-way select.

Russ

[smosher]

On Nov 23, 8:50 pm, Russ Cox <r...@golang.org> wrote:
> This isn't fair.  You've set up a false choice between an O(n)-select
> and n goroutines, but the right way is not to create those n channels
> in the first place.  Instead, arrange for those n writers to be writing
> on a single channel.  If you look at the Go method in the rpc package
> (http://golang.org/pkg/rpc/#Client.Go) it takes a channel as an argument
> for precisely this reason: if you want to wait for n RPCs, you just kick
> them all off with the same done channel and use an ordinary receive
> instead of an n-way select.

You're completely right, it's not fair. I brought it up to point out
how bad it is to go about launching goroutines as a solution to this
particular problem, so it was never really fair to begin with. I still
think that there are cases where I'll want to select over a slice,
particularly given the tendency (my observations) for people to prefer
returning channels over accepting them as arguments. (Perhaps
allocating collections of channels should be discouraged, but there
should also be a reasonable solution to this problem--it's likely to
arise with the way things are going. OTOH, I do like the idea of using
channels as map keys, but so far I've only had to send from them in
the hash in that scenario.) I want to be clear here: I presented no
false choice, the choice was presented to me and I shot it down. (I
realize that we could go back and correct this in the original post by
not assigning to C, but pre-allocating and using C as the exceptions
channel and things start looking nicer.)

On the other hand, I'm surprised this has gotten so far off track, it
seems more important (to me) to really consider whether channels are
truly the right medium. I have spent some time building a demo
(unpublished) using channels, just to test the behavior. My
conclusions were that channels muddy up the concept of exception
handling somewhat, and add clutter to the code (not as much as
encapsulating in blocks, but still more than necessary.) I'm still
fleshing out my work on this.

The problem with relating channels to Erlang messages is that channels
fall woefully short as replacements for Erlang messages. This should
be fairly obvious, and I really think we lose some power in modeling
exceptions this way as a result. On the other hand, it may be useful
for a non-Erlang approach, or it may "suffice", or maybe something
else should be done instead. (This isn't to say that I find channels
disappointing, but it's undeniable that they are more restrictive (on
the reading end) than Erlang messaging. They're novel like a new
hammer, and "the exceptions problem" looks like a nail perhaps because
people with hammers try to fit everything they see into the "nails"
category.)

Another problem with this comparison doesn't relate to capabilities
but to practice. In Erlang you will commonly match on messages at the
top of your routine and recurse, with state changes implemented by
calling different functions. Go development doesn't seem too
interested in making any kind of constant-space recursion guarantees
so this pattern is out (for now anyway.) The result is that
transitions are handled differently, which to me really changes the
criteria for elegance.

[roger peppe]

2009/11/24 smosher <dark.n...@gmail.com>:

> The problem with relating channels to Erlang messages is that channels
> fall woefully short as replacements for Erlang messages.

actually, i think this could be said the other way around too.

i haven't used erlang, but it's my understanding that erlang
channels are not first class objects.

go's are, which opens up all kinds of interesting possibilities.
it can take a while to work out the useful idioms though.

[Jonathan Amsterdam]

> I'm surprised this has gotten so far off track...

Thank you for bringing it back!

> (I realize that we could go back and correct this in the original post by
> not assigning to C, but pre-allocating and using C as the exceptions
> channel and things start looking nicer.)

That was my intent:

C := new(chan *ExitInfo);
go f(args), C;

The second position in a go statement is an ordinary, evaluated
position, not something magic that creates and assigns a channel. So
you can use the same channel for all your goroutines. But you'd still
have the problem of distinguishing them. The child could pass
identifying info in the argument to runtime.GoexitWith, but that
wouldn't help for built-in errors. So ultimately I think you would
need a channel per goroutine, with a merge. However, I think both your
concerns about this idiom are unfounded. As Ryanne Dolan pointed out,
a little goroutine like this can be created in a handful of bytes. And
as for clutter, this is why we have abstraction. You'd expect to see
functions like the following popping up in libraries, just as Erlang
has standard patterns that are captured in code:

--------------------
// Handle a channel of requests by spawning a goroutine for each one.
Each Request contains
// a reply channel. If a request goroutine exits non-normally,
exitHandler is called with the Request that
// failed and the ExitInfo of the failed goroutine.
// UNCOMPILED, UNTESTED
func perRequestServer(in <-chan *Request,
requestHandler func(*Request),
exitHandler func(*Request, *ExitInfo)) {

type Pair struct {
req *Request;
einfo *ExitInfo;
}

exits := make(chan Pair);

for { // main loop
select {
case req := <-in:
if req == nil && closed(in) {
close(exits);
break
}
c := make(chan *ExitInfo);
go requestHandler(req), c;
go func() { // merge exit channels
ei := <-c;
if ei != nil { exits <- Pair{req, ei} }
}();

case p := <-exits:
exitHandler(p.req, p.einfo);
}
}
// Drain exits.
for !closed(exits) {
if p := <-exits; p.einfo != nil {
exitHandler(p.req, p.einfo)
}
}
}
--------------------------------

Sure, that's ugly, but you only have to write it once (assuming
Request is an interface, or you have generics).

> Another problem with this comparison doesn't relate to capabilities
> but to practice. In Erlang you will commonly match on messages at the
> top of your routine and recurse, with state changes implemented by
> calling different functions. Go development doesn't seem too
> interested in making any kind of constant-space recursion guarantees
> so this pattern is out (for now anyway.) The result is that
> transitions are handled differently, which to me really changes the
> criteria for elegance.

In Go, you'd write a loop with a select inside. That's a bit less
expressive, to be sure, but I don't think it makes a huge difference.

[Mike Zraly]

On Tue, Nov 24, 2009 at 10:40 AM, Jonathan Amsterdam <jbams...@gmail.com> wrote:

> I'm surprised this has gotten so far off track...

Thank you for bringing it back!

>  (I realize that we could go back and correct this in the original post by
> not assigning to C, but pre-allocating and using C as the exceptions
> channel and things start looking nicer.)

That was my intent:

   C := new(chan *ExitInfo);
   go f(args), C;

The second position in a go statement is an ordinary, evaluated
position, not something magic that creates and assigns a channel. So
you can use the same channel for all your goroutines. But you'd still
have the problem of distinguishing them. The child could pass
identifying info in the argument to runtime.GoexitWith, but that
wouldn't help for built-in errors. So ultimately I think you would

need a channel per goroutine, with a merge. [...]

Why not simply accept a 3rd argument optional for the identifying expression?
Pardon any syntax errors in the code below:

c := some common channel for reporting
n := 10

// launch n goroutines
for i := 0; i < n; i++ {
    go f(args), c, i
}

// wait for them all to complete, get exit status and identifying data
for i := 0; i < n; i++ {
    exitStatus, id <- c
    // report any goroutines that terminated abnormally
    ...
}

[Jonathan Amsterdam]

On Nov 24, 5:51 am, roger peppe <rogpe...@gmail.com> wrote:
> 2009/11/24 smosher <dark.nowh...@gmail.com>:

But in Erlang, processes are first-class, whereas Go goroutines are
not.

I think there are simple translations:

To build a channel in Erlang, you'd write a process that accepted send
and receive messages. The pid of that process would act like a Go
channel.

To make a goroutine that behaved like an Erlang process, you'd write
something like:

mailbox := make(chan interface{}, 1000);
go func() {
for {
msg := <-mailbox;
switch ...
}
}();

The mailbox channel plays the role of an Erlang pid.

These translations aren't exact (e.g. Erlang mailboxes are unbounded),
but they capture the gist.

The typing regime actually plays an important role in which mechanism
to prefer: since Go is statically typed and does not have algebraic
types (think safe unions) or pattern matching, typed channels make
more sense. Erlang is dynamically typed with powerful pattern
matching, so it's not at all uncomfortable to have a single mailbox
that accepts messages of all different types.

The more important difference between the languages with respect to my
proposal is that in Erlang, if a process's recieve statement doesn't
explicitly have a pattern that matches the {'EXIT', ...}, it will
crash on receiving exit, whereas in Go, you can simply not read from
the exit channel. So it is easier to inspect Erlang code for correct
handling of exits than it is to inspect Go code.

[Jonathan Amsterdam]

> Why not simply accept a 3rd argument optional for the identifying
> expression?
> Pardon any syntax errors in the code below:
>
> c := some common channel for reporting
> n := 10
>
> // launch n goroutines
> for i := 0; i < n; i++ {
>     go f(args), c, i
> }

Sure. But the language is minimalist in spirit: there are very few
features that are present merely for convenience (the := syntax being
the whopping exception, but the benefits there are huge). My proposal
was in that spirit: it is doing something you cannot do any other way.
There is another way to do the identifying expression.

[roger peppe]

2009/11/24 Jonathan Amsterdam <jbams...@gmail.com>:

> The more important difference between the languages with respect to my
> proposal is that in Erlang, if a process's recieve statement doesn't
> explicitly have a pattern that matches the {'EXIT', ...}, it will
> crash on receiving exit, whereas in Go, you can simply not read from
> the exit channel. So it is easier to inspect Erlang code for correct
> handling of exits than it is to inspect Go code.

it seems that it's common for erlang processes to have a single
inner loop, reading and processing messages.

while that idiom isn't unusual using go-style channels,
it's also common to just write channel-receives as
part of the control flow of the process.

for instance, a classic mouse tracking routine:

m: Mousestate;
for {
m := <-mouse;
if(m.buttons != 0){
break;
}
for m.buttons != 0 {
trackmouse(m.point);
m = <-mouse;
}

and of course, the tracking could be handed off
to another goroutine as desired, without the sender
of the events being any the wiser.

this is one of the nicest things about using channels - the
divorcing of control flow between sender and receiver:
each thinks it is the master! it's rare to have to write
any kind of state machine.

but i'm sure erlang has its own idioms for this kind of thing.

FWIW, i like the specify-channel-with-go-statement idea.
i also like the idea of specifying an extra value as a local
identifier - but i think that just sending a process id
(and providing access to a process's id) is probably
simpler, and saves two pointers' worth of space per routine.

[smosher]

On Nov 24, 10:40 am, Jonathan Amsterdam <jbamster...@gmail.com> wrote:
> >  (I realize that we could go back and correct this in the original post by
> > not assigning to C, but pre-allocating and using C as the exceptions
> > channel and things start looking nicer.)
>
> That was my intent:

Oh, I have gotten the wrong impression somehow.

> The second position in a go statement is an ordinary, evaluated
> position, not something magic that creates and assigns a channel. So
> you can use the same channel for all your goroutines. But you'd still
> have the problem of distinguishing them. The child could pass
> identifying info in the argument to runtime.GoexitWith, but that
> wouldn't help for built-in errors. So ultimately I think you would
> need a channel per goroutine, with a merge.

I don't see why this is necessary. If the runtime is going to start
supporting the exceptions model, it should go all the way. Personally,
I would go as far as using a specified type with identifying
information (perhaps including a member of interface{} type) about the
goroutine, since this is not Erlang and we do not have pattern
matching. For now I'm assuming this "identifying information" to be a
process id which is returned from "go ..." statements, but on the
other hand there's nothing conceptually preventing the identifier from
being provided to the goroutine just as the channel is.

> However, I think both your
> concerns about this idiom are unfounded. As Ryanne Dolan pointed out,
> a little goroutine like this can be created in a handful of bytes. And
> as for clutter, this is why we have abstraction.

I still find this unacceptable. This could double the stack used by
supervisors in the worst case, and all for such a simple thing.
Besides, what if my embedded device runs out of memory because a
goroutine used to handle all this channeling couldn't allocate stack.
Anyone running into that problem will find that their team forbids
that practice going forward.

Besides, even on a large system it's just plain wasteful. It's not
insignificant if you are running service processes which are almost as
simple as their channeling helpers, especially if you're running a lot
of them.

> Sure, that's ugly, but you only have to write it once (assuming
> Request is an interface, or you have generics).

If you were doing it this way why not just send the handler to the
goroutine instead of a channel? It's more direct and accomplishes the
same thing, i.e. contingency code. What it doesn't do (without a whole
lot more work) is propagate the error back to the installing caller,
which is something I think people want from exceptions. (On the other
hand, changing the game is something I would not be opposed to.)

> In Go, you'd write a loop with a select inside. That's a bit less
> expressive, to be sure, but I don't think it makes a huge difference.

I left that one open. I'm concerned that the equivalent in Go will be
ugly for many cases. I think it's something to think about.

[Mike Zraly]

Yes you're right -- I can call go f(args), c within an outer goroutine that
reads the exit status into a dedicated exit status channel, then reports
that exit status with arbitrary other data (not jsut a goroutine id) to a
common channel.  No need for new syntax here.

This would be easier if the go keyword could be followed by a code block
instead of just a function or method call, or even in place of a function or
method call, but that's not a huge burden at all.

[Jonathan Amsterdam]

> For now I'm assuming this "identifying information" to be a
> process id which is returned from "go ..." statements, but on the
> other hand there's nothing conceptually preventing the identifier from
> being provided to the goroutine just as the channel is.

Yes, if they made a goroutine unique ID visible, then that would get
put into the ExitInfo, and problem solved.

> If you were doing it this way why not just send the handler to the
> goroutine instead of a channel?

In some ways, I really like this idea. It's much easier to see by
inspection that you're not ignoring the error, and if you want to
simulate my idea, just do

exits := make(chan *ExitInfo);
go f(args), func(ei *ExitInfo) { exits <- ei };

On the downside, you now have asynchronous inter-thread communication,
which is very error-prone. The handler function is closed over your
variables, but can be called any time from another thread -- exactly
the kind of shared-memory pattern that the Go designers are
recommending be replaced by channels.

[Jonathan Amsterdam]

> This would be easier if the go keyword could be followed by a code block
> instead of just a function or method call, or even in place of a function or
> method call, but that's not a huge burden at all.

Because Go has full closures, you can write

go func() {
...whatever you would have put into the block ....
}();

A little noisier than a simple block, but not too bad.

[smosher]

On Nov 24, 3:14 pm, Jonathan Amsterdam <jbamster...@gmail.com> wrote:
> In some ways, I really like this idea. It's much easier to see by
> inspection that you're not ignoring the error, and if you want to
> simulate my idea, just do
>
>     exits := make(chan *ExitInfo);
>     go f(args), func(ei *ExitInfo) { exits <- ei };
>
> On the downside, you now have asynchronous inter-thread communication,
> which is very error-prone. The handler function is closed over your
> variables, but can be called any time from another thread -- exactly
> the kind of shared-memory pattern that the Go designers are
> recommending be replaced by channels.

It doesn't seem to be functionally different than handling the errors
inside a perRequestServer() type goroutine, though there is less
overhead and less clutter. The closure problem can occur in both
cases, but only if you're not careful.

I still don't like it. All of the workable fixes I've come up with so
far could fill pages, and they're all bad in one way or another. I'm
avoiding the wish for messages and pattern matching since that would
just confuse with existing features (to say nothing of implementation
details), but it's really hard to dream up the missing piece to make
everything happy.

[Jonathan Amsterdam]

> It doesn't seem to be functionally different than handling the errors
> inside a perRequestServer() type goroutine, though there is less
> overhead and less clutter. The closure problem can occur in both
> cases, but only if you're not careful.

In my code, the calls to exitHandler happen in the same goroutine that
called perRequestServer. In your idea, the calls happen in another
thread.