This proposal differs from the usual model of exceptions as a control structure, but that is a deliberate decision.  We don't want to encourage the conflation of errors and exceptions that occur in languages such as Java.

Instead, this proposal uses a slight change to the definition of defer and a couple of runtime functions to provide a clean mechanism for handling truly exceptional conditions.

Another proposal may happen regarding handling/monitoring exceptions in goroutine A from the safety of goroutine B, but that is not part of this proposal and may not happen at all, although we believe it may be achievable entirely by library code given this proposal.

Proposal
-----

First, we tweak the definition of defer.  At the moment, it's defined to run deferred functions after the result values are copied back to the caller's stack. We change the definition slightly to say they run after the return values are evaluated but before they are copied to the caller's stack.  This makes the output variables available to deferred closures.  Thus a deferred closure can modify the return values of the caller of the deferring function.

The function

        func numDefer() (n int) {
                for i := 0; i < 100; i++ {
                        defer func() {
                                n++
                        }()
                }
                return
        }

will return 100.   With the existing spec, it would return 0, if it weren't also illegal (the closure was not allowed to refer to n at all due to the prohibition on taking the address of return values; that has been lifted too).

[This much is now checked in to the spec.]

Next we define a pair of predefined functions:

        func panic(v interface{})
        func recover() (v interface{})

[This is a redefinition of panic, and we delete panicln altogether.]

The function panic begins a process called panicking.  Panicking unwinds the deferred calls, executing each in turn.  In other words, each function on the stack, including the one that calls panic, is terminated, in reverse order, with its deferred calls executing as they terminate.

During panicking, if a deferred function invocation calls recover, recover returns the value passed to panic and stops the panicking.  At any other time, or inside functions called by the deferred call, recover returns nil.  After stopping the panicking, a deferred call can panic with a new argument, or the same one, to continue panicking.  Alternately, the deferred call might edit the return values for its outer function, perhaps returning an error.

Here is a simple example.

        func f(dopanic bool) {
                defer func() {
                        if x := recover(); x != nil {
                                fmt.Printf("panicking with value %v\n", v)
                                panic(x)  // go back to panicking
                        }
                        fmt.Printf("function returns normally")
                }()
                fmt.Printf("before")
                p(dopanic)
                fmt.Printf("after")
        }

        func p(dopanic bool) {
                if dopanic {
                        panic(3)
                }
        }

If the function f is called with dopanic=true, it will print

        before
        panicking with value 3

and generate a stack dump showing that f called function p which called panic. Otherwise, it will print

        before
        after
        function returns normally

and return without crashing.

If we remove the call to panic in f, and call it with dopanic=true, it will print

        before
        panicking with value 3
        function returns normally

and not generate a stack trace.

Since a deferred function may modify named return parameters, a function may recover from a panic and signal that it has done so through its return value.  For instance, given

        func f(dopanic bool) (err os.Error) {
                defer func() {
                        if x := recover(); x != nil {
                                err = x.(os.Error)
                        }
                }()
                q(dopanic)
                return
        }

        func q(dopanic bool) {  
                if dopanic {
                        panic(os.ErrorString("bad karma"))
                }

        }

If dopanic is true, f will return the error "bad karma"; otherwise it will return nil.

As mentioned above, the recover function can return non-nil only when it is executed directly by a deferred function while panicking.  This allows panics to stack and avoids a problem in cases such as the one illustrated here:

        func foo() {
                defer func() {
                        if bar() != nil {
                                os.Exit(0)
                        }
                }()
                panic(3)
        }

        func bar() (err os.Error) {
                defer func() {
                        if x := recover(); x != nil {
                                fmt.Printf("bar is panicking")
                                err = x.(os.Error)
                        }
                }()
                if horribleNewProblemHappens() {
                        panic("disaster")
                }
                return nil
        }

Here we see foo call panic, which then calls bar during the deferred function.  We do not want bar to act as if it has triggered a panic itself; it should only return an error if there has been a horrible new problem.  This is the reason that recover can return non-nil only when executed directly in a deferred closure while the stack is unwinding and not while functions are called in defer blocks.

Implementation of defer, panic, and recover in gc
-----

Defer
-----

This is gc's implementation of the original defer (before any of the recent spec changes):

Each goroutine has a deferred call frame stack on which deferred calls are pushed.
Each entry records:

        SP at time of defer, aka SP of func that did the defer
        PC of function to call
        raw frame holding function arguments

A function that contains a defer statement ends with a special epilogue.  Instead of
the normal

        ADDQ $framesize, SP
        RET

it ends with

        CALL rundefer(SB)
        ADDQ $framesize, SP
        RET

Rundefer processes each entry on the top of the deferred call frame stack
that matches the current SP, writing each to the runtime stack and then
running that function.

If a deferred call inspects the call stack, it looks like it was called directly from f.

Allowing function return values to be addressed or stored in closures
means copying them to the heap on function entry (just like addressed
parameters) and then copying them back to the caller before the RET.
The copy back to the caller must happen after the call to rundefer.
That is the only change to support the editing of return values.

Panic and recover
-----

Each goroutine has a panic stack on which values passed to panic are pushed.
Each entry records:

        SP at time of panic
        value passed to panic

To panic, a goroutine starts processing entries on the defer stack, one at a time,
just as rundefer would.  Recover consults the panic stack: if the caller's SP matches
the SP of the entry atop the stack, recover pops the entry off the stack and returns
the value from that entry.  Otherwise recover returns nil.

The processing of a particular panic stops when a deferred call pops the
corresponding entry off the panic stack and does not replace it (by invoking panic).
When that happens, the original call stack is unwound to the SP associated with
the deferred call that stopped the panic, making it appear that the function that
executed the corresponding defer statement is returning.  (This might in turn
cause the execution of other deferred code, in a non-panicking context.)

The processing of a panic also stops when the deferred call stack has emptied.
If the latter happens, the program crashes, printing the value passed to panic
and then the full stack trace at the time the panic started.

Recursive panics
-----

If a child of a deferred call panics, or if a deferred call panics without first calling
recover, a new panic begins that applies to the current call stack, including the
deferred call that was handling the first panic.  Thus:

        func f() {
                defer func() {
                        panic("second")
                        print("not reached")
                }
                panic("first")
        }

panics twice: the "first" panic runs the deferred func, which causes the "second"
panic.  The second panic aborts execution of the code in which it appears,
like panic always does, so the print is not reached.  In this program, the second
panic will empty the deferred call stack, causing the program to crash.
When it does, it prints the stacked panic values, oldest first, and then the stack trace.

Stack traces
-----

runtime.Caller works inside deferred code just as it does inside normal code.
During normal execution, the caller of deferred code is the function that
executed the defer statement.  During a panic, the caller of deferred code
is panic itself.  During a recursive panic, earlier panics are visible on the
call stack.  For example, this program:

        package main

        import (
                "fmt"
                "runtime"
                "strings"
        )

        // Print names of functions on call stack.
        func stack(msg string) {
                fmt.Print(msg, ":")
                var stk [10]uintptr
                n := runtime.Callers(2, stk[0:])
                for _, pc := range stk[0:n] {
                        s := runtime.FuncForPC(pc).Name()
                        if i := strings.Index(s, "."); i >= 0 {    // cut package name
                                s = s[i+1:]
                        }
                        fmt.Print(" ", s)
                        if s == "main" {
                                break
                        }
                }
                fmt.Println()
        }

        func main() {
                f()
        }

        func f() {
                stack("in f")
                defer g()
                panic("first")
        }

        func g() {
                stack("in g")
                defer h()
                panic("second")
        }

        func h() {
                stack("in h")
                panic("third")
        }

prints

        in f: f main
        in g: g panic f main
        in h: h panic g panic f main

and then the program crashes, printing

        panic: first
        panic: second
        panic: third

        main.h+0x3e /home/rsc/x.go:35
                main.h()
        runtime.panic+0x45 /home/rsc/g/go/src/pkg/runtime/runtime.c:25
                runtime.panic()
        main.g+0x36 /home/rsc/x.go:29
                main.g()
        runtime.panic+0x45 /home/rsc/g/go/src/pkg/runtime/runtime.c:25
                runtime.panic()
        main.f+0x36 /home/rsc/x.go:23
                main.f()
        main.main+0x1d /home/rsc/x.go:18
                main.main()
        mainstart+0xf /home/rsc/g/go/src/pkg/runtime/386/asm.s:83
                mainstart()

Note that the panics are printed oldest first, but the stack trace is, as always,
newest frame first.

Being able to print the stack trace after running the deferred calls does not
require expensive operations like copying the stack when a panic happens:
all the deferred calls run like ordinary function calls, leaving the original
state at the time of panic intact.  The unwinding of caller frames only happens
when a deferred call stops the panic, if at all.

nice.

will this mechanism enable handling of runtime.Goexit?

I like it.

Do defer statements work inside of anonymous functions? In other
words, can we simulate a try/catch block by doing something like

func foo() {
        doSomethingHere()
        func() {
                defer func() {
                        if x := recover(); x != nil {
                                fmt.Printf("panic caught with value %v\n", x)
                        }
                }()
                doSomethingThatCouldPanic()
        }()
        fmt.Println("finished foo")

If so, we could even write a function to do  the catching for us, as in

func catch(f func()) (v interface{}) {
        defer func() {
                if x := recover(); x != nil {
                        v = x
                }
        }()
        f()

This would then be used to rewrite foo like

func foo() {
        doSomethingHere()
        x := catch(func() {
                doSomethingThatCouldPanic()
        })
        if x != nil {
                fmt.Printf("panic caught with value %v\n", x)
        }
        fmt.Println("finished foo")

--
Kevin Ballard
http://kevin.sb.org
kball...@gmail.com

Rob 'Commander' Pike <r...@google.com> wrote:

Ab initio disclaimer: I confess I am not a fan of exceptions.
Personally I'd be just as happy if exceptions are not added to
Go.

I _especially_ don't want exceptions to become an oft-used
alternative to multiple levels of error return, as at that
point they deteriorate to action at a distance and make
understanding large code bases much harder.  Been there, done
that, got the scars to prove it.  (Mostly from C++ and not
from Java, but I've seen enough Java to have a healthy fear of
runtime exceptions.)

Rob -- I've only read your proposal twice so far, and will
read it again and think about it some more after some sleep,
but I have two questions already after that long winded
introduction, based on the immediately practical question:
"What would (or should) change in my existing code if this
proposal was implemented?"

Herewith:

a) Will there be a way other than relying on package
   documentation or reviewing a package's source code to tell
   whether a package may call panic(), and in particular which
   function(s) in the package might do so?

b) Will there be any indication (e.g. a compiler warning) when
   I call a function which may panic without setting up an
   appropriate defer() beforehand?

Specific to the details of the proposal, the recovery
mechanism looks sane, and the whole proposal looks clear and
is admirably short for what is usually a language feature with
tricky semantics and chapters worth of description. I salute
you.

I look forward to reviewing the proposal again and following
the mailing list discussion, and may have more comments later.

In the meantime, if anyone can either point me to answers to
my questions in the proposal (in which case I'll say "Oops,
sorry" very nicely) or if Rob (and/or the Google Go team?)
would consider the questions, I'll be appreciative.

I am extremely impressed with Go so far: I've written a few
thousand lines (reimplementing some small applications,
essentially as a learning exercise) and with a few exceptions
(sorry, pun unintended) to do with signal handling and pushing
past cgo's limits almost each time I touch that tool, I have
found Go to be an easy and productive language to write in.

(I have some ideas about the signal package which I'll write
up once I'm finished thinking them through.  I'm not so sure
about cgo; my ideas there still need some more time to
mature.)

Best regards,

Giles
--

Yes.

Yes.

You could also write a similar wrapper around the go statement,
sending the panic value back on a channel.

Ian

I agree.

That is a fair question, and the answer is no.  In particular,
although it is not part of this proposal, I think that if this
proposal is adopted it is fairly likely that a runtime error like an
out of bounds access to a slice or an erroneous type assertion would
become a panic.  (Perhaps division by zero would also become a panic
if that can be done efficiently.)

Probably not.  After all, there is nothing wrong with letting the
panic propagate upward in the call stack, nor even with letting it
crash your program if that is a reasonable course of action.

The name panic was chosen in part because it connotes a truly unusual
condition.  As you say, wide use of an exception mechanism makes a
program harder to understand.  At the same time, particularly in the
parallel server model which is easy to write in Go, there is clearly
an advantage to giving a program a way to let a goroutine fail without
causing the entire program to crash.  The panic function also provides
a way to return an error up several levels in a call stack without
testing the error at each step, but the expectation there is certainly
that some closely related function would recover and return a more
reasonable error indication.

I personally think that a library function which intentionally panics
and forces the caller to handle the panic is poorly designed.  Of
course nothing in this proposal prevents somebody from writing such a
function.  It's reasonable to ask whether the language can provide
some mechanism to prevent it.  This is just a proposal at this point,
and if anybody has a clever suggestion here we would certainly
consider it.

That said, still speaking purely personally, I don't think exception
specifications as in C++ or Java are appropriate for Go.  I don't
think they have been succesful in those languages, and indeed they are
being considered for deprecated in the new C++0x standard.  Exceptions
specifications can work well in theory; in practice, however, their
main effect seems to be to force people to write extra code to evade
the specification requirements.  There is no reason to import that
approach into a new language.

(The C++0x committee is considering adding a noexcept keyword which
may be applied to a function.  I think that what it means in effect is
that the function is wrapped in a catch handler which crashes the
program.  We could such a feature to Go, but I'm not sure it really
carries its weight.)

Ian

Hi Ian,

Thanks for your answers.

Ian Lance Taylor <i...@google.com> wrote:

Thanks, question answered, and it's probably the appropriate
choice.

Now, those could be useful: I can imagine a defer() function picking one
of those up and giving useful (and likely different) data to both the
user of the application (who probably can't fix it) and the programmer
(who usually wants to).

Users find stack traces very unappealing, however useful they are
to programmers!

(Ian -- are you trying to sell me on exceptions being useful after
all?  Hmm ....)

Pretty much what I expected.  Indeed, I suspect if this proposal is
adopted it'll be common to declare a suitable defer() function early in
a goroutine's existence, just to take over control of the error
reporting so a compiler warning probably wouldn't help much: if there's
no appropriate defer(), there probably wasn't meant to be one.

I'm familiar -- one might say overly familiar -- with the term from 8+
years of kernel support for a Unix vendor.  If used in similar
circumstances and as little as possible, I don't really object.  If
you've painted yourself into a corner and can't levitate, you're out of
options, after all.

I would like to think there are fewer cases for libraries to panic than
for kernels to panic, of course, but good library design is non-trivial.

(And certainly during development panic wouldn't hurt in a language
without assertions; making it blatantly obvious to developers and Q&A
departments that "yes, we know this doesn't work yet" isn't nutty when
they hit something that's not implemented yet. Woe to the developer and
reviewers who let one of those out into a product though!)

I concur.  (I've fought similar arguments about library functions that
spat error messages out to what they *thought* was standard error.
Library functions shouldn't do that, either.)

I suspect nobody will come up with a clever enough suggestion, because
sometimes it may be the library that has something panic-worthy happens.
One example would be a "can't happen" case in a library triggered by
hardware data corruption: a panic would sometimes not be out of order
there, any more than it would be within a kernel.

(A "can't happen" based on corrupt input would be another matter:
that would just be an error, IMHO.  Programs should expect and be
prepared for corrupt input.)

+1

In both languages I've found them a PITA, as I've had to code all this
extra "stuff" without much knowledge of what any of the individual
exception types was going to mean if I got one, at least until I buit up
an understanding of the underlying code.  (Yeah, I have worked on too
many projects that have had totally inadequate documentation.)

+1 also, but your knowledge of C++ and Java is well ahead of mine.

Thanks for the response.  It answered of both my questions, and provided
more background to Rob's proposal (which I now suspect is not just Rob's
proposal :-).

Cheers,

Giles

SEND MORE

2010/3/25 Giles Lean <giles.l...@pobox.com>:

On Mar 25, 2010, at 1:06 AM, roger peppe wrote:

Hadn't thought of it but it shouldn't be hard to do.  Goexit is like  
panic without failure.

-rob

yeah, i'd imagined that there might be a special type or
value (or both) (e.g. runtime.Exiting) which means "don't really panic"
when the stack is unwound completely.

Goexit would just be:

func Goexit() {
     panic(runtime.Exiting)

if you have this then no-one can evade a defer
without taking down the whole program.

On 25 March 2010 16:21, Rob 'Commander' Pike <r...@google.com> wrote:

Will there still be a way to obtain the stack trace that it currently prints
out? (Especially important if the panic source is a divide by zero or other
runtime failure condition)

On Thu, Mar 25, 2010 at 11:21 AM, Rob 'Commander' Pike <r...@google.com> wrote:

--
Daniel Smith
http://www.schaumburggoclub.org/

I think this is a solid plan. In my experience, the two types of
crashes I see most often are invalid map access, and null pointer
exceptions. Would these be recoverable? What kind of data would they
carry?

- Mike

On Mar 25, 7:26 am, Ian Lance Taylor <i...@google.com> wrote:

On Mar 25, 2010, at 9:33 AM, Daniel Smith wrote:

Absolutely.  Part of the design (see the implementation discussion) is  
to guarantee you don't lose stack trace information.

-rob

Ah, sorry, I somehow missed that section. This will be a very important
feature for anyone who doesn't want their webserver to go down...

On Thu, Mar 25, 2010 at 11:36 AM, Rob 'Commander' Pike <r...@google.com> wrote:

--
Daniel Smith
http://www.schaumburggoclub.org/

recoverable in the sense that you could abort some
number of function call frames and move on with
the rest of your program.

not recoverable in the sense that you could do something
to let the goroutine continue at the point of fault.

you shouldn't see many more invalid map accesses,
by the way.  the next release will change the behavior
of m["no-such-key"] to return a zero value instead of
crash/panic.  that change is already in tip.

russ

On Mar 25, 6:36 pm, "Rob 'Commander' Pike" <r...@google.com> wrote:

Will it be possible to get the stack trace programmatically? My use
case is a testing framework, where the code under test throws an
exception, and the testing framework must collect the stack trace into
a string, which will then some time later be shown in the test results
(the stack trace can't be printed to stdout immediately because of the
way that the framework works).

Can I use runtime.Callers() and runtime.FuncForPC() to generate
exactly the same stack trace as would happen when the program
terminates? Where is the code that Go is currently using for
formatting the stack trace? I would like my testing framework to print
stack traces using the same format, because it's familiar to the
developers.

Looks great to me, too! I especially like how it reuses the existing
concept of the defer function, making it considerably more powerful,
in the process.  And the Goexit approach sounds great, so that when
there is important cleanup to be done, one can pretty well guarantee
it'll get done.  e.g. setting the terminal mode back to plain old
cooked mode... there's little more annoying than having one's terminal
messed up by a program dying.  Well, it's not so annoying if you know
to just type "reset" blind, but for users who don't know that, it's
pretty annoying.

David

Almost.  Not quite, because you can't get at the argument values.
But the argument values printed in the panic traces aren't useful
anyway, unless you're comfortable reading the binary representation
of the various data structures.  You can certainly make it look
substantially similar.  The loss of the argument values is not such
a big deal since they're not really readable as it is.

I think it's safe to say that the interface to the active call stack
is still evolving.  It's a little too ad hoc right now.

src/pkg/runtime/amd64/traceback.c (even on 386)

Russ

On Mar 25, 9:57 pm, Russ Cox <r...@golang.org> wrote:

Thanks.

Also the exception proposal looks promising. :)

Yeah, this is the lite (and fixed) version of the proposal I just
proposed on 2009.12.09... (panic == exception.Throw and recover ==
exception.GetCurrent + exception.Swallow in my proposal) Your fix is the
modified "defer" semantics, I like it!

However there are some problems with it:

1. There is no default defer mechanism (which would wrap every goroutine
automatically).

2. If a defer is handling a panic and there is a nested panic happens
deep down in the called functions (which can easily happen while logging
the panic), then this nested panic will be re-thrown. So even if some
panic handler defer higher in the call stack could analyze the panic
value and do some error recovery (sometimes it can happen), all it will
recover() the useless nested panic value. This was the reason my
proposal used exception.GetCurrent() and exception.Swallow() to signal
that the exception was handled and kept the nested panic value in a
panic value array and it returned the original one for GetCurrent().
Your proposal swallows the exception at recover() even if it is not yet
handled properly by the defer... While I 100% agree that you should
discourage writing a lot of exception handling code, the result seems a
little bit too much for me:

func f1() {
        defer func() {
                if x := recover(); x != nil {
                        defer func() {
                                // nothing, just catch the panic of logging
                        }
                        print("panic happened") // just log, which can panic inside
                }
        }
        panic("error")

or this one

func f2() {
        defer func() {
                if x := recover(); x != nil {
                        defer func() {
                                if y := recover(); y != nil {
                                        if x != nil {
                                                panic(x)
                                        }
                                        panic(y)
                                }
                        }
                        print("panic happened") // just log, which can panic inside
                        x = nil
                }
        }
        panic("error")

3. This last one is not a problem with the text of your proposal but
about the ideology of exception handling.
Go gives us sufficient tools to return and check expected errors. Your
proposal is sufficient to return and check for unexpected errors.
Unfortunately it still leaves the definition of expected/unexpected
error to the library which is called. Now there is this little problem
that only the caller code can decide (I mean the developer of the caller
code) what is expected and what is not. Because there is no mechanism in
Go to automatically check errors (this was the longer part of my
proposal), the default code will either just ignore unexpected errors or
check and panic() every failed function call which is ugly... (If you
look at Go code without exceptions, it also just ignores possible error
return values which is a little bit C like.)

Does 'matches' there mean something more than simple equality?  From the
rest of the proposal, it seems clear that functions are meant to be able
to catch panics that occur in the functions they call; but in that case
the stack pointers would not be equal.  A greater-than comparison could
distinguish between SPs of called functions (from which panics can be
caught) and SPs of callers (from which they're not supposed to be
caught), provided that the stack is a normal stack, with stack frames
ordered by call depth.  So maybe that is what is meant by 'matches'?

The SP of the function that executes the defer statement is not relevant.
It's the SP of the deferred invocation that must match, because
that's how you can tell it's the top of a deferred invocation as opposed
to a sub-call.  It sounds like you understand the intent.  The details
may be off slightly - it's a sketch of an implementation that doesn't
yet exist.

Russ

On Mar 24, 11:36 pm, "Rob 'Commander' Pike" <r...@google.com> wrote:

Very elegant.

It looks like testing for panics would incur very little overhead (as
compared to creating a try-catch frame).

Do you have a general idea of what runtime errors could be detected
and what would just crash no matter what?  A few were discussed, div
by 0, array index out of bounds...

Regarding:

This would also be very cool.  Right now I feel the syntax proposed is
a little bulky, but it is very precise.  It seems tempting to creating
a generic "catch" syntax such as:
recover(x) {
  printError()
rather then defer func() { x :=recover() }
but that may be taken as a C++, Java, C# style error handling.  I
guess by writing out exactly what's happening there's much more
control, and these should be few and far between.

If this is able to handle handling and monitoring exceptions in other
goroutines, it sounds great!  Otherwise, there may be a syntax that
can work with arbitrary goroutine exceptions...  Hmm that thought
needs work.

I don't know... mere naming certainly isn't going to deter me from
misusing the mechanism for ordinary errors.  Nor would anything else I
noticed in the proposal.  Indeed, with an empty interface as the error
code, I could define my own types for exceptions, and write code that
checked the type returned from recover(), handled only the types of
exceptions that I myself had defined (doing type comparisons via
reflect.TypeOf), and passed anything else on by re-calling panic().
That'd be pretty much guaranteed to not mess with other uses of panic().
It might not even really be worth calling 'misuse'.

This is true but I think it's OK.  A simple library function can
implement this for those who need it.  Perhaps something like

func Go_and_recover(f func()) chan interface{} {
        c := make(chan interface{})
        go func() {
                defer func() {
                        c <- recover()
                }()
                f()
        }()
        return c

This example, and the other one, seems slightly odd to me.  If you are
going to log something, then presumably there is no higher level
caller which can handle a panic.  So I don't see how the distinction
between GetCurrent and Swallow helps in such a case.  Also, you are
assuming that 1) logging something can panic; 2) if it does panic, you
still want to continue program execution.  It's not obvious to me that
either assumption is likely to be true.  And, as you demonstrate with
your program, if either is in fact true, Go can still handle it,
albeit more verbosely.

I think that one can build GetCurrent and Swallow out of the proposed
panic/recover mechanism, and I think the reverse is true as well.
Since the mechanisms are equally powerful, the question is which is
more likely to be useful in the common case.  My initial thought is
that a nested panic is not the common case, so it is not the one to
optimize for.  Do you have other examples which argue otherwise?

I'm not sure I entirely understand what you are saying.  The caller of
a function can always ignore any error that function may be generate.
That does not change with the panic/recover proposal.

Ian