defer not guaranteed to run!

[Hǎiliàng]

I've just found that defer is not guaranteed to run at all!
Any goroutine without a recover will terminate the program and no deferred functions will be called except the ones in the panicking goroutines.

It means that basically there is no guarantee that the clean up code will get called especially when a 3rd party package is used, and it happens to have a panicking goroutine without a recover.

Is my understanding correct? Is it a problem? Any solutions?

Hǎiliàng

[Dmitry Vyukov]

It's impossible to do.

Consider that a goroutine has created an external resource but had not a chance to setup a defer or store the resource descriptor somewhere. At this point another goroutine crashes. There is no way to clean up the resource.

[Patrick Mylund Nielsen]

> when a 3rd party package is used, and it happens to have a panicking goroutine without a recover

Best bet is to not use packages that do that, or to do your own recover.

Hǎiliàng

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[bryanturley]

Just a guess, but I am betting you are over using panic().
I think the only way to not have defers run is to not exit a function when your program stops, which only happens in extreme circumstances like panic().

Hǎiliàng

[Jeremy Wall]

http://golang.org/ref/spec#Handling_panics

"When a function F calls panic, normal execution of F stops immediately. Any functions whose execution was deferred by the invocation of F are run in the usual way, and then F returns to its caller. To the caller, F then behaves like a call to panic, terminating its own execution and running deferred functions. This continues until all functions in the goroutine have ceased execution, in reverse order. At that point, the program is terminated and the error condition is reported, including the value of the argument to panic. This termination sequence is called panicking."

I don't know what exact behaviour you are seeing but the spec clearly states that panics run all deferred functions for that function. The only way a defer in a function wouldn't be run is if the panic happens before the defer call. So for example if I do this

func f() {
  f, err := os.Open(...)

  panic("AAAAAAhhhhh")

  if err != nil {

   defer f.Close() // this defer won't run because the the function panics before the defer is scheduled.

  }

}

This is programmer error though since they are panicking before scheduling any cleanup to happen.

recover isn't a magical syntax that forces it deferred's to run it works because deferred calls run when you panic.

Hǎiliàng

[bryanturley]

On Thursday, February 21, 2013 1:45:28 PM UTC-6, Jeremy Wall wrote:

http://golang.org/ref/spec#Handling_panics

"When a function F calls panic, normal execution of F stops immediately. Any functions whose execution was deferred by the invocation of F are run in the usual way, and then F returns to its caller. To the caller, F then behaves like a call to panic, terminating its own execution and running deferred functions. This continues until all functions in the goroutine have ceased execution, in reverse order. At that point, the program is terminated and the error condition is reported, including the value of the argument to panic. This termination sequence is called panicking."

I don't know what exact behaviour you are seeing but the spec clearly states that panics run all deferred functions for that function. The only way a defer in a function wouldn't be run is if the panic happens before the defer call. So for example if I do this

func f() {
  f, err := os.Open(...)

  panic("AAAAAAhhhhh")

  if err != nil {

   defer f.Close() // this defer won't run because the the function panics before the defer is scheduled.

  }

}

This is programmer error though since they are panicking before scheduling any cleanup to happen.

recover isn't a magical syntax that forces it deferred's to run it works because deferred calls run when you panic.

http://play.golang.org/p/QQyVVOFXz4  this illustrates his problem, only one of these 10 defers gets run.
Now if instead of panic()'ing there was error detection all of the defers could run.
Or the panic could get recovered and running could continue as normal.

 

[bryanturley]

On Thursday, February 21, 2013 1:56:05 PM UTC-6, bryanturley wrote:

On Thursday, February 21, 2013 1:45:28 PM UTC-6, Jeremy Wall wrote:

http://golang.org/ref/spec#Handling_panics

"When a function F calls panic, normal execution of F stops immediately. Any functions whose execution was deferred by the invocation of F are run in the usual way, and then F returns to its caller. To the caller, F then behaves like a call to panic, terminating its own execution and running deferred functions. This continues until all functions in the goroutine have ceased execution, in reverse order. At that point, the program is terminated and the error condition is reported, including the value of the argument to panic. This termination sequence is called panicking."

I don't know what exact behaviour you are seeing but the spec clearly states that panics run all deferred functions for that function. The only way a defer in a function wouldn't be run is if the panic happens before the defer call. So for example if I do this

func f() {
  f, err := os.Open(...)

  panic("AAAAAAhhhhh")

  if err != nil {

   defer f.Close() // this defer won't run because the the function panics before the defer is scheduled.

  }

}

This is programmer error though since they are panicking before scheduling any cleanup to happen.

recover isn't a magical syntax that forces it deferred's to run it works because deferred calls run when you panic.

http://play.golang.org/p/QQyVVOFXz4  this illustrates his problem, only one of these 10 defers gets run.
Now if instead of panic()'ing there was error detection all of the defers could run.
Or the panic could get recovered and running could continue as normal.

Heh, Println() screws with my printf() hardcoded brain...
http://play.golang.org/p/uSqIzV3CJ6
 

[Jeremy Wall]

On Thu, Feb 21, 2013 at 1:57 PM, bryanturley <bryan...@gmail.com> wrote:

On Thursday, February 21, 2013 1:56:05 PM UTC-6, bryanturley wrote:

On Thursday, February 21, 2013 1:45:28 PM UTC-6, Jeremy Wall wrote:

http://golang.org/ref/spec#Handling_panics

"When a function F calls panic, normal execution of F stops immediately. Any functions whose execution was deferred by the invocation of F are run in the usual way, and then F returns to its caller. To the caller, F then behaves like a call to panic, terminating its own execution and running deferred functions. This continues until all functions in the goroutine have ceased execution, in reverse order. At that point, the program is terminated and the error condition is reported, including the value of the argument to panic. This termination sequence is called panicking."

I don't know what exact behaviour you are seeing but the spec clearly states that panics run all deferred functions for that function. The only way a defer in a function wouldn't be run is if the panic happens before the defer call. So for example if I do this

func f() {
  f, err := os.Open(...)

  panic("AAAAAAhhhhh")

  if err != nil {

   defer f.Close() // this defer won't run because the the function panics before the defer is scheduled.

  }

}

This is programmer error though since they are panicking before scheduling any cleanup to happen.

recover isn't a magical syntax that forces it deferred's to run it works because deferred calls run when you panic.

http://play.golang.org/p/QQyVVOFXz4  this illustrates his problem, only one of these 10 defers gets run.
Now if instead of panic()'ing there was error detection all of the defers could run.
Or the panic could get recovered and running could continue as normal.

Heh, Println() screws with my printf() hardcoded brain...
http://play.golang.org/p/uSqIzV3CJ6

Since the goroutines never return or panic themselves then the defer never runs. I'm actually not sure if that's a bug or not. The spec doesn't directly address this by my reading :-p I wonder if this is what the Go authors intended?

[bryanturley]

On Thursday, February 21, 2013 2:28:33 PM UTC-6, Jeremy Wall wrote:

On Thu, Feb 21, 2013 at 1:57 PM, bryanturley <bryan...@gmail.com> wrote:

On Thursday, February 21, 2013 1:56:05 PM UTC-6, bryanturley wrote:

On Thursday, February 21, 2013 1:45:28 PM UTC-6, Jeremy Wall wrote:

http://golang.org/ref/spec#Handling_panics

"When a function F calls panic, normal execution of F stops immediately. Any functions whose execution was deferred by the invocation of F are run in the usual way, and then F returns to its caller. To the caller, F then behaves like a call to panic, terminating its own execution and running deferred functions. This continues until all functions in the goroutine have ceased execution, in reverse order. At that point, the program is terminated and the error condition is reported, including the value of the argument to panic. This termination sequence is called panicking."

I don't know what exact behaviour you are seeing but the spec clearly states that panics run all deferred functions for that function. The only way a defer in a function wouldn't be run is if the panic happens before the defer call. So for example if I do this

func f() {
  f, err := os.Open(...)

  panic("AAAAAAhhhhh")

  if err != nil {

   defer f.Close() // this defer won't run because the the function panics before the defer is scheduled.

  }

}

This is programmer error though since they are panicking before scheduling any cleanup to happen.

recover isn't a magical syntax that forces it deferred's to run it works because deferred calls run when you panic.

http://play.golang.org/p/QQyVVOFXz4  this illustrates his problem, only one of these 10 defers gets run.
Now if instead of panic()'ing there was error detection all of the defers could run.
Or the panic could get recovered and running could continue as normal.

Heh, Println() screws with my printf() hardcoded brain...
http://play.golang.org/p/uSqIzV3CJ6

Since the goroutines never return or panic themselves then the defer never runs. I'm actually not sure if that's a bug or not. The spec doesn't directly address this by my reading :-p I wonder if this is what the Go authors intended?

 

If you think of panic() on the level of a segfault or other major kill signal it makes sense.
If you think of it as an exception try/catch mechanism perhaps not?
Pretty sure it was the intended to be more like a fault than a java style exception.

[Jens Alfke]

On Thursday, February 21, 2013 12:44:10 PM UTC-8, bryanturley wrote:

If you think of panic() on the level of a segfault or other major kill signal it makes sense.
If you think of it as an exception try/catch mechanism perhaps not? 

Actually this sounds just like C++ exception handling. If a thread throws an exception without any active 'catch' block to handle it, that's a fatal error and the runtime support library will abort the process (I think it literally calls abort().) And when a process exits, background threads just stop instead of going through any kind of cleanup like running 'catch' blocks. (I don't even know of any mechanism in C++ for forcing another thread to clean up and unwind its stack.)

As for Java, it's been way too long since I did any serious work in it, but IIRC, exiting a process doesn't make any attempt to stop background threads — in fact the whole Thread.stop mechanism was deprecated back in 1.2 or so because throwing asynchronous exceptions at threads led to so many problems with stuff not getting cleaned up properly.

The moral here, I think, is that if you want fail-safe cleanup even in the face of panics, you have to make sure that all goroutines have top-level handlers that recover from the panics.

--Jens

[Greg Ward]

On 21 February 2013, Jens Alfke said:
> The moral here, I think, is that if you want fail-safe cleanup even in the
> face of panics, you have to make sure that all goroutines have top-level
> handlers that recover from the panics.

Why is this even a problem? If the process terminates, then files will
be closed, database connections closed, transactions aborted, locks
released, etc.

The only resource you need to worry about are persistent resources,
e.g. that 3 GB temporary file I don't want to delete yet. But there's
*no point* in trying to handle that, because even if no goroutine ever
panics in the history of the world, users will continue tripping over
power cords and overzealous sysadmins will continue to use "kill -9"
inappropriately.

Well-written software recovers from users tripping over the power
cord.

Greg
--
Greg Ward http://www.gerg.ca
<gr...@gerg.ca> @gergdotca

[bryanturley]

Not a problem for me. I initially (1.5-2yrs ago) expected panic to not even run the local defers.
I think I started using go before they added recover though, I don't remember entirely.

And note I said the problem was "Just a guess, but I am betting you are over using panic()."