Finalizers

[Ian Lance Taylor]

In https://golang.org/issue/13347 , I raised a problem with
finalizers: if the finalizer of a value destroys some field of that
value, it's essential to ensure that the value is alive across any use
of that field. If the value becomes dead while the field is being
used, a GC might run, destroying the field and causing unpredictable
behavior. In cases involving C code, it might even lead to a program
crash.

Some existing Go programs today use finalizers incorrectly because of
this, and many are only correct because the compiler currently ensures
that all function arguments (and the receiver) are live for the extent
of the function. That compiler feature is undocumented and we don't
want to depend on it forever.

To address this problem, we (the Go runtime team) propose to add a new
function: runtime.KeepAlive. This function will take a single
argument and will simply ensure that its value is alive at that point
in the code. This may then be used in a function that uses a value
with a finalizer, to ensure that that value is alive for as long as is
necessary.

This is essentially what the syscall package does today, using an
unexported function syscall.use. It is also similar to what cgo does
today, using the unexported function runtime.cgoUse. Both those
functions could be replaced with runtime.KeepAlive.

The runtime.KeepAlive function is trivial:

//go:noinline
func KeepAlive(interface{}) {
}

Since this change is mainly documentation, we propose to do this for Go 1.7.

In future releases, the compiler may recognize the runtime.KeepAlive
function specially and implement it more efficiently, by simply
changing the liveness information recorded by the compiler. That
change will not be in 1.7 (and may never happen, who knows).

This approach is the same as that taken by C#:
https://msdn.microsoft.com/en-us/library/system.gc.keepalive(v=vs.110).aspx
.

Ian

[brainman]

Thank you Ian for explaining. But I would like to see more details.

Will runtime.KeepAlive be used to ensure that external (to Go) code can safely use Go memory?

If yes, what will my code look like? Lets say I need to pass 10 byte buffer to my external library, and I want my external code to have access to these bytes until my Go program exists. Do I do:

var b [10]byte
runtime.KeepAlive(b)
syscall.Syscall(externalFunction, 1, uintptr(unsafe.Pointer(@b[0])), 0, 0)

do that?

What if my external code stops using b before my program completes? Can I tell GC to free 'b' somehow?

Will runtime.KeepAlive ensure that @b[0] (address of first byte in b) not change?

If runtime.KeepAlive will provide all that functionality, maybe runtime.KeepAlive name is not explaining what it does. Unfortunetly I don't have better name suggestions.

Thank you.

Alex

[Matthew Dempsky]

The idea is that you write:

    var p *X = foo()

    bar(p.f)

    runtime.KeepAlive(p)

The problem is that without the runtime.KeepAlive(p) call, p isn't necessarily alive anymore within the bar(p.f) call. So p's finalizer might run during bar(p.f), which could invalidate p.f.  E.g., imagine it's a file descriptor or handle, which p's finalizer closes.

Adding the runtime.KeepAlive(p) call simply guarantees that p's finalizer doesn't run until after bar returns.

It's not a manual malloc/free mechanism.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[brainman]

On Wednesday, 11 May 2016 12:14:43 UTC+10, Matthew Dempsky wrote:

It's not a manual malloc/free mechanism.

That does it for me. Thank you for explaining.

Alex 

[Dave Cheney]

This is kind of gross, but I guess that's the fault of finalisation in general.

Would it be possible to change the spec to force the underscore
assignment to keep a value alive ?

_ = p // p is alive until the end of this statement ?

[minux]

On Tue, May 10, 2016 at 10:28 PM, Dave Cheney <da...@cheney.net> wrote:

This is kind of gross, but I guess that's the fault of finalisation in general.

Would it be possible to change the spec to force the underscore
assignment to keep a value alive ?

_ = p // p is alive until the end of this statement ?

This might be unrelated to the current topic. But I think for the current compilers,

blank identifier assignment doesn't always force the RHS to be computed....

(that's the reason why in benchmark you need to sink into a global variable.)

Along the same line, I'd like to see the semantics of blank identifier assignments

formally defined in the spec. I.e. the RHS will always be computed --- or the

"virtual" store won't be optimized away. Then not only we have a solution to this

problem, benchmarks are also simplified (even assigning to global variable

might be problematic given more advanced global program analysis --- if the

compiler could prove that no one is going to read from the global variable, the

stores will become dead, and can be eliminated.)

[Dave Cheney]

I like this suggestion.

[jimmy frasche]

If you rolled me out of bed and shoved a laptop into my hands because
the servers are on fire I'd have a notion of what runtime.KeepAlive
meant, even if I didn't know Go.

If I saw _ = p I'd wonder why no one bothered to delete the line.

[Ian Lance Taylor]

On Tue, May 10, 2016 at 7:28 PM, Dave Cheney <da...@cheney.net> wrote:
> This is kind of gross, but I guess that's the fault of finalisation in general.
>
> Would it be possible to change the spec to force the underscore
> assignment to keep a value alive ?
>
> _ = p // p is alive until the end of this statement ?

I thought about that. I don't personally prefer that approach because
1) I've seen that people already use _ = p to mark their variables as
used during development, so we would see the same thing with two
different meanings; 2) the current compilers do not support it, but
they do support runtime.KeepAlive (because function arguments
naturally must be live); 3) in an area this subtle there is some
benefit to following an existing example, in this case C#.

Ian

[Keith Randall]

To answer Minux's question, "_ = x" does have to evaluate x, always.  Of course, it is often the case that the evaluation of x is not observable, so the compiler can optimize it away without changing the behavior of the program.  But if x contains a function call, or a bounds check that might fail, etc. then those side effects must be preserved.

[Ian Lance Taylor]

I've sent out https://golang.org/cl/23102 to implement runtime.KeepAlive.

Last change for objections.

Ian