Local variable escapes to heap

[alex.b...@gmail.com]

Hi all,

I've read a lot about escape analysis in golang, but still couldn't find an answer to this question

here is the sample:

package main

type IMeta interface {
    Index() int
}

type Builder struct {
    buf  []byte
    meta IMeta
}

func main() {
    var buffer [512]byte

    b := Builder{buffer[:0], nil}

    if b.meta != nil {
        b.meta.Index()
    }
}

escape analyzer says:

.\main.go:15:21: buffer escapes to heap
.\main.go:13:6: moved to heap: buffer
<autogenerated>:1:0: leaking param: .this

Could anyone please clarify why local buffer escapes here?

Thx in advance

[Dave Cheney]

If you pass more -m's to the compiler it will explain why

Daves-MacBook-Pro(~/src) % go build -gcflags=-m=2 buffer.go

# command-line-arguments

./buffer.go:12:6: cannot inline main: function too complex: cost 108 exceeds budget 80

./buffer.go:15:21: buffer escapes to heap

./buffer.go:15:21:      from Builder literal (struct literal element) at ./buffer.go:15:14

./buffer.go:15:21:      from b (assigned) at ./buffer.go:15:4

./buffer.go:15:21:      from b.meta.Index() (receiver in indirect call) at ./buffer.go:18:15

./buffer.go:13:6: moved to heap: buffer

<autogenerated>:1: leaking param: .this

<autogenerated>:1:      from .this.Index() (receiver in indirect call) at <autogenerated>:1

I recommend raising a bug https://golang.org/issue/new if you feel the compiler could prove that main.buffer did not escape.

[jfcg...@gmail.com]

Hi,

I have the following:

package myf

func F1(x *int, ch chan bool) {
    *x += 1
    ch <- false
}

func F2() {
    var x int
    ch := make(chan bool) // or with buffer
    go F1(&x, ch)
    <-ch
}

I get this when I build with go 1.15.5 via go build -gcflags '-m -m' :

3:6: can inline F1 with cost 7 as: func(*int, chan bool) { *x += 1; ch <- false }
8:6: cannot inline F2: unhandled op GO
3:9: x does not escape
3:17: ch does not escape
9:6: x escapes to heap:
9:6:   flow: {heap} = &x:
9:6:     from &x (address-of) at ./heap.go:11:8
9:6:     from F1(&x, ch) (call parameter) at ./heap.go:11:7
9:6: moved to heap: x

So x is allocated on the heap and needs gc when F2() returns. I know F2() will wait for F1() and passing &x is safe if it was local. So how can I tell this to go compiler and avoid allocation/gc costs? Do we need a new go:local directive to mark such variables?

Thanks..

[Ian Lance Taylor]

It would help to have a more realistic example. I don't yet see any
reason why people would write code like this, so it doesn't seem worth
optimizing.

If something like this is worth optimizing, the first thing to look
into would be whether the compiler's escape analysis can improve to
handle that case. A go:local directive seems very easy to misuse, and
doesn't seem like a good fit for the Go language.

Ian

[jfcg...@gmail.com]

In sorty (commit e4fb296daf1d90037d) I see:

$ go build -gcflags -m |& grep -i heap
./sortyI8.go:319:2: moved to heap: sv
./sortyU8.go:319:2: moved to heap: sv
./sortyF4.go:319:2: moved to heap: sv
./sortyF8.go:319:2: moved to heap: sv
./sortyI4.go:319:2: moved to heap: sv
./sortyLsw.go:338:2: moved to heap: sv
./sortyS.go:319:2: moved to heap: sv
./sortyU4.go:319:2: moved to heap: sv

Local variable sv for synchronization (that I know would be safe to stay local like the simplified example) escapes to heap. It is the one and only thing that escapes to heap and I want to get rid of it with something like go:local :)

[jake...@gmail.com]

For me, the example you gave of sorty is a strong argument against adding go:local. If I understand correctly, using go:local, if a variable marked this way actually does escape it would cause undefined behavior, possibly in unrelated code. This is the type of bug that is very, very hard to find and fix. Using your example of  ./sortyI8.go:319:2, if I were doing a code review, or reading the code later, it would take a minute to realize that the intent was for sv, and especially sv.done not to live past the end of the function. But then, as a reviewer, I would want to make sure that was actually the case. Turns out sv, and its members are used in a tangled web of function calls and goroutines, some of which are multiple layers deep. I gave up trying to track all of it after about 12 code jumps in my browser. One of the benefits of go, and the "go style" is that it should be easy to read and understand.

I'm not saying that there is no benefit to adding something like go:local, just that the bar would be very, very high in my opinion. 

[jfcg...@gmail.com]

I found this in runtime/chan.go:

// Sends and receives on unbuffered or empty-buffered channels are the
// only operations where one running goroutine writes to the stack of
// another running goroutine. The GC assumes that stack writes only
// happen when the goroutine is running and are only done by that
// goroutine. Using a write barrier is sufficient to make up for
// violating that assumption, but the write barrier has to work.

So keeping a variable local but sharing its address with another goroutine opens the possibility of writing to the variable (from the non-owner goroutine) which seems to violate the above assumption. Probably too much work to implement..