Memory usage of goroutines

[Ryan Slade]

Hi

Why does memory usage (only tested on OSX) rise and then not decrease when I run the following code?

package main

import "time"

func main() {

    for i := 0; i < 10000; i++ {

        go func() {

        }()

    }

    time.Sleep(1 * time.Minute)

}

I also tried importing the runtime package and running the garbage collecter but it made no difference.

Thanks

Ryan

[Paul Borman]

The OS does not know what pages of memory you are no longer using.  Also, addressing even a single byte of a page will keep that page in memory.  If you change your run to create the the goroutine and then wait for them to complete and then repeat you should not see any significant increase in memory.

    -Paul

[Ryan Slade]

Does that mean that the memory can never be reclaimed?

On Tuesday, 17 July 2012 21:53:35 UTC+1, Paul Borman wrote:

The OS does not know what pages of memory you are no longer using.  Also, addressing even a single byte of a page will keep that page in memory.  If you change your run to create the the goroutine and then wait for them to complete and then repeat you should not see any significant increase in memory.

    -Paul

[Brad Fitzpatrick]

Go does return memory to the OS, but your example (allocating ~40 MB once and never again) might not be enough (or often enough) for Go to get around to it.  I forget the details of how the knobs were initially tuned, but it might only return memory during future allocations.

Does this affect you?

[Glenn Brown]

Here's how to see what's going on:
http://play.golang.org/p/RaSrm4_P8_
Note that HeapObjects does stabilize, so goroutines are collected. This loop does not leak memory.

However, if you remove the call to runtime.Gosched(), HeapObjects keeps growing.
That's because goroutines can't be freed until they run, and they can't run until the scheduler is called, and the scheduler runs automatically only for blocking I/O operations. Similarly, memory is automatically collected only during memory allocation attempts. For your program, that means the goroutines can't run until time.Sleep() and even then may not be collected before the program exits because no allocation is attempted.

--Glenn

[Ryan Slade]

Thanks for your help.

I was concerned that it might be possible, given a spike of activity, to consume more memory than was necessary that would then never be given back to the OS. I'm happy as long as it is eventually freed.

Cheers

Ryan