go routines not being deallocated

[Trig]

Can somebody tell me why this is?  It won't work on the playground; however, put something like time.Sleep(time.Second * 1) and put the last allocs in a loop... and you'll see it's literally MBs of Allocs used that never get released.

What am I doing wrong, or not understanding here?  I thought go routines got completely removed after they've returned (assuming they're not blocked from doing so).

https://play.golang.org/p/JfaOVnpMUpf

[Trig]

Sorry, for clarification... the above should be, if you 'put something like time.Sleep(time.Second * 1) in the go routine...'

[Brian Candler]

What do you mean by "It won't work on the playground"?  It runs for me.  Are you saying you get different results when running locally?  If so, what version of go are you running locally, on what platform, and what do you see?

Or are you saying the problem is really with something like this?

https://play.golang.org/p/Q91Qd-DvvKx

That indeed fails to run in the playground - it exits with signal 9 (128+9=137)

Running locally with go 1.16.3 under macOS (Intel) I see:

Allocs before: 82696

Allocs while running: 51496728

Allocs after: 40384112

Allocs after: 40384112

Allocs after: 40384112

Allocs after: 40384112

Allocs after: 40384112

That is 40MB that apparently isn't reclaimed, but it does get reused if you run more goroutines:

https://play.golang.org/p/YcMTlLYxXAP

I don't know why that is.  It's about 400 bytes per goroutine.  Stack perhaps?

[Trig]

Correct... the example using time.Sleep didn't run on the playground.  The original post I made, I provided the link with just a goroutine with an empty function.

At least you were able to reproduce what I'm inquiring about.  I'll look further into this, but surely unused and finished goroutines don't leave stuff laying around like that.

[Trig]

Also, I don't think it's the Stack.  If you replace Alloc with HeapAlloc... it's all there.

[Trig]

From further research (and anybody correct me if this is wrong), when the GC does collect memory the profile shrinks, but no memory is returned to the system. Any future allocations will try to use memory from the pool of previously collected objects before asking the system for more.  This is why things are behaving this way.

[Ian B]

Just to note, from a Perl background (learning Go atm so this may not apply, just trying to indicate it may be "normal"), I think this isn't necessarily unusual for a process to not free up memory, and it's an OS limitation (again I may be barking up the wrong tree). Elsewhere (i.e not Go), I've got around this problem by running a new process that eats up the memory, processes, and then dies, returning the response to the original process (so memory isn't allocated for a long term, we have processes that sometimes run for best part of a whole day).

Ian

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f42e270a-7ae9-4f11-bb85-d94bcbd331f1n%40googlegroups.com.

[Brian Candler]

This is memory allocated as reported by go itself, not by the OS.

[Keith Randall]

This might be https://github.com/golang/go/issues/34457 (at least, Brian's repro). When a goroutine finishes, we deallocate its stack, and that deallocation will eventually be given back to the OS.

The Goroutine descriptor, however, will live forever. We'll reuse it for new goroutines, but it never deallocates. Goroutine descriptors are currently 376 bytes (on 64-bit archs).