Why does the GC copy the stack when shrinking it ?

371 views
Skip to first unread message

Dave Cheney

unread,
Jan 13, 2016, 4:42:14 PM1/13/16
to golang-dev
This is undoutably a naive question, but having seen several CLs related to the complication around shrinking a stack via copying, I wonder why the GC does not simply reclaim the extra pages of the stack by marking them free.

I guess this would limit the lower bound a stack could be shrunk to a system package, or whatever is the size above the last shared size class. 

There may also be an impact of fragmentation, but in that respect a stack frame is no different from a large string retained on the heap. 

Is this work necessary for a future moving collector ?

Would it be simpler/safer in 1.6 to shrink stacks via lopping off their tails, rather than copying them?

Thanks for your time

Dave

Austin Clements

unread,
Jan 13, 2016, 5:15:49 PM1/13/16
to Dave Cheney, golang-dev
On Wed, Jan 13, 2016 at 4:41 PM, Dave Cheney <da...@cheney.net> wrote:
This is undoutably a naive question, but having seen several CLs related to the complication around shrinking a stack via copying, I wonder why the GC does not simply reclaim the extra pages of the stack by marking them free.
 
This may improve the performance of shrinking stacks, but given that stack growth would still have to be done by copying, I don't think it would actually reduce the complexity of the system.

Most of the recent CLs I can think of off the top of my head that related to stack copying are really more about the complexity of profiling signals happening at any point. For the most part, those issues apply equally to stack growth. Are there other CLs or classes of problems you're thinking of?

I guess this would limit the lower bound a stack could be shrunk to a system package, or whatever is the size above the last shared size class. 

This could be tricky, though there are some situations where it wouldn't be hard. If we're shrinking a large stack, we can certainly return the pages that are no longer used either to the heap (if we're not GC'ing) or to the large stack pool (broken up in to power-of-two regions). If we shrink a large stack down to a small stack, I think we could reclassify the remaining stack span as whatever size class the new stack is and make the rest of the span available for other small stacks of the same class.

If we're shrinking an already small stack to a smaller stack, things are harder, since currently we assume all stacks in a stack span are the same size class. We could lift that assumption using a different free list structure or even a buddy allocator, but it's definitely trickier.

There may also be an impact of fragmentation, but in that respect a stack frame is no different from a large string retained on the heap. 

Is this work necessary for a future moving collector ?

I don't think so.

Would it be simpler/safer in 1.6 to shrink stacks via lopping off their tails, rather than copying them?

I don't think it would be simpler or safer. However, it may be an answer to a real performance problem: https://github.com/golang/go/issues/12967. That's definitely for 1.7, though.

Thanks for your time

Dave

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dave Cheney

unread,
Jan 13, 2016, 5:53:14 PM1/13/16
to Austin Clements, golang-dev

Thanks for your reply Austin. I wasn't really concerned with  performance, my question was spurred on by reading Russ' recent CL about teaching the compiler to understand the subtlety of stack allocated arguments passed to long running syscalls.

Keith Randall

unread,
Jan 13, 2016, 6:07:02 PM1/13/16
to Dave Cheney, Austin Clements, golang-dev
I actually hacked up a buddy allocator about a year ago for stacks.  It would allow copyless shrinking which does make some problems easier to solve*.

My worry was fragmentation.  If you allocate a large stack, then shrink to a small one and block, then do that repeatedly (with a different goroutine each time), you end up with lots of memory wastage because you can't compact all of those small stacks.  I'm not convinced this is a serious problem, but I can't convince myself otherwise either.  So I've tabled it for now.

* for instance, SudoGs for channel/select operations could be allocated on the stack.

Austin Clements

unread,
Jan 13, 2016, 6:38:01 PM1/13/16
to Keith Randall, Dave Cheney, golang-dev

That's a more realistic scenario than the one I'd come up with. I could see servers with long lived connections or goroutine pools doing this.

Reply all
Reply to author
Forward
0 new messages