a safe free of memory proposal, runtime.FreeMemory()

[Jason E. Aten]

I'm considering how to allow performance optimization of memory freeing in Go; my current GC pause times for one application average 750 - 1500 msec, which makes these pauses a bit troublesome.  It seems to me like programmers should be able to provide hints to the garbage collector. Often times I know I'm done with this block of memory, and my knowledge is currently wasted, resulting in long GC pauses.

What if there was a runtime.FreeMemory(*T) operation in Go, that depended on three states of the system:

In Sandboxed environments, runtime.FreeMemory() is a noop. That way it is impossible to crash the program in a sandbox by early free of memory.

In non-sandboxed environments, suppose we have (I'm making up a variable name:) unsafe.MemoryFreeAllowed = CHECK_FREE. If this variable is set to CHECK_FREE, then the garbage collector will simply note the FreeMemory() invocation stack trace, and on the next scan, if that memory was not in fact free, will panic with a report indicating that the FreeMemory() call was erroneous, and give the saved stack trace. The effect would be to allow the program to run with production loads, and to verify that, under those loads, the hints were always correct.

Finally, also in non-sandboxed environments, one can set the performance setting of unsafe.MemoryFreeAllowed = FREE_INSTANTLY. With FREE_INSTANTLY (also a non-zero not-default value), then the memory is freed immediately without waiting for GC to verify. This is unsafe, as it can crash your program, but could readily and radically reduce pause times when the programmer is providing correct FreeMemory() advice.

It's optional. It's safe when sandboxed.  It lets Go venture into application territory that is owned by C++ currently.

-Jason

[Jan Mercl]

On Mon, Nov 24, 2014 at 5:38 PM, Jason E. Aten <j.e....@gmail.com> wrote:
> I'm considering how to allow performance optimization of memory freeing in
> Go; my current GC pause times for one application average 750 - 1500 msec,
> which makes these pauses a bit troublesome.

What kind of objects form the most of the garbage? Are thy under your
control? If so, have you tried sync.Pool and/or allocation arenas,
...?

What does mem profiling[0] reveal?

[0]: $ go test -c ; ./foo.test -test.memprofile mem.out ; go tool
pprof --lines --alloc_space foo.test mem.out

-j

[Jason E. Aten]

Even better: If unsafe.MemoryFreeAllowed == CHECK_FREE, then *immediately* do a garbage collection and check the free, rather than wait for the next round of GC. This is both easier to implement and would catch more errors.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/cmpiArv10f4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tahir]

I would guess it is unlikely to be needed post 1.5 but the idea is interesting nonetheless.

[Ian Lance Taylor]

On Mon, Nov 24, 2014 at 8:38 AM, Jason E. Aten <j.e....@gmail.com> wrote:
>

> I'm considering how to allow performance optimization of memory freeing in
> Go; my current GC pause times for one application average 750 - 1500 msec,
> which makes these pauses a bit troublesome. It seems to me like programmers
> should be able to provide hints to the garbage collector. Often times I know
> I'm done with this block of memory, and my knowledge is currently wasted,
> resulting in long GC pauses.

When you know the lifetime of memory, use some form of pool
allocation. That is simple and safe and avoids this issue.

By the way, at present, GC pause time is mainly affected by the size
of live memory that may contain pointers. It's not significantly
affected by the size of unreferenced memory that will be freed.

Future GC plans (http://golang.org/s/go14gc) should significantly
reduce GC pause times.

Ian

[Jason E. Aten]

On Mon, Nov 24, 2014 at 9:16 AM, Ian Lance Taylor <ia...@golang.org> wrote:

On Mon, Nov 24, 2014 at 8:38 AM, Jason E. Aten <j.e....@gmail.com> wrote:
>
> I'm considering how to allow performance optimization of memory freeing in
> Go; my current GC pause times for one application average 750 - 1500 msec,
> which makes these pauses a bit troublesome.  It seems to me like programmers
> should be able to provide hints to the garbage collector. Often times I know
> I'm done with this block of memory, and my knowledge is currently wasted,
> resulting in long GC pauses.

When you know the lifetime of memory, use some form of pool
allocation.  That is simple and safe and avoids this issue.

Hi Ian,

could you elaborate on how you see user doing pool allocation in Go?  For example, if I pre-allocate a Go slice/array to be the pool, I don't see how that avoids having to have the GC scan that array every time.

Thanks!

Jason

[Jason E. Aten]

Hi Jan,

Thanks for the suggestions. Yes, most objects are under my control. I haven't explored sync.Pool, reading the description in the docs, I'm not sure how it avoids the garbage collector. Are there facilities in the std lib or elsewhere for doing an Arena that avoids inspection by GC?

Thanks!

Jason

 

[Ian Lance Taylor]

On Mon, Nov 24, 2014 at 9:59 AM, Jason E. Aten <j.e....@gmail.com> wrote:
> On Mon, Nov 24, 2014 at 9:16 AM, Ian Lance Taylor <ia...@golang.org> wrote:
>>
>> On Mon, Nov 24, 2014 at 8:38 AM, Jason E. Aten <j.e....@gmail.com> wrote:
>> >
>> > I'm considering how to allow performance optimization of memory freeing
>> > in
>> > Go; my current GC pause times for one application average 750 - 1500
>> > msec,
>> > which makes these pauses a bit troublesome. It seems to me like
>> > programmers
>> > should be able to provide hints to the garbage collector. Often times I
>> > know
>> > I'm done with this block of memory, and my knowledge is currently
>> > wasted,
>> > resulting in long GC pauses.
>>
>> When you know the lifetime of memory, use some form of pool
>> allocation. That is simple and safe and avoids this issue.
>

> could you elaborate on how you see user doing pool allocation in Go? For
> example, if I pre-allocate a Go slice/array to be the pool, I don't see how
> that avoids having to have the GC scan that array every time.

Unless you also have a fixed number of elements, typically a pool is
implemented as a linked list of elements. When you need an element,
get it from the pool; allocate it if the pool is empty. When you are
done with an element, put it back in the pool.

Yes, the GC will scan the pool each time. The point of a pool is that
you don't allocate memory each time you need it. Most of the time you
get it from the pool. So the overall total of allocated memory does
not increase, so GC runs less often, so you have fewer GC pauses.

Ian

[Carlos Castillo]

GC currently only runs when the amount of allocated memory will exceed a certain threshold, ie: on an allocation. If your objects are in a pool, they are re-used directly, instead of needing an allocation (which could trigger GC). 

Once most/all your memory is managed by pools, only the following will result in allocations and eventually cause GC (albeit slower):

• non-pool memory use, but since it's only the smaller/less frequently used objects the collections will be farther appart
  
• empty pools, if you need more objects at once then are in the pool
• the scavenger will force a GC every 5 minutes or so

In case you were wondering, here's how I usually use sync.Pool: http://play.golang.org/p/o862V8ELzg

The important thing is that you have to remember that after calling pool.Get, you may get either a previously used object, or if none are available, a new zeroed one, so you must write code which is able to handle both cases. In the example, I set every field in the NewLargeObject "constructor" so that after the call, both a freshly allocated object or a re-used one produce a functionally identical result. You could alternatively zero the object in FreeLargeObject before putting it on the list, and thus only need to set fields in the constructor which is probably a better idea anyway since it is slightly more efficient, and immediately zeroes any pointers in the object which may have a slight positive effect on GC times and memory use.

Here's how that would look (note: in this case, It results in more code, as nearly every field is set by the constructor): http://play.golang.org/p/V5m9uY79o6

[Kevin Malachowski]

If you need an implementation of FREE_INSTANTLY for your code could you use C.malloc and C.free directly? The GC doesn't scan memory outside of the Go heap so if you really know the exact lifetimes of objects you could do that to prevent long pauses.

[minux]

If you want to do manual memory management, fine, write your own memory allocator

or port existing ones using syscall.Mmap (VirtualAlloc on Windows). GC won't ever scan

any objects allocated from that heap.

But I don't think unsafe memory management primitives should ever be introduced to the

Go runtime. There are much better ways than adding unsafe free primitive to reduce GC

pauses times, and one of Go 1.5's goal is to reduce the pause time using concurrent GC.

Btw, having explicit frees actually complicates, not simplifies, the GC implementation.

Look through the history of runtime changes to see how many lines are deleted when

explicit frees are removed.

[Jason E. Aten]

On Mon, Nov 24, 2014 at 3:48 PM, minux <mi...@golang.org> wrote:

If you want to do manual memory management, fine, write your own memory allocator

or port existing ones using syscall.Mmap (VirtualAlloc on Windows). GC won't ever scan

any objects allocated from that heap.

I didn't realize that I could allocate Go structs on another heap if they contained other Go structs, slices, and strings. I assumed, perhaps incorrectly, that this would require a C++ style "placement new", but I wasn't aware of any such thing. How does one tell the memory allocator to recursively use a specific heap for all recursive allocations?

 

But I don't think unsafe memory management primitives should ever be introduced to the

Go runtime. There are much better ways than adding unsafe free primitive to reduce GC

pauses times, and one of Go 1.5's goal is to reduce the pause time using concurrent GC.

I don't really understand the reservations. It is optional. If you don't need it, don't use it. For some programs, pausing for 1500 msec is a real problem.

The state of art in garbage collection currently has its glutimus maximus kicked to the curb by simple exploitation of programmer knowledge about when to free memory.  I don't see any proposal to advance the state of the art. Moreover, I don't see any reason not to use that knowledge when it is available. 

Doing complete manual memory management -- without the assistance of the garbage collector -- is not the goal.  We have a garbage collector that can tell if a free() was incorrect.  We have a programmer who cheaply knows (or wishes to assert; a completely separate and highly useful use case) that memory can be freed.  These are two hands: one hand washes the other.  With hints and GC able to verify and exploit hints, you get the best of both worlds: you can check that your hints are correct, and you can exploit them when you need to for performance.

[Ian Lance Taylor]

On Mon, Nov 24, 2014 at 6:34 PM, Jason E. Aten <j.e....@gmail.com> wrote:
>
> On Mon, Nov 24, 2014 at 3:48 PM, minux <mi...@golang.org> wrote:
>>
>> If you want to do manual memory management, fine, write your own memory
>> allocator
>> or port existing ones using syscall.Mmap (VirtualAlloc on Windows). GC
>> won't ever scan
>> any objects allocated from that heap.
>
>
> I didn't realize that I could allocate Go structs on another heap if they
> contained other Go structs, slices, and strings. I assumed, perhaps
> incorrectly, that this would require a C++ style "placement new", but I
> wasn't aware of any such thing

As you know, Go does not have constructors, so there is no need for
placement new. syscall.Mmap will return a []byte. To convert that
into a pointer to some type T, write something like
p := (*T)(unsafe.Pointer(&b[0]))

> How does one tell the memory allocator to
> recursively use a specific heap for all recursive allocations?

You can't. You would have to manage that yourself, as in C.

>> But I don't think unsafe memory management primitives should ever be
>> introduced to the
>> Go runtime. There are much better ways than adding unsafe free primitive
>> to reduce GC
>> pauses times, and one of Go 1.5's goal is to reduce the pause time using
>> concurrent GC.
>
>
> I don't really understand the reservations. It is optional. If you don't
> need it, don't use it.

Off the top of my head:

* It constrains all future GC implementations.

* It makes the GC API more complex (the current GC API is about as
simple as possible, which is a good thing).

* It adds a rarely used code path to some of the most complex runtime
code.

* The current GC measures every bit that it uses; this new feature
would add a new feature that would need to be tracked in some way,
effectively causing the GC to slow down and/or use more memory for
something that would be rarely used.

Of course these problems would be inconsequential if the feature were
really needed, but given the future GC plans I don't think it is.

> For some programs, pausing for 1500 msec is a real
> problem.

Yes. Hence the plan, described at http://golang.org/s/go14gc, to make
GC pauses very very short in future releases.

Ian

[Tahir]

If the GC has to recheck anyway what's the point of hints ? for some sort of static analysis ?

In that case, that shouldn't be a runtime thing. And manual GC triggering is likely to end up into underperformance

[Tamás Gulácsi]

Maybe you use pointers instead of values?

[Dmitry Vyukov]

In fact, I've just recently deleted that code (free) from runtime,
because it was constant source of very subtle and hard to reproduce
bugs.

> * The current GC measures every bit that it uses; this new feature
> would add a new feature that would need to be tracked in some way,
> effectively causing the GC to slow down and/or use more memory for
> something that would be rarely used.
>
> Of course these problems would be inconsequential if the feature were
> really needed, but given the future GC plans I don't think it is.
>
>
>> For some programs, pausing for 1500 msec is a real
>> problem.
>
> Yes. Hence the plan, described at http://golang.org/s/go14gc, to make
> GC pauses very very short in future releases.
>
> Ian
>

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Frank Schröder]

Are your GC pause times stemming from lots of pointers? What really helped us was converting all map[string]*Foo to map[string]Foo to get the GC under control. We had a couple hundred million of live pointers. 

Frank

[Jason E. Aten]

MMMmmmm. Very interesting suggestion. Yes, I have many millions of live pointers, perhaps going to maps of values instead of maps of pointers could help. Thanks for the suggestion, Frank.

- Jason

On Mon, Nov 24, 2014 at 10:45 PM, Frank Schröder <frank.s...@gmail.com> wrote:

Are your GC pause times stemming from lots of pointers? What really helped us was converting all map[string]*Foo to map[string]Foo to get the GC under control. We had a couple hundred million of live pointers. 

Frank

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/cmpiArv10f4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

[Jason E. Aten]

On Mon, Nov 24, 2014 at 9:09 PM, Tahir <welcometot...@gmail.com> wrote:

If the GC has to recheck anyway what's the point of hints ?

Read the description of FREE_INSTANTLY. There's no recheck in this case. The point of recheck under CHECK_FREE is to allow you to verify that your hints are correct, before running at full speed under FREE_INSTANTLY.

[aruma...@gmail.com]

My plan is to refactor the runtime to an extent so that the GC can be pluggable. For many applications, especially those short-lived, a 'dumb'  GC  could be the optimal(plus that the shrunk runtime size could be helpful for the loading time). For GC sensitive ones, on the other hand, a tailored GC could be profitable. Personally I don't like a super-smart one-size-fits-all GC solution, which would also violate Go's spirit in my opinion.

[Jan Mercl]

On Tue, Nov 25, 2014 at 1:40 PM, <aruma...@gmail.com> wrote:
> My plan is to refactor the runtime to an extent so that the GC can be
> pluggable.

Go designers avoided most GC knobs, modulo debugging stuff, for a good
reason. Once Go programs start to work terribly under particular load
without being given specific values of GC knobs, that reason will
proven again.

-j

[Tahir]

But then you have path dependence, branching issues.. and add on top multiple threads and caches... You mentioned it is not safe, well it is not safe at all.

Seems to me that the combination of escape analysis + concurrent/parallel GC is a better approach and is way safer.

And in the meanwhile, as the posts above indicated, you can avoid allocating, avoid large datastructures of pointers etc. The sync/pool package is quite handy to recycle your data structures and if you preallocate, you also avoid heap fragmentation.

[aruma...@gmail.com]

That makes sense. Your excuse (just kidding) sounds like the one suggested by Linux kernel developers about pluggable scheduler.
Just in case if you misinterpreted my intention. What I mean is not a configurable GC by tuning the parameters, instead, just making it replaceable by another one like a library. I wholeheartedly hate something like conditional compilation macros too.

To be honest, it's very likely the Go designers will receive complaints from some application no matter how good the default GC is.

[Jan Mercl]

On Tue, Nov 25, 2014 at 2:47 PM, <aruma...@gmail.com> wrote:
> What I mean is not a
> configurable GC by tuning the parameters, instead, just making it
> replaceable by another one like a library.

$ go build -gc foo
or
$ prog -gc bar

is as much of a GC knob as any other one is

-j

[Frank Schröder]

My personal experience was that the lack of tuning options made me look harder for issues in our code and in the end with a good result. With Java I didn't look that hard but tried more knobs on the different GCs. More knobs mean more models to reason about. "Good enough" might be just that: good enough.

It depends a bit on what you mean by short lived apps but if I'm not mistaken the GC kicks in after two seconds for the first time.

Frank
--
Frank Schröder
Amsterdam, NL
PGP - DDA53977

> On 25 Nov 2014, at 13:40, aruma...@gmail.com wrote:
>
> My plan is to refactor the runtime to an extent so that the GC can be pluggable. For many applications, especially those short-lived, a 'dumb' GC could be the optimal(plus that the shrunk runtime size could be helpful for the loading time). For GC sensitive ones, on the other hand, a tailored GC could be profitable. Personally I don't like a super-smart one-size-fits-all GC solution, which would also violate Go's spirit in my opinion.
>

[aruma...@gmail.com]

On Tuesday, November 25, 2014 10:35:57 PM UTC+8, Frank Schröder wrote:

My personal experience was that the lack of tuning options made me look harder for issues in our code and in the end with a good result. With Java I didn't look that hard but tried more knobs on the different GCs. More knobs mean more models to reason about. "Good enough" might be just that: good enough.  

 

It depends a bit on what you mean by short lived apps but if I'm not mistaken the GC kicks in after two seconds for the first time.  

Like some simple CLI utilities. A 1.5M sized runtime.a looks not so pretty in some fields.

By the way, I am not trying to convince others to modify the code for me.
It's just my hobby plan and I care more about someone making it happen before me :D

My real intention is to play Go in kernel space and bare metal. And something like a special interface type which allows programmers to implement their own GC algorithms in application code.
They are not expected to be beneficial to the world and humankind.
 

--Frank

[Jason E. Aten]

On Mon, Nov 24, 2014 at 9:06 PM, Ian Lance Taylor <ia...@golang.org> wrote:

As you know, Go does not have constructors, so there is no need for
placement new.  syscall.Mmap will return a []byte.  To convert that
into a pointer to some type T, write something like
    p := (*T)(unsafe.Pointer(&b[0]))

> How does one tell the memory allocator to
> recursively use a specific heap for all recursive allocations?

You can't.  You would have to manage that yourself, as in C.

Hi Ian,

Thank you for the explanation and hints.  

Jason

[Jason E. Aten]

On Mon, Nov 24, 2014 at 10:43 PM, Dmitry Vyukov <dvy...@google.com> wrote:

In fact, I've just recently deleted that code (free) from runtime,
because it was constant source of very subtle and hard to reproduce
bugs.

Hi Dmitry,

I'm puzzled, how does the garbage collector free unused memory then?  Could you point me at the CLI so I can read the change?

Jason

[Ian Lance Taylor]

On Tue, Nov 25, 2014 at 10:32 AM, Jason E. Aten <j.e....@gmail.com> wrote:
> On Mon, Nov 24, 2014 at 10:43 PM, Dmitry Vyukov <dvy...@google.com> wrote:
>>
>>
>> In fact, I've just recently deleted that code (free) from runtime,
>> because it was constant source of very subtle and hard to reproduce
>> bugs.
>
>

> I'm puzzled, how does the garbage collector free unused memory then? Could
> you point me at the CLI so I can read the change?

What Dmitry got rid of was an explicit free function that freed the
memory immediately rather than waiting for it to be garbage collected,
similar to what you are suggesting. The change is
https://codereview.appspot.com/116390043 .

Ian

[Jason E. Aten]

Thanks Dmitry, thanks Ian.  An odd coincidence that the feature I was thinking of was already there, I just didn't know about it before it was removed. Reading the code, one thing does seem slightly odd; I see at src/pkg/runtime/proc.c:1929 on

https://codereview.appspot.com/116390043/diff/120001/src/pkg/runtime/proc.c?context=10&column_width=80

that although the runtime*free() calls are gone, there are still calls to runtime·malloc().

are the mallocs simply leaked now there are no frees?  Or how are they handled?

Thanks,

Jason

[k...@golang.org]

In that CL, runtime.malloc just forwards to mallogc, the garbage-collected malloc.  runtime.malloc has been completely removed at tip.  All call sites were switched to use mallocgc directly.

[Dmitry Vyukov]

changeset: 20519:6acc2dd545b2
user: Dmitriy Vyukov <dvyukov>
date: Thu Jul 31 12:55:40 2014 +0400
summary: runtime: get rid of free