Memory allocator / Giving back some memory to system.

1152 views
Skip to first unread message

Sébastien Paolacci

unread,
Nov 28, 2011, 3:06:22 PM11/28/11
to golan...@googlegroups.com
Hello,

Never ever releasing unused memory can be painful for long running
processes, more especially when they encounter rare peak loads that
come with significant memory overhead.

I'd like to smooth that situation by adding a kind of high-level,
possibly temporary, runtime.ReleaseUnusedMemory(ratio float64) that
would clean-up some of the free spans up to (or around) ~ratio x
heap_idle. To keep it simple, and give those spans a decent chance of
being remapped, considering spans larger than HeapAllocChunck (1MB)
might even be fine enough as a first step.

Not really a fine-tuned magical incremental scavenge system, but I
guess it should do the job: either callable once every "when the
system is idle" (or when a specific phase is reached), or bindable to
a timer to get-back some control on used memory half-life.

Does that sound reasonable to you?

Thanks,
Sebastien

John Asmuth

unread,
Nov 28, 2011, 3:13:52 PM11/28/11
to golan...@googlegroups.com
I don't know about having this be something that a programmer has to request, but some way to give the memory back seems like a really important feature for Go 1.

Brad Fitzpatrick

unread,
Nov 28, 2011, 3:22:50 PM11/28/11
to golan...@googlegroups.com
The definition of Go 1 isn't about having every thing done & wonderful.

The definition of Go 1 is more about cleaning up and axing old cruft and having a stable base to move forward from.

Go 1 code should be valid Go 1.1 code
Go 1 code should be valid Go 2 code.

If Go 1 doesn't release memory to the OS but Go 1.1 does, that doesn't invalidate people's programs or authors' books.

That said, this is a known issue and recent CLs have referenced it (such as the recent CL moving time & times into the runtime).

Sebastien Paolacci

unread,
Nov 28, 2011, 4:16:13 PM11/28/11
to golang-nuts
I have to admit that I'm bit confuse with the Go 1 portion, as well as
with the time one...

My point was just about investigating a patch to fix an issue I'm
having trouble with, not about complaining about the situation nor
stating it can't wait.
Do you mean that automatic incremental scavenging is the preferred (or
only) way to go because of the non api alteration? I could give a try
to it too, but having something more manual (and optional) to play
with looks to me like a worthwhile cheap intermediary step.

Brad Fitzpatrick

unread,
Nov 28, 2011, 4:28:48 PM11/28/11
to Sebastien Paolacci, golang-nuts
I actually was replying to John and trying to squash the fear I've seen a few times now that "if X doesn't go into Go 1, it'll never go in".

Russ would have actual comments on your proposal.  He's been thinking about it and working towards it.  See:


... "(for example, for the garbage collector to hand back unused memory to the OS on a time delay)"

John Asmuth

unread,
Nov 28, 2011, 5:25:40 PM11/28/11
to golan...@googlegroups.com
On Monday, November 28, 2011 4:28:48 PM UTC-5, Brad Fitzpatrick wrote:
I actually was replying to John and trying to squash the fear I've seen a few times now that "if X doesn't go into Go 1, it'll never go in".

Success :) 

Russ Cox

unread,
Nov 28, 2011, 9:00:56 PM11/28/11
to Sébastien Paolacci, golan...@googlegroups.com
On Mon, Nov 28, 2011 at 15:06, Sébastien Paolacci
<sebastien...@gmail.com> wrote:
> Never ever releasing unused memory can be painful for long running
> processes, more especially when they encounter rare peak loads that
> come with significant memory overhead.

Yes, this is on the TO DO list.

Russ

unread,
Nov 29, 2011, 7:30:59 AM11/29/11
to golang-nuts
On Nov 28, 9:06 pm, Sébastien Paolacci <sebastien.paola...@gmail.com>
wrote:

> I'd like to smooth that situation by adding a kind of high-level,
> possibly temporary, runtime.ReleaseUnusedMemory(ratio float64) that
> would clean-up some of the free spans up to (or around) ~ratio x
> heap_idle. To keep it simple, and give those spans a decent chance of
> being remapped, considering spans larger than HeapAllocChunck (1MB)
> might even be fine enough as a first step.

How would "runtime.ReleaseUnusedMemory(ratio float64)" be different
from the already existing "runtime.GC()" function?

Jan Mercl

unread,
Nov 29, 2011, 7:58:31 AM11/29/11
to golan...@googlegroups.com
On Tuesday, November 29, 2011 1:30:59 PM UTC+1, ⚛ wrote:
On Nov 28, 9:06 pm, Sébastien Paolacci <sebastien...@gmail.com>

How would "runtime.ReleaseUnusedMemory(ratio float64)" be different
from the already existing "runtime.GC()" function?

I think runtime.GC() will release unreachable memory objects from the allocated part of the heap (but keep all of the heap allocated from the kernel's POV). runtime.ReleaseUnusedMemory() will release a ratio of unused memory back to the kernel.

Anyway I'm not sure how much really is this needed. I expect the kernel to purge unused/not recently touched pages from the working set under high memory load. Except for writing them possibly unnecessarily to swap, there should be no big difference. And when the memory load is low then nothing happens when the process keeps it virtual memory space. Probably it's all more complicated than I think it is.

André Moraes

unread,
Nov 29, 2011, 8:00:51 AM11/29/11
to golang-nuts
> How would "runtime.ReleaseUnusedMemory(ratio float64)" be different
> from the already existing "runtime.GC()" function?

I guess the current GC don't give back memory to the OS, later, that
free memory can be used by another allocation without the need to ask
for more memory from the OS.

--
André Moraes
http://andredevchannel.blogspot.com/

Johann Höchtl

unread,
Nov 29, 2011, 8:09:46 AM11/29/11
to golan...@googlegroups.com
Under high load and pressure the Linux vm subsystem will start to kill processes. It does so by a definedand in margings customizable way:

http://www.codeproject.com/KB/TipsnTricks/linux-memory-management.aspx (OOM killer)

Processes occupying a lot of memory will get a higher badness than others. Thus releasing memory back to the kernel VM subsystem is a good thing.


Jan Mercl

unread,
Nov 29, 2011, 8:27:11 AM11/29/11
to golan...@googlegroups.com
From the link:

"The Linux kernel assumes that you're not going to use all of the memory you asked for. Real allocation occurs only when you get access to the allocated memory, and if the system does not have enough free memory, then it will run Out-of-memory (OOM) killer. OOM killer will try to free some memory for you, killing other processes and releasing their memory or it will kill your process if it deems the task you are performing to have a lower priority."

If I understand the above correctly, the OOM killer steps in iff the process actually uses *more* allocated memory from the kernel. The situation per topic is when the process uses *less* memory than it has already got (and used before) from the kernel. One more time, the working set management is probably much more complex than I do imagine.

chris dollin

unread,
Nov 29, 2011, 8:53:34 AM11/29/11
to golan...@googlegroups.com
On 29 November 2011 13:27, Jan Mercl <jan....@nic.cz> wrote:

> From the link:
>
> "The Linux kernel assumes that you're not going to use all of the memory you
> asked for. Real allocation occurs only when you get access to the allocated
> memory, and if the system does not have enough free memory, then it will run
> Out-of-memory (OOM) killer. OOM killer will try to free some memory for you,
> killing other processes and releasing their memory or it will kill your
> process if it deems the task you are performing to have a lower priority."
>
> If I understand the above correctly, the OOM killer steps in iff the process
> actually uses *more* allocated memory from the kernel. The situation per
> topic is when the process uses *less* memory than it has already got (and
> used before) from the kernel. One more time, the working set management is
> probably much more complex than I do imagine.

The OOM killer may be looking on some /other/ process's
behalf to find killable processes for more memory. It may find
your Go process to be a tasty treat because it (the Go process)
has acquired a lot of virtual memory.

So if (and I speculate) the OOMK has a preference for killing
memory-rich processes on the grounds that that will give back
a lot of memory to the system, one should try and keep one's
processes small enough to not show over the parapet.

Chris

--
Chris "allusive" Dollin

Jan Mercl

unread,
Nov 29, 2011, 9:09:09 AM11/29/11
to golan...@googlegroups.com
On Tuesday, November 29, 2011 2:53:34 PM UTC+1, ehedgehog wrote:

The OOM killer may be looking on some /other/ process's
behalf to find killable processes for more memory. It may find
your Go process to be a tasty treat because it (the Go process)
has acquired  a lot of virtual memory.

So if (and I speculate) the OOMK has a preference for killing
memory-rich processes on the grounds that that will give back
a lot of memory to the system, one should try and keep one's
processes small enough to not show over the parapet.

I agree it could be like that. I hope that it is not. The situation per topic should be IMO preferably solved by the kernel (in the situation presented by you when other process wants more memory under higher load), by purging unused (not recently touched) pages from the working set (taking them from e.g. this Go process which had some old memory consumption peak) and giving them to that other process which is now in demand for more of them. Only when there are no such "old" pages left the OOM killer should step in to avoid thrashing. I really don't know the details of the Linux implementation, maybe it is simpler than that - or much more complex.

(Just to be clear, by working set I mean the dynamic set of all pages of all processes virtual memory spaces which currently do have the physical memory page backing.)

David Roundy

unread,
Nov 29, 2011, 11:03:42 AM11/29/11
to golan...@googlegroups.com
On Tue, Nov 29, 2011 at 6:09 AM, Jan Mercl <jan....@nic.cz> wrote:
> I agree it could be like that. I hope that it is not. The situation per
> topic should be IMO preferably solved by the kernel (in the
> situation presented by you when other process wants more memory under higher
> load), by purging unused (not recently touched) pages from the working set
> (taking them from e.g. this Go process which had some old memory consumption
> peak) and giving them to that other process which is now in demand for more
> of them. Only when there are no such "old" pages left the OOM killer should
> step in to avoid thrashing. I really don't know the details of the Linux
> implementation, maybe it is simpler than that - or much more complex.
> (Just to be clear, by working set I mean the dynamic set of all pages of all
> processes virtual memory spaces which currently do have the physical memory
> page backing.)

I believe the OOM killer is even more conservative than you describe:
it only comes in play when there are (essentially) no clean pages in
memory and all swap has been used. i.e. at this point you've already
been thrashing for a while. There is no way to make memory free
without killing some sort of process, since we can't write dirty pages
to swap and we can't simply discard clean pages. At this point, odds
are that there is a process which has begun leaking memory, and the
OOM killer's job is to find that process. If it kills the wrong
process, then the one that was leaking memory will keep leaking memory
until the OOM killer gets a second try.

Anyhow, I wouldn't see the OOM killer as a major reason to hand back
memory (presume with madvise MADV_DONTNEED). However, avoiding
writing dirty unused memory to swap does seem like a good reason.

David Roundy

ChrisLu

unread,
Nov 29, 2011, 7:23:28 PM11/29/11
to golang-nuts
Seems languages either support GC or not. However, is it possible to
divide the memory into two,
One is on-demand GC, one is automatic GC?

We as developers know which part of memory is big and is used only
once.
But for most of the time, we don't want to bother with the details.
However, if the memory is big and only used once, we really want to
release it.

With the on-demand GC, we can put the big-and-used-once into a special
section of memory.

Is this just a stupid dead-end idea?

Chris

On Nov 28, 6:00 pm, Russ Cox <r...@golang.org> wrote:
> On Mon, Nov 28, 2011 at 15:06, Sébastien Paolacci
>

Chris Wedgwood

unread,
Nov 29, 2011, 7:47:21 PM11/29/11
to ChrisLu, golang-nuts
> Seems languages either support GC or not. However, is it possible to
> divide the memory into two,
> One is on-demand GC, one is automatic GC?

if you don't want the GC to manage you memory in some cases, don't use
it

using unsafe it's not hard to create your own allocator


it's not clear for most people that's much of a win (if at all)

unread,
Nov 30, 2011, 4:55:21 AM11/30/11
to golang-nuts
On Nov 30, 1:23 am, ChrisLu <chris...@gmail.com> wrote:
> Seems languages either support GC or not. However, is it possible to
> divide the memory into two,
> One is on-demand GC, one is automatic GC?
>
> We as developers know which part of memory is big and is used only
> once.
> But for most of the time, we don't want to bother with the details.
> However, if the memory is big and only used once, we really want to
> release it.
>
> With the on-demand GC, we can put the big-and-used-once into a special
> section of memory.
>
> Is this just a stupid dead-end idea?
>
> Chris

It may be very hard to implement in the compiler and in the run-time.

If all programs are supposed to be safe (that is: the programming
language makes it impossible (by design) for the programmer to make
programming errors such as to accidentally free the same memory area
twice), all invocations of Free(x) need to be checked by the garbage
collector. So, even if you call Free(x) and you are sure that it is
safe to deallocate 'x', the garbage collector has to check for itself
whether deallocating 'x' is actually safe. This means that 'x' cannot
be freed when you call Free(x), it will be freed later (during the
next GC cycle).

The "on-demand GC" you mentioned has basically the same properties/
issues as an invocation of Free(x).

If the assumption is to maintain memory safety, then the only way to
go is advanced escape analysis. If the compiler is unable to tell
whether it is safe to free 'x', then calling Free(x) would need to
generate a compile-time error - or at least a warning - saying that
the compiler failed to determine that it is safe to deallocate 'x'.

When putting "big-and-used-once object into a special section of
memory" and later requesting the garbage collector to deallocate the
object, *the* question is: what method should the compiler use to
determine that it is actually possible to deallocate the object?

Johann Höchtl

unread,
Nov 30, 2011, 9:35:07 AM11/30/11
to golan...@googlegroups.com
not at all and this idea is in effect what others generational GCs do. Memory requested is first kept in the nursery. If the allocated memory there is active for some time it's propagated to the second generation memory area. The infant arena is swept very often. This arena suffers of high fragmentation and thus get's compacted from time to time.Typically small amounts of memory will be in the infant arena, while long lived objects in the second generation are most of the time large objects. If they die, the memory can be released back to the OS, as this memory area is not fragmented.

A very good and understandable explanation of a modern GC is given there:

http://mono-project.com/Generational_GC

http://schani.wordpress.com/2010/12/20/sgen/
http://schani.wordpress.com/2010/12/29/sgen-the-nursery/
http://schani.wordpress.com/2011/01/10/sgen-the-major-collectors/
http://schani.wordpress.com/2011/02/22/sgen-%E2%80%93-finalization-and-weak-references/

Sébastien Paolacci

unread,
Dec 3, 2011, 6:10:35 AM12/3/11
to golan...@googlegroups.com, r...@golang.org
Hello,

I was actually talking about releasing free (unused) memory to the
system. I don't think counting on some OS housekeeping trick for
auto-discovering that you actually don't need the memory you asked
for, and doing something clever/predictable with that, is really a
viable option.

Swap area should also not be used as waste collector but as an
optimization toward infrequently accessed (but useful) pages, as well
as nice protection against huge and abrupt memory spike because of the
anticipatory collection.

Not to mention that overcommitting/swapping is not always possible (or
turned-on) and that the OOM killer will always try hard to shoot the
process the most valuable to you (kind of punishment for not dealing
with the situation yourself;)).

As previously mentioned, my issue is only with long running processes.
In such situations it's not uncommon to get peak memory usages that
are few orders of magnitude away from the steady ones.

Go's current memory allocator (tcmalloc based) does never ever give
back any allocated - but unused anymore - memory to the OS, so
processes have to bear any extra peak consumption for all their live
which you sometime expect to be as long as possible. All of that make
your knapsack problem a bit harder to solve since there's far less
(no) temporal inter-process memory mutualization to be expected.


Having said that, my idea would be to smooth the situation with the
following constraints / considerations:

- keep gc / mem allocator intact: I'm fine with both of them (and gc
is out of scope here).
- no impact on very short lived processes (<<sec), nor noticeable
side-effect on the overall memory management expected behavior.
- being able to release memory when the process is asleep.
- avoid additional fragmentation: long running processes don't like that too.
- no api alteration: transparent / automatic scavenging.
- simple and possibly predictable behavior.


My proposal is as follow: Every regular Tick period a background
goroutine inspect MHeap's stats and decide whether to give back (or
not) some unused memory to the OS, unused memory is released
proportionally to its current value (exponential decay).

- Keep gc / mem allocator intact -> don't force any additional
collection nor move around of free objects.

- No impact on short lived processes / small footprint on the current
memory management behavior -> large tick period, do positionate on the
high-end central heap side, don't flush all the margin in one single
shot.

- Be able to release memory when the process is asleep (e.g waiting
for new connections) -> don't hook the MHeap_Alloc/Free calls since
they're not solicited is such situations, have a dedicated background
goroutine instead.

- Avoid additional fragmentation -> don't give backs spans smaller
than MHeap_Grow's minimum HeapAllocChunk size such that they keep a
decent chance of being (possibly immediately) remapped.

- No api alteration -> no additional runtime call.

- Simple and predictable -> don't try to be clever, just make sure
unused memory will eventually tend to something reasonable. Avoid
(/limit) path-dependant auto-tuning, process intention guessing, I.A
and the likes.


There's obviously a bunch of personal judgment here, as well as room
for subjective parameterization and/or few nasty details like eligible
spans selection, transient allocations handling or hysteresis control.

A more formal sketch up is available on
http://groups.google.com/group/golang-dev/browse_thread/thread/bb8b7aae1d783894

Comments/suggestions warmly welcomed, please however try to separate
ideas from potential implementations.

Thanks,
Sebastien

Reply all
Reply to author
Forward
0 new messages