Never ever releasing unused memory can be painful for long running
processes, more especially when they encounter rare peak loads that
come with significant memory overhead.
I'd like to smooth that situation by adding a kind of high-level,
possibly temporary, runtime.ReleaseUnusedMemory(ratio float64) that
would clean-up some of the free spans up to (or around) ~ratio x
heap_idle. To keep it simple, and give those spans a decent chance of
being remapped, considering spans larger than HeapAllocChunck (1MB)
might even be fine enough as a first step.
Not really a fine-tuned magical incremental scavenge system, but I
guess it should do the job: either callable once every "when the
system is idle" (or when a specific phase is reached), or bindable to
a timer to get-back some control on used memory half-life.
Does that sound reasonable to you?
Thanks,
Sebastien
My point was just about investigating a patch to fix an issue I'm
having trouble with, not about complaining about the situation nor
stating it can't wait.
Do you mean that automatic incremental scavenging is the preferred (or
only) way to go because of the non api alteration? I could give a try
to it too, but having something more manual (and optional) to play
with looks to me like a worthwhile cheap intermediary step.
I actually was replying to John and trying to squash the fear I've seen a few times now that "if X doesn't go into Go 1, it'll never go in".
Yes, this is on the TO DO list.
Russ
How would "runtime.ReleaseUnusedMemory(ratio float64)" be different
from the already existing "runtime.GC()" function?
On Nov 28, 9:06 pm, Sébastien Paolacci <sebastien...@gmail.com>
How would "runtime.ReleaseUnusedMemory(ratio float64)" be different
from the already existing "runtime.GC()" function?
I guess the current GC don't give back memory to the OS, later, that
free memory can be used by another allocation without the need to ask
for more memory from the OS.
--
André Moraes
http://andredevchannel.blogspot.com/
> From the link:
>
> "The Linux kernel assumes that you're not going to use all of the memory you
> asked for. Real allocation occurs only when you get access to the allocated
> memory, and if the system does not have enough free memory, then it will run
> Out-of-memory (OOM) killer. OOM killer will try to free some memory for you,
> killing other processes and releasing their memory or it will kill your
> process if it deems the task you are performing to have a lower priority."
>
> If I understand the above correctly, the OOM killer steps in iff the process
> actually uses *more* allocated memory from the kernel. The situation per
> topic is when the process uses *less* memory than it has already got (and
> used before) from the kernel. One more time, the working set management is
> probably much more complex than I do imagine.
The OOM killer may be looking on some /other/ process's
behalf to find killable processes for more memory. It may find
your Go process to be a tasty treat because it (the Go process)
has acquired a lot of virtual memory.
So if (and I speculate) the OOMK has a preference for killing
memory-rich processes on the grounds that that will give back
a lot of memory to the system, one should try and keep one's
processes small enough to not show over the parapet.
Chris
--
Chris "allusive" Dollin
The OOM killer may be looking on some /other/ process's
behalf to find killable processes for more memory. It may find
your Go process to be a tasty treat because it (the Go process)
has acquired a lot of virtual memory.So if (and I speculate) the OOMK has a preference for killing
memory-rich processes on the grounds that that will give back
a lot of memory to the system, one should try and keep one's
processes small enough to not show over the parapet.
I believe the OOM killer is even more conservative than you describe:
it only comes in play when there are (essentially) no clean pages in
memory and all swap has been used. i.e. at this point you've already
been thrashing for a while. There is no way to make memory free
without killing some sort of process, since we can't write dirty pages
to swap and we can't simply discard clean pages. At this point, odds
are that there is a process which has begun leaking memory, and the
OOM killer's job is to find that process. If it kills the wrong
process, then the one that was leaking memory will keep leaking memory
until the OOM killer gets a second try.
Anyhow, I wouldn't see the OOM killer as a major reason to hand back
memory (presume with madvise MADV_DONTNEED). However, avoiding
writing dirty unused memory to swap does seem like a good reason.
David Roundy
We as developers know which part of memory is big and is used only
once.
But for most of the time, we don't want to bother with the details.
However, if the memory is big and only used once, we really want to
release it.
With the on-demand GC, we can put the big-and-used-once into a special
section of memory.
Is this just a stupid dead-end idea?
Chris
On Nov 28, 6:00 pm, Russ Cox <r...@golang.org> wrote:
> On Mon, Nov 28, 2011 at 15:06, Sébastien Paolacci
>
if you don't want the GC to manage you memory in some cases, don't use
it
using unsafe it's not hard to create your own allocator
it's not clear for most people that's much of a win (if at all)
It may be very hard to implement in the compiler and in the run-time.
If all programs are supposed to be safe (that is: the programming
language makes it impossible (by design) for the programmer to make
programming errors such as to accidentally free the same memory area
twice), all invocations of Free(x) need to be checked by the garbage
collector. So, even if you call Free(x) and you are sure that it is
safe to deallocate 'x', the garbage collector has to check for itself
whether deallocating 'x' is actually safe. This means that 'x' cannot
be freed when you call Free(x), it will be freed later (during the
next GC cycle).
The "on-demand GC" you mentioned has basically the same properties/
issues as an invocation of Free(x).
If the assumption is to maintain memory safety, then the only way to
go is advanced escape analysis. If the compiler is unable to tell
whether it is safe to free 'x', then calling Free(x) would need to
generate a compile-time error - or at least a warning - saying that
the compiler failed to determine that it is safe to deallocate 'x'.
When putting "big-and-used-once object into a special section of
memory" and later requesting the garbage collector to deallocate the
object, *the* question is: what method should the compiler use to
determine that it is actually possible to deallocate the object?
I was actually talking about releasing free (unused) memory to the
system. I don't think counting on some OS housekeeping trick for
auto-discovering that you actually don't need the memory you asked
for, and doing something clever/predictable with that, is really a
viable option.
Swap area should also not be used as waste collector but as an
optimization toward infrequently accessed (but useful) pages, as well
as nice protection against huge and abrupt memory spike because of the
anticipatory collection.
Not to mention that overcommitting/swapping is not always possible (or
turned-on) and that the OOM killer will always try hard to shoot the
process the most valuable to you (kind of punishment for not dealing
with the situation yourself;)).
As previously mentioned, my issue is only with long running processes.
In such situations it's not uncommon to get peak memory usages that
are few orders of magnitude away from the steady ones.
Go's current memory allocator (tcmalloc based) does never ever give
back any allocated - but unused anymore - memory to the OS, so
processes have to bear any extra peak consumption for all their live
which you sometime expect to be as long as possible. All of that make
your knapsack problem a bit harder to solve since there's far less
(no) temporal inter-process memory mutualization to be expected.
Having said that, my idea would be to smooth the situation with the
following constraints / considerations:
- keep gc / mem allocator intact: I'm fine with both of them (and gc
is out of scope here).
- no impact on very short lived processes (<<sec), nor noticeable
side-effect on the overall memory management expected behavior.
- being able to release memory when the process is asleep.
- avoid additional fragmentation: long running processes don't like that too.
- no api alteration: transparent / automatic scavenging.
- simple and possibly predictable behavior.
My proposal is as follow: Every regular Tick period a background
goroutine inspect MHeap's stats and decide whether to give back (or
not) some unused memory to the OS, unused memory is released
proportionally to its current value (exponential decay).
- Keep gc / mem allocator intact -> don't force any additional
collection nor move around of free objects.
- No impact on short lived processes / small footprint on the current
memory management behavior -> large tick period, do positionate on the
high-end central heap side, don't flush all the margin in one single
shot.
- Be able to release memory when the process is asleep (e.g waiting
for new connections) -> don't hook the MHeap_Alloc/Free calls since
they're not solicited is such situations, have a dedicated background
goroutine instead.
- Avoid additional fragmentation -> don't give backs spans smaller
than MHeap_Grow's minimum HeapAllocChunk size such that they keep a
decent chance of being (possibly immediately) remapped.
- No api alteration -> no additional runtime call.
- Simple and predictable -> don't try to be clever, just make sure
unused memory will eventually tend to something reasonable. Avoid
(/limit) path-dependant auto-tuning, process intention guessing, I.A
and the likes.
There's obviously a bunch of personal judgment here, as well as room
for subjective parameterization and/or few nasty details like eligible
spans selection, transient allocations handling or hysteresis control.
A more formal sketch up is available on
http://groups.google.com/group/golang-dev/browse_thread/thread/bb8b7aae1d783894
Comments/suggestions warmly welcomed, please however try to separate
ideas from potential implementations.
Thanks,
Sebastien