Memory overcommit
David Given
I was rather startled recently to notice that the standard Android
kernel appears to have the memory overcommit setting set to 1. This is
--- as far as I can tell, the numbering got changed not long ago and not
all the documentation has been updated --- these means 'allow all memory
allocations even if no RAM+swap is available'.
The knock on effect of this is that running out of memory will cause
either a memory trap in the application or else the OOM killer will nuke
your entire process without warning.
Can anyone comment on this? I know, for example, that the Android patch
has modified the OOM killer. I would have thought that memory overcommit
should be disabled on this kind of embedded device?
--
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│
│ "They laughed at Newton. They laughed at Einstein. Of course, they
│ also laughed at Bozo the Clown." --- Carl Sagan
David Turner
Hello,
I was rather startled recently to notice that the standard Android
kernel appears to have the memory overcommit setting set to 1. This is
--- as far as I can tell, the numbering got changed not long ago and not
all the documentation has been updated --- these means 'allow all memory
allocations even if no RAM+swap is available'.
The knock on effect of this is that running out of memory will cause
either a memory trap in the application or else the OOM killer will nuke
your entire process without warning.
Can anyone comment on this? I know, for example, that the Android patch
has modified the OOM killer. I would have thought that memory overcommit
should be disabled on this kind of embedded device?
memory overcommit is used by design, there is no plan to remove it. The fact
that the OOM killer will nuke processes to make room for others is part of the
platform's design. As far as I know, the OOM changes were to make the killer
a bit smarter about what kind of processes it would try to kill first.
David Turner
Hello,
I was rather startled recently to notice that the standard Android
kernel appears to have the memory overcommit setting set to 1. This is
--- as far as I can tell, the numbering got changed not long ago and not
all the documentation has been updated --- these means 'allow all memory
allocations even if no RAM+swap is available'.
The knock on effect of this is that running out of memory will cause
either a memory trap in the application or else the OOM killer will nuke
your entire process without warning.
Can anyone comment on this? I know, for example, that the Android patch
has modified the OOM killer. I would have thought that memory overcommit
should be disabled on this kind of embedded device?
To make things cleared, memory overcommit is a good thing in environments
where there are tons of copy-on-write shared pages between processes (which
is typical of Android), with very few copies in practice.
If we were to disable it, the system would barely run with the current amount
of memory.
David Given
[...]
> memory overcommit is used by design, there is no plan to remove it. The fact
> that the OOM killer will nuke processes to make room for others is part
> of the
> platform's design. As far as I know, the OOM changes were to make the killer
> a bit smarter about what kind of processes it would try to kill first.
Yes, but that doesn't really answer my question --- *why* is memory
overcommit used on Android? It's normally used on systems with huge
amounts of swap to make more efficient use of physical memory, but
Android devices don't have any swap.
The specific problem I've got here is that an app can successfully
allocate memory which it then can't use. Memory overcommit hides memory
allocation failures from the app. The first the app knows that the
memory isn't available is when the OOM killer terminates it without warning!
--
David Given
d...@cowlark.com
Dianne Hackborn
Yes, but that doesn't really answer my question --- *why* is memory
overcommit used on Android? It's normally used on systems with huge
amounts of swap to make more efficient use of physical memory, but
Android devices don't have any swap.
The specific problem I've got here is that an app can successfully
allocate memory which it then can't use. Memory overcommit hides memory
allocation failures from the app. The first the app knows that the
memory isn't available is when the OOM killer terminates it without warning!
Dianne Hackborn
Android framework engineer
hac...@android.com
Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails. All such questions should be posted on public forums, where I and others can see and answer them.
David Given
Hash: SHA1
Dianne Hackborn wrote:
[...]
> If you turn off over-commit, I believe Android won't even boot on a G1,
> because Linux would need to assume that RAM is needed for every mmapped()
> thing, and we run out of it well before the system is fully up.
Not necessarily --- my intuitive understanding would be that overcommit
would only applies to anonymous storage and not to file storage, as
mmapped() files are backed by actual *files* and not by the pool of swap
pages. (Besides, most of those mmapped() files that Android's using are
read-only and their pages can be silently discarded at any point.)
[...]
> Put another way: if you are really using so much memory that the kernel
> needs to kill your app, you are well past the line of being a good citizen
Oh, absolutely. But it would be rather nice to have a medium line
between happy-happy-fun-joy and the boys coming round saying, "'ere, Mr.
Torvalds would like a *word* with you..."
(What we've actually got is a portable API implementation for native
apps. The memory allocation function is defined to return NULL on
allocation failure. It had never occurred to us that *making* it return
NULL on allocation failure could be hard.)
- --
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│
│ "People who think they know everything really annoy those of us who
│ know we don't." --- Bjarne Stroustrup
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iD8DBQFKrslpf9E0noFvlzgRAuuOAKCAEARJSEPcHxcrEowsxwcMBvkxGwCcC1Jz
kuny8oLQqn1S8mBDmqp9Bwk=
=iCwo
-----END PGP SIGNATURE-----
David Turner
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Dianne Hackborn wrote:
[...]
> If you turn off over-commit, I believe Android won't even boot on a G1,Not necessarily --- my intuitive understanding would be that overcommit
> because Linux would need to assume that RAM is needed for every mmapped()
> thing, and we run out of it well before the system is fully up.
would only applies to anonymous storage and not to file storage, as
mmapped() files are backed by actual *files* and not by the pool of swap
pages. (Besides, most of those mmapped() files that Android's using are
read-only and their pages can be silently discarded at any point.)
That can only be said for files that are mapped read-only. It doesn't apply for
anything that is shared copy-on-write (e.g. the Zygote process pages, as well
as initial heap) and non-file mappings. This would increase the memory
pressure tremendously, given that every VM process has several megabytes
of shared copy-on-write pages.
(What we've actually got is a portable API implementation for native
apps. The memory allocation function is defined to return NULL on
allocation failure. It had never occurred to us that *making* it return
NULL on allocation failure could be hard.)
But it *is* hard to define what "allocation failure" really means if you have
memory overcommit enabled. This behaviour also happens on all modern operating
systems these days. If you have a swap file, you will simply make it happen later,
after much much disk chugging that will make the user totally nuts anyway.
David Given
Hash: SHA1
David Turner wrote:
[...]
> That can only be said for files that are mapped read-only. It doesn't apply
> for
> anything that is shared copy-on-write (e.g. the Zygote process pages, as
> well
> as initial heap) and non-file mappings.
Ah, I'd forgotten about copy-on-write.
[...]
> But it *is* hard to define what "allocation failure" really means if you
> have
> memory overcommit enabled.
Yes, that's precisely my point!
In fact, I find the OOM killer behaviour deeply suspect. I do not
believe that randomly killing processes is *ever* the right thing to do
on a reliable system, as it completely denies the application the
ability to properly clean up after itself. What happens if the app is in
the middle of a file operation at the time? You lose, that's what. (Not
*everything* can be journalled.)
If I was going to have overcommit enabled, I'd much rather have the
allocation failures exposed to the application in the form of the
appropriate signal when the application fails to access the page. SIGBUS
('bad memory access') would seem to be the suitable one here. That
allows the application to catch it and do any applicable cleanup, and
any application that doesn't care will be killed anyway.
This would also have the advantage that you'd be much more likely to
terminate the offending process; in my experience the OOM killer has a
nasty tendency to home in on the wrong process. Hopefully that's been
fixed in the Android patches.
Incidentally, does Android use rlimits to impose per-process memory limits?
- --
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ───── http://www.cowlark.com ─────
│
│ "People who think they know everything really annoy those of us who
│ know we don't." --- Bjarne Stroustrup
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iD8DBQFKrth5f9E0noFvlzgRAjblAJ44qz7poU+R6hKbmAJTGrO5FwwerwCeKVSb
6ucGqn6l8sWJ8mlanQ8PE5s=
=Y3tU
-----END PGP SIGNATURE-----
Dianne Hackborn
In fact, I find the OOM killer behaviour deeply suspect. I do not
believe that randomly killing processes is *ever* the right thing to do
on a reliable system, as it completely denies the application the
ability to properly clean up after itself. What happens if the app is in
the middle of a file operation at the time? You lose, that's what. (Not
*everything* can be journalled.)
We don't randomly kill processes, we kill only things in the background unless memory is getting extremely tight in which case we need to resort to increasingly more valuable processes.
And yes, this absolutely denies the application the ability to clean up itself. More importantly, it denies the application the ability to delay or prevent itself from going away, which is very key to the system being able to keep itself running well, managing these processes, without the user having to worry about it.
If I was going to have overcommit enabled, I'd much rather have the
allocation failures exposed to the application in the form of the
appropriate signal when the application fails to access the page. SIGBUS
('bad memory access') would seem to be the suitable one here. That
allows the application to catch it and do any applicable cleanup, and
any application that doesn't care will be killed anyway.
There generally are no allocation failures happening, there is just the system doing what it needs to do to prevent itself from getting into too low a memory state and thus paging or otherwise not performing well.
This would also have the advantage that you'd be much more likely to
terminate the offending process; in my experience the OOM killer has a
nasty tendency to home in on the wrong process. Hopefully that's been
fixed in the Android patches.
We don't use the stock Linux out of memory killer, because it pretty universally does the wrong thing.
If you want to understand how this part of Android works, you should first read the docs on process management here:
http://developer.android.com/guide/topics/fundamentals.html#lcycles
fadden
> If I was going to have overcommit enabled, I'd much rather have the
> allocation failures exposed to the application in the form of the
> appropriate signal when the application fails to access the page.
probably get the behavior you desire by doing so.
% adb shell ulimit -a -H
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) unlimited
coredump(blocks) unlimited
memory(kbytes) unlimited
locked memory(kbytes) 64
process(processes) 808
nofiles(descriptors) 1024