Debugging Memory Leaks in system_process

[Jonathan Herriott]

Hi,

I am modifying the keyguard in a way that requires me to use an
AsyncTask, ServiceConnection, etc., and when I run my code, if I keep
interacting/opening/closing the keyguard, memory is leaked and
eventually kills the system_process due to an out-of-memory exception.

On cleanup, I am both cancelling the AsyncTask as well as unbinding
the ServiceConnection. Beyond this, it is my understanding that after
cleanUp() is called on the keyguardView, there will no longer be a
pointer to this object, so it can be reclaimed by the VM as well as
any other object this object points to that no longer has a pointer
outside of my object. I do pass Callback objects to the service I am
binding to, but my understanding is that after unbinding to the
service, I need not worry about these since at this point, they are
outside of my control.

So, because I don't know what could be causing the memory leak (either
through a lack of understanding or whatever), I'm trying to debug
this, but DDMS doesn't seem to have a good way of debugging the
system_process. My question is, what is the standard method for
debugging memory leaks in the system_process? it seems that DDMS
really only shows the heap size and how much memory is allocated, and
from there I verify the leak, but I was wondering if there is anything
else that could be done to help debug my leak.

Thanks,
Jonathan Herriott

[fadden]

On Mar 29, 1:29 pm, Jonathan Herriott <herri...@gmail.com> wrote:
> I am modifying the keyguard in a way that requires me to use an
> AsyncTask, ServiceConnection, etc., and when I run my code, if I keep
> interacting/opening/closing the keyguard, memory is leaked and
> eventually kills the system_process due to an out-of-memory exception.

You can generate an hprof dump with
android.os.Debug.dumpHprofData(filename), convert it to "normal"
format with hprof-conv, and then examine with a tool like jhat or
Eclipse MAT to see where your memory went.

On most apps you can ask DDMS to do most of this for you, but it
requires the write-external-storage permission, and system_server
isn't allowed to open files on /sdcard. (The dump mechanism will be
fixed in a future release.)

Are there any messages in logcat at the point of the OOM?

[Jonathan Herriott]

Yes, actually, I get

135424-byte external allocation too large for this process.
VM won't let us allocate 135424 bytes
threadid=39: thread exiting with uncaught exception (group=0x4001b188)
Uncaught handler: thread WindowManagerPolicy exited due to uncaught
exception
*** EXCEPTION IN SYSTEM PROCESS. System will crash.
android.view.InflateException: Binary XML file line #7: Error
inflating class <unknown>
...
(More Stack Dump)
...
Caused by: java.lang.OutOfMemoryError: bitmap size exceeds VM budget
at android.graphics.BitmapFactory.nativeDecodeAsset (Native
Method)
at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:
447)

But that doesn't help since the memory leak isn't necessarily
occurring here. It is occurring in WindowManagerPolicy since that is
where my modified code resides. The bulk of the code actually resides
in my extension of the KeyguardViewBase. The only modification of the
code outside of this is just passing a string around and into my
implementation of the KeyguardView. So, I expect the memory leak in
my implementation of the KeyguardView.

Anyways, I'll check out generating the hprof you were talking about
and see if that helps at all.

Thanks,
Jonathan Herriott

[Jonathan Herriott]

What I've found using Eclipse MAT is my class that I am loading from a
separate package via a custom ContextWrapper, is leaking even though
it is an object with no incoming references. This shouldn't happen.
Am I misunderstanding something?

So, what I am doing is upon the KeyguardView being created, I wrap the
context with my own wrapper, so I can load classes/inflate views from
a specific package. It seems that any class I load from the
PathClassLoader leaks even though, if I am reading the results
correctly, there are no incoming references to the object.

I load the class in the following manner (not including the try/catch
blocks):

ClassLoader l = mMyContextWrapper.getClassLoader();
Class c = l.loadClass("com.mypackage.SomeClass");
mSomeClass = (SomeClass)c.newInstance();

Now, once the object that holds the member variable mSomeClass no
longer has any references to it, my assumption is that mSomeClass
should also be released from memory, but from what I am experiencing,
it is not, and so, the PathClassLoader returned by
mMyContextWrapper.getClassLoader() also is leaked. What leads me to
believe this is when I select one of the leaked SomeClass instances in
Eclipse MAT, select "List objects with incoming references", and then
I try and expand the class; it shows no incoming references. My
assumption is that I am misunderstanding something, so I just need to
figure out what it is.

[Jonathan Herriott]

Well, it is alleviated by making the PathClassLoader static as well as
the SomeClass instance static. If there is only a single reference to
each, it can't increase memory any more. My question, however, is
what can cause it to leak like this? It makes absolutely no sense to
me.

[Jonathan Herriott]

Another thing I find interesting is the fact that when I create an
hprof file, I can't seem to get it to crash, but as soon as I remove
the creation of the hprof file, it will crash. Does a GC run as soon
as an hprof file is created? Also, when I get that memory error
creating the bitmap, the system_process heap only has a size of about
4MB. Also, it seems to die creating a bitmap everytime, but I have
been chalking this up to the fact that the bitmaps will allocate a
large amount of memory at a single time, and the crash happens when
the system_process is taking up about 82% of its heap space. I'm
assuming the failure is due to fragmentation, but when the heap is
only about 4MB, why isn't it increasing the heap size for the
allocation?

[Romain Guy]

A GC is run as part of the hprof generation, otherwise you would see
"garbage" in the heap dump.

>  I'm
> assuming the failure is due to fragmentation, but when the heap is
> only about 4MB, why isn't it increasing the heap size for the
> allocation?

Bitmaps are not allocated in the Java heap but counted towards the
limit of the Java heap.

> --
> You received this message because you are subscribed to the Google Groups "android-platform" group.
> To post to this group, send email to android-...@googlegroups.com.
> To unsubscribe from this group, send email to android-platfo...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/android-platform?hl=en.
>
>

--
Romain Guy
Android framework engineer
roma...@android.com

Note: please don't send private questions to me, as I don't have time
to provide private support.  All such questions should be posted on
public forums, where I and others can see and answer them

[Jonathan Herriott]

Hey Romain,

The interesting part is when I run hprof in the code, it has no issues
with memory; the size stays stable at a certain point when I keep
opening and closing the keyguard, and I've even sat there opening/
closing it for 15+ minutes with it staying stables. I had tested the
same operation sans hprof; and via ddms, I can easily see the memory
leaking, and it crashes pretty quickly. Now, assuming the only
difference between the two versions of code is essentially a GC from
the heap's perspective, I'd assume that manually running a GC would
fix the issue (or rather put a band-aid on it). However, when I
manually run a GC instead of dumping the hprof, I still can watch the
used heap increase in size and eventually crash. This makes it
extremely difficult to figure out where the memory leak is coming
from.

That being said, I'd prefer not to put a band-aid on this and actually
get it right, so I'd prefer not to do a manual GC or anything similar.

In KeyguardViewManager, there are two methods which are important to
showing the Keyguard: show() and hide(), which I have left completely
unchanged from the original code. The only things I have changed are
the fact that KeyguardViewProperties.createKeyguardView() returns my
custom KeyguardView which extends KeyguardViewBase. If you look at
these functions, hide() calls mKeyguardHost.removeView(mKeyguardView)
and then sets mKeyguardView to null (not necessarily in that order).
At this point, on the next GC, mKeyguardView and anything it
references that is not referenced outside of it, should be GC'ed since
I don't give references to anything inside my custom KeyguardView to
anything outside of its instance. That being said, my custom
KeyguardView does some asynchronous stuff as well as bind to an
interface, which should be fine because I've verified that both are
taken care of during cleanUp() and there aren't any more instances in
the hprof than I'd expect (but again, the heap management with hprof
is different).

I've also verified, again, through opening and closing of the keyguard
for 15+ minutes, that the default keyguard that Android standardly
returns has no issues with memory leaking as far as opening/closing
the keyguard for long periods of time. Is there something inherently
wrong with threading, binding to a service, and/or creating a
PathClassLoader and loading classes in the window policy when it comes
to the Keyguard UI? Can I not stack UI on top of the KeyguardView
that were generated from a different Context by loading resources/
layouts/classes from an apk?

Thanks,
Jonathan Herriott

> > For more options, visit this group athttp://groups.google.com/group/android-platform?hl=en.

>
> --
> Romain Guy
> Android framework engineer

> romain...@android.com

[Jonathan Herriott]

Nevermind about the System.gc(). I think I had forgotten to compile
after that. It appears to be working as a band-aid at the moment.
Well, one way or the other, it is really too premature to say.

[fadden]

On Mar 30, 3:10 pm, Jonathan Herriott <herri...@gmail.com> wrote:
> Nevermind about the System.gc().  I think I had forgotten to compile
> after that.  It appears to be working as a band-aid at the moment.
> Well, one way or the other, it is really too premature to say.

There are (sadly) many threads about the interaction of bitmaps,
"external allocation", and GCs on the Android mailing list (search
android-developers for the error message you're seeing). In
particular note discussions about Bitmaps and how the storage can be
kept around while waiting for finalization to occur.

You can use "gclog.py" to see how your heap and the external
allocation area are growing. (http://android.git.kernel.org/?
p=platform/dalvik.git;a=blob_plain;f=tools/gclog.py;hb=HEAD is one
source for the script.) This may give you some insight into where the
bits are going.

We're actively working to bring sanity to bitmaps. An upcoming
release will also see the removal of unnecessary finalizers from some
fundamental classes (like some parts of the view hierarchy), which
should help. (Imagine one of your classes has a finalizer and holds a
Bitmap. We GC, and your class is marked finalizable. Later, we
finalize it. Next GC, we discard it, and mark Bitmap as finalizable.
Later, we finalize it, releasing the "externally allocated" pixels.
So that's two GCs and two runs through the finalization queue before
the no-longer-needed Bitmap data can go away. The more often you GC,
the sooner it can be discarded.)

FWIW, hprof doesn't simply involve a GC; it *is* a GC. The analysis
needed to perform an hprof dump is the same as a garbage collection
pass, so the VM just generates some additional output when requested.