Possible memory leak?

[Harm]

Hi all,

I am using the method fromJson( Reader reader, Type t) on a static
Gson object in android application.
I have a service that runs in the background and that will contact a
webservice every minute. This webservice returns a Json inputstream
from which I will create an InputStreamReader that I will pass on to
the fromJson method of my Gson static instance.

However after doing a memory dump of my application after several
minutes have passed, I can see a steadily increase in
com.google.gson.stream.JsonReader objects that can't seem to be
garbage collected.

I have ran several tests and each time the Eclipse Memory Analyzer
notifies me of a possible leak, all instances of JsonReader classes.

The following is an excerpt of the Eclipse Memory Analyzer output:

63 instances of "com.google.gson.stream.JsonReader", loaded by
"dalvik.system.PathClassLoader @ 0x452e61f0" occupy 12,820,880
(83.22%) bytes.

Biggest instances:
•com.google.gson.stream.JsonReader @ 0x45395cc8 - 213,144 (1.38%)
bytes.
•com.google.gson.stream.JsonReader @ 0x4539f8c8 - 213,144 (1.38%)
bytes.
•com.google.gson.stream.JsonReader @ 0x453ce468 - 213,144 (1.38%)
bytes.
...

So in this above case, my webservice method has been called 63 times,
and each time the Json result stream was passed to a static Gson
instance that uses the fromJson method with the reader and type
parameter.
Each time a 208kb instance can't seem to be garbage collected. However
the memory analyzer can't seem to pin down the GC root that keeps them
in memory.

Can somebody guide me in the right direction on how to tackle this
problem.
Is it a problem with my code that is preventing the JsonReader
instances from being garbage collected or is there something in the
gson library that keeps them around ?
I'm new to memory analysis so go easy on me ;)

But yes, eventually I get an OutOfMemory exception.

Thanks,
Harm

[Jesse Wilson]

I suspect this is a problem in your application code. There isn't anything in Gson that keeps a JsonReader as a field. Your best bet is to get your application in a state where there are several JsonReader instances leaked. Then call dump a heap and analyze it. Here's a post covering the steps to create and analyze an Android heap dump:

http://discuz-android.blogspot.com/2009/06/android-memory-usage-with-hprof.html

[Harm]

Thanks Jesse,

The output came from an Android heap dump analyzed with the Eclipse
memory analyzer.

However, I did some more testing with the following code,
I hope the formatting doesn't mess it up.

Basically, I created a loop that called my webservice 200 times.

for(int i = 0; i < 200; i++)
{
InputStream stream = task.performRequestAndReturnStream(message, url,
context);
InputStreamReader reader = new InputStreamReader(stream);
items = (LogisticsCollection[])
JsonUtils.getJsonLogisticsCollection(reader);
stream.close();
reader.close();
reader = null;
stream = null;
}

In the method JsonUtils.getJsonLogisticsCollection I use a Gson object
and call fromJson passing the InputStreamReader. After the Gson object
has parsed the stream, I set it explicitly to null so it can be
garbage collected.
Here is the method from the JsonUtils class:

public static Object getJsonLogisticsCollection(Reader reader) {
Gson gson = GetGsonParser();
Type type= new
TypeToken<DotNetWebserviceData<LogisticsCollection[]>>() {}.getType();
DotNetWebserviceData<LogisticsCollection[]> data =
(DotNetWebserviceData<LogisticsCollection[]>)gson.fromJson(reader,
type);
gson = null;
return data.d;
}

Now here is the strange part, if I run this loop, after a while it
will get an out of memory exception and the heap dump shows tons of
JsonReader objects that can't get garbage collected.

But if I create a JsonReader object myself and pass that to my
JsonUtils method (after changing its signature to accept a JsonReader
instead of a Reader object), then all is fine and I get no leaks and
the loop finishes without problems.

for(int i = 0; i < 200; i++)
{
InputStream stream = task.performRequestAndReturnStream(message, url,
context);
JsonReader reader = new JsonReader( new InputStreamReader(stream));
items = (LogisticsCollection[])
JsonUtils.getJsonLogisticsCollection(reader);
stream.close();
reader.close();
reader = null;
stream = null;
}

When I dump the heap after executing this second loop, there are no
traces of JsonReader objects to be found.
So, this leads me to believe my code is fine and something in the gson
library prevents the JsonReaders from being garbage collected when a
Reader object is being passed.

Harm

[Harm]

I forgot to mention,
I'm using version 2.0 of the Gson library.

Thanks
Harm

[Jesse Wilson]

Can you explain why the JsonReader cannot be collected? Ie. can you
show the path from the root set? We can't fix a problem that we can't
see!

[Harm]

Sorry Jesse,

I'm afraid I can't explain why it can't be collected. The memory
analyzer does not give an indication as to who is keeping a reference
to the objects (it says System?). I'm not getting any wiser from it :(

All I can provide you with is the simplest Android activity class to
reproduce the problem. Perhaps this is not a Gson problem but an
android problem, I'm afraid I'm not knowledgeable enough to come to a
conclusion.
Not sure if you have access to an android testing environment, but
perhaps another member on the Gson team has and can try to reproduce
the problem by using the code below.

Here is the code that will create the OutOfMemory exception when you
run it (Android 2.2) .

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.StringBufferInputStream;
import java.lang.reflect.Type;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import android.app.Activity;
import android.os.AsyncTask;
import android.os.Bundle;

public class GsonOutOfMemoryActivity extends Activity {

private class Test<T> {
public T var1;
public T var2;
}

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
AsyncJsonTest test = new AsyncJsonTest();
test.execute();
}


private class AsyncJsonTest extends AsyncTask<Void, Void, Void>
{

@Override
protected Void doInBackground(Void... arg0) {
for(int i = 0; i < 10000; i++)
{
InputStream stream = new StringBufferInputStream("{\"var1\": \"blah\",
\"var2\": \"bloe\"}");

InputStreamReader reader = new InputStreamReader(stream);

Gson gson = new Gson();
Type type= new TypeToken<Test<String>>() {}.getType();
Test<String> test = (Test<String>)gson.fromJson(reader, type);
try {
stream.close();
reader.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();

}
reader = null;
stream = null;
}

return null;
}
}
}

Again, after I change the method and create the JsonReader object
myself using JsonReader(new InputStreamReader(stream)) and pass that
into the Gson.fromJson method, it works fine and no
OutOfMemoryException and my android heap remains constant (around 3mb)
But by passing the InputStreams directly the heap will just keep
incrementing until OutOfMemoryException

I can also provide you with a hprof heap dump if you want to have a
look yourself.
If you give me an email address to send it to, I can send you a
relatively small dump (3mb) that shows the lingering JsonReader
instances. Perhaps you can spot something that I can't.

Regards,
Harm

[Jesse Wilson]

Harm,

Thanks I'll try to reproduce this today. My day job is working on
Dalvik (Android's runtime and library), so I should be able to fix the
bug either way.

Cheers,
Jesse

[Jesse Wilson]

I can't reproduce this. I ran it on a 2.2 emulator just fine; it
completed after 10,000 reps.

Are you seeing this on the Android emulator or a particular device?
Perhaps you're running into the bug where gson is included by default
on the boot class path of certain HTC devices? If that's your problem
you'll need to use jar jar or proguard to repackage Gson with your
application.
http://code.google.com/p/google-gson/issues/detail?id=255

[Harm]

I'm seeing the exact same behavior on a 2.2 emulator as well as on an
actual device (Huawei Ideos X5).
I'm getting outofmemory every single time.
Very surprised by the fact that you can't reproduce it. A glitch on my
emulator and on my phone ?

At least I know how to work around the issue, I'll see if I can get
hold of another device and test on it as well.
Thank you for looking at this and have a great 2012.

[Harm]

Ok, I've just had confirmation that this also occurs on an actual
device on android 2.3 (HTC Desire S) using my sample Activity that I
posted earlier on.
So problem occurs on 2 actual devices and my emulator...
Is there anywhere I should file an official bug report ?

Thanks