OutOfMemoryError when parsing large response

[Mehmet Arikan]

Hi I'm using volley with Gson. it works fine, but if response data is a little bit large(~240K characters length), then I get OOME when parsingNetworkResponse. I get different stack traces, but all of them are related to volley. How can I solve this OOME problems? Only suggestion I can find is to use Json stream reader, but I think volley doesn't support it. Following are the stack traces I encountered: 

java.lang.OutOfMemoryError

       at java.lang.AbstractStringBuilder.enlargeBuffer(AbstractStringBuilder.java:94)

       at java.lang.AbstractStringBuilder.append0(AbstractStringBuilder.java:145)

       at java.lang.StringBuilder.append(StringBuilder.java:216)

       at com.pozitron.bilyoner.volleystack.PZTRequest.parseNetworkResponse(SourceFile:147)

       at com.android.volley.NetworkDispatcher.run(SourceFile:116)

PZTRequest.parseNetworkResponse:147 has the following code :

String jsonString = new String(response.data, parseCharset(response.headers));

----------------------------

java.lang.OutOfMemoryError

       at java.lang.String.(String.java:255)

       at java.lang.String.(String.java:228)

       at com.pozitron.bilyoner.volleystack.PZTRequest.parseNetworkResponse(SourceFile:157)

       at com.android.volley.NetworkDispatcher.run(SourceFile:116)

PZTRequest.parseNetworkResponse:157 has the following code :

Crashlytics.log("service name = " + getServiceName() + " response length=" + jsonString.length());

----------------------------

java.lang.NullPointerException

       at com.pozitron.bilyoner.volleystack.PZTRequest.deliverResponse(SourceFile:35)

       at com.android.volley.ExecutorDelivery$ResponseDeliveryRunnable.run(SourceFile:99)

       at android.os.Handler.handleCallback(Handler.java:615)

       at android.os.Handler.dispatchMessage(Handler.java:92)

       at android.os.Looper.loop(Looper.java:137)

       at android.app.ActivityThread.main(ActivityThread.java:4898)

       at java.lang.reflect.Method.invokeNative(Method.java)

       at java.lang.reflect.Method.invoke(Method.java:511)

       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1008)

       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:775)

       at dalvik.system.NativeStart.main(NativeStart.java)

This is the strangest exception. PZTRequest.deliverResponse(SourceFile:35) is the class definition line :

public abstract class PZTRequest<T extends Aesop.BaseActionResponse> extends Request<T> {

----------------------------

java.lang.OutOfMemoryError

       at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:122)

       at com.android.volley.toolbox.BasicNetwork.entityToBytes(SourceFile:219)

       at com.android.volley.toolbox.BasicNetwork.performRequest(SourceFile:104)

       at com.android.volley.NetworkDispatcher.run(SourceFile:105) 

----------------------------

java.lang.OutOfMemoryError

       at com.android.volley.toolbox.ByteArrayPool.getBuf(SourceFile:101)

       at com.android.volley.toolbox.PoolingByteArrayOutputStream.expand(SourceFile:76)

       at com.android.volley.toolbox.PoolingByteArrayOutputStream.write(SourceFile:84)

       at com.android.volley.toolbox.BasicNetwork.entityToBytes(SourceFile:217)

       at com.android.volley.toolbox.BasicNetwork.performRequest(SourceFile:104)

       at com.android.volley.NetworkDispatcher.run(SourceFile:105)

[Ficus Kirkpatrick]

Volley holds all responses in memory. Are you sure it's the right library for you?

--
You received this message because you are subscribed to the Google Groups "Volley Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to volley-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mehmet Arikan]

Is it holds all the previous responses, or the only the last one? There is only one service call that has such big responses. So in general, volley fits perfect. If there aren't any solution, I may use something else. 

[Ficus Kirkpatrick]

Any response in flight. Could be multiple if things are happening in parallel.

[Ralph Bergmann | the4thFloor.eu]

Am 27.03.14 11:53, schrieb Mehmet Arikan:

> Hi I'm using volley with Gson. it works fine, but if response data is a
> little bit large(~240K characters length), then I get OOME when
> parsingNetworkResponse.

Are you sure you did it the right way?
I tried it with a ~900kb JSON an a Samsung Galaxy S2 and it works.


final StringRequest test = new
StringRequest("http://www.dasralph.de/bilder/test.json", new
Listener<String>() {

@Override
public void onResponse(final String response) {

final Gson gson = new Gson();
final Person[] persons = gson.fromJson(response, Person[].class);

for (final Person p : persons) {
Log.i("test", "Person: " + p.name);
}
}
}, new ErrorListener() {

@Override
public void onErrorResponse(final VolleyError error) {

Log.i("test", "onErrorResponse: " + error.getLocalizedMessage());
}
});
queue.add(test);



public class Person {

private long id;
private String guid;
private boolean isActive;
private String balance;
private String picture;
private int age;
private String name;
private String gender;
private String company;
private String email;
private String phone;
private String address;
private String about;
private String registered;
private double latitude;
private double longitude;
}

[Mehmet Arikan]

Yeah, I thought I'm doing it right. Here is my code: 

    @Override

    protected Response<TRes> parseNetworkResponse(NetworkResponse response) {

        try {

            String jsonString = new String(response.data, parseCharset(response.headers));

            res = AppInstance.gson.fromJson(jsonString, responseClass);

            return Response.success(res,

                    HttpHeaderParser.parseCacheHeaders(response));

        } catch (Exception e) {

            return Response.error(new ParseError(e));

        }

    }

The problem is it is not always crashing. Actually, I can't reproduce myself but Crashlytics is integrated into project and I get crash reports from users. I'm not even sure whether the problem is here or somewhere else. But all crash reports includes volley and network operations. So I started to investigate from here. 

[Mehmet Arikan]

Now, i am stopping using volley for service calls that may return large responses and writing my own network class using AsyncTask and JSON stream reader. We will see whether this will fix or not. 

[余川]

     Hi，all：

     I also encountered a problem with  oom，can  you  help  me  solve   the OutOfMemoryError。   The  log  is that：

  

     java.lang.OutOfMemoryError

	at com.disk.network.toolbox.ByteArrayPool.getBuf(ByteArrayPool.java:101)
	at com.disk.network.toolbox.PoolingByteArrayOutputStream.expand(PoolingByteArrayOutputStream.java:76)
	at com.disk.network.toolbox.PoolingByteArrayOutputStream.write(PoolingByteArrayOutputStream.java:84)
	at com.disk.network.toolbox.BasicNetwork.entityToBytes(BasicNetwork.java:223)
	at com.disk.network.toolbox.BasicNetwork.performRequest(BasicNetwork.java:108)
	at com.disk.network.NetworkDispatcher.run(NetworkDispatcher.java:110)

在 2014年3月27日星期四UTC+8下午6时53分53秒，Mehmet Arikan写道：

[Joe McGinley]

Mehmet,

Did you ever figure this out? I use the almost the same code as you do below with parseNetworkResponse and sometimes it will throw a OOME.

Thanks

Joe