Feature Requst: GZIP response functionality

[josh.w...@gmail.com]

Hi,

Thanks so much to everyone who contributed to Volley, it's amazing!

I am working on a project where I read JSON from a server and I want to have the response sent gzip'd...

I couldn't find gzip encoding support anywhere in the code so I added my own but I was wondering if it could be incorporated into the main branch?

This is what I did in case anyone is interested, also, this might not be the correct way to go about it so if anyone has any corrections please let me know:

First of all I had to get access to the headers in the request so I subclassed JsonObjectRequest and added a headers field (like in the GsonRequest example) and then added the header "Accept-Encoding":"gzip".

Then I found where the response input stream is created in HurlStack.java (entityFromConnection() - line 134) and I added this:

try {
        inputStream = connection.getInputStream();
        String contentEncoding = connection.getHeaderField("Content-Encoding");
        if(contentEncoding != null && contentEncoding.equals("gzip")) {
            inputStream = new GZIPInputStream(inputStream);
        }
} catch (IOException ioe) {
        inputStream = connection.getErrorStream();
}

so far this works perfectly.

hope this is useful to someone and Thanks again!

joshua pierce

[josh.w...@gmail.com]

OK, I was looking around some more and I realized that I could use a GZIPInputStream on the NetworkResponse object's data in the parseNetworkResponse() method of a Response subclass, so that is probably a better solution than modifying the core framework and one that I am now using; however it does require about 20 more lines of code than the fix I posted above and I still think it would be nice to have the framework handle gzipped responses transparently...though probably in a more robust way than that hack I was using...

joshua pierce

[Ficus Kirkpatrick]

You shouldn't have to do anything. Both the Apache and HttpURLConnection clients should handle this for you (which is why you don't see code explicitly handling it in Volley).

Were you running into problems? Are you sure your server's response headers are correct?

Ficus

On Jul 29, 2013 4:46 AM, <josh.w...@gmail.com> wrote:

OK, I was looking around some more and I realized that I could use a GZIPInputStream on the NetworkResponse object's data in the parseNetworkResponse() method of a Response subclass, so that is probably a better solution than modifying the core framework and one that I am now using; however it does require about 20 more lines of code than the fix I posted above and I still think it would be nice to have the framework handle gzipped responses transparently...though probably in a more robust way than that hack I was using...

joshua pierce

--
You received this message because you are subscribed to the Google Groups "Volley Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to volley-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[josh.w...@gmail.com]

Yes, I am experiencing problems when I pass in "Accept-Encoding":"gzip" in the request header I get back the gzip content in the Listener<JSONObject>'s onErrorResponse:

org.json.JSONException: Value ?M?M?@?????@?DoQ"?N of type java.lang.String cannot be converted to JSONObject

It seems to me that the input stream is not correctly decompressing the gzip content and thus it can't be parsed as JSON.  I can decode the NetworkResponse.data byte array myself using a GZIPInputStream in a subclass of Request<T> and it works fine. As for the server configuration, it is possible that something is wrong there, I am not responsible for configuring the server but it has gzip enabled via a php module (I think?).  The headers I receive for the call that produced the above error are:

{
Vary=Accept-Encoding,
Transfer-Encoding=chunked,
Date=Mon, 29 Jul 2013 17:36:56 GMT,
Expires=Thu, 19 Nov 1981 08:52:00 GMT,
Content-Encoding=gzip,
Set-Cookie=LIMONADE0x5x0=ij1v50uid7dmtne6n45dscimo7;
path=/, Content-Type=application/json;
charset=utf-8,
Connection=keep-alive,
X-Powered-By=PHP/5.4.16,
X-Limonade=Un grand cru qui sait se faire attendre,
Server=nginx/1.4.1,
Pragma=no-cache,
Cache-Control=no-store, no-cache, must-revalidate, post-check=0, pre-check=0
}

As far as I can tell the Content-Encoding: gzip is the field that should tell the Android clients to automatically decompress it, is that correct? So by the time I get the NetworkResponse object in parseNetworkResponse() the data byte array should be correctly decompressed?

Thanks for your help!

joshua pierce 

[Ficus Kirkpatrick]

What are you using to set up Volley? Volley.newRequestQueue?

--

[josh.w...@gmail.com]

Yeah, I get a new request queue inside an activity with Volley.newRequestQueue() and pass in the activity as the context, then I instantiate a new request and add it to the queue.  If I set the Accept-Encoding header I get the error in onErrorResponse(), if I don't set the header it works just fine, parses the JSON successfully.

[Joris Bolsens]

Did this ever get added?

I am running into the same problem, the byte[] that's being put into response.data is still compressed, I have checked the header and Content-Encoding is correctly set to gzip.

I am creating the queue with Volley.newRequestQueue(mContext.getApplicationContext()); 

[Joris Bolsens]

if you explicitly set the Accepts-Encoding header on the request, the http clients wont auto decompress, however if you don't set it, it automatically gets set and handled by the client.

tl;dr don't set it or worry about it, happens in background.

[Joris Bolsens]

I made a change and uploaded to gerrit that handles the case where the header is set explicitly and the httpclient doesn't handle it.

I tested this and it works both when the header is explicitly set on the request, and without so should not cause anything to break

https://android-review.googlesource.com/#/c/161310/

[Joshua Pierce]

Ok, that's good to know.  But I don't understand (due to a lack of knowledge about HTTP protocol) how the server knows to send a GZIPd response if the request doesn't have the Accepts-Encoding?  What if the client can't handle zipped responses?  or is it part of the HTTP 1.1 specification that clients must be able to handle zipped responses?  Thanks for your investigation!

[Joris Bolsens]

So what happens is if you do not manually add the header, the httpclient will automatically add it, then unzip it and pass normally on to you as if nothing happened. If you set the header yourself then the httpclient will assume you want to handle the unzipping yourself and pass it on to you untouched.