Is it possible to un-gzip data on the fly, before the complete set has been received?
Dimitris Tsitses
iPhone project that does network access and pulls down and parses JSON
data. ASIHTTPRequest allows gzipped HTTP responses by default, which
is great, but I'm using the streaming parser ability of YAJL (SAX-
like) and it rightfully chokes on gzipped data. I can wait until the
HTTP request has finished then un-gzip and parse the response, but I'm
going for speed here and would like to parse the gzipped data as it
downloads.
So the question is, is it possible to un-gzip data on the fly before
the complete set has been received? The intention is to parse the JSON
within, then forget about that chunk of gzipped data.
If this last part could be solved, this setup seems like it would make
for a great system:
YAJL is one of the fastest JSON parsers around, and it supports SAX-
like, event driven processing
ASIHTTPRequest is easy, asynchronous, with gzip support
Response bodies could be gzipped, saving on-the-wire traffic
JSON could be parsed without loading the whole tree into constrained
device memory
Any guidance would be greatly appreciated, many thanks!
Ben Copsey
> I'm successfully using yajl-objc along with ASIHTTPRequest in an
> iPhone project that does network access and pulls down and parses JSON
> data. ASIHTTPRequest allows gzipped HTTP responses by default, which
> is great, but I'm using the streaming parser ability of YAJL (SAX-
> like) and it rightfully chokes on gzipped data. I can wait until the
> HTTP request has finished then un-gzip and parse the response, but I'm
> going for speed here and would like to parse the gzipped data as it
> downloads.
>
> So the question is, is it possible to un-gzip data on the fly before
> the complete set has been received? The intention is to parse the JSON
> within, then forget about that chunk of gzipped data.
This comes up every now and again. It certainly isn't possible to do this at the moment, but having spent a bit of time experimenting with it this morning, it looks like this should be possible. I'll add it to the to-do list.
Out of interest, how large (uncompressed) are the JSON responses you are downloading?
Best
Ben
Ben Copsey
>> the complete set has been received? The intention is to parse the JSON
>> within, then forget about that chunk of gzipped data.
>
> This comes up every now and again. It certainly isn't possible to do this at the moment, but having spent a bit of time experimenting with it this morning, it looks like this should be possible. I'll add it to the to-do list.
There's a new branch here that includes on the fly inflating of gzipped responses:
http://github.com/pokeb/asi-http-request/tree/newgzipstuff
I've rewritten most of the gzip stuff, and moved it into two new classes, ASIDataDecompressor, and ASIDataCompressor. To inflate responses as they come in, you just need to do this:
[request setShouldWaitToInflateCompressedResponses:NO];
If you're using didReceiveDataSelector to process the response yourself, you should get uncompressed data passed to your delegate instead, but it also works if you're downloading to memory or using downloadDestinationPath.
Hasn't had a lot of testing yet, I'd appreciate any feedback.
Best
Ben
Dimitris Tsitses
and will report back to let you know how it goes. And will definitely
make a donation when my app starts making some $$, ASIHTTPRequest is a
great piece of work!! ;-)
The uncompressed payload varies in size, it can be a few Kb to a few
Mb (i.e. during the initial sync, the very first time the app is used)
Many thanks!
Dimitris
Dan Zeitman
--
You received this message because you are subscribed to the Google Groups "ASIHTTPRequest" group.
To post to this group, send email to asihttp...@googlegroups.com.
To unsubscribe from this group, send email to asihttpreques...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/asihttprequest?hl=en.
--
Dan Zeitman
Founder, FilmFest ®
Mission:
The mission of FilmFest® is to use technology to connect audiences with film festivals and filmmakers. FilmFest: A Festival In Your Pocket ®
Ben Copsey
> GZip:
>
> I've got a data file. it's a zip - it's very small - contains 9 plists. I want to avoid 9 requests, so I'd like to try unzipping it and saving it to the (ios) documents directory.
>
> I've gotten this far -using your example code project:
> The file transfers and is saved as a zip in the documents dir.
>
> What do I need to get it to unzip on the fly or unzip into that dire after the download is completed.
ASIHTTPRequest only supports automatic inflating (decompression) of responses that are gzip encoded, I'm afraid it can't do anything with zip files.
Sam Soffes wrote this:
https://github.com/samsoffes/ssziparchive
...which sounds like it should meet your needs - just set a downloadDestinationPath on your request, and have SSZipArchive unzip the file from your requestFinished delegate method.
> Etag:
> I'm hosting files on AWS - which does support e-tag - I could use a simple example on how to implement that to test if data is stale.
The short answer:
[request setDownloadCache:[ASIDownloadCache sharedCache]];
Or, if you want to handle etags automatically for all requests:
[ASIHTTPRequest setDefaultCache:[ASIDownloadCache sharedCache]];
Either of these will have the download cache do all the work for you. By default, requests will only contact the server if their cached response has expired, and even then, they'll only download the actual data again if the etags don't match (otherwise, they'll just download the headers, reset the cached data's expiry date, and stop).
You can force ASIHTTPRequest to ask the server if the cached data is stale every time you request it by doing:
[request setCachePolicy:ASIAskServerIfModifiedCachePolicy];
Best
`
Ben