zlib.unzip gives me an empty result

1,384 views
Skip to first unread message

Rambo

unread,
Feb 1, 2012, 11:01:55 AM2/1/12
to nod...@googlegroups.com
Please help I don't know what's happening. 
I'm connecting to Gnip stream api which emits json messages using gzip. But when I try to unzip incoming data, I get an empty buffer!

This is the code that makes the request. Basically it's the same as the example given in http://nodejs.org/docs/latest/api/zlib.html#zlib.Unzip

var options = {
host : 'stream.gnip.com',
port : 443,
method : 'get',
path : '/track/prod.json',
headers : {
'Authorization' : self._basicAuth(self.options.username, self.options.password),
'User-Agent' : self.options.userAgent,
'Accept-Encoding' : 'gzip,deflate'
}
};

self._req = https.request(options, function(res) {
self.emit('ready');
res.on('data', function(chunk) {
console.log(chunk.toString()) // this prints a lot of strange stuff
zlib.unzip(chunk, function(err, res) {
console.log(res.toString()) // empty!
if (err) self.emit('error', err);
else self.parser.receive(res);
});
});
res.on('end', function() {
self.emit('end');
});
});
self._req.on('error', function(e) {
self.emit('error', e);
self.end();
});
self._req.end();

Isaac Schlueter

unread,
Feb 1, 2012, 2:03:17 PM2/1/12
to nod...@googlegroups.com
The zlib.unzip() method assumes that you have *all* the data. HTTP
requests in node send the data bit by bit. It's possible that what
you have in that first chunk isn't the whole gzip'ed file.

If you want to process it one chunk at a time (which is probably best)
use a zlib.Unzip() object rather than the convenience method.

var unzipper = zlib.Unzip()
req.on("data", function (c) {
unzipper.write(c)
})
req.on("end", function () {
unzipper.end()
})
unzipper.on("data", function (c) {
// some unzipped data
})
unzipper.on("end", function () {
// no more unzipped data is coming
})

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

Isaac Schlueter

unread,
Feb 1, 2012, 2:04:07 PM2/1/12
to nod...@googlegroups.com
Oops, pedantic error. Should be:

var unzipper = new zlib.Unzip()

(Though, the function detects this and does the right thing anyway.)

Demián Andrés Rodriguez

unread,
Feb 1, 2012, 2:12:19 PM2/1/12
to nod...@googlegroups.com
Thanks. I've just done that and it seems to work!
Reply all
Reply to author
Forward
0 new messages