Re: [spdy-dev] decompressing the name-value header block

[Mike Belshe]

On Mon, Feb 13, 2012 at 7:32 PM, Osama M. <sir_...@hotmail.com> wrote:

Hello,

The SPDY spec says:
###
There is a single zlib stream (context) for all name value pairs in
one direction on a connection.   SPDY uses a SYNC_FLUSH between frame
which uses compression.
###

I wanted to confirm that the z_stream context is directional per SPDY
*connection* (and not per SPDY stream)?

Per connection, yes.

 

This means that concurrent SPDY streams (on the same SPDY connection)
will have to decompress under lock because they operate on the same
z_stream.  That is non-ideal but acceptable.

Only under a lock if you're passing to separate threads before compress/decompress happens.  There is no reason you couldn't decompress before passing to a secondary thread, then you don't need a lock.

 

More critically, does this necessitate that all blocks must be
decompressed in the same order in which they were received from the
TCP connection?  For example, if SYN_STREAM-1 arrived first and
SYN_STREAM-2 arrived a few microseconds later, does this require that
SYN_STREAM-1 must run through the decompressor first?  This becomes
problematic if the stream processing is packaged and queued for later
execution because the system scheduler is free to execute them in
arbitrary order.

Absolutely - you must decompress exactly in the order which they came off the stream.

 

Skimming the zlib manual, it appears a deflate(SYNC_FLUSH) is to
completely write the current block output so that inflate(Z_BLOCK) can
stop at the end of the name-value block.  But I don't see whether the
inflate order can be changed. It appears Z_FULL_FLUSH might do the
trick.

The Z_FULL_FLUSH would ditch the built up dictionary, thus eliminating compression gains by 'learning' from multiple frames.  This would make the current stateful compression stateless, but would also reduce the compression from ~85-90% compression to ~45-50% compression (or less in the presence of large cookies)

Mike

 

Thanks,
--Osama.

[Osama M.]

Hello,

The SPDY spec says:
###
There is a single zlib stream (context) for all name value pairs in
one direction on a connection. SPDY uses a SYNC_FLUSH between frame
which uses compression.
###

I wanted to confirm that the z_stream context is directional per SPDY
*connection* (and not per SPDY stream)?

This means that concurrent SPDY streams (on the same SPDY connection)
will have to decompress under lock because they operate on the same
z_stream. That is non-ideal but acceptable.

More critically, does this necessitate that all blocks must be
decompressed in the same order in which they were received from the
TCP connection? For example, if SYN_STREAM-1 arrived first and
SYN_STREAM-2 arrived a few microseconds later, does this require that
SYN_STREAM-1 must run through the decompressor first? This becomes
problematic if the stream processing is packaged and queued for later
execution because the system scheduler is free to execute them in
arbitrary order.

Skimming the zlib manual, it appears a deflate(SYNC_FLUSH) is to
completely write the current block output so that inflate(Z_BLOCK) can
stop at the end of the name-value block. But I don't see whether the
inflate order can be changed. It appears Z_FULL_FLUSH might do the
trick.

Thanks,
--Osama.