SPDY Spec

[Mike Belshe]

I'm getting ready to finalize the SPDY specification, so I have posted it on github.  This is a significant change since the last draft, and I hope is getting close to a version worthy of submission to the ietf.  But there is still a lot of work to do.

I welcome and invite anyone willing to go through it to checkout a copy, make edits, and contribute them back.  github makes this process very easy.

You can view the current spec here:  http://mbelshe.github.com/SPDY-Specification/

And the git repo can be found here:    https://github.com/mbelshe/SPDY-Specification/branches/gh-pages

Thanks to everyone for your help so far and your help ongoing :-)

Mike

[Luke-Jr]

On Tuesday, May 17, 2011 12:58:16 PM Mike Belshe wrote:
> I'm getting ready to finalize the SPDY specification, so I have posted it
> on github. This is a significant change since the last draft, and I hope
> is getting close to a version worthy of submission to the ietf. But there
> is still a lot of work to do.
>
> I welcome and invite anyone willing to go through it to checkout a copy,
> make edits, and contribute them back. github makes this process very easy.
>
> You can view the current spec here: http://mbelshe.github.com/SPDY-
> Specification/

Is there a human-readable version somewhere?

[Mike Belshe]

This should be human readable on FF, Chrome, or IE9.

Mike

 

[Gabriel Ciuloaica]

Hi Mike,

There is something wrong in the SPDY spec. In the section 7. "Incompatibilities with SPDY draft #2", you are mentioning that Name/value block will be included only in HEADERS frame, but the description of SYN_FRAME and SYN_REPLY frame still mentions Name/Value block...

Thanks,

Gabi

[Mike Belshe]

On Tue, May 17, 2011 at 11:41 AM, Gabriel Ciuloaica <gciul...@gmail.com> wrote:

Hi Mike,

There is something wrong in the SPDY spec. In the section 7. "Incompatibilities with SPDY draft #2", you are mentioning that Name/value block will be included only in HEADERS frame, but the description of SYN_FRAME and SYN_REPLY frame still mentions Name/Value block...

Fixed!  There was a time we discussed that, but after partially implementing, we backed that out

Mike

[Luke-Jr]

I don't like bloated browsers. Is there standard HTML/CSS?

[Mike Belshe]

No. 

[James Cloos]

>>>>> "MB" == Mike Belshe <mbe...@google.com> writes:

MB> This should be human readable on FF, Chrome, or IE9.

There is no stylesheet so, no, it is not readable.

In fact it is worse in a browser than the raw xml.

(I looked at both a local checkout and the posted url.)

-JimC
--
James Cloos <cl...@jhcloos.com> OpenPGP: 1024D/ED7DAEA6

[James Cloos]

JC> There is no stylesheet so, no, it is not readable.

I should have continued:

but (the current version of) xml2rfc renders it propperly.

And xml2rfc's xml2html variant generates html output which
looks great in all browsers.

[Mike Belshe]

So maybe we're looking at two different things.  It is certainly readable for a lot of people.  I'm talking about this link:

    http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml

It uses a rfc2629.xml stylesheet to render, which is also present there.

Mike

[Fedor Indutny]

Everything works fine, Mike!

Sorry, no comments from me for awhile, but I'll review it soon

Cheers,

Fedor.

[James Cloos]

>>>>> "MB" == Mike Belshe <mbe...@google.com> writes:

MB> http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml

That was what I tested.

I tried again with more browsers. FF3 and opera do render it. But
both arora (webkit) and SM21 (same rendering engine as FF4) fail to
use a stylesheet.

The SM21 rendering should be identical to a FF4 rendering.

[Roy T. Fielding]

On May 17, 2011, at 9:58 AM, Mike Belshe wrote:

> I'm getting ready to finalize the SPDY specification, so I have posted it on github. This is a significant change since the last draft, and I hope is getting close to a version worthy of submission to the ietf. But there is still a lot of work to do.
>
> I welcome and invite anyone willing to go through it to checkout a copy, make edits, and contribute them back. github makes this process very easy.

Er, not much time for editing at the moment, but editorially I suggest
replacing "receiver" with "recipient" everywhere it occurs.

....Roy

[Mike Belshe]

Thanks. done.  Curiosity:  I suspect you have experience which tells you why.   Is there a reason you recommend this?  or just preference?

Mike

 

....Roy

[Leen Besselink]

Hi,

As I was reading part about Server Push Transactions of the
specification I noticed it mentioned browser same-origin policy as the
way for the browser to check if the pushed response should be accepted.

Recently I read a study Microsoft had done called HTTPi [1] * where they
used the X-Content-Security-Policy from the Mozilla CSP specification. [2]

I'm just thinking out loud, as I'm not sure how useful it would be, but
could something like not also be added to the SPDY-specification to
allow for content from other domains to be pushed as well ?

Have a nice day,
Leen.

* the HTTPi study [0] [1] is kind of interresting as it is trying to
solve things SPDY does not solve, to allow intermediate servers
(proxies/CDN) to cache/serve static content.

[0] http://research.microsoft.com/apps/pubs/default.aspx?id=148963
[1] http://research.microsoft.com/pubs/148963/htti-techreport.pdf
[2]
https://developer.mozilla.org/en/Security/CSP/Introducing_Content_Security_Policy

[Mike Belshe]

Hi Leen,

Thanks for thinking about this.

Hopefully, SPDY works just fine with all of these proposals.  We haven't been trying to change the semantics of HTTP itself very much, so these schemes which overlay on top of existing HTTP headers should just work, even with SPDY.  

I suppose SPDY could make them mandatory, but we have been trying to avoid those types of restrictions in general.

Mike

[Alexandre Prates]

Been reading the specification and stay with one doubts the frame
syn_reply field must contain the same block name / value? For the
specification given that this frame has fixed length of 4 bytes.

Regards,

Alexandre Prates

[James A. Morrison]

Comments:
2.3.1: Why is it a session error not "not increase" the stream id for
new streams vs a stream error if a stream
id is used twice in a row? These seem like the same error to me. I'm
fine with either having to close the session
or simply resetting offending stream.

2.3.1: The specification doesn't say that -1 or -2324 are invalid, it's only
sort of implied since the top bit of a 32 bit number is removed for the
control frame bit.

Better than mentioning negative numbers anywhere else, you could say
"All integer values, ... are in unsigned in network byte order." in section 2.2.

2.3.2: Please link to where priorities are defined or the SYN_STREAM/SYN_REPLY
frames. You still have a TODO about the trickle priority. Is this
still useful? You
haven't defined anything else about the priorities, so you can
probably get rid of the
TODO.

2.3.3: You said you got rid of the headers frame, correct?

2.3.4: More uses of the HEADERS frame.

2.6.2: Should STREAM_IN_USE or STREAM_ALREADY_CLOSED be listed in
2.6.3?

2.6.6: Do you have more errors to put in?

2.6.7: So the headers frame is staying, ok.

2.7: I think you should explicitly state that the spec is for version = 3.

3.2.1: Do you mean SYN_STREAM instead of HEADERS?

3.x: I think you should deal with the 100 response codes
explicitly.

3.3: You may as well say that pushed content must at least
be privately cacheable.

If you would like me to make any of these changes, my github user is phython. I
can make a branch that you can pull from.

--
Thanks,
Jim
http://phython.blogspot.com

[Roy T. Fielding]

The word receiver was invented for the telephone (IIRC) and later
extended to radio to be the listening device. Recipient is the
traditional English noun for the recipient of a message, letter, etc.
In a spec, it is just my preference, though the noun pairs should be
consistently transmitter/receiver or sender/recipient. NICs are
sometimes referred to as transceivers, but most app-level specs
use the terms sender/recipient.

Not important -- it just stood out on a quick scan (all I had time
for before heading out on vacation).

....Roy

[Alexandre Prates]

Hi, Another suggestion, the field has fixed length of 8 bytes in the
frame-update windows because there are only two fixed field of 32 bits
after the Length field.

Regards,
Alexandre Prates

[William Chan (陈智昌)]

Comments:

2.2.2: I just realized that DATA frame compression is specified on a frame
by frame basis. Is this flexibility really intended / desired? If so, what's
the use case?
2.3.5: We should probably mandate sending a RST_STREAM with a PROTOCOL_ERROR
if further data is sent after a FLAG_FIN.
2.3.6: Likewise, we should mandate sending a RST_STREAM with a
PROTOCOL_ERROR if further data is sent after the stream is closed.
2.6.2: You don't mention STREAM_IN_USE/STREAM_ALREADY_CLOSED elsewhere.
Maybe STREAM_ALREADY_CLOSED should be used for 2.3.5/2.3.6 errors.
2.6.9: Have you defined what the version number is in this draft? 3?
2.6.10: How should non-lowercased headers be handled? RST_STREAM with
PROTOCOL_ERROR?
2.6.10.1: "The Name/Value Header Block is the portion of the HEADERS frame
which follows the Stream-ID field." Why only the HEADERS frame?
3.2.1: I'm not sure why we document that
Connection/Keep-Alive/Proxy-Connection must be ignored, rather than must not
be sent. "If a client sends a HEADERS without all of the method, host, path,
scheme, and version headers, the server must reply with a HTTP 400 Bad
Request reply." Probably should capitalize MUST.
3.2.2: I find it interesting that if the request leaves out certain HTTP
headers (method/scheme/etc), the response must be a HTTP 400 Bad Request,
but if the response leaves out the status/version header

[Fedor Indutny]

William, Mike

Will ID in settings frame stored in network byte order in next protocol implementation in Chrome?

If not - I would like to see that exception in documentation

Cheers,

Fedor.

[William Chan (陈智昌)]

We will work to fix bugs like this for the next time we bump up the
SPDY version in Chrome and Google servers. We don't plan on making
exceptions.

[Mike Belshe]

On Wed, May 25, 2011 at 9:34 PM, Alexandre Prates <alexandre...@gmail.com> wrote:

Hi, Another suggestion, the field has fixed length of 8 bytes in the
frame-update windows because there are only two fixed field of 32 bits
after the Length field.

updated text.

 

Regards,
Alexandre Prates

[Mike Belshe]

On Thu, May 26, 2011 at 6:40 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

Comments:

2.2.2: I just realized that DATA frame compression is specified on a frame
by frame basis. Is this flexibility really intended / desired? If so, what's
the use case?

It's a side effect of not having other good options.  We definitely want to be asynchronous and avoid a negotiation of it.  You could put the flag on the SYN_STREAM, but that requires a decision about compression very early in stream creation.  Alternatively, we could add a whole frame for turning on stream-level flags.  But that seemed overkill.

Nonetheless, I'm thinking of striking all data compression from the protocol, because as far as I can tell, nobody is going to use it.

 

2.3.5: We should probably mandate sending a RST_STREAM with a PROTOCOL_ERROR
if further data is sent after a FLAG_FIN.

done, made this STREAM_ALREADY_CLOSED

 

2.3.6: Likewise, we should mandate sending a RST_STREAM with a
PROTOCOL_ERROR if further data is sent after the stream is closed.

done, made this PROTOCOL_ERROR.  I don't want the recipient to track state of fully closed streams.

 

2.6.2: You don't mention STREAM_IN_USE/STREAM_ALREADY_CLOSED elsewhere.
Maybe STREAM_ALREADY_CLOSED should be used for 2.3.5/2.3.6 errors.

added 

 

2.6.9: Have you defined what the version number is in this draft? 3?

Added this to section 2.2.1.

 

2.6.10: How should non-lowercased headers be handled? RST_STREAM with
PROTOCOL_ERROR?

Good question.  Should we be strict?

 

 

2.6.10.1: "The Name/Value Header Block is the portion of the HEADERS frame
which follows the Stream-ID field." Why only the HEADERS frame?

Bug.  Fixed.

 

3.2.1: I'm not sure why we document that
Connection/Keep-Alive/Proxy-Connection must be ignored, rather than must not
be sent.

ok.

 

"If a client sends a HEADERS without all of the method, host, path,
scheme, and version headers, the server must reply with a HTTP 400 Bad
Request reply." Probably should capitalize MUST.

fixed

 

 

3.2.2: I find it interesting that if the request leaves out certain HTTP
headers (method/scheme/etc), the response must be a HTTP 400 Bad Request,
but if the response leaves out the status/version header

Looks like you got cut off here :-)

Mike

[Mike Belshe]

On Thu, May 26, 2011 at 6:45 AM, Fedor Indutny <fe...@indutny.com> wrote:

William, Mike

Will ID in settings frame stored in network byte order in next protocol implementation in Chrome?

yes

[Yoav]

I have a few comments regarding compression:
1. As we have learned from the HTTP spec, defining "zlib" compression
is not enough. The original zlib terminology refers to the compression
algorithm as "deflate" (http://www.gzip.org/zlib/rfc-deflate.html)
when "zlib" (http://www.gzip.org/zlib/rfc-zlib.html) and
"gzip" (http://www.gzip.org/zlib/rfc-gzip.html) are wrappers using
"deflate" internally. The HTTP spec defined the "deflate" content
encoding as using "zlib". That caused a lot of confusion in various
implementations and made "deflate" practically unreliable.
IMO the SPDY spec should explicitly define which compression it uses,
and stick to the original terminology. I.e. if we want to use raw
deflate (since we don't need a checksum), we should call that
compression "zlib's deflate algorithm as defined in RFC 1951" (or
something of that nature).
2. Regarding compression contexts for data:
There can be scenarios where a global compression context can give
better results then a compression context per resource (One example is
a series of small & similar JS/CSS files).
OTOH, that will force using the same stream for multiple resources in
situations where multiple streams may have given better results (in
terms of speed).
Another problem is that the content author and server side (who may
have prior knowledge as to what compression context usage would be
more beneficial) have no saying in the matter. Assigning resources to
stream (as far as I understand it) in a traditional browsing scenario,
is the client-side's job. The client can use heuristics (such as
assigning dedicated streams according to requesting tag type) but
naturally, it has less info regarding the actual data that's going to
come in.
I think the "static, up to 16 compression contexts" proposal from
section 4.5 could have resolved the above issues if the compression
context number could be given on a per resource/frame basis from the
server side. I believe the benefits for such a compression scheme
would outweigh the extra implementation complexity/cost. A short, non
scientific test I ran shows 5-10% compression improvement from mutual
ZLIB context between text resources.
We should take the opportunity of introducing SPDY to also enable
newer compression algorithms such as bzip (which was removed from
chrome due to intermediary proxies trouble as far as I understand) and
7zip. A zlib variant with a larger window might also be beneficial in
mutual context scenarios. In yet another non-scientific test I ran, I
see 7zip compressing HTML,CSS & JS, producing files that are 30%
smaller then gzip (with a mutual context for both), with relatively
similar decompression speeds. The server can then decide regarding the
CPU vs. BW tradeoff.
All in all, IMO removing data compression from the specification would
be a wasted opportunity to provide significant data reduction to the
web.

Yoav

http://blog.yoav.ws

On May 26, 5:44 pm, Mike Belshe <mbel...@google.com> wrote:
> On Thu, May 26, 2011 at 6:40 AM, William Chan (陈智昌)

> <willc...@chromium.org>wrote:

[Simon B.]

+1 specific on which compression to use. Though if SPDY anyways will
get wrapped in SSL, maybe all compression can be done by SSL?

If SPDY will do compression -- things to consider when choosing:
* There might be a point in using the same compression as for HTTP, in
that a web server can precompress all its static resources. (Maybe
servers like nginx and varnish already does this?)
* Is there risk of degrading overall performance if compressed SPDY is
tunnelled through SSL that also does compression?
* For non-static content there is no need to allow pre-compression,
thus a more modern compression can be pushed to gain adoption. On
http://stackoverflow.com/questions/124239/fastest-c-file-compression-library-available
I found hints towards LZMA. Either way, any new algorithm must come
with well-tested code and no "problems" of copyright, patents, etc.
Also Google Chrome want's to have a smaller binary (just speed, or
perhaps aiming for embedded use?) and thus may reject adding big new
libs to its codebase, making deflate an easy choice to avoid adding
more code.

[Simon B.]

> * For non-static content there is no need to allow pre-compression,
> thus a more modern compression can be pushed to gain adoption.

Looking at benchmarks shows freearc 0.666 in lead - perhaps because
it combines several different compression methods apart from LZMA -
and freearc also gives some trouble license-wise.

Google internally supposedly uses http://code.google.com/p/snappy/
which seems tiny enought and may come with benefits regarding
licensing.
More on snappy: http://code.google.com/p/snappy/source/browse/trunk/README

[William Chan (陈智昌)]

s/HEADER frame/HEADERS frame/

[William Chan (陈智昌)]

I think the stream half-close descriptions in the spec are
confusing/wrong. Examples:

* http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#StreamHalfClose
says: "If an endpoint receives a data frame after the stream is
half-closed, it must send a RST_STREAM to the sender with the status
STREAM_ALREADY_CLOSED." This is vague and should be clarified. It
could be interpreted such that, if the client has set the FIN flag in
its SYN_STREAM, receives a SYN_REPLY without a FIN, and then receives
data frames, that the client should respond to the data frames with a
RST_STREAM.
* http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#SYN_REPLY
says: "If an endpoint receives a SYN_REPLY frame for a stream already
in the half-closed state, it MUST issue a stream error (Section 2.4.2)
with the error code STREAM_ALREADY_CLOSED." I'm confused by this. If
the client's SYN_STREAM already had a FIN, and thus the stream is
half-closed, it seems perfectly reasonable to send a SYN_REPLY. I'm
also not sure when this is applicable. Perhaps it's meant to cover the
case when the server has already sent a SYN_REPLY with a fin or a data
frame with a FIN, but then sends another SYN_REPLY for the same
stream? But that would be covered in the previous sentence which says:
"If an endpoint receives multiple SYN_REPLY frames for the same active
stream ID, it MUST issue a stream error (Section 2.4.2) with the error
code STREAM_IN_USE." There is also some ambiguity about which of these
two sentences applies to the fully closed state, STREAM_ALREADY_CLOSED
or STREAM_IN_USE?

Perhaps I'm being too anal, but I think the descriptions could be clarified.

Also, I see that you've updated the spec wrt the invalid headers like
Connection. The request headers section notes that these headers MUST
not be sent: "The Connection, Keep-Alive, and Proxy-Connection headers
are not valid and MUST not be sent."
(http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.3.2.1)
However, the response headers section does not include the same "MUST
not be sent" language: "The Connection and Keep-alive response headers
are no longer valid."
(http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.3.2.2).

And for both the requests and responses, the spec notes that
Transfer-Encoding is no longer valid. How about we add the same
language and say that it must not be sent?

On Sun, Jun 12, 2011 at 4:01 PM, William Chan (陈智昌)

[William Chan (陈智昌)]

When a SYN_STREAM/SYN_REPLY/HEADERS frame has mismatching frame length
vs name/value header block lengths, in the Chromium SPDY
implementation (http://codesearch.google.com/codesearch#OAMlx_jo-ck/src/net/spdy/spdy_framer.cc&exact_package=chromium&q=SpdyFramer&type=cs&l=233)
we return RST_STREAM
(http://codesearch.google.com/codesearch#OAMlx_jo-ck/src/net/spdy/spdy_session.cc&type=cs&l=1213).

The problem here is, since we compress the name/value header block,
isn't the compression context horked now? Should the spec document
this case and treat it as a session error instead of a stream error?

On Mon, Jun 13, 2011 at 4:01 PM, William Chan (陈智昌)

[William Chan (陈智昌)]

For stream errors, we send a RST_STREAM. For session errors, we send
GOAWAY and close the connection. RST_STREAM has a status code to
indicate what happened. Should GOAWAY also have a status code? I
should note that GOAWAY is also sent in normal shutdown (client or
server decides to free up the connection) so a status code should
include that. Just a thought.

On Mon, Jun 13, 2011 at 9:45 PM, William Chan (陈智昌)

[William Chan (陈智昌)]

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.2.3
says:
"Streams optionally carry a set of name/value header pairs."

This directly contradicts
http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.3.2.1
which says:
"""
The first line of the request is unfolded into name/value pairs like
other HTTP headers and MUST be present:
":method" - the HTTP method for this request (e.g. "GET", "POST", "HEAD", etc)
":path" - the url-path for this url with "/" prefixed. (See RFC1738
[RFC1738]). For example, for "http://www.google.com/search?q=dogs" the
path would be "/search?q=dogs".
":version" - the HTTP version of this request (e.g. "HTTP/1.1")
"""

Actually, I realized now that this is probably OK since it section 3
only discusses HTTP layering over SPDY, whereas the comment in section
2 made no such assumption that this was HTTP.

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#HTTPLayer
has a typo:
"SPDY is intended to be as compatible as possible with current
web-based applications. This means that, from the perspective of the
server business logic or application API, the features of HTTP are
change."

I think the desired word is unchanged instead of change.

Cheers.

On Tue, Jun 14, 2011 at 5:47 PM, William Chan (陈智昌)

[William Chan (陈智昌)]

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#RST_STREAM
is has 3 different status codes all with the value of 6.

"""
6 - FLOW_CONTROL_ERROR. The endpoint detected that its peer violated
the flow control protocol.
6 - STREAM_IN_USE. The endpoint received a SYN_REPLY for a stream already open.
6 - STREAM_ALREADY_CLOSED. The endpoint received a data or SYN_REPLY
frame for a stream which is half closed.
"""

I should also note that "Note: 0 is not a valid status code for a
RST_STREAM.", yet
http://codesearch.google.com/codesearch#OAMlx_jo-ck/src/net/spdy/spdy_session.cc&exact_package=chromium&q=SpdySession&type=cs&l=1271.

On Thu, Jun 16, 2011 at 2:09 PM, William Chan (陈智昌)

[Costin Manolache]

Hi, 

One not frequently implemented or used feature of HTTP is sending headers at the end of the body,

if chunked encoding is used.  (http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1)

I guess the use case would be to send a CRC or signature for the body, or any other info that was not known in advance.

There is also support for 'chunk extensions' - i.e. name/value pairs associated with a chunk. Again, a use case would be to send some metadata - like a CRC.

To support 'chunk extensions', one option would be to extend the 'Data' frame - add a Flag 'extra headers', and if the flag is set the first part of the data frame should be interpreted like Name/Value pair section in SYNC_STREAM.

For headers-at-end: relax the  the 'only one SYNC_STREAM' or add a separate frame that can be used to close the stream and send additional headers. 

I think the ability to extend the protocol to include either chunk or full body signatures might be nice, in particular if SPDY over non-SSL is allowed. There are other use cases - probably feature parity with http is not a major issue since both features are rarely used.

Costin

[Mike Belshe]

Will -

Thanks for all the great comments.  Sorry to take so long, finally getting to them now.

On Mon, Jun 13, 2011 at 6:01 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

I think the stream half-close descriptions in the spec are
confusing/wrong. Examples:

* http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#StreamHalfClose
says: "If an endpoint receives a data frame after the stream is
half-closed, it must send a RST_STREAM to the sender with the status
STREAM_ALREADY_CLOSED." This is vague and should be clarified. It
could be interpreted such that, if the client has set the FIN flag in
its SYN_STREAM, receives a SYN_REPLY without a FIN, and then receives
data frames, that the client should respond to the data frames with a
RST_STREAM.

Gotcha.  You're pointing out that it is unclear which direction is half-closed.

Retrying with the following:

If an endpoint receives a data frame after the stream is half-closed from the sender (e.g. the endpoint has already received a prior frame for the stream with the FIN flag set), it MUST ...

 

* http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#SYN_REPLY
says: "If an endpoint receives a SYN_REPLY frame for a stream already
in the half-closed state, it MUST issue a stream error (Section 2.4.2)
with the error code STREAM_ALREADY_CLOSED." I'm confused by this. If
the client's SYN_STREAM already had a FIN, and thus the stream is
half-closed, it seems perfectly reasonable to send a SYN_REPLY. I'm
also not sure when this is applicable. Perhaps it's meant to cover the
case when the server has already sent a SYN_REPLY with a fin or a data
frame with a FIN, but then sends another SYN_REPLY for the same
stream?

Agree - I think this is redundant.  Removed.

 

But that would be covered in the previous sentence which says:
"If an endpoint receives multiple SYN_REPLY frames for the same active
stream ID, it MUST issue a stream error (Section 2.4.2) with the error
code STREAM_IN_USE." There is also some ambiguity about which of these
two sentences applies to the fully closed state, STREAM_ALREADY_CLOSED
or STREAM_IN_USE?

Perhaps I'm being too anal, but I think the descriptions could be clarified.

No - this is great stuff.  Thank you!!!
 

Also, I see that you've updated the spec wrt the invalid headers like
Connection. The request headers section notes that these headers MUST
not be sent: "The Connection, Keep-Alive, and Proxy-Connection headers
are not valid and MUST not be sent."
(http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.3.2.1)
However, the response headers section does not include the same "MUST
not be sent" language: "The Connection and Keep-alive response headers
are no longer valid."
(http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.3.2.2).

Fixed. 

And for both the requests and responses, the spec notes that
Transfer-Encoding is no longer valid. How about we add the same
language and say that it must not be sent?

Done.

[Mike Belshe]

On Fri, May 27, 2011 at 5:13 AM, Yoav <yoav.weiss.fr@gmail.com> wrote:

I have a few comments regarding compression:
1. As we have learned from the HTTP spec, defining "zlib" compression
is not enough. The original zlib terminology refers to the compression
algorithm as "deflate" (http://www.gzip.org/zlib/rfc-deflate.html)
when "zlib" (http://www.gzip.org/zlib/rfc-zlib.html) and
"gzip" (http://www.gzip.org/zlib/rfc-gzip.html) are wrappers using
"deflate" internally. The HTTP spec defined the "deflate" content
encoding as using "zlib". That caused a lot of confusion in various
implementations and made "deflate" practically unreliable.
IMO the SPDY spec should explicitly define which compression it uses,
and stick to the original terminology. I.e. if we want to use raw
deflate (since we don't need a checksum), we should call that
compression "zlib's deflate algorithm as defined in RFC 1951" (or
something of that nature).

Of course, you are right.

I've locked this in as RFC1950, which is what I think we're currently using.

 

2. Regarding compression contexts for data:
There can be scenarios where a global compression context can give
better results then a compression context per resource (One example is
a series of small & similar JS/CSS files).
OTOH, that will force using the same stream for multiple resources in
situations where multiple streams may have given better results (in
terms of speed).
Another problem is that the content author and server side (who may

Not true - this is the SPDY protocol specification.  The protocol does not create a contract between the web server and the content author - that is an independent issue.  The same exists for HTTP today - the web server can be configured to automatically compress certain types of content.

 

have prior knowledge as to what compression context usage would be
more beneficial) have no saying in the matter. Assigning resources to
stream (as far as I understand it) in a traditional browsing scenario,
is the client-side's job. The client can use heuristics (such as
assigning dedicated streams according to requesting tag type) but
naturally, it has less info regarding the actual data that's going to
come in.
I think the "static, up to 16 compression contexts" proposal from
section 4.5 could have resolved the above issues if the compression
context number could be given on a per resource/frame basis from the
server side. I believe the benefits for such a compression scheme
would outweigh the extra implementation complexity/cost. A short, non
scientific test I ran shows 5-10% compression improvement from mutual
ZLIB context between text resources.

This is an interesting idea, and I'd love to see more scientific data on it.

The purpose of the compression contexts was part of privacy review.  It's a way to ensure that nothing can be gleaned from a second request about what might have been transmitted across a prior request.  I hadn't really considered it for cross-resource compression optimizations.

From an implementation point of view, compression contexts are difficult to code around - especially if two resources are being transmitted using the same context.  The implementation has to decide when to compress data.  Once it is compressed, the order in which you can transmit that data is FIXED.   E.g, if I compress blocks in the order,   A, C, D, B, E, then I'm committed to transmitting them in that order, even if a higher priority block comes along.  This sounds trivial, but it actually is quite limiting and is one of the more subtle points about spdy.

 

We should take the opportunity of introducing SPDY to also enable
newer compression algorithms such as bzip (which was removed from
chrome due to intermediary proxies trouble as far as I understand) and
7zip. A zlib variant with a larger window might also be beneficial in
mutual context scenarios. In yet another non-scientific test I ran, I
see 7zip compressing HTML,CSS & JS, producing files that are 30%
smaller then gzip (with a mutual context for both), with relatively
similar decompression speeds. The server can then decide regarding the
CPU vs. BW tradeoff.

I'm strongly against compression type negotiation, but I am not against use of multiple types.  If you can show real benefits here, I am open to them.

 

All in all, IMO removing data compression from the specification would
be a wasted opportunity to provide significant data reduction to the
web.

I am inclined to agree.

Mike

[Mike Belshe]

On Sat, May 28, 2011 at 4:53 AM, Simon B. <simon....@gmail.com> wrote:

+1 specific on which compression to use. Though if SPDY anyways will
get wrapped in SSL, maybe all compression can be done by SSL?

I don't know of any SSL Servers that advertise any compression support (look at the ServerHello handshake, the compressor list is always null)

 

If SPDY will do compression -- things to consider when choosing:
* There might be a point in using the same compression as for HTTP, in
that a web server can precompress all its static resources. (Maybe
servers like nginx and varnish already does this?)

Not sure what compression you're referring to here?

 

 

* Is there risk of degrading overall performance if compressed SPDY is
tunnelled through SSL that also does compression?

Yes, but I suspect this is trivial and also optimizable.

 

* For non-static content there is no need to allow pre-compression,
thus a more modern compression can be pushed to gain adoption. On
http://stackoverflow.com/questions/124239/fastest-c-file-compression-library-available
I found hints towards LZMA. Either way, any new algorithm must come
with well-tested code and no "problems" of copyright, patents, etc.
Also Google Chrome want's to have a smaller binary (just speed, or
perhaps aiming for embedded use?) and thus may reject adding big new
libs to its codebase, making deflate an easy choice to avoid adding
more code.

There are several factors:

Latency:

    time to compress

    compression ratio  (which has a latency to transmit)

    time to decompress 

Memory

    minimum memory usage for servers/proxies.

Many studies have been done on this, and I've looked at it extensively, although it has now been over a year.  I don't mean to sound stubborn, but I'm past the point of looking for new, better compressors.  If someone wants to do the full study and 

provide data, I'm very open to that.  But otherwise, declaring that "we should use the best compression algorithm" isn't helpful.   A full study of these 4 attributes needs to be done.... ;-(

Personally, I think we're at the point of "better is the enemy of good" for compressor choices.

Mike

[Mike Belshe]

On Sun, Jun 12, 2011 at 6:01 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

s/HEADER frame/HEADERS frame/

done

[Mike Belshe]

On Mon, Jun 13, 2011 at 11:45 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

When a SYN_STREAM/SYN_REPLY/HEADERS frame has mismatching frame length
vs name/value header block lengths, in the Chromium SPDY
implementation (http://codesearch.google.com/codesearch#OAMlx_jo-ck/src/net/spdy/spdy_framer.cc&exact_package=chromium&q=SpdyFramer&type=cs&l=233)
we return RST_STREAM
(http://codesearch.google.com/codesearch#OAMlx_jo-ck/src/net/spdy/spdy_session.cc&type=cs&l=1213).

The problem here is, since we compress the name/value header block,
isn't the compression context horked now? Should the spec document
this case and treat it as a session error instead of a stream error?

There are two types of failures here.

a) You could have sent extra bytes in the header-block portion of the compressed frame.  In this case, the compression context may or may not be hosed, and we could recover with only resetting the stream.

b) You could have sent too few bytes in the header-block portion of the compressed frame.  In this case, the receiver will just wait for more data (potentially forever), and when it does receive it, the entire stream will likely be corrupt.  There is no good answer here, except don't do that.  Severe PROTOCOL_ERROR.

Mike

[Mike Belshe]

On Tue, Jun 14, 2011 at 7:47 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

For stream errors, we send a RST_STREAM. For session errors, we send
GOAWAY and close the connection. RST_STREAM has a status code to
indicate what happened. Should GOAWAY also have a status code? I
should note that GOAWAY is also sent in normal shutdown (client or
server decides to free up the connection) so a status code should
include that. Just a thought.

GOAWAY does carry a status code already?  :-)  I think we already added this in draft 3.  It might not have been in draft 2.

Mike

[Mike Belshe]

On Thu, Jun 16, 2011 at 4:09 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.2.3
says:
"Streams optionally carry a set of name/value header pairs."

This directly contradicts

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#rfc.section.3.2.1

which says:
"""
The first line of the request is unfolded into name/value pairs like
other HTTP headers and MUST be present:
":method" - the HTTP method for this request (e.g. "GET", "POST", "HEAD", etc)
":path" - the url-path for this url with "/" prefixed. (See RFC1738
[RFC1738]). For example, for "http://www.google.com/search?q=dogs" the
path would be "/search?q=dogs".
":version" - the HTTP version of this request (e.g. "HTTP/1.1")
"""

Actually, I realized now that this is probably OK since it section 3
only discusses HTTP layering over SPDY, whereas the comment in section
2 made no such assumption that this was HTTP.

Exactly.  Section 2 attempts to define a fairly application-agnostic version of SPDY's framing.  It isn't a goal to be reusable, but it does make logical sense (I think).

 

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml#HTTPLayer
has a typo:
"SPDY is intended to be as compatible as possible with current
web-based applications. This means that, from the perspective of the
server business logic or application API, the features of HTTP are
change."

I think the desired word is unchanged instead of change.

All your base are belong to us.

[Mike Belshe]

Thanks, Costin -

We have been "thinking about" chunked headers for some time.  I'm at the point of dropping them, because while we can imagine cases to use them, nobody has a serious case where they actually will use them.  It also creates more burden for doing a SPDY -> HTTP gateway.  What to do if your HTTP server doesn't support any form of chunked headers?

I'm sure we could churn for a long time about what they could do, but it doesn't seem to matter much if nobody is going to use them anyway?

Mike

[Dzonatas Sol]

On 06/18/2011 04:06 PM, Mike Belshe wrote:
>
> Personally, I think we're at the point of "better is the enemy of
> good" for compressor choices.
>
> Mike
>
>

Does that mean in the inverse way or the converse; bad or evil; ugly? Mud.

M.U.D., quantum movements, patent-flows.... you can't get better then
the license in the language let you. You have to make sure
java/javascript is not being used in the connection. That restriction
does not exists with those that developed their own VM.

--
--- http://twitter.com/Dzonatas_Sol ---
Web Development, Software Engineering
Ag-Biotech, Virtual Reality, Consultant

[Mark Nottingham]

On 18/06/2011, at 4:01 PM, Mike Belshe wrote:

> 1. As we have learned from the HTTP spec, defining "zlib" compression
> is not enough. The original zlib terminology refers to the compression
> algorithm as "deflate" (http://www.gzip.org/zlib/rfc-deflate.html)
> when "zlib" (http://www.gzip.org/zlib/rfc-zlib.html) and
> "gzip" (http://www.gzip.org/zlib/rfc-gzip.html) are wrappers using
> "deflate" internally. The HTTP spec defined the "deflate" content
> encoding as using "zlib". That caused a lot of confusion in various
> implementations and made "deflate" practically unreliable.
> IMO the SPDY spec should explicitly define which compression it uses,
> and stick to the original terminology. I.e. if we want to use raw
> deflate (since we don't need a checksum), we should call that
> compression "zlib's deflate algorithm as defined in RFC 1951" (or
> something of that nature).
>
> Of course, you are right.
>
> I've locked this in as RFC1950, which is what I think we're currently using.

FYI, see:
http://trac.tools.ietf.org/wg/httpbis/trac/ticket/73
http://trac.tools.ietf.org/wg/httpbis/trac/changeset/806#file1

--
Mark Nottingham http://www.mnot.net/

[Mark Nottingham]

Trailer support would be nice; people are actually starting to look at them with interest (e.g., the Varnish guys have use cases that they'd be interested in, if browsers would support them).

Chunk extensions haven't been used, AFAIK. We haven't discussed them much in HTTPbis, but I wouldn't be surprised if we deprecated them.

Cheers,

--
Mark Nottingham http://www.mnot.net/

[Costin Manolache]

On Sat, Jun 18, 2011 at 4:21 PM, Mike Belshe <mbe...@google.com> wrote:

Thanks, Costin -

We have been "thinking about" chunked headers for some time.  I'm at the point of dropping them, because while we can imagine cases to use them, nobody has a serious case where they actually will use them.  It also creates more burden for doing a SPDY -> HTTP gateway.  What to do if your HTTP server doesn't support any form of chunked headers?

I have no problem with dropping them in SPDY->HTTP gateways in case the http server or client would get confused.

But in a 'spdy only' environment they would provide an extension point that may avoid a backward-incompatible change. At worse they'll have the same fate as HTTP trailing and chunk headers. 

At least define an 'extra space / padding' flag - if the bit is set, after the data frame there is a length-prefixed space that implementations should know to skip. 

 

I'm sure we could churn for a long time about what they could do, but it doesn't seem to matter much if nobody is going to use them anyway?

I think they might be used, at least on intranets.  

I don't know what's the final word on mandatory SSL, but if SPDY gets used over non-SSL it would be nice to have at least a MAC for data integrity. Even with mandatory SSL - it may not be end-to-end SSL, so it's good to have an extra layer. 

 

If chrome would support a CRC / MAC - it might get deployed on servers as well. 

Costin

[William Chan (陈智昌)]

It seems like the protocol versioning section has been removed in
draft 3. IIRC, this may be because we currently rely on NPN /
Alternate-Protocol to handle version negotiation. If that's the case,
do we need RST_STREAM with UNSUPPORTED_VERSION? I think the spec
should address protocol versioning.

[Mike Belshe]

On Mon, Jun 20, 2011 at 7:45 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

It seems like the protocol versioning section has been removed in
draft 3. IIRC, this may be because we currently rely on NPN /
Alternate-Protocol to handle version negotiation. If that's the case,
do we need RST_STREAM with UNSUPPORTED_VERSION? I think the spec
should address protocol versioning.

I removed all the versioning sections in my last set of updates (I sent email to the list about this).

The problem is that it is unproven and unimplemented.  Without pragmatic implementation, it seemed best to remove.

I should remove the reference to unsupported_version too.

Mike

PS - Let's move new comments to new threads, so that we don't just generate a huge centithread! :-)

[William Chan (陈智昌)]

Oops, sorry. I fail at using gmail search =/