ordering of required headers

[Patrick McManus]

for spdy/4 I'd like to recommend we make :version and :status required
(MUST) to be the first and second response headers and
similarly :method, :path, and :version to be the first 3 request
headers.

Those elements are required by HTTP/1 to be on the first line of an HTTP
transaction, so lack of proscribed ordering of them creates a potential
buffering problem for anything doing spdy<>http translation. And I don't
see any upside to the ordering flexibility. This is a bigger problem in
spdy/3 than in spdy/2 because of the (re)introduction of the HEADERS
frame - the buffering could go on for a while :)

-Pat

[Roberto Peon]

Sgtm.

[Matthew Steele]

If there's going to be a requirement that ":version" and ":status" are
the first two fields, please either allow them to be in either order,
or require that ":status" (rather than ":version") comes first.

Reason: implementation is much simpler in some cases if one can always
assume that it's valid to order the headers in any given frame in
ASCII-betical order. (Thankfully, ':' sorts before alphabetic
characters.)

[Patrick McManus]

"HTTP/1.1 200 OK" is {:version, :status} so I actually think it should
be that way because the HTTP mapping requires that order and the mapping
is the reason to do this.

[Adam Langley]

On Fri, Apr 27, 2012 at 9:03 AM, Matthew Steele <mdst...@google.com> wrote:
> Reason: implementation is much simpler in some cases if one can always
> assume that it's valid to order the headers in any given frame in
> ASCII-betical order.  (Thankfully, ':' sorts before alphabetic
> characters.)

That would preclude us adding :toffee in the future, because it would
sort between :status and :version.


Cheers

GAL

[Matthew Steele]

Well, I guess I meant to propose that the rule simply be that all the
special ":foo" headers must come first. Assuming we don't add very
many more ":foo" headers in the future (oh ho ho), that still
guarantees that ":status" and ":version" will come very early on. But
on further reflection I suppose the convenience gained for writing
frames is not worth the potential buffering issues, so I retract the
proposal.

[William Chan (陈智昌)]

Sounds like there's reasonable consensus here, and FWIW I agree with this suggestion too. Thanks Pat. Do you want to send out a suggested wording?

[Roberto Peon]

No order dependence on headers in generally, however, headers beginning with ':' go first.

agreed.

-=R

[Patrick McManus]

On Fri, 2012-04-27 at 09:56 -0700, William Chan (陈智昌) wrote:
> Sounds like there's reasonable consensus here, and FWIW I agree with
> this suggestion too. Thanks Pat. Do you want to send out a suggested
> wording?
>

Change 3.2.1
"The first line of the request is unfolded into name/value pairs like
other HTTP headers and MUST be present:"
to
"The first line of the request is unfolded into three name/value pairs,
like the other HTTP headers, and they MUST be present as the first three
request headers of the stream in :method, :path, :version order."

likewise in 3.2.2 Change
"The response status line is unfolded into name/value pairs like other
HTTP headers and must be present:"
to
"The response status line is unfolded into two name/value pairs, like
the other HTTP headers, and the MUST be present as the first two
response headers of the stream in :version, :status order."

and then swap the bullet points so they are in :version, :status order.

[William Chan (陈智昌)]

Sounds good. Feel free to submit a pull request with a committed edit, otherwise I'll just edit it myself tomorrow morning and commit it.

I assume "the MUST" is supposed to be "they MUST".

[Mike Belshe]

I'm not opposed to this - I like it :-) 

Caution, however.  Early versions of the SPDY spec had several "cleanup" and "better" restrictions on header ordering and format.  They've basically all been removed because they offered no tangible benefit but did provide ways for clients/servers to be incompatible and the error handling was confusing.

So if you do go down this path, the most important part is to define the error handling, and make all clients handle it the same.  If *one* browser starts accepting the undesirable ordering, then then this feature will fall apart...

Overall, while this seems like a good idea, its so minor, I'm not sure its worth the time.

But don't feel the need to debate me on it - its just my opinion.

Mike

[William Chan (陈智昌)]

Hah, I forgot to commit this. Too busy with other stuff. Thanks Mike for the comment, I agree with your concern. Let's define the error handling...I think we should RST_STREAM with PROTOCOL_ERROR. Thoughts?

[Hark Lee]

I don't think the order of headers important.it is very tiny thing that no improvement of Performance,and gives more complexity

2012/5/19 William Chan (陈智昌) <will...@chromium.org>

[Alek Storm]

Is it too late to object to this? If we're going to mandate a specific ordering of some headers, isn't their "name" field superfluous?

Alek

[Roberto Peon]

No and yes (at least for those whose positions are predefined).
Spdy/3 has already sailed, but there will be a spdy/4.

-=R

[Hark Lee]

even if Ordering is needed,I think the NAME field is useful if there will be a new field inserted between two old fields.

2012/5/23 Roberto Peon <fe...@google.com>

[Patrick McManus]

Remember this isn't general ordering of all headers, this is just
ordering of the headers that comprise the first request and response
line of HTTP - which is the exact same amount of ordering that HTTP
requires. Enforcing the ordering allows gatewaying between the protocols
without having to buffer multiple HEADERS frames on receipt - and it
doesn't present a burden to the sender to buffer because HTTP already
has that requirement. It's basically saying don't break that property
when tunneling across SPDY, which seems pretty reasonable.

[Alek Storm]

Yes, I understand the benefits, but it seems unnecessarily confusing for implementors to require some headers to be ordered, and not the others. Can these special headers be extracted from the Headers block and get their own section, as simply length-delimited strings?

Alek

[Patrick McManus]

You're basically arguing that well known headers can be identified by
indicies rather than redundantly including the header name. Instead of
doing that spdy relies on compression to identify and largely remove
repetitive names from the bytestream without having to specify all the
details - and this works very very well. You're not going to
meaningfully improve the transfer rate by fishing for a dozen
pre-compression bytes here. If you'd like to ignore the names in your
implementation (which one is that?) I guess you simply could. As you
point out - they are redundant but I'd rather have that tiny redundancy
than multiple data formats.

[Alek Storm]

I wasn't justifying my point with claims of smaller frame sizes - it would simply be intuitively clearer for consumers of the spec if these "ordered headers" weren't headers at all, since no such ordering requirement exists in HTTP. It's obviously not crucial; I have no problem dropping it if you disagree.

I'm currently finishing up spdy/2 for Tornado (http://tornadoweb.org).

Alek

[Patrick McManus]

On Thu, 2012-05-24 at 13:24 -0500, Alek Storm wrote:

>
>
> I'm currently finishing up spdy/2 for Tornado (http://tornadoweb.org).
>

awesome!

[Eric C]

Can we just require that ':' headers come before non-colon headers?

This ensures the important data is upfront while not making ordering completely strict.

[Patrick McManus]

that doesn't solve the problem the proposal is meant to solve (i.e. it
might still disable streaming).

it also doesn't help the sender as the data is already presented to it
in the right order when gatewaying because HTTP requires the same thing.

[Alek Storm]

That would defeat the purpose. AFAIK, there's nothing in the spec that prevents clients from adding arbitrary ':'-prefixed headers to their requests. Proxies would require arbitrarily-large buffers to store these headers while scanning for :status and :version.

Alek

[Eric Ceres]

Hi,

So just require that ':' can only be used for headers specifically defined in the spec. The ':' is used to ensure it does not collide with http headers which can not include a ':'.

- Eric Ceres

[Eric C]

Sorry addition to address Pattrick's response:

"that doesn't solve the problem the proposal is meant to solve (i.e. it might still disable streaming).

it also doesn't help the sender as the data is already presented to it in the right order when gatewaying because HTTP requires the same thing."

It does, the headers starting with ':' are very specific spdy protocol headers.  If there are a lot of headers sent with ':' that stops from streaming then there is something wrong in the server/client sending so many ':' headers.

If people want flexibility to expand spdy by adding other ':' headers that will be ignored if unknown that should be specified in the spec.  In that case we can just have a maximum size allowed for special ':' headers, which gives a guaranteed maximum read size before streamining.  If someone sends more then that size they violated protocol and should get an error.

- Eric Ceres

[Alek Storm]

That sounds much more complicated than just mandating a certain order for two headers. There's no reason other SPDY-specific headers should come first; :status and :version are needed only because they allow proxies to stream responses.

Alek

[Patrick McManus]

On 5/24/2012 8:56 PM, Eric C wrote:

Sorry addition to address Pattrick's response:

"that doesn't solve the problem the proposal is meant to solve (i.e. it might still disable streaming).

it also doesn't help the sender as the data is already presented to it in the right order when gatewaying because HTTP requires the same thing."

It does, the headers starting with ':' are very specific spdy protocol headers.  If there are a lot of headers sent with ':' that stops from streaming then there is something wrong in the server/client sending so many ':' headers.

it might still disable streaming if the colon headers are not in the same order that HTTP requires them. Particularly if they are spread amongst different spdy frames.

[Eric C]

Maybe I am missing something but wouldn't all spdy specific headers be mandatory to come before an http protocol headers?  

Right now the only spdy specific ones are for the request line/status line.  

I would think other spdy specific headers (if there are every any) would be beneficial to come before http.  This is one protocol encapsulating another, so it makes sense for the protocol encapsulating the other to need information pertinent to it before information about the protocol it is carrying.  All of this is actually not important as right now any other ':' header would either cause an error or be discarded (spec is not clear).

As such there are only the few, so which way it is implemented does not really matter.

Adding in Patrick's response:

"it might still disable streaming if the colon headers are not in the same order that HTTP requires them. Particularly if they are spread amongst different spdy frames."

I think the core difference in thinking is whether spdy is being made to as efficiently carry HTTP as possible or be a flexible protocol that can expand over time and carry other types of data.  I am not sure how either approach solves splitting frames.  Wouldn't it be valid for me to make initial frame with a single header (the first one) and then provide the second header in next frame?  If we want all required headers (the 2-3 specific ones defined or all ':' headers - doesn't matter) in the first frame then that should be be included in spec.  If you want them in the same order as HTTP why have multiple headers?  Seems better to just have a header of ':status' that encapsulates the entire status line.

Sorry bit long response, and all of this does not really matter unless it is clear wether other ':' headers are allowed and how they should be handled.

- Eric Ceres

[Alek Storm]

On Thu, May 24, 2012 at 8:27 PM, Eric C <eric...@gmail.com> wrote:

Maybe I am missing something but wouldn't all spdy specific headers be mandatory to come before an http protocol headers?  

Right now the only spdy specific ones are for the request line/status line.  

I would think other spdy specific headers (if there are every any) would be beneficial to come before http.  This is one protocol encapsulating another, so it makes sense for the protocol encapsulating the other to need information pertinent to it before information about the protocol it is carrying.

SPDY does not encapsulate HTTP, it frames it. An example of one protocol encapsulating another would be TCP on top of IP - an IP packet is structured as headers+payload, where the payload is an entire TCP packet. Payload data is interchangeable - IP can carry UDP as well.

SPDY is a way of converting the concept of an "HTTP message" into bytes to be sent over the wire. This is confusing because the method of doing this currently in widespread use is also called HTTP. As an example, an HTTP message contains a list of headers, each of which is a name/value pair. In SPDY, this is serialized as a series of length-delimited strings, the block of which is then gzipped. In HTTP, this is serialized as a series of CRLF-delimited strings, where names and values are separated by a colon (and isn't gzipped).

All of this is actually not important as right now any other ':' header would either cause an error or be discarded (spec is not clear).

If the SPDY spec is silent on an issue, I believe you can assume HTTP semantics. All headers are allowed, ':'-initial or not.

I think the core difference in thinking is whether spdy is being made to as efficiently carry HTTP as possible or be a flexible protocol that can expand over time and carry other types of data.

I think (and I believe others will agree) that the first priority of designing SPDY should be to carry HTTP as efficiently as possible. It's nice if others find uses for it outside of HTTP, but its design should not be sacrificed to accommodate them.

I am not sure how either approach solves splitting frames.  Wouldn't it be valid for me to make initial frame with a single header (the first one) and then provide the second header in next frame?  If we want all required headers (the 2-3 specific ones defined or all ':' headers - doesn't matter) in the first frame then that should be be included in spec.

No, it would not be valid, as is specifically stated in both the v2 and v3 specs. However, the headers that are required to be in the first header block are individually named, not generalized as ':'-initial; this seems to indicate that more ':'-initial headers may be added in the future that do not have such a requirement.

If you want them in the same order as HTTP why have multiple headers?  Seems better to just have a header of ':status' that encapsulates the entire status line.

So that we only have to do one round of parsing, instead of splitting the status line on whitespace. In fact, I'm in favor of splitting :status into :status and :reason, and :host into :host and :port, to save more string-splitting operations, but haven't actually proposed it yet.

[Hark Lee]

--So that we only have to do one round of parsing, instead of splitting the status line on whitespace. In fact, I'm in favor of splitting :status into :status and :reason, and :host into :host and :port, to save more string-splitting operations, but haven't actually proposed it yet. 

@Alek.Strom I agree with you , I think the Status And Reason should be seperated,Becuase Most of implementation are splitting these fields.

2012/5/25 Alek Storm <alek....@gmail.com>

[William Chan (陈智昌)]

This thread died out, although it appears there was rough consensus.
We should push this forward for SPDY/4. Volunteers to take this on and
make the spec change? If no one picks it up in the next few days, I'll
drive this, but I'd like to defer.

[Roberto Peon]

I'll have to do this, since it interacts with compression in annoying ways.

-=R

[William Chan (陈智昌)]

This never happened, but I'm taking care of it right now, since I'm copying this section from HTTP2: http://http2.github.io/http2-spec/#HttpRequest. It says:

"""

Header fields whose names begin with ":" (whether defined in this document or future extensions to this document) MUST appear before any other header fields.

"""

If anyone has a problem with that text, you should raise it in ietf-http-wg@. I'm going to copy it in to the SPDY/4 draft spec now.

[Roberto Peon]

This is still potentially problematic for the compression. There has been no resolution of/for that yet.

--
 
---
You received this message because you are subscribed to the Google Groups "spdy-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spdy-dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[William Chan (陈智昌)]

Hrm, I forgot about that. Do you have a planned resolution there? Or is it an unsolved problem that you think will remain unsolved (and we'll have to go back to HTTP2 to ask them to remove this requirement despite its other benefits)? If you think it'll stay unsolved, I'll go ahead and add a commit to remove the relevant clauses (it's also in the HttpResponse section of the spec).

[Roberto Peon]

It will stay unsolved 'till we've agreed on the basics of compression for HTTP/2 (which, hopefully Herve and I have now done), at which point we can see how to attempt to shoehorn something into the state machine or keep a separate context or something.

-=R

[James Cloos]

>>>>> "RP" == Roberto Peon <fe...@google.com> writes:

RP> or keep a separate context or something.

A separate context seems reasonable.

Outside of the compression, the :-headers, the rest of the headers and
the body can been seen already to be separate.

-JimC
--
James Cloos <cl...@jhcloos.com> OpenPGP: 1024D/ED7DAEA6