Intent to ship: Alternative Services

[Bence Béky]

== Contact emails ==

"Bence Béky" <b...@chromium.org>

"Ryan Hamilton" <r...@chromium.org>

== Specification ==

https://tools.ietf.org/html/rfc7838

== Summary ==

Alternative Services allow an origin serving an http:// or

https:// resource to nominate additional (protocol, host,

port) tuples that the client can choose to request the

resource from instead of the origin when making subsequent

requests.  This can be used, for example, as a protocol

upgrade mechanism, for connection pooling, or for load

balancing.

== Link to "Intent to implement" discussion ==

https://groups.google.com/a/chromium.org/forum/#!topic/net-dev/wBWmoQ00Ysc

== Platforms ==

This feature is supported on all six Blink platforms

(Windows, Mac, Linux, Chrome OS, Android, and Android

WebView).

== Debuggability ==

about:net-internals/#events shows "Alt-Svc" response

headers, also potential requests to alternative services.

about:net-internals/#alt-svc lists the active Alternative

Service Mappings.

== Interoperability and compatibility risk ==

Alternative service advertisements are sent by the server.

From the client side, there is no compatibility risk,

because a server not supporting alternative services will

not send such advertisements.  From the server side, it has

to be considered that clients not supporting alternative

services but otherwise compliant with the HTTP or HTTP/2

protocol specification will ignore such advertisements.

== Status ==

Alternative Services experiments have been running since

release 51 on Canary and Dev channel, used instead of

Alternate Protocol as a protocol upgrade mechanism to QUIC.

The latency and error count metrics of the experiment group

are identical to that of the control group, as expected.

== OWP launch tracking bug ==

https://crbug.com/615115

== Technical tracking bug ==

https://crbug.com/392575

== Entry on the feature dashboard ==

http://www.chromestatus.com/feature/5754540846481408

[Rick Byers]

The interop risk we're concerned with here is that we end up regretting supporting this protocol because it fails to gain wide adoption in other browsers and servers, but enough adoption that we can't just rip out our support without some negative consequences.  Or perhaps worse, that another browser ships an incompatible implementation.  Is see your Chromestatus entry says this is in development in Firefox, any details (status / bug) you can point to?  Are the Mozilla engineers working on it aware that we're shipping (and so breaking changes to the spec are about to become more difficult)?  The entry also says it's been shipped in Opera - is that really true?  Regardless, l imagine the interop risk is lower in general for protocols than other browser APIs - this is just a hint and we could always rip it out later if we determine it was a mistake, right?

In terms of the compat risk, I imagine the real risk is that there are servers out there already sending incorrect Alt-Svc headers which would cause apparent breakage to Chrome users.  Assuming there's no reason to believe this is likely, then I don't think it's anything we need to worry about proactively.

[Elliott Sprehn]

This seems like it has the same risks as service worker for allowing an attacker who MiTM's your connection to persistently intercept all your requests in the future. Why is this not restricted to https only?

[Anne van Kesteren]

On Thu, May 26, 2016 at 8:08 PM, Bence Béky <b...@chromium.org> wrote:
> == Interoperability and compatibility risk ==
>
> Alternative service advertisements are sent by the server.
> From the client side, there is no compatibility risk,
> because a server not supporting alternative services will
> not send such advertisements. From the server side, it has
> to be considered that clients not supporting alternative
> services but otherwise compliant with the HTTP or HTTP/2
> protocol specification will ignore such advertisements.

Having read through
https://bugzilla.mozilla.org/show_bug.cgi?id=1148357 again (and
followed the link in comment 26 and some follow up discussion) it
sounds like Chrome has a different approach here to Firefox,
potentially compromising the same-origin policy. Could you perhaps
explain what the current situation is with respect to that?


--
https://annevankesteren.nl/

[Samir Jafferali]

>potentially compromising the same-origin policy

The Alt-svc is effectively a more secure DNS CNAME (because of altName validation). How does it compromise the same-origin policy when DNS CNAMEs don't?

-Samir

[Bence Béky]

Hi Rick,

Thank you for your questions.  Responses inline below.

On Thu, May 26, 2016 at 2:20 PM, Rick Byers <rby...@chromium.org> wrote:
>
> The interop risk we're concerned with here is that we end up regretting supporting this protocol because it fails to gain wide adoption in other browsers and servers, but enough adoption that we can't just rip out our support without some negative consequences.  Or perhaps worse, that another browser ships an incompatible implementation.

This is entirely possible, although an RFC having been published makes it less likely that another browser would implement it an incompatible way (or at least would make it clear what the right behavior should be).

>  Is see your Chromestatus entry says this is in development in Firefox, any details (status / bug) you can point to?

Good question.  Bug 1003448 "http/2 alt-svc support (RESOLVED FIXED)" would be a good example of a bug, also change log of AlternateServices.h points to some other relevant bugs.  Also, mnot's (Alternative Services RFC co-author) blog entry from this March claims that "Mozilla have already implemented Alternative Services in Firefox".

> Are the Mozilla engineers working on it aware that we're shipping (and so breaking changes to the spec are about to become more difficult)?

I just sent Patrick McManus (Mozilla developer, another co-author on the Alternative Services RFC) a heads up, thank you for reminding me.

> The entry also says it's been shipped in Opera - is that really true?

This is a mistake on my part.  In the webform I specified that the implementation status is "in experimental framework" for Chrome 51, which autopopulated the Opera release with 38.  Indeed the code must be there behind field trial flags in Opera, but it certainly has not shipped.  I edited the feature dashboard entry to reflect this.

> Regardless, l imagine the interop risk is lower in general for protocols than other browser APIs - this is just a hint and we could always rip it out later if we determine it was a mistake, right?

That is correct.  Note, however, that Alternative Services is the only mechanism to upgrade to QUIC (besides Alternate-Protocol which is removed in M53), therefore it is unlikely that it would be removed.

> In terms of the compat risk, I imagine the real risk is that there are servers out there already sending incorrect Alt-Svc headers which would cause apparent breakage to Chrome users.  Assuming there's no reason to believe this is likely, then I don't think it's anything we need to worry about proactively.

My understanding is that Mozilla has already shipped this feature, which would help exert pressure on server operators not to send incorrect Alt-Svc headers.

Cheers,

Bence

[Rick Byers]

Thanks for the responses Bence!

If Mozilla has already shipped that definitely changes both the interop and compat equations.  Can you point to a test / demo page that we can use to try it out and see for sure?

Cheers,

Bence

[Anne van Kesteren]

On Fri, May 27, 2016 at 8:19 PM, Samir Jafferali <pub...@samir.ca> wrote:
>>potentially compromising the same-origin policy
>
> The Alt-svc is effectively a more secure DNS CNAME (because of altName
> validation). How does it compromise the same-origin policy when DNS CNAMEs
> don't?

Did you read through the bug I referenced? It explains it quite well.
(A more secure DNS CNAME doesn't mean much...)

The mailing list discussion I referenced explains how the Firefox and
Chrome implementation differ, at least at that point, with Chrome
allowing cross-port communication and Firefox not. So saying "Mozilla
shipped this feature" isn't really quite sufficient to mitigate all
concerns, especially if the implementations differ.


--
https://annevankesteren.nl/

[Bence Béky]

On Fri, May 27, 2016 at 3:52 AM, Anne van Kesteren <ann...@annevk.nl> wrote:
>
> Having read through
> https://bugzilla.mozilla.org/show_bug.cgi?id=1148357 again (and
> followed the link in comment 26 and some follow up discussion) it
> sounds like Chrome has a different approach here to Firefox,
> potentially compromising the same-origin policy. Could you perhaps
> explain what the current situation is with respect to that?
>

Hi Anne,

There is a cross-origin issue mentioned at https://bugzilla.mozilla.org/show_bug.cgi?id=1148357.  My understanding is that the root of the issue is that Firefox does not require that the alternative's certificate matches the origin.  They seem to refer to this as "OE", which I believe stands for opportunistic encryption.  Chromium is not affected by this issue, because it requires that the alternative service is encrypted and that its certificate matches the origin, and I do certainly not indend to weaken this security requirement.

There is another issue discussed on ieft-h...@w3.org, that of priviliged ports, which is linked in comment 26.  As Ryan Hamilton stated, Chromium requires the alternative port to be < 1024 if the origin port is < 1024, which seemed to be the final conclusion of that e-mail thread anyway.

Does this answer your question?

Cheers,

Bence

[Bence Béky]

On Thu, May 26, 2016 at 2:20 PM, Elliott Sprehn <esp...@chromium.org> wrote:
>
> This seems like it has the same risks as service worker for allowing an attacker who MiTM's your connection to persistently intercept all your requests in the future. Why is this not restricted to https only?
>

Hi Elliott,

This is a very good point. After some discussion, the decision has
been made to restrict Alternative Services to https origins. This
makes is easier to reason about the security properties of the code,
while there is no compelling use case that the current implementation
would serve but restricting to https origins would not. I filed
https://crbug.com/615497 to track this.

For the record, the current implementation allows origins with "http"
scheme, but requires that the alternative service is encrypted, and
that the alternative service certificate is valid for the origin. For
this to be useful, the site operator already needs to have a valid
certificate for the origin anyway. However, a MITM could inject an
Alt-Svc response header in the unencrypted traffic, potentially with a
long expiration time, causing the client to initiate TLS connections
with a server of the attacker's choice, eating up resources.

Another potential issue with the current implementation is that if
there is a bug in the server treating http requests over a secure
connection as if they were of https origin, sensitive information
could be communicated to the client, which the client treats as
associated with an http origin and thus might leak it in a subsequent
request on an unencrypted channel.

Both of these issues are mitigated by restricting Alt-Svc to secure
origins. This is also consistent with the proposition at
https://www.chromium.org/Home/chromium-security/prefer-secure-origins-for-powerful-new-features.

Thank you,

Bence

[Anne van Kesteren]

On Tue, May 31, 2016 at 6:14 PM, Bence Béky <b...@chromium.org> wrote:
> There is a cross-origin issue mentioned at
> https://bugzilla.mozilla.org/show_bug.cgi?id=1148357. My understanding is
> that the root of the issue is that Firefox does not require that the
> alternative's certificate matches the origin. They seem to refer to this as
> "OE", which I believe stands for opportunistic encryption. Chromium is not
> affected by this issue, because it requires that the alternative service is
> encrypted and that its certificate matches the origin, and I do certainly
> not indend to weaken this security requirement.

Hmm, I'm probably not understanding. I thought the issue was multiple
origins sharing a certificate and then one of those origins launching
a cross-origin attack on the other through usage of this header. Since
in those scenarios whenever A talks to B, the certificate B presents
will cover A as well as itself. The fact that they share a certificate
doesn't mean you should be able to bypass CORS.

> There is another issue discussed on ieft-h...@w3.org, that of priviliged
> ports, which is linked in comment 26. As Ryan Hamilton stated, Chromium
> requires the alternative port to be < 1024 if the origin port is < 1024,
> which seemed to be the final conclusion of that e-mail thread anyway.

--
https://annevankesteren.nl/

[Samir Jafferali]

>The fact that they share a certificate doesn't mean you should be able to bypass CORS.

Today, can’t we already bypass CORS in Chrome with DNS rebinding (updating DNS records to point to another end point AFTER preflight)?  DNS rebinding is not a hack, many companies use DNS rebinding in production today for load balancing (short ttls ranging from 20 seconds to 5 minutes).   

One of the wonderful purposes of Alt-Svc is to avoid using DNS altogether; allowing the http server to do that work.  Which is why I stated that Alt-svc is effectively a more secure DNS CNAME (because of altName validation).   I don’t see how it compromises the same-origin policy anymore than DNS does.

I believe Anne’s point is that headers are easier to hack than DNS and that some organizations share TLS certs.  He has at least 4 concerns (Anne correct me if I’m wrong):

1. Alt-svc remapping scheme + port (not host)
2. Alt-svc also remapping across hosts (host + scheme + port)
3. Organizations sharing TLS certs
4. Implementation differences between browsers


For the above points:

1. Hasn't this already been implemented in Chrome as "Alternate-Protocol" since 2013 and used by services like Google and Youtube to switch to quic?
 
2. Alt-Services replaces "Alternate-Protocol" by enabling host alternatives. Is this the only addition on top of "Alternate-Protocol"? I'm trying to understand how this adds vulnerabilities to CORS preflight requests not already present with DNS rebinding.  

3. As for SAN certs containing altNames for multiple organizations, those certs are already ripe for abuse.  Also shared certs tend to be on CDN services that offer caching of static resources.  For dynamic data (proxying/DSA/DWA), TLS certs are generally not shared.

4. I don't have comments on this. 

-Samir 

[Bence Béky]

On Tue, May 31, 2016 at 1:35 PM, Samir Jafferali <pub...@samir.ca> wrote:
>
> [...]

>
> 2. Alt-Services replaces "Alternate-Protocol" by enabling host alternatives. Is this the only addition on top of "Alternate-Protocol"?

While arguably this is the most powerful addition to Alt-Svc with
respect to Alternate-Protocol, this is not the only one. For example,
Alt-Svc allows multiple entries, has a keyword for removing previously
advertised entries. Also Alt-Svc allows the server to specify an
expiration, while Alternate-Protocol headers did not include an
expiration field.

Bence

[Anne van Kesteren]

On Tue, May 31, 2016 at 7:35 PM, Samir Jafferali <pub...@samir.ca> wrote:
> Today, can’t we already bypass CORS in Chrome with DNS rebinding (updating
> DNS records to point to another end point AFTER preflight)? DNS rebinding
> is not a hack, many companies use DNS rebinding in production today for load
> balancing (short ttls ranging from 20 seconds to 5 minutes).
>
> One of the wonderful purposes of Alt-Svc is to avoid using DNS altogether;
> allowing the http server to do that work. Which is why I stated that
> Alt-svc is effectively a more secure DNS CNAME (because of altName
> validation). I don’t see how it compromises the same-origin policy anymore
> than DNS does.

That assumes the attacker has access to DNS.

Some entities use SAN certificates for many origins. With this
feature, if one of those origins is compromised in some way, e.g., the
attacker gaining access to part of the server, they can use that to
launch an attack on the other origins. As I understand it, anyway.

It does seem that if a target protects itself against DNS rebinding
through carefully checking the Host header, they would also be
protected against Alt-Svc attacks. Usually that kind of argument does
not carry enough weight to add to the problem.


--
https://annevankesteren.nl/

[Bence Béky]

Hi,

First, to tie off Elliott's question about restricting Alt-Svc to secure
origins, just a heads up that https://crbug.com/615497 is Fixed: Chrome now only
respects Alt-Svc headers over https.

Anne: on your question about the cross-origin scenario described in the linked
Bugzilla bug, I think I understand the scheme now. The discussion on that bug
is very thorough, and I do not think there is any virtue in revisiting any of
the arguments unless there is new piece of information available. It seems like
the issue has not been deemed severe enough to warrant a change in the
specification, even though the discussion happened early enough for that.
Therefore in this respect I intend to follow Firefox's implementation in
Chromium.

Rick: I am sorry, I cannot point you to a demo page, but a Firefox developer has
confirmed in private that they have shipped Alt-Svc based on an earlier draft of
the specification and they are working on updating to the RFC.

I believe I have answered Anne's question about Chrome's requirements on port
changing. I understand that Firefox and Chrome have different requirements
regarding alternative service port, and because of this, the fact that Firefox
launched Alt-Svc a while ago does not mitigate all concerns about Chrome's
implementation. On the other hand, my reading is that no particular security
concerns were raised on the working group e-mail thread about this specific
issue. Please let me know if I'm missing something here.

Cheers,

Bence

[David Benjamin]

Are we implementing the spec quite right yet? I recall there was an issue due to our original implementation treating Alt-Svc as an extension of the HttpStream pooling logic rather than a funny CNAME. It ended up manifesting as the SNI being "off" (in reality the SNI is correct, it's just that the layers above were off), among other behaviors. It seems the TCP version of this bug is still open: https://crbug.com/615413

Do you think it would be good to resolve that before shipping the feature?

David

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CACMu3ton_OS3Qqh%2BLseauD3LPNH4Ry421Q2fYpAmq0tYLGbO8A%40mail.gmail.com.

[Bence Béky]

Hi David,

Excellent question. Indeed https://crbug.com/615413 is still open,
and because of this, TCP Alt-Svc is currently restricted to
alternatives with a hostname identical to the hostname of the origin.
This restriction was implemented in https://crbug.com/615637. Once
the SNI issue is fixed for the TCP case, TCP Alt-Svc to different
hosts can be re-enabled.

On the other hand, QUIC Alt-Svc works correctly with respect to SNI
and pooling, this was solved in https://crbug.com/585876. Therefore
there are benefits to shipping Alternative Services now with the above
temporary restriction.

Cheers,

Bence

[Elliott Sprehn]

On Mon, Jun 6, 2016 at 9:25 AM, Bence Béky <b...@chromium.org> wrote:

Hi,

First, to tie off Elliott's question about restricting Alt-Svc to secure
origins, just a heads up that https://crbug.com/615497 is Fixed: Chrome now only
respects Alt-Svc headers over https.

Yay, thanks! :) 

- E 

[Chris Bentzel]

This seems to have stalled. 

Are there unanswered questions, or is this waiting for Blink API owners to take another look?

--

You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CAO9Q3i%2Ba44dLMuRqgEbay3-9GeSugDnvENLXsOd%3DrA1XkbPgjQ%40mail.gmail.com.

[Chris Harrelson]

Hi Chris (Bentzel),

I was waiting for a resolution to the question from David Benjamin a few messages back about resolving a particular issue before shipping. Bence Béky indicated they didn't disagree. Could someone please clarify the current state?

Apologies if I misunderstood and stalled out the thread.

Chris (Harrelson)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Rick Byers]

On Tue, Jun 21, 2016 at 1:53 PM, Chris Harrelson <chri...@chromium.org> wrote:

Hi Chris (Bentzel),

I was waiting for a resolution to the question from David Benjamin a few messages back about resolving a particular issue before shipping. Bence Béky indicated they didn't disagree. Could someone please clarify the current state?

I was confused by this too, but re-reading the thread now it sounds like Bence wants to proceed with shipping with the temporary limitation that avoids that issue.

So it sounds like there aren't any outstanding major objections, so LGTM1 from me.

[Bence Béky]

Hi,

I am sorry that I sent out this Intent to ship late in the shipping process.

Yes, I can confirm that I do not intend to enable TCP alternative
service with a host different from the origin until I fix
https://crbug.com/615413.

On the other hand, I believe QUIC alternative service with a host
different from the origin works correctly and can be shipped. This
will allow QUIC deployments to use alternative services for either
connection pooling or load balancing. I see no reason to wait with
shipping the QUIC case until I fix the TCP case.

Also, alternative service is replacing alternate protocol as an
upgrade mechanism for QUIC. The earlier alternative service ships (at
least with the above restriction), the earlier servers that currently
advertise redundant Alternate-Protocol and Alt-Svc headers can stop
advertising the former.

Thank you,

Bence

[Chris Harrelson]

Ok. LGTM2 then.

[Philip Jägenstedt]

LGTM3

[Bence Béky]

Thank you all. Please feel free to let me know if any further questions arise.

Bence

[pub...@samir.ca]

>Yes, I can confirm that I do not intend to enable TCP alternative service with a host different from the origin until I fix https://crbug.com/615413. 

Hi Bence, we were curious when TCP alt-svc would be shipped. Any timelines you can share would be appreciated.

-Samir 

[Bence Béky]

Hi Samir,

Unfortunately I cannot give you an estimate timeline at this point.  Sorry.

Bence

To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+unsubscribe@chromium.org.

To post to this group, send email to net...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/9df239c3-0bfd-46b7-b100-b2ade01ef023%40chromium.org.