Intent to Ship: Accept-CH-Lifetime support for requesting client hints

[Tarun Bansal]

Contact emails

tba...@chromium.org, igri...@chromium.org, be...@chromium.org

Spec

https://tools.ietf.org/html/draft-ietf-httpbis-client-hints-05#section-2.2.2

Discussion threads:

• https://github.com/httpwg/http-extensions/issues/306 (feature)

• https://github.com/httpwg/http-extensions/issues/372 (privacy)

 

Blink-dev i2i thread:

https://groups.google.com/a/chromium.org/d/topic/blink-dev/QHI3sio6--Q/discussion

 

Summary and Motivation:

Client Hints enables user agent to provide device-specific preferences in the HTTP request headers, which can be used for proactive content negotiation. As currently implemented, hints are only available for subresource requests of a response that provides the explicit opt-in signal, either via Accept-CH response header or equivalent HTML markup. The opt-in preference is not persisted across navigations, and hints are thus not available on the initial (navigation) request.

 

Based on developer feedback, inability to receive hints on navigation requests has proven to be a major hurdle for adoption: many critical negotiation decisions need to be made on the initial request. To address this, the client hints spec was recently updated to allow origins to persist their opt-in policy for a specified period of time via the Accept-CH-Lifetime HTTP response header. One limitation of this approach is that the hints are not available on the “very first” navigation request to the origin, before the opt-in is discovered and persisted; hints are available on repeat navigation requests. However, given past concerns with respect to header bloat (if we were to enable all hints by default on all requests to all origins), we believe this is a good and workable tradeoff.

 

During development and privacy review of Accept-CH-Lifetime, we also converged on:

 

1. Client Hints delivery is restricted to Secure Contexts

2. Opt-in is origin-scoped and by default extends to 1P resources only.

1. 1P will have to explicitly delegate permission for 3P origins to opt-in (via Feature Policy)

3. Delivery is conditional on 1P origin having permission to store cookies and run JavaScript.

 

Above updates are “breaking” with respect to previous implementation: hints are no longer available over HTTP and 3P parties will not receive hints by default. However, CH usage is currently low (<= 0.03% of page loads) and we believe that the potential breakage is (a) low, and (b) outweighted by long-term security and privacy benefits of the new model.

 

Finally, this i2s does not address (2a): this will be solved via integration with Feature Policy and will ship separately later in the year.

 

Interoperability and Compatibility Risk:

Compat risk: low. Positive signals from web developers for Accept-CH-Lifetime.

Interop risk: small. Only Blink-based browsers support CH today.

 

Ongoing technical constraints:

None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Requesting Approval to Ship?

Yes.

 

Link to TAG discussion

https://github.com/w3ctag/design-reviews/issues/206

 

Privacy & Security

The headers are sent by the user agent only on secure HTTPS connections. Further, the client hints are attached to the request headers only if the origin has permissions to store cookies and run JavaScript. To prevent third party origins from accessing information that might be unavailable to them otherwise, the client hints are attached only to the requests to the first party origins.

 

Related discussions:

• https://github.com/httpwg/http-extensions/issues/372 (privacy)

• https://github.com/WICG/feature-policy/issues/129 (feature policy integration)

 

Is this feature fully tested by web-platform-tests?

Feature is well tested within Chromium, but web platform tests are missing.

Chromium issue to add web platform tests.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=735518

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5713139295322112

[Ilya Grigorik]

+Yoav Weiss +Alex Russell 

Yay, thanks Tarun for all the great spec+eng work to make this happen! Eager to see this ship, it will unblock a lot of important CH use cases.

[Yoav Weiss]

Thanks for working on this! LGTM1

On Wed, Mar 7, 2018 at 10:20 PM Tarun Bansal <tba...@chromium.org> wrote:

Contact emails

tba...@chromium.org, igri...@chromium.org, be...@chromium.org

Spec

https://tools.ietf.org/html/draft-ietf-httpbis-client-hints-05#section-2.2.2

Discussion threads:

• https://github.com/httpwg/http-extensions/issues/306 (feature)

• https://github.com/httpwg/http-extensions/issues/372 (privacy)

 

Blink-dev i2i thread:

https://groups.google.com/a/chromium.org/d/topic/blink-dev/QHI3sio6--Q/discussion

 

Summary and Motivation:

Client Hints enables user agent to provide device-specific preferences in the HTTP request headers, which can be used for proactive content negotiation. As currently implemented, hints are only available for subresource requests of a response that provides the explicit opt-in signal, either via Accept-CH response header or equivalent HTML markup. The opt-in preference is not persisted across navigations, and hints are thus not available on the initial (navigation) request.

 

Based on developer feedback, inability to receive hints on navigation requests has proven to be a major hurdle for adoption: many critical negotiation decisions need to be made on the initial request. To address this, the client hints spec was recently updated to allow origins to persist their opt-in policy for a specified period of time via the Accept-CH-Lifetime HTTP response header. One limitation of this approach is that the hints are not available on the “very first” navigation request to the origin, before the opt-in is discovered and persisted; hints are available on repeat navigation requests. However, given past concerns with respect to header bloat (if we were to enable all hints by default on all requests to all origins), we believe this is a good and workable tradeoff.

 

During development and privacy review of Accept-CH-Lifetime, we also converged on:

 

1. Client Hints delivery is restricted to Secure Contexts

2. Opt-in is origin-scoped and by default extends to 1P resources only.

1. 1P will have to explicitly delegate permission for 3P origins to opt-in (via Feature Policy)

3. Delivery is conditional on 1P origin having permission to store cookies and run JavaScript.

(1) seems important to prevent MITM fingerprinting. 

(2) will be breaking an important CH use-case, but I agree that it's required for privacy reasons. (2a) seems like the right path forward.

(3) seems important to prevent active fingerprinting in environments where it is not possible by other means.

 

Above updates are “breaking” with respect to previous implementation: hints are no longer available over HTTP and 3P parties will not receive hints by default. However, CH usage is currently low (<= 0.03% of page loads) and we believe that the potential breakage is (a) low, and (b) outweighted by long-term security and privacy benefits of the new model.

Agreed. Worth noting that "breakage" here means serving less-suited images to the user's device, but no broken content. Also, this will only happen on some subset of the overall usage, as only 3P hints will be affected.

 

 

Finally, this i2s does not address (2a): this will be solved via integration with Feature Policy and will ship separately later in the year.

Looking forward to (2a).

 

Interoperability and Compatibility Risk:

Compat risk: low. Positive signals from web developers for Accept-CH-Lifetime.

Interop risk: small. Only Blink-based browsers support CH today.

Would have been great to see more cross-browser support for this. Hopefully this change will result in higher usage and renew interest of other vendors. (e.g. this addition enables use-cases that I heard Mozilla folks express interest in)

 

Ongoing technical constraints:

None.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Requesting Approval to Ship?

Yes.

 

Link to TAG discussion

https://github.com/w3ctag/design-reviews/issues/206

That's quite a discussion :)

I agree with Ilya's conclusion:

* Sending hints without a server opt-in is a no-go due to bandwidth concerns by other vendors, so having an opt-in increases chances of cross-browser adoption. Passive fingerprinting concerns are also legitimate. 

* 's/Accept/Allow/' will result in breakage and force authors to modify content without being a clear win.

* `Accept-CH` and `Accept-CH-Lifetime` do not incentivize redirects more than say Service Workers. The "very first page" case may need resolution, but it shouldn't block this opt-in addition.

 

Privacy & Security

The headers are sent by the user agent only on secure HTTPS connections. Further, the client hints are attached to the request headers only if the origin has permissions to store cookies and run JavaScript. To prevent third party origins from accessing information that might be unavailable to them otherwise, the client hints are attached only to the requests to the first party origins.

 

Related discussions:

• https://github.com/httpwg/http-extensions/issues/372 (privacy)

• https://github.com/WICG/feature-policy/issues/129 (feature policy integration)

 

Is this feature fully tested by web-platform-tests?

Feature is well tested within Chromium, but web platform tests are missing.

Chromium issue to add web platform tests.

Do you intend to add WPT tests for `Accept-CH-Lifetime` before shipping?

FWIW, I don't think this intent should block on a full Client Hints test suite (although it would be great to add that as well. I should have added one with the initial implementation :/).

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=735518

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5713139295322112

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/63065789-64a7-44cd-a02f-cdbdb7538906%40chromium.org.

[Tarun Bansal]

I am actively working on adding the WPT tests. That also includes adding tests for client hints without ACL header.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=735518

Link to entry on the feature dashboard

https://www.chromestatus.com/features/5713139295322112

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Philip Jägenstedt]

Hi, what are the Chromium-internal tests we have for this, is it unit tests in the network stack and such? Wondering how much of this could be exercised as WPT, and from the description in this intent it sounds like we could also test that it doesn't work on non-secure contexts, and that it interacts with Feature Policy in the way you describe? +Ian Clelland has written lots of those tests and may have pointers.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/63065789-64a7-44cd-a02f-cdbdb7538906%40chromium.org.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/822104b2-e1ee-473a-9c9e-e5f686ce50b8%40chromium.org.

[Philip Jägenstedt]

Sorry, sloppy reading on my part, FP will come later. I'll comment on https://crbug.com/817049 on some things I think ought to be tested in addition, but LGTM2 to proceed and close that issue when done.

[Tarun Bansal]

> Hi, what are the Chromium-internal tests we have for this, is it unit tests in the network stack and such? 

There are Chromium browser_tests that cover all the possible variations (I think!) including secure context vs. non-secure contexts, if the origin has permission to store cookies and/or JavaScript. The browser_tests also cover the original Accept-CH behavior as well as the behavior of Accept-CH-Lifetime.

The tests are here: https://cs.chromium.org/chromium/src/chrome/browser/client_hints/client_hints_browsertest.cc

> Wondering how much of this could be exercised as WPT,

I think there are parts of the spec which can be covered. I am still exploring that, and adding tests gradually.  Adding test that it doesn't work on non-secure contexts is definitely possible, and I am going to do that very soon.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/63065789-64a7-44cd-a02f-cdbdb7538906%40chromium.org.

[Philip Jägenstedt]

Sounds like this is a case where leaning mostly on unit tests and browser tests is required to have confidence in some important aspects, and then that's probably going to be true for other implementers as well. In WPT test that check the basic claims of the spec and as the header parsing might suffice, going too far into the hard-to-test territory is probably not the best trade-off if other vendors can't lean mostly on WPT anyway.

Thanks for testing what you can, though, it's fun to see IETF specs making use of WPT as well!

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/63065789-64a7-44cd-a02f-cdbdb7538906%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/822104b2-e1ee-473a-9c9e-e5f686ce50b8%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8b2e22b4-bb20-4f5e-a5b2-1ac76e49c20a%40chromium.org.

[Rick Byers]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYczGc-HKjQG_%3DZTM9KvgEA_4_tU5P79y_RNxuKdeSnK_g%40mail.gmail.com.

[co...@bendell.ca]

After discussion with Rick Byers, Ilya and Tarun, I would like to propose that we delay the deprecation of cross-origin CH until some point after we have the FP mechanism shipped. There is notable compatibility impact and pain for developers that should be considered. 

Examining the last month of user traffic on Cloudinary's platform, we see a notable volume of requests with client hints enabled. For Cloudinary, Chrome 67's deprecation of cross-origin CH will impact our customers and the end-user experience. Specifically it will impact the developers that use our service in two ways:

1. Reduced quality for customers using client hints for DPR enhancement. These customers will get a blurrier, lower quality image with a default of DPR 1. These customers typically use Viewport-Width and DPR to minimize the size increase. For a DPR3 image, instead of a 300% file size increase, the file is 120% compared to DPR1.

2. Increased file size for customers using client hints for responsive-image byte reduction. These customers typically see file sizes 50% to 20% smaller compared to what is sent to Safari. For these customers, Chrome67 will result in net-increase of data transfers and a slower user experiences decreased conversion. 

On Cloudinary, we see ~800 customers who have client hints enabled on their webpages (meaning DPR/Viewport-Width/Width). Of those ~400 are using it with the Cloudinary's necessary transformation features (w_auto, dpr_auto) to improve the UX quality (DPR enhancement) or to improve performance (right-sized dimensions). The other customers are likely using client hints on other parts of their site, just not for our service.

These are customers who use Cloudinary as a third party - either diretly through res.coudinary.com or indirectly through their own custom domain such as images.example.com. Deprecating cross-origin CH without FP as an alternative will mean that these customers will have an unexpected degradation of their web experience and increased bills for transfer costs. 

Manual inspection of the client hints implementation reveals that the the vast majority (all?) use the html <meta> tags to enable CH. Implementing FP will require code changes for our customers and will require time to coordinate with their own development life cycle.

Therefore, I would like to propose that we delay the deprecation of cross origin CH until the FP mechanism is implemented and sufficient time is given to enable developers to adopt this new syntax. 

Thanks for your consideration,

/colin

Additional insights that might be relevant:

* most use the DPR client hints (though DPR + ViewPort-Width are usually together - 75% of the time)

* only 5% of client-hints presented use Width in any combination 

* 20% of client-hints are non-TLS (which is higher than our global traffic of 15% non-TLS)

* of this 95% of TLS CH is H2. The remaining 5% which are H1+TLS appears to be AV, and corporate proxies

[Yoav Weiss]

Thank you for that data!

The reason for those CH changes (removal of 3rd party and non-secure-context support) was privacy concerns. Specifically, there were concerns that as implemented, CH gave passive third parties as well as passive observers ways to fingerprint users that were not available to them otherwise.

I was convinced by Artur (CCed) that these risks justify rapid removal.

Artur - do you think we can (from a privacy perspective) revert those changes until the FP solution is ready?

Tarun - you talked about implementing the FP bits. Can you estimate a timeframe in which that can happen?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c2798dad-aaf8-416d-a9eb-79f17712141b%40chromium.org.

[Artur Janc]

On Fri, May 18, 2018 at 5:58 AM Yoav Weiss <yo...@yoav.ws> wrote:

Thank you for that data!

The reason for those CH changes (removal of 3rd party and non-secure-context support) was privacy concerns. Specifically, there were concerns that as implemented, CH gave passive third parties as well as passive observers ways to fingerprint users that were not available to them otherwise.

I was convinced by Artur (CCed) that these risks justify rapid removal.

Artur - do you think we can (from a privacy perspective) revert those changes until the FP solution is ready?

I would like for this decision to be made by someone from Chromium who can weigh the privacy risk (exposing fingerprintable information such as viewport dimensions to third parties) against compatibility concerns (the file size/quality concerns on 3p images mentioned by Colin below). Either option has its downsides so this boils down to making a product decision as to which of them to prioritize in this particular case.

Best,

-Artur

[Colin Bendell]

The value of DPR and Viewport-Width is greatest on mobile devices.
While DPR and ViewportWidth are privacy concerns for desktops, they
are less so on mobile devices. These client hints are generally static
on mobile devices. A pixel 2 DPR is consistent for all pixel2 and
derivable through the UA.

While not ideal, an acceptable compromise would be to delay `DPR` and
`Viewport-Width` client hints for only mobile and retain the planned
deprecation for desktop. There are customers that use client hints for
TV and dashboard installations and will get a inferior quality
experience. But the majority of cases will be benefiting mobile users.

`Width` is only ever present when the developer has implemented `<img
sizes>` markup. This would indicate developer intention since `Width`
will only be sent to a 3p when the developer has intentionally pointed
the resource to the 3p in the `srcset` or `src` attributes. I propose
that cross origin client hint `Width` not be deprecated for this
reason.

/colin

[Rick Byers]

Thanks for your feedback Colin, and I'm sorry we didn't more seriously consider this scenario originally.

These seem like reasonable compromises to me.  In general backing off a bit here until the FP API has been shipping for at least a milestone seems very consistent with our principle of avoiding taking functionality away from the platform completely.  We just need to weigh that off against the privacy risk and implementation timeframe (given that M67 stable release is immanent).

Mike/Jochen: can you provide your thoughts on the privacy tradeoff of this temporary compromise?

Tarun, what's your take from the implementation perspective?  Is it even possible that we could make a change like this in time for M67 at this point?

Rick

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL12FVEBW0qxwdYUF-Pm1HSnC5Hr%2B2P%3D2sJx44abEJTT-YgizQ%40mail.gmail.com.

[Jochen Eisinger]

Delaying the removal until the FP version has shipped for a milestone seems like a reasonable compromise to me.

[Tarun Bansal]

Thanks everybody. As proposed above by Colin, we are going to hold on the deprecation of DPR, Viewport-Width client hints on Android until we have the feature-policy based implementation in place. Note that the https-only restriction will still apply. I am hoping to merge back the fix to M-67.

>>> email to blink-dev+unsubscribe@chromium.org.

[tba...@google.com]

Thanks everybody. As proposed above by Colin, we are going to hold on the deprecation of DPR, Viewport-Width client hints on Android until we have the feature-policy based implementation in place. Note that the https-only restriction will still apply. I am hoping to merge back the fix to M-67.

On Monday, May 21, 2018 at 1:56:37 AM UTC-7, Jochen Eisinger wrote:

[Rick Byers]

Thank you for jumping on this Tarun, I hope we can still make the M-67 cutoff!

Thanks again for raising the compat concern Colin - that's exactly why we have these "intent" threads in public :-)

[Colin Bendell]

Thanks!

Just to close the loop - what was the consensus about the `Width` client hint?

As I mentioned, currently this is only present when the developer
signals with `sizes` in the html tag. In this case, the developer has
intentionally signalled if a cross-origin host should be used or not
used. For example:

<img src="//images.example.com/logo.jpg" sizes="30vw">

regards,
/colin

[tba...@google.com]

The Width client should also work for 3p origins.

[tba...@google.com]

Of course, I meant: Width client *hint*