Contact emails
yoav...@chromium.org, aaro...@chromium.org
Summary
We want to freeze and unify (but not remove) the User Agent string in HTTP requests as well as in `navigator.userAgent`
Motivation
The User-Agent string is an abundant source of passive fingerprinting information about our users. It contains many details about the user’s browser and device as well as many lies ("Mozilla/5.0", anyone?) that were or are needed for compatibility purposes, as servers grew reliant on bad User Agent sniffing.
On top of those privacy issues, User-Agent sniffing is an abundant source of compatibility issues, in particular for minority browsers, resulting in browsers lying about themselves (generally or to specific sites), and sites (including Google properties) being broken in some browsers for no good reason.
The above abuse makes it desirable to freeze the UA string and replace it with a better mechanism. There have been past attempts at UA string freezing from the Safari team, but without an alternative way to perform UA based content-negotiation, they had to be partially reverted.
The User Agent Client Hints (UA-CH) feature provides an alternate source for the information the User-Agent string provides, both in its request header form as well as its JS API one.
Its main advantages are:
It provides the required information only when the server requests it, over secure connections, making any fingerprinting that relies on it be active fingerprinting, which enables such use to be audited, as well as acted-upon by the browser (e.g. in a future implementation of the Privacy Budget).
It provides the information in small increments, so servers are only exposed to the information they need and request, rather than being exposed to the full gamut of the UA string even if they are just trying to figure out one detail about the browser. (e.g. brand and major version)
Since it provides the information via dedicated fields, it enables better ergonomics and makes it less likely for servers to get it wrong and cause compatibility issues.
And finally, starting fresh will enable us to drop a lot of the legacy baggage that the UA string carries (“Mozilla/5.0”, “like Gecko”, “like KHTML”, etc) going forward.
Once UA-CH ships as an alternative means for browser-specific content adaptation, we would like to freeze the User-Agent string.
We propose to deprecate at M81 (starting to emit console warnings in pages that read that string in JS), freeze its version information at M83, and unify strings of different devices at M85. See detailed freezing plan below.
This timeline provides 3 months for developers to move to the new mechanism for their future browser and OS version needs, and 6 months for more sophisticated OS or device specific targeting.
Freezing plan
Different parts of the UA string have different compatibility implications.
Some parts of it, such as the browser version and the OS version, can be frozen without any backwards compatibility implications. Values that worked in the past will continue to work in the future.
Other parts, such as the model (for mobile devices) and the OS platform, can have implications on sites that tailor their UI to the underlying OS or that target a very specific model in their layout. Such sites will need to migrate to use UA-CH.
As such we are planning to freeze the parts that are amenable to freezing fairly early, and gradually unify the rest.
(*) For the mobile value, we may split it further based on common device dimensions, as a one-time exercise, to reduce the compatibility risk of unification.
Interoperability and Compatibility Risk
The compatibility risk varies at different stages.
For the freezing planned for M83, the compatibility risk is low. Existing UA sniffing code will continue to work as expected. It is only future UA sniffing code that will need to change and use the UA client hints instead.
For the unification planned for M85, the compatibility risk is medium. Some sites can modify their responses based on the OS and device model, and those sites will have to change their UA sniffing code to use UA-CH. We expect such sites to be well maintained (otherwise, how can they keep up with OS UI and device model changes?). Therefore, having 4 releases to modify their code seems sufficient.
In the long term, we expect this change to improve compatibility, as UA sniffing based on UA-CH is bound to be more reliable than the current status quo.
As for interoperability, we have other vendors on board with UA freezing, but not necessarily with the UA Client Hints mechanism, that is supposed to replace it. That can create a tricky situation, where developers would need to rely on the User-Agent string for some browsers and on UA-CH for others.
Edge: Public support
Firefox: Public support for freezing the UA string - “freezing the User Agent string without any client hints—seems worth-prototyping”
Safari: Shipped to some extent. Safari has attempted to completely freeze the UA string in the past, without providing an alternative mechanism. That got a lot of pushback, which resulted in somewhat reverting that decision. Nowadays, their UA string seems frozen, other than updates to the OS version and the browser major version.
Alternative implementation suggestion for web developers
For many (most?) uses of UA sniffing today, a better tool for the job would be to use feature detection. Where feature detection fails developers, UA Client Hints are the right path forward.
Potential deployment hurdles compared to status quo:
Third party services that rely on the UA string would need to convince the sites that include them to delegate that information to them using Feature Policy.
The opt-in based mechanism of Client Hints currently suffers from the fact that on the very-first view, the browser have not yet received the opt-in. That means that requiring specific UA information (e.g. device model, platform) on the very-first navigation request may incur a delay. We are exploring options to enable the opt-in to happen further down the stack to avoid that delay.
Usage information from UseCounter
If we were going to remove the User-Agent string outright, the answer to that would’ve been “a lot!!!!111”.
Given that we are planning to freeze it, we expect the “removal” to be mostly backwards compatible, barring cases of specific OS or device adaptation.
Client-side UA sniffing use counters show it’s being used by ~90% of sites. But again, since we’re talking about freezing, it should not break most uses.
Entry on the feature dashboard
Hey Yoav, Aaron:Glad to see the CH-UA intent to out to ensure that similar information is provided no matter where this intent lands. Can you confirm that, if given approval for this, you will not take action to change UA behavior until CH-UA is shipped?
Presumably, in the context of privacy budgets, a model has been developed that discusses the bits of entropy reduction by default that are gained and the performance tradeoffs of requiring Accept-CH redirectors on nearly all top-level entrypoints for major services that wish to avoid overpolyfilling. Can you publish that analysis?
What work has gone into discussing this change with latency-sensitive properties and services? I'm aware that other UAs have implemented similar changes, but they have also been UAs that tend to benefit from high-end devices and networks by dint of their users being wealthy. If this change had been made 2 years ago, what would the polyfill bloat impact be on our user-base for those serving optimally (e.g., using polyfill.io)?
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/fbfd9b0b-97a4-4775-b4e2-a4da9f87660f%40chromium.org.
a) If the server doesn't get the info on first hit isn't there a performance penalty in that it has to respond with what it wants, then wait for the response just to give the optimal experience?
b) I'm a little unclear on mobileness, what values should be given for a phone, tablet, desktop?
c) Depending on b), it would be nice to at least have resolution buckets so we can return the most appropriate experience. Right now the experience for universal applications having to leverage UA sniffing (server) and resolution (client) is really bad.
On Tuesday, January 14, 2020 at 5:22:00 AM UTC-6, Yoav Weiss wrote:
Contact emails
yoav...@chromium.org, aaro...@chromium.org
Summary
We want to freeze and unify (but not remove) the User Agent string in HTTP requests as well as in `navigator.userAgent`
Motivation
The User-Agent string is an abundant source of passive fingerprinting information about our users. It contains many details about the user’s browser and device as well as many lies ("Mozilla/5.0", anyone?) that were or are needed for compatibility purposes, as servers grew reliant on bad User Agent sniffing.
On top of those privacy issues, User-Agent sniffing is an abundant source of compatibility issues, in particular for minority browsers, resulting in browsers lying about themselves (generally or to specific sites), and sites (including Google properties) being broken in some browsers for no good reason.
The above abuse makes it desirable to freeze the UA string and replace it with a better mechanism. There have been past attempts at UA string freezing from the Safari team, but without an alternative way to perform UA based content-negotiation, they had to be partially reverted.
The User Agent Clicsscssent Hints (UA-CH) feature provides an alternate source for the information the User-Agent string provides, both in its request header form as well as its JS API one.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5e5bd3db-0662-4de7-8bbb-004091b780f9%40chromium.org.
Hey Conrad,I think it's a little clearer in the explainer: https://github.com/WICG/ua-client-hints#a-proposal proposes exposing a NavigatorUAData interface via navigator.getUserAgent()
On Tue, 14 Jan 2020 at 17:08, conrad via blink-dev <blink-dev@chromium.org> wrote:
Thanks for sharing this. Maybe I missed this in reading through the spec, but is the same information as sent via the Sec-CH- headers going to be available to JavaScript running on the page?
I ask because at Superhuman we parse the userAgent string client side for several things:
1. If the operating system is Linux or Windows we increase the contrast of the CSS to better fit with user expectations.
2. Depending on the device (e.g. iPhone vs Android) we display a different error message to users, and to direct them to the appropriate App Store.
3. We use the minor version number of the browser to block known ‘bad’ versions of Chrome Canary and explain the situation (in the rare cases it contains bugs that break Superhuman fundamentally)
Currently we load the HTML and JavaScript from Google Cloud Storage, and so this logic could not live on the server for us.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5e5bd3db-0662-4de7-8bbb-004091b780f9%40chromium.org.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8a0dbd52-2793-4398-ac15-bb9afe64769b%40chromium.org.
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BjkKumWa7az2jgGLNbyXAkLyeAffLwdh0gOeN80boRZMc4yCA%40mail.gmail.com.
Can you point where the API meets the //content code for this path? I'm fairly sure that this would overwrite the frozen string, but I can make sure for you. I'm just not as familiar with the webview part of the codebase.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJwB4XarrBWCMpAPEkD3PCL29bwnzCQ2Ah%3DJK7JHwssRL5FQJQ%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CA%2BjkKukk_NJbJZz7W-yrm3wUCkXfxEj6Mioe9Yxr94U_3%3DmFww%40mail.gmail.com.
Summary
We want to freeze and unify (but not remove) the User Agent string in HTTP requests as well as in `navigator.userAgent`
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEjpz2AZ4x5nVm8X5%2BHrjdzH1-7f5y7qf6CmGMLTFVU4CQ%40mail.gmail.com.
I'm glad to see an effort towards freezing the UA string and finding alternatives. UA sniffing is at the root of countless web compat issues and these disproportionately affect browsers the developer didn't test in or think about.At the same time, this header has been evolving and mutating since Mosaic, and freezing it is bound to have effects all across the ecosystem and cause a lot of frustration. I'd like to see some of the risks explored more. Some concrete concerns:This depends on Intent to Ship: Client Hints infrastructure and UA Client Hints, but even if that ships first I don't think we can be confident that this alternative will be sufficient. Would it not be more prudent to ship that first, and work with a number of real sites to see what kind of issues come up?
That's not going to shake out some of the long tail issues, but would reveal any common problems. Seeing other vendors adopt UA Client Hints would also help a lot.
The timeline for this seems a fair bit too short given that it's a non-trivial change and can span multiple layers of software, and so need trickle through multiple planning and release cycles. What are the tradeoff of letting this bake 6 months vs. 1 year, say? (Chrome 99 around mid-2022 seems like an upper bound because Chrome 100 will very likely break some sniffing code, just like how Opera's UA string got stuck at 9.)
Showing a deprecation message on 90% of page views is a lot. Sites will likely keep accessing navigator.userAgent if they've determined nothing will break, or they'd have to use another form of sniffing to avoid it. (I have found UA sniffing to avoid triggering deprecate messages in past httparchive digging.)
How essential is it to have this message? It looks like Safari doesn't have a console warning.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJwB4XaR92xCbfp3emxkfRHG7y2Ns%3DCLYJcvL1eXv3oCw7-_KA%40mail.gmail.com.
Showing a deprecation message on 90% of page views is a lot. Sites will likely keep accessing navigator.userAgent if they've determined nothing will break, or they'd have to use another form of sniffing to avoid it. (I have found UA sniffing to avoid triggering deprecate messages in past httparchive digging.)If they'd use `getUserAgent()` for that, that's a win, no? :D
}
if(/Mobi/.test(navigator.userAgent) && !/^iPad/.test(navigator.platform) || window.operamini) {
/* linkify phone numbers */
No need to change this code, no reason for a spammy warning.
Contact emails
yoav...@chromium.org, aaro...@chromium.org
Summary
We want to freeze and unify (but not remove) the User Agent string in HTTP requests as well as in `navigator.userAgent`
FYI, there is also the "viewport-width" client hint, which will tell you the CSS px width of the viewport. If you want to go even crazier, you can request the "model" client hint, get the device model, and look up the width, height, and any other information based on that. That's not as recommended though unless you need very fine-grained information, as it's always better to get the least amount of entropy needed.
I am trying to understand that: The data about the browser is still available, but we have to send an additional request (for Sec-CH) to the user-agent? Or can the (Accept-CH) header be sent right away during the initial server response?
How and why is this preventing finger printing?
Contact emails
yoav...@chromium.org, aaro...@chromium.org
On Wed, Jan 15, 2020 at 11:45 AM Philip Jägenstedt <foo...@chromium.org> wrote:I'm glad to see an effort towards freezing the UA string and finding alternatives. UA sniffing is at the root of countless web compat issues and these disproportionately affect browsers the developer didn't test in or think about.At the same time, this header has been evolving and mutating since Mosaic, and freezing it is bound to have effects all across the ecosystem and cause a lot of frustration. I'd like to see some of the risks explored more. Some concrete concerns:This depends on Intent to Ship: Client Hints infrastructure and UA Client Hints, but even if that ships first I don't think we can be confident that this alternative will be sufficient. Would it not be more prudent to ship that first, and work with a number of real sites to see what kind of issues come up?It certainly would be more prudent. The urgency comes from the fact that right now, we are broadcasting a lot of information about our users, in clear text, to all servers. We would like to stop doing that.Our assumption was that 4 releases (between shipping UA CH and unifying the UA string) would be enough time for us to make sure the replacement is sufficient before pulling the plug on the legacy string.
That's not going to shake out some of the long tail issues, but would reveal any common problems. Seeing other vendors adopt UA Client Hints would also help a lot.I agree that other vendors adopting would help. Unfortunately, Mozilla don't seem to be interested in implementing the CH part. I've asked Apple to re-review, so one can remain hopeful.
The timeline for this seems a fair bit too short given that it's a non-trivial change and can span multiple layers of software, and so need trickle through multiple planning and release cycles. What are the tradeoff of letting this bake 6 months vs. 1 year, say? (Chrome 99 around mid-2022 seems like an upper bound because Chrome 100 will very likely break some sniffing code, just like how Opera's UA string got stuck at 9.)Letting this bake for a year would mean continuing to send user information, in the clear, to all servers, for 6 more months.
Showing a deprecation message on 90% of page views is a lot. Sites will likely keep accessing navigator.userAgent if they've determined nothing will break, or they'd have to use another form of sniffing to avoid it. (I have found UA sniffing to avoid triggering deprecate messages in past httparchive digging.)If they'd use `getUserAgent()` for that, that's a win, no? :DHow essential is it to have this message? It looks like Safari doesn't have a console warning.We want to let developers know that the API they are using is about to become useless and move them to the new API. Are there other tools that fit that purpose other than console warnings?
Some additional issues with WebView here:1) I don't think that client hints are fully implemented in WebView right now (I don't think it is wired up to store the requests for hints from servers anywhere), so we might need to deal with that before shipping?
2) WebView apps which override the user agent string might be doing so in order to *conceal* information that's present by default, which might mean that they would also want to control what's exposed via client hints - we might need to consider exposing APIs for that.
3) In general many WebView apps aren't prepared for potentially-breaking changes to web-facing behaviour like this one, and the developers of those apps might not be participating in the larger web ecosystem in a way that would give them any visibility into these kinds of upcoming changes. This is not to say that we shouldn't make this change for WebView, but it may be riskier.
4) WebView's useragent string format is prescribed by Android's Compatibility Definition Document (CDD) and Compatibility Test Suite (CTS). We can't ship changes to WebView that would violate the rules set out in CDD/CTS unless we first update them, which in almost all cases is only permitted for *future* versions of Android not yet released, and can't be changed retroactively unless it's to fix actual errors in the definitions/tests. Freezing most aspects of the UA at their current values without changing the format would not violate the current implementation of CDD/CTS, with one exception: the Android version is expected to be the same value that's exposed to Android apps in general (i.e. something like "Android 5.1" - not a specific build ID or anything, just the "marketing" version number of the OS).
5) There's been past discussion about removing the Android device model name from the UA for either/both of WebView and Chrome on Android, but last I checked this hasn't been resolved. Will the freezing proposal have any impact on this?
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEV-rjf-bMK9sUqYgZ8PQ1V2GH8EeFChMEJ7w_JnUnoJqgfFng%40mail.gmail.com.
From a light reading, this seem detrimental to the "Chromium forks". There are many rebrandings of "Chromium" (Edge and Opera being some of the more prominent examples) that we simply ignore and use the Chromium versioning. It seems, as I've read it, to force browsers forks to masquerade as Chrome if they can't be recognized as part of the Chromium family. Did I miss something that would allow this grouping of "families"? Generally, server side rendering requires a reasonable user-agents identification.
I'm also concerned that "freezing" the UA prevents folks from getting the feedback that you need to validate that you have solved the right problems (as nothing is broken "today").
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/7b2fd069-ab61-4257-bc8c-0b5d73a6bb1e%40chromium.org.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/62082d91-ce30-4455-806a-0b68eee531ab%40chromium.org.
A second roundtrip to get more information is definitely a performance issue especially with high latency times. From our perspective the platform e.g. win or macos would be helpful as default information as certain things behaves differently on the platform. Just to make a very simple example tooltips/menu entries for hotkeys (like ctrl+s) are differently on a mac or win (ctrl vs. cmd key) and already part from server in the first roundtrip.
Depending whether the frozen UserAgent string and JS interface remains it is highly incompatible and would result especially for enterprise customer to upgrade the complete landscape. 3 months are maybe enough to implement but not to update a large onprem landscape with hundreds of mission critical servers. We currently have this issue with the dismissal of syncXHR.Enough time between releasing the new interface and deprecation of the old behavior would be necessary to give enterprise customers enough time to upgrade their systems or like in the syncXHR a possibility to opt-in the user-agent via flag and group policy
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/98bb9c26-91c5-47b6-8165-dea0b2b4c7aa%40chromium.org.
> The opt-in based mechanism of Client Hints currently suffers from the fact that on the very-first view, the browser have not yet received the opt-in. That means that requiring specific UA information (e.g. device model, platform) on the very-first navigation request may incur a delay. We are exploring options to enable the opt-in to happen further down the stack to avoid that delay.
This drawback seems significant and ought to block the release of this feature until it is solved.
Moving fast and breaking things is not the way of the web.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d7deba3b-2075-4759-81b5-e0c41b944539%40chromium.org.
Apologies if this is the wrong place to post feedback, please point me to somewhere more appropriate if it exists.
I have a few points of feedback:
1. It seems here as if there are still unanswered questions about how to achieve parity with the features provided by the UA like how to get browser version on the first pageload, which seems like a very important use case.
2. The use case of preventing fine grained fingerprinting via CH is a good goal, but can't that be addressed by freezing the UA except for vendor, OS, and version, similar to Safari?
3. The public consensus section seems to indicate that FF and Safari have possibly different approaches in play or in mind, that would be compatible with partial UA freeze but not with CH.
4. The fact that freezing the UA is BC is good, but there's tons of existing material and libraries for parsing UA. Switching browser detection mechanism is a significant piece of churn for the Platform, and thus should come with a big benefit that can't be achieved otherwise.
Taking these points all together, it seems the main benefit of switching to CH over partial UA freeze is "drops legacy bits of text from the string" and "provides better ergonomics for sniffing". Neither of these IMO seem like a high enough bar to introduce a second system here and invalidate all of the existing libraries and documentation on the web. Secondly, it would seem you should solve the "first pageload" problem first.
Thanks for considering these points, and I'm happy to hear counterpoints and be convinced this feature is worth the churn.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/927e2f0a-bbc1-4a2c-9cbe-ea05f59afb78%40chromium.org.
FYI polyfill.io currently serves 38 billion requests per month, which depend on accurately identifying browsers (including minot version) via the UA string.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d5087dcc-aa14-4907-85c2-085a5ca3b5b5%40chromium.org.
Yoav,Would the freezing of the UA and introduction of Client Hints assist in anyway of combating the fraudulent use of stolen fingerprints? I am assuming that the server requests to the browser for more detailed information which the "illicit" browser would provide would therefore not go towards helping prevent such fraudulent use?If this is the case, are there any plans to help prevent this type of activity?
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/4733a289-7417-4c96-8758-ef1578ca2142%40chromium.org.
On Tue, Jan 14, 2020 at 7:26 PM <fer...@gmail.com> wrote:> The opt-in based mechanism of Client Hints currently suffers from the fact that on the very-first view, the browser have not yet received the opt-in. That means that requiring specific UA information (e.g. device model, platform) on the very-first navigation request may incur a delay. We are exploring options to enable the opt-in to happen further down the stack to avoid that delay.
This drawback seems significant and ought to block the release of this feature until it is solved.The thought is that the parts that are sent by default (UA brand and meaningful version, as well as most-probably the mobileness bit) would be sufficient for most cases of UA based content negotiation to not require any delays.Do you have common use-cases where this information will not be sufficient?
On Wed, Jan 15, 2020 at 9:42 PM Yoav Weiss <yo...@yoav.ws> wrote:On Wed, Jan 15, 2020 at 11:45 AM Philip Jägenstedt <foo...@chromium.org> wrote:I'm glad to see an effort towards freezing the UA string and finding alternatives. UA sniffing is at the root of countless web compat issues and these disproportionately affect browsers the developer didn't test in or think about.At the same time, this header has been evolving and mutating since Mosaic, and freezing it is bound to have effects all across the ecosystem and cause a lot of frustration. I'd like to see some of the risks explored more. Some concrete concerns:This depends on Intent to Ship: Client Hints infrastructure and UA Client Hints, but even if that ships first I don't think we can be confident that this alternative will be sufficient. Would it not be more prudent to ship that first, and work with a number of real sites to see what kind of issues come up?It certainly would be more prudent. The urgency comes from the fact that right now, we are broadcasting a lot of information about our users, in clear text, to all servers. We would like to stop doing that.Our assumption was that 4 releases (between shipping UA CH and unifying the UA string) would be enough time for us to make sure the replacement is sufficient before pulling the plug on the legacy string.It could be enough time to show that migrating is possible in most scenarios before the UA string freezes, but it would also be unsurprising if trying to migrate a real-world complex case will reveal a number of issues that need to be resolved. Before we have confidence in CH as a replacement, I don't think we should be deciding on a timeline or communicating dates for UA freezing to web developers.That's not going to shake out some of the long tail issues, but would reveal any common problems. Seeing other vendors adopt UA Client Hints would also help a lot.I agree that other vendors adopting would help. Unfortunately, Mozilla don't seem to be interested in implementing the CH part. I've asked Apple to re-review, so one can remain hopeful.Paging +Michael Taylor to clarify the linked issue. Would Mozilla consider freezing the UA string and only provide the same information via JS APIs? That seems pretty risky to me...The timeline for this seems a fair bit too short given that it's a non-trivial change and can span multiple layers of software, and so need trickle through multiple planning and release cycles. What are the tradeoff of letting this bake 6 months vs. 1 year, say? (Chrome 99 around mid-2022 seems like an upper bound because Chrome 100 will very likely break some sniffing code, just like how Opera's UA string got stuck at 9.)Letting this bake for a year would mean continuing to send user information, in the clear, to all servers, for 6 more months.Yes, it's 6 more months, but what is the tradeoff? How bad for users is the UA string vs. how many sad developers and broken sites? I get there's no exact answer, but the impact on the ecosystem looks like the more concrete risk.
It should also give us pause if other vendors aren't planning to freeze their UA strings, it calls the urgency into question.
Showing a deprecation message on 90% of page views is a lot. Sites will likely keep accessing navigator.userAgent if they've determined nothing will break, or they'd have to use another form of sniffing to avoid it. (I have found UA sniffing to avoid triggering deprecate messages in past httparchive digging.)If they'd use `getUserAgent()` for that, that's a win, no? :DHow essential is it to have this message? It looks like Safari doesn't have a console warning.We want to let developers know that the API they are using is about to become useless and move them to the new API. Are there other tools that fit that purpose other than console warnings?To spell out the concern, it is about warning fatigue. However, I have to agree that if we commit to a timeline, then we'll have to broadcast this very widely. A lot of the usage won't be practical to remove, however, so it would have to be for a limited time.
On Fri, Jan 17, 2020 at 11:56 AM Yoav Weiss <yo...@yoav.ws> wrote:On Tue, Jan 14, 2020 at 7:26 PM <fer...@gmail.com> wrote:> The opt-in based mechanism of Client Hints currently suffers from the fact that on the very-first view, the browser have not yet received the opt-in. That means that requiring specific UA information (e.g. device model, platform) on the very-first navigation request may incur a delay. We are exploring options to enable the opt-in to happen further down the stack to avoid that delay.
This drawback seems significant and ought to block the release of this feature until it is solved.The thought is that the parts that are sent by default (UA brand and meaningful version, as well as most-probably the mobileness bit) would be sufficient for most cases of UA based content negotiation to not require any delays.Do you have common use-cases where this information will not be sufficient?Learning about such use cases would be valuable, but I don't think we can assume the change to be safe in the absence of such examples. And at the scale we're talking about, usage doesn't need to be common to pose a serious compat problem.
Given that use counters will fail us here, some different form of compat analysis is needed. https://gist.github.com/foolip/38e3ea6f060bad4638c2aea6fac63e80 is an example of what this might look like, there varying the Accept-Language header and looking at how the result varies.
Rough sketch of what it might look like for the UA header, in line with Blink principles of web compatibility:
- Crawl the same sites with multiple forms of the UA header
- Identify the sites where some of the responses vary (could also compare code paths taken and rendering)
- Confirm the differences by re-crawling to reduce noise
- Look at a sample of the candidate sites, assessing the severity to users and the difficulty to fix for users
This is time consuming, but it's what it will take to understand the risk.
On Thu, Jan 16, 2020 at 4:22 PM Philip Jägenstedt <foo...@chromium.org> wrote:On Wed, Jan 15, 2020 at 9:42 PM Yoav Weiss <yo...@yoav.ws> wrote:On Wed, Jan 15, 2020 at 11:45 AM Philip Jägenstedt <foo...@chromium.org> wrote:I'm glad to see an effort towards freezing the UA string and finding alternatives. UA sniffing is at the root of countless web compat issues and these disproportionately affect browsers the developer didn't test in or think about.At the same time, this header has been evolving and mutating since Mosaic, and freezing it is bound to have effects all across the ecosystem and cause a lot of frustration. I'd like to see some of the risks explored more. Some concrete concerns:This depends on Intent to Ship: Client Hints infrastructure and UA Client Hints, but even if that ships first I don't think we can be confident that this alternative will be sufficient. Would it not be more prudent to ship that first, and work with a number of real sites to see what kind of issues come up?It certainly would be more prudent. The urgency comes from the fact that right now, we are broadcasting a lot of information about our users, in clear text, to all servers. We would like to stop doing that.Our assumption was that 4 releases (between shipping UA CH and unifying the UA string) would be enough time for us to make sure the replacement is sufficient before pulling the plug on the legacy string.It could be enough time to show that migrating is possible in most scenarios before the UA string freezes, but it would also be unsurprising if trying to migrate a real-world complex case will reveal a number of issues that need to be resolved. Before we have confidence in CH as a replacement, I don't think we should be deciding on a timeline or communicating dates for UA freezing to web developers.That's not going to shake out some of the long tail issues, but would reveal any common problems. Seeing other vendors adopt UA Client Hints would also help a lot.I agree that other vendors adopting would help. Unfortunately, Mozilla don't seem to be interested in implementing the CH part. I've asked Apple to re-review, so one can remain hopeful.Paging +Michael Taylor to clarify the linked issue. Would Mozilla consider freezing the UA string and only provide the same information via JS APIs? That seems pretty risky to me...The timeline for this seems a fair bit too short given that it's a non-trivial change and can span multiple layers of software, and so need trickle through multiple planning and release cycles. What are the tradeoff of letting this bake 6 months vs. 1 year, say? (Chrome 99 around mid-2022 seems like an upper bound because Chrome 100 will very likely break some sniffing code, just like how Opera's UA string got stuck at 9.)Letting this bake for a year would mean continuing to send user information, in the clear, to all servers, for 6 more months.Yes, it's 6 more months, but what is the tradeoff? How bad for users is the UA string vs. how many sad developers and broken sites? I get there's no exact answer, but the impact on the ecosystem looks like the more concrete risk.IMO, the main risk for breakage is from the OS and device unification.The former seems less risky for server side detection, but maybe carries some risk on the client side (e.g. the wrong keyboard shortcuts picked)For the latter, my assumption was that folks that are tailoring their layouts to specific devices have to remain on their toes and update that constantly, as new devices arrive to market all the time.
On Tue, Jan 14, 2020 at 10:51 PM <thorste...@sap.com> wrote:A second roundtrip to get more information is definitely a performance issue especially with high latency times. From our perspective the platform e.g. win or macos would be helpful as default information as certain things behaves differently on the platform. Just to make a very simple example tooltips/menu entries for hotkeys (like ctrl+s) are differently on a mac or win (ctrl vs. cmd key) and already part from server in the first roundtrip.Sending different code to each platform just to handle different keys sounds like an overkill. Do you have a real-life example where OS related changes can't be reasonably done on the client-side?
The keycode was just an example. The issues are server side rendering frameworks which are used to do the stuff on the server side without many JS coding. First of all they must use more Javascript to get the things done on client-side. Of course this is manageable.
But they also provide information derived from the user agent which are normally the browser type, version number and OS (at least in our cases) to the application. It depends what the application is doing with it but it is complete out of control of the framework provider.
Considering the maintenance window of > 10 years for business systems it is very hard to find all this cases and fix them especially as the application are also developed by customers. I’m not arguing completely against such changes but it should somehow compliant or enough time to change the onprem systems of enterprise customers.
This is the challenge as enterprise customer with mission critical systems thinks in years to upgrade the complete landscape whereas the browsers vendors thinks in months. Something in between would be great.
Getting the correct OS in the first roundtrip would be helpful to reduce the risk of failures.
Depending whether the frozen UserAgent string and JS interface remains it is highly incompatible and would result especially for enterprise customer to upgrade the complete landscape. 3 months are maybe enough to implement but not to update a large onprem landscape with hundreds of mission critical servers. We currently have this issue with the dismissal of syncXHR.Enough time between releasing the new interface and deprecation of the old behavior would be necessary to give enterprise customers enough time to upgrade their systems or like in the syncXHR a possibility to opt-in the user-agent via flag and group policyAdding an enterprise policy flag sounds reasonable.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.
On Wed, Jan 15, 2020 at 4:51 PM Torne (Richard Coles) <to...@chromium.org> wrote:Some additional issues with WebView here:1) I don't think that client hints are fully implemented in WebView right now (I don't think it is wired up to store the requests for hints from servers anywhere), so we might need to deal with that before shipping?That's a good point, and indeed something we need to make sure happens before freezing, at least in WebViews.2) WebView apps which override the user agent string might be doing so in order to *conceal* information that's present by default, which might mean that they would also want to control what's exposed via client hints - we might need to consider exposing APIs for that.Interesting. What would exposing such an API entail?
3) In general many WebView apps aren't prepared for potentially-breaking changes to web-facing behaviour like this one, and the developers of those apps might not be participating in the larger web ecosystem in a way that would give them any visibility into these kinds of upcoming changes. This is not to say that we shouldn't make this change for WebView, but it may be riskier.What would be a good way to increase awareness to this?
4) WebView's useragent string format is prescribed by Android's Compatibility Definition Document (CDD) and Compatibility Test Suite (CTS). We can't ship changes to WebView that would violate the rules set out in CDD/CTS unless we first update them, which in almost all cases is only permitted for *future* versions of Android not yet released, and can't be changed retroactively unless it's to fix actual errors in the definitions/tests. Freezing most aspects of the UA at their current values without changing the format would not violate the current implementation of CDD/CTS, with one exception: the Android version is expected to be the same value that's exposed to Android apps in general (i.e. something like "Android 5.1" - not a specific build ID or anything, just the "marketing" version number of the OS).OK, so this may be somewhere where the WebView's UA string and Chrome's would have to diverge...
5) There's been past discussion about removing the Android device model name from the UA for either/both of WebView and Chrome on Android, but last I checked this hasn't been resolved. Will the freezing proposal have any impact on this?This proposal intends to unify Android devices under a single common UA string, so under a single "model". Maybe we can get away (compatibility-wise) with removing the model entirely.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEjYiVxS044Ns7D1BfTqLs4XPBo0R8prtS5bmdTkw5ZZmw%40mail.gmail.com.
On Wed, Jan 15, 2020 at 4:55 PM jj <fri...@jeka.info> wrote:Showing a deprecation message on 90% of page views is a lot. Sites will likely keep accessing navigator.userAgent if they've determined nothing will break, or they'd have to use another form of sniffing to avoid it. (I have found UA sniffing to avoid triggering deprecate messages in past httparchive digging.)If they'd use `getUserAgent()` for that, that's a win, no? :DA lot of UA sniffing code will be happy with the frozen String for years.Think on things like that, not written by the "sophisticated developers" mentioned in the spec:}
if(/Mobi/.test(navigator.userAgent) && !/^iPad/.test(navigator.platform) || window.operamini) {
/* linkify phone numbers */
No need to change this code, no reason for a spammy warning.While I'd have preferred such uses to move to getUserAgent() (cheaper and more accurate), I see your point.
If(document.documentMode || /Edge\//.test(navigator.userAgent) {
/* double box-shadow blur values for IE an EdgeHTML to match other browsers */
}
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blin...@chromium.org.
Hi Yoav,
As stakeholders in the web and following discussions with colleagues, clients and industry professionals I’ve prepared a summary of questions and observations concerning your proposed changes to the established usage of the HTTP User-Agent header field. This usage has been established since the beginning of the web and the purpose of the field is noted in the documentation of HTTP from 1990s.
While anybody must be concerned about possibly compromising the privacy of the users of the web we feel that the nature and extent of those compromises needs to be more clearly established. The effect of the proposed changes is to damage quite seriously the user experience enjoyed by users on a significant number of websites that require the User-Agent to be in its present form.
Further we are most concerned about the possible damaging effects on the accessibility of those sites and the availability of those sites in developing nations as a result of the proposed changes.
Here
follows a number of more detailed comments and we’d be most grateful for your
feedback on these comments.
1. Where is the evidence to support the primary assertion that the User-Agent is an “abundant source of passive fingerprinting information”?
Suggestion: A very large web site or network provider using session tracking - such as first party cookies -would be able to provide the analysis needed. Group a large cohort of web sessions by IP and User-Agent and count the number of sessions. Is the relation 1 to 1 or 1 to many? If 1 to many, how many? Does it vary by different formats of User-Agent? Is the analysis sufficient to validate the assertion?
2. In response to your second assertion User-Agent field values are poorly structured. It is our experience that a high degree of accuracy can be achieved from the current state of these strings.
We are not the only practitioners. Experts in the field of device identification and therefore User-Agents strings include:
51Degrees (this authors company “we”)
Device Atlas (owned by Afilias)
ScientiaMobile via the WURFL solution.
Between us support is provided to millions of web sites including blue chip businesses as varied as Google, Facebook, Oracle, eBay, Amazon, Disney, Tencent, IBM, Naspers, Verizon and countless others many of which operate outside North America and Europe.
We provide free and open source solutions. Access to first request device information is essential to the services they support. We have not been consulted about this change.
3. Has thought been given to the impact on non-technical web professionals such as marketing professionals who use analytics tools? As a result of the proposed changes their website may break. As a minimum their web site analytics will be incorrect. They will report incorrect information and form invalid conclusions. This will be frustrating to them and waste countless days head scratching.
4. Have network operators been consulted about the deployment of IPv6 and networks in general? The fingerprinting argument depends at least 50% on the IP address. Depending on deployment a client IP address could persist with a device forever; effectively forming a unique identifier. Frequently changing the client IP address will lead to inefficiencies for network operators.
5. Has the change been validated with web site owners large and small who operate in many different markets and geographies, many of whom make extensive use of the User-Agent to deliver their services? How do developing countries differ to north America and Europe?
Consider news services from Naspers and Tencent targeting Asia and Africa. These services use the information obtained from the User-Agent in the very first request to optimise content ensuring feature phones, powerful smartphones and budget smartphones all receive optimised content cost effectively. Many consumers outside north America and Western Europe do not enjoy “all you can eat” data plans or high-end electronics. They need this feature to receive cost effective and optimised news.
Facebook Zero inspired many businesses to operate this “stripped down” approach. They tend not to be covered in the English language technology press or outside their home markets.
6. In our experience many websites are bloated with MBs of JavaScript, constrained by a single threaded execution engine, and perform poorly. Google invented Accelerated Mobile Pages (AMP) to offer readers a faster way of consuming publishers’ content. It is often seen deployed in Android where knowledge of the device informs the information presented. Companies that are not Google who host AMP will lack the necessary information to provide their service.
Many publishers including Reach, Mail Online and News International are still plagued by JavaScript performance issues over 5 years after Google shown a light on the performance problem.
From an engineering perspective optimising content on the web server at first request to reduce the amount of data transmitted and the computation the user agent needs to perform is very sensible. The client and web server work more harmoniously together rather than the server being a dumb file server. Making the information needed available in the first request is essential to enable this performance improvement. The User-Agent isn’t ideally formatted, but it works.
It works so well http://chromium.org uses the technique with Google Sites. The User-Agent is everywhere.
7. Does the process of agreeing this include a sufficiently broad cross section of engineers and web professionals? For example the User-Agent is used extensively within the HTTP protocol by CDNs and within edge networks both with and without SSL. Such engineers are key stakeholders in such a change. Should the IEFT be consulted? Is this not part of HTTP 3?
Current contributions appear to be skewed to be from people with a web front end / JavaScript centric view.
8. Is Safari a good comparator for this performance degrading change? Users are skewed towards higher socio-economic groups, high end chipsets and specific geographic markets. Chromium is the ubiquitous foundation of the web browser globally. It’s used with $10 MediaTek A53 chips to the most powerful Qualcomm SoCs.
9. Is Firefox a good comparator? In many cases Firefox is not bundled with Android by default. It is installed by a user who wishes to benefit from its features. As such a typical user is more privacy aware. As Chromium is the default for most Android devices it should support the widest set of requirements out of the box. Perhaps Google will offer a choice of web browser when Android is first setup? Microsoft did a similar thing in the past with Windows.
10. Is this a feature that should be up to the browser vendor to form a view on rather than sit within the Chromium project? Brave already take one approach, Microsoft another. Privacy conscious users can easily switch between browsers.
11. Public support seems to be in the form of other web browser engineers via a series of tweets and git issues posts operating within a “bubble” or “echo chamber”. This tweet is cited as support (https://twitter.com/_scottlow/status/1206831008261132289). Is this sufficient support for such a wide-reaching change impacting almost every digital citizen and business?
12. What is the view of “minority” browser vendors who benefit extensively via marketing and analytics by appending their browser details to the User-Agent string? What is the view of Android device vendors who wish to be able to gain insight into device usage?
13. Even if a large company has the resources to make the necessary changes within the timescales proposed what are the changes required to maintain the current fine grain of optimisation enjoyed by their users? The majority of small and medium sized businesses and other non-profit web site operators do not. They are all impacted.
They are not of the size and scale of the relatively small number of AdTech / tracking businesses who need to adapt to cookie changes. These companies were given 2 years notice.
This change impacts every website operator and owner. It is impractical to assume the vast majority can or want to make changes to their sites.
14. The AdTech business, of which Google is a major player, have spent a long time producing the standards used to communicate between each other in real time. Has an assessment been performed on the impact to OpenRTB and AdCom where User-Agent is the primary method of validating and unifying device information?
https://github.com/InteractiveAdvertisingBureau/AdCOM/blob/master/AdCOM%20v1.0%20FINAL.md#object_device
In this case there are many use cases where a clear alternative does not exist. Consider the previously mentioned web site adapting on first request to the device with a granularity beyond just desktop or mobile. This change will remove this possibility and new designs will be required.
Or the web professionals wishing to understand why performance is better or worse by different users or web sessions. The IETF HTTP 1.1 specification states “This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations.”
https://tools.ietf.org/html/rfc2616
21 years later this is exactly how it’s being used. So widely the Chromium.org web site uses the User-Agent to adapt its content.
Have the IETF been consulted? Should this change be made by the IETF within the HTTP 3 specification?
16. Is feature detection really a good solution? If it were why does Chromium.org make use of the User-Agent? It’s not a very complex web site.
- If 90% of web sites are making use of the User-Agent, as cited in the post – why change anything? It clearly works.
- Given the dominance of Google services, and Google’s control over Chromium, the change will benefit Google economically by removing information from the wider web. Only Google via its dominance in Search, Android and the Play Store will understand device and browser usage. Will Google be making this data available in aggregate as open data? If not, then there exists an argument concerning abuse of dominate market position that Google would be left open to. Has this been considered?
There are a lot of implications to this proposal which do not as yet appear to have been thought through. We sincerely hope the governance model for such a change considers all stakeholders, particularly those who benefit from optimised content in developing countries and do not engage in these discussions.
Regards,
James Rosewell – For self and 51Degrees
On Wed, Jan 15, 2020 at 8:35 PM <ad...@gamerperfection.com> wrote:Yoav,Would the freezing of the UA and introduction of Client Hints assist in anyway of combating the fraudulent use of stolen fingerprints? I am assuming that the server requests to the browser for more detailed information which the "illicit" browser would provide would therefore not go towards helping prevent such fraudulent use?If this is the case, are there any plans to help prevent this type of activity?Once we turn the information from being passively sent out to one that's actively asked for by the server, the browser could keep track of who is asking for it. Then a mechanism like the Privacy Budget could be used to restrict access to that information in cases where the browser suspects that the access to the information is for nefarious purposes.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/d45f46c1-8c49-4249-95e7-8d6d940d9a26%40chromium.org.
Hi Yoav,
As stakeholders in the web and following discussions with colleagues, clients and industry professionals I’ve prepared a summary of questions and observations concerning your proposed changes to the established usage of the HTTP User-Agent header field. This usage has been established since the beginning of the web and the purpose of the field is noted in the documentation of HTTP from 1990s.
While anybody must be concerned about possibly compromising the privacy of the users of the web we feel that the nature and extent of those compromises needs to be more clearly established. The effect of the proposed changes is to damage quite seriously the user experience enjoyed by users on a significant number of websites that require the User-Agent to be in its present form.
Further we are most concerned about the possible damaging effects on the accessibility of those sites and the availability of those sites in developing nations as a result of the proposed changes.
Here follows a number of more detailed comments and we’d be most grateful for your feedback on these comments.
1. Where is the evidence to support the primary assertion that the User-Agent is an “abundant source of passive fingerprinting information”?
Suggestion: A very large web site or network provider using session tracking - such as first party cookies -would be able to provide the analysis needed. Group a large cohort of web sessions by IP and User-Agent and count the number of sessions. Is the relation 1 to 1 or 1 to many? If 1 to many, how many? Does it vary by different formats of User-Agent? Is the analysis sufficient to validate the assertion?
2. In response to your second assertion User-Agent field values are poorly structured. It is our experience that a high degree of accuracy can be achieved from the current state of these strings.
We are not the only practitioners. Experts in the field of device identification and therefore User-Agents strings include:
51Degrees (this authors company “we”)
Device Atlas (owned by Afilias)
ScientiaMobile via the WURFL solution.
Between us support is provided to millions of web sites including blue chip businesses as varied as Google, Facebook, Oracle, eBay, Amazon, Disney, Tencent, IBM, Naspers, Verizon and countless others many of which operate outside North America and Europe.
We provide free and open source solutions. Access to first request device information is essential to the services they support. We have not been consulted about this change.
3. Has thought been given to the impact on non-technical web professionals such as marketing professionals who use analytics tools? As a result of the proposed changes their website may break. As a minimum their web site analytics will be incorrect. They will report incorrect information and form invalid conclusions. This will be frustrating to them and waste countless days head scratching.
4. Have network operators been consulted about the deployment of IPv6 and networks in general? The fingerprinting argument depends at least 50% on the IP address. Depending on deployment a client IP address could persist with a device forever; effectively forming a unique identifier. Frequently changing the client IP address will lead to inefficiencies for network operators.
5. Has the change been validated with web site owners large and small who operate in many different markets and geographies, many of whom make extensive use of the User-Agent to deliver their services? How do developing countries differ to north America and Europe?
Consider news services from Naspers and Tencent targeting Asia and Africa. These services use the information obtained from the User-Agent in the very first request to optimise content ensuring feature phones, powerful smartphones and budget smartphones all receive optimised content cost effectively. Many consumers outside north America and Western Europe do not enjoy “all you can eat” data plans or high-end electronics. They need this feature to receive cost effective and optimised news.
Facebook Zero inspired many businesses to operate this “stripped down” approach. They tend not to be covered in the English language technology press or outside their home markets.
6. In our experience many websites are bloated with MBs of JavaScript, constrained by a single threaded execution engine, and perform poorly. Google invented Accelerated Mobile Pages (AMP) to offer readers a faster way of consuming publishers’ content. It is often seen deployed in Android where knowledge of the device informs the information presented.
Companies that are not Google who host AMP will lack the necessary information to provide their service.
Many publishers including Reach, Mail Online and News International are still plagued by JavaScript performance issues over 5 years after Google shown a light on the performance problem.
From an engineering perspective optimising content on the web server at first request to reduce the amount of data transmitted and the computation the user agent needs to perform is very sensible. The client and web server work more harmoniously together rather than the server being a dumb file server. Making the information needed available in the first request is essential to enable this performance improvement. The User-Agent isn’t ideally formatted, but it works.
It works so well http://chromium.org uses the technique with Google Sites. The User-Agent is everywhere.
7. Does the process of agreeing this include a sufficiently broad cross section of engineers and web professionals? For example the User-Agent is used extensively within the HTTP protocol by CDNs and within edge networks both with and without SSL.
Such engineers are key stakeholders in such a change. Should the IEFT be consulted? Is this not part of HTTP 3?
Current contributions appear to be skewed to be from people with a web front end / JavaScript centric view.
8. Is Safari a good comparator for this performance degrading change? Users are skewed towards higher socio-economic groups, high end chipsets and specific geographic markets. Chromium is the ubiquitous foundation of the web browser globally. It’s used with $10 MediaTek A53 chips to the most powerful Qualcomm SoCs.
9. Is Firefox a good comparator? In many cases Firefox is not bundled with Android by default. It is installed by a user who wishes to benefit from its features. As such a typical user is more privacy aware. As Chromium is the default for most Android devices it should support the widest set of requirements out of the box. Perhaps Google will offer a choice of web browser when Android is first setup? Microsoft did a similar thing in the past with Windows.
10. Is this a feature that should be up to the browser vendor to form a view on rather than sit within the Chromium project? Brave already take one approach, Microsoft another. Privacy conscious users can easily switch between browsers.
11. Public support seems to be in the form of other web browser engineers via a series of tweets and git issues posts operating within a “bubble” or “echo chamber”. This tweet is cited as support (https://twitter.com/_scottlow/status/1206831008261132289). Is this sufficient support for such a wide-reaching change impacting almost every digital citizen and business?
12. What is the view of “minority” browser vendors who benefit extensively via marketing and analytics by appending their browser details to the User-Agent string? What is the view of Android device vendors who wish to be able to gain insight into device usage?
13. Even if a large company has the resources to make the necessary changes within the timescales proposed what are the changes required to maintain the current fine grain of optimisation enjoyed by their users? The majority of small and medium sized businesses and other non-profit web site operators do not. They are all impacted.
They are not of the size and scale of the relatively small number of AdTech / tracking businesses who need to adapt to cookie changes. These companies were given 2 years notice.
This change impacts every website operator and owner. It is impractical to assume the vast majority can or want to make changes to their sites.
14. The AdTech business, of which Google is a major player, have spent a long time producing the standards used to communicate between each other in real time. Has an assessment been performed on the impact to OpenRTB and AdCom where User-Agent is the primary method of validating and unifying device information?
https://github.com/InteractiveAdvertisingBureau/AdCOM/blob/master/AdCOM%20v1.0%20FINAL.md#object_device
15. Where a change is needed there is an obvious solution? Consider the Y2K problem. Like this proposed change it impacted every business. The solution was clear; add two digits to the year and test everything.
In this case there are many use cases where a clear alternative does not exist. Consider the previously mentioned web site adapting on first request to the device with a granularity beyond just desktop or mobile. This change will remove this possibility and new designs will be required.
Or the web professionals wishing to understand why performance is better or worse by different users or web sessions. The IETF HTTP 1.1 specification states “This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations.”
https://tools.ietf.org/html/rfc2616
21 years later this is exactly how it’s being used. So widely the Chromium.org web site uses the User-Agent to adapt its content.
Have the IETF been consulted? Should this change be made by the IETF within the HTTP 3 specification?
16. Is feature detection really a good solution? If it were why does Chromium.org make use of the User-Agent? It’s not a very complex web site.
- If 90% of web sites are making use of the User-Agent, as cited in the post – why change anything? It clearly works.
- Given the dominance of Google services, and Google’s control over Chromium, the change will benefit Google economically by removing information from the wider web. Only Google via its dominance in Search, Android and the Play Store will understand device and browser usage. Will Google be making this data available in aggregate as open data? If not, then there exists an argument concerning abuse of dominate market position that Google would be left open to. Has this been considered?
There are a lot of implications to this proposal which do not as yet appear to have been thought through. We sincerely hope the governance model for such a change considers all stakeholders, particularly those who benefit from optimised content in developing countries and do not engage in these discussions.
Regards,
James Rosewell – For self and 51Degrees
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/49485ca1-646e-4161-a6db-a2fd5c6396de%40chromium.org.
On Tue, Jan 14, 2020 at 10:51 PM <thorste...@sap.com> wrote:A second roundtrip to get more information is definitely a performance issue especially with high latency times. From our perspective the platform e.g. win or macos would be helpful as default information as certain things behaves differently on the platform. Just to make a very simple example tooltips/menu entries for hotkeys (like ctrl+s) are differently on a mac or win (ctrl vs. cmd key) and already part from server in the first roundtrip.Sending different code to each platform just to handle different keys sounds like an overkill. Do you have a real-life example where OS related changes can't be reasonably done on the client-side?
- Matt
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/80c7a29f-9b96-4d61-b531-2744e5562e66%40chromium.org.
Interoperability and Compatibility Risk
The compatibility risk varies at different stages.
For the freezing planned for M83, the compatibility risk is low. Existing UA sniffing code will continue to work as expected. It is only future UA sniffing code that will need to change and use the UA client hints instead.
For the unification planned for M85, the compatibility risk is medium. Some sites can modify their responses based on the OS and device model, and those sites will have to change their UA sniffing code to use UA-CH. We expect such sites to be well maintained (otherwise, how can they keep up with OS UI and device model changes?). Therefore, having 4 releases to modify their code seems sufficient.
In the long term, we expect this change to improve compatibility, as UA sniffing based on UA-CH is bound to be more reliable than the current status quo.
As for interoperability, we have other vendors on board with UA freezing, but not necessarily with the UA Client Hints mechanism, that is supposed to replace it. That can create a tricky situation, where developers would need to rely on the User-Agent string for some browsers and on UA-CH for others.
Edge: Public support
Firefox: Public support for freezing the UA string - “freezing the User Agent string without any client hints—seems worth-prototyping”
Safari: Shipped to some extent. Safari has attempted to completely freeze the UA string in the past, without providing an alternative mechanism. That got a lot of pushback, which resulted in somewhat reverting that decision. Nowadays, their UA string seems frozen, other than updates to the OS version and the browser major version.
Alternative implementation suggestion for web developers
For many (most?) uses of UA sniffing today, a better tool for the job would be to use feature detection. Where feature detection fails developers, UA Client Hints are the right path forward.
Potential deployment hurdles compared to status quo:
Third party services that rely on the UA string would need to convince the sites that include them to delegate that information to them using Feature Policy.
The opt-in based mechanism of Client Hints currently suffers from the fact that on the very-first view, the browser have not yet received the opt-in. That means that requiring specific UA information (e.g. device model, platform) on the very-first navigation request may incur a delay. We are exploring options to enable the opt-in to happen further down the stack to avoid that delay.
Hi Yoav - can you please request a TAG review of this proposal? I think this has enough far-reaching consequences for the web that this is really something the TAG can help with in terms of wide review. Please also see the minutes of our calls this week where we've had some discussions on this topic. Thanks, Dan
With respect, I think a specific proposal to freeze the UA string is different enough in scope from a high level proposal to replace "some headers" with client hints that I think this deserves a new review item. The design is not the issue. The impacts on the rest of the web ecosystem is the issue - and I think this is the part that is making us nervous.I am also concerned about how this impacts "minority browsers" - especially as web developers learn what browsers are accessing their sites (and therefore what browsers they need to test on) based on information that may partly be derived from the UA string.
I want to know that you have done the work to understand every way the UA string is being used in the wild and that your proposal accommodates those use cases - especially what some might consider the edge cases such as delivery of content to feature phone browsers.
With respect, I think a specific proposal to freeze the UA string is different enough in scope from a high level proposal to replace "some headers" with client hints that I think this deserves a new review item. The design is not the issue. The impacts on the rest of the web ecosystem is the issue - and I think this is the part that is making us nervous.
I am also concerned about how this impacts "minority browsers" - especially as web developers learn what browsers are accessing their sites (and therefore what browsers they need to test on) based on information that may partly be derived from the UA string.
I want to know that you have done the work to understand every way the UA string is being used in the wild and that your proposal accommodates those use cases - especially what some might consider the edge cases such as delivery of content to feature phone browsers.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALiHrgkvpkW3B7ybgYBNj6VD0ugbJvD3vGZ00wOB88hzyyTdOA%40mail.gmail.com.
On Thu, Jan 23, 2020 at 7:56 PM Daniel Appelquist <appel...@gmail.com> wrote:With respect, I think a specific proposal to freeze the UA string is different enough in scope from a high level proposal to replace "some headers" with client hints that I think this deserves a new review item. The design is not the issue. The impacts on the rest of the web ecosystem is the issue - and I think this is the part that is making us nervous.The design review was rather specific about the fact that it targets the `User-Agent` and the `Accept-Language` headers.But, if that is somehow helpful, I'm happy to open a new issue.
sec-ch-ua: Vivaldi 79
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.131 Safari/537.36
sec-ch-ua: Vivaldi 79
Contact emails
If I have 2 servers on 2 different sub domains, and there's a request to one of the domains that return the UA information request header, will Chrome add the Client Hints to the next request on the other domain?
Will Chrome automatically add the Client Hints on the request header on redirected requests (HTTP status 302) ?I mean when the first HTTP response has the `Location` header and the UA information request headers. In such case, when the browser automatically launch the second request, will it include the Clients Hints ?
Thanks for the notice.
Hi Yoav,
could you please provide an update regarding time line of both projects.
1) Implementing UA Client Hints
2) Freezing and Removing user agent string
Could you please make an example of the freeze user agent string. It is very important to decide on the criticality of the change.
Please be aware that we would need enough time between 1) and 2) to implement and ship and for customers to implement the fixes. Especially on-prem customer running mission critical systems needs at least one year to get the systems updated. A time plan like initially told within half a year is not realistic for business customers running onprem systems.
Thanks Thorsten
What does unify mean? Does it mean that existing user agents will be changed?
As part of our close monitoring of the COVID-19 impact on the web ecosystem and our recent announcements about changes to Chrome’s upcoming release schedule, I wanted to share an update on the upcoming User-Agent Client Hints project.
First, the phase of the project that reduces the amount of available information in the traditional User-Agent string to browser (e.g. “Chrome”), major browser version, and mobile/desktop, has been deferred until at least 2021 to provide additional time for the ecosystem to evaluate the new UA-CH capabilities.
Second, the initial phase of the UA-CH project which introduces new Client Hints focused on providing similar informational capabilities to the traditional User-Agent HTTP header string, is now expected to be ready to ship for Chrome version 84 which is scheduled to be released to stable on July 14th. This release is intended to allow for developers to experiment and provide feedback. This introduction is entirely additive, with no expected impact on existing systems.
While work on UA-CH continues, we don’t currently know in what ways or for how long COVID-19 will impact the web ecosystem’s ability to test and implement support for this change. We will continue to monitor the situation and revisit this topic as the situation evolves.
Thanks,
Yoav
Hi Yoav,
Is there an updated timeline anywhere?
What about any documentation of responses to the challenges to the plan raised in the thread?
Is there a better place than this thread to follow the developments?
--
This message (including any attachments) may contain confidential,
proprietary, private and/or privileged information. The information is
intended to be for the use of the individual or entity designated above. If
you are not the intended recipient of this message, please notify the
sender immediately, and delete the message and any attachments. Any
disclosure, reproduction, distribution or other use of this message or any
attachments by an individual or entity other than the intended recipient is
strictly prohibited. In addition, emails sent from and to this
integralads.com <http://integralads.com>domain are monitored, archived,
and subject to disclosure, including in connection with regulatory or other
legal processes.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/17fc95fa-60c7-46f1-b294-39de80aac074o%40chromium.org.
Any specific issue you feel wasn't addressed?
--
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/b1ada7d1-afc7-4474-aafe-f1a0a5d01bdeo%40chromium.org.
Because Chrome for iOS uses Webkit, is it out-of-topic that if the UA String will be frozen in Chrome for iOS or not?
> Our limited testing in April showed a few differences between the spec and the implementation at that point.We've done quite a bit of work on the internals of client hints in the past few months; if you can recall, do you remember anything specific?
> Our limited testing in April showed a few differences between the spec and the implementation at that point.We've done quite a bit of work on the internals of client hints in the past few months; if you can recall, do you remember anything specific?--On Sat, Jun 6, 2020 at 4:15 PM micah via blink-dev <blin...@chromium.org> wrote:Any specific issue you feel wasn't addressed?Not exactly. I've only been following this project tangentially. Everywhere I look I see long lists of issues and discussions that I'm not eager to try to digest in their entirety :). Our limited testing in April showed a few differences between the spec and the implementation at that point and raised a bunch of questions for how we'll adapt to this change. Also, some of the related blog posts, such as DeviceAtlas's, seem to raise significant questions. I'm just trying to find a definitive place from which to watch this project and to get a better understanding of what exactly is going to be shipped when.
This message (including any attachments) may contain confidential, proprietary, private and/or privileged information. The information is intended to be for the use of the individual or entity designated above. If you are not the intended recipient of this message, please notify the sender immediately, and delete the message and any attachments. Any disclosure, reproduction, distribution or other use of this message or any attachments by an individual or entity other than the intended recipient is strictly prohibited. In addition, emails sent from and to this integralads.com domain are monitored, archived, and subject to disclosure, including in connection with regulatory or other legal processes. --
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/b1ada7d1-afc7-4474-aafe-f1a0a5d01bdeo%40chromium.org.
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJwB4XZ1BxiNoA1DjGmK1YgDHts1go5WmT9cu7UCf6p0du5bzQ%40mail.gmail.com.
> Our limited testing in April showed a few differences between the spec and the implementation at that point.We've done quite a bit of work on the internals of client hints in the past few months; if you can recall, do you remember anything specific?I may chime in here with a few questions/comments I’ve encountered playing with the current implementation and feedback from the wild:
2) Feature-Policy - used to delegate hints to 3rd parties - is about to change name and structure. How/when will this be addressed?
3) …and last but not least; The first navigation request, will the User-Agent be deprecated without a real alternative for content negotiation on the first navigation request/page load?
--
From: Aaron Tagliaboschi <aaro...@chromium.org>
Reply: Aaron Tagliaboschi <aaro...@chromium.org>
Date: 9 June 2020 at 01:27:15
To: mi...@integralads.com <mi...@integralads.com>
Cc: blink-dev <blin...@chromium.org>
Subject: Re: [blink-dev] Re: Intent to Deprecate and Freeze: The User-Agent string
> Our limited testing in April showed a few differences between the spec and the implementation at that point.We've done quite a bit of work on the internals of client hints in the past few months; if you can recall, do you remember anything specific?--On Sat, Jun 6, 2020 at 4:15 PM micah via blink-dev <blin...@chromium.org> wrote:Any specific issue you feel wasn't addressed?Not exactly. I've only been following this project tangentially. Everywhere I look I see long lists of issues and discussions that I'm not eager to try to digest in their entirety :). Our limited testing in April showed a few differences between the spec and the implementation at that point and raised a bunch of questions for how we'll adapt to this change. Also, some of the related blog posts, such as DeviceAtlas's, seem to raise significant questions. I'm just trying to find a definitive place from which to watch this project and to get a better understanding of what exactly is going to be shipped when.
This message (including any attachments) may contain confidential, proprietary, private and/or privileged information. The information is intended to be for the use of the individual or entity designated above. If you are not the intended recipient of this message, please notify the sender immediately, and delete the message and any attachments. Any disclosure, reproduction, distribution or other use of this message or any attachments by an individual or entity other than the intended recipient is strictly prohibited. In addition, emails sent from and to this integralads.com domain are monitored, archived, and subject to disclosure, including in connection with regulatory or other legal processes. --
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/b1ada7d1-afc7-4474-aafe-f1a0a5d01bdeo%40chromium.org.
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJwB4XZ1BxiNoA1DjGmK1YgDHts1go5WmT9cu7UCf6p0du5bzQ%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADzD%3DaMz0FqWY%3DSKnRKtM-s%3DyL8WhD%2BmQnWXTLFoC9XhhC5Zdg%40mail.gmail.com.
On Tue, Jun 16, 2020 at 6:02 PM Jon Arne Sæterås <set...@gmail.com> wrote:> Our limited testing in April showed a few differences between the spec and the implementation at that point.We've done quite a bit of work on the internals of client hints in the past few months; if you can recall, do you remember anything specific?I may chime in here with a few questions/comments I’ve encountered playing with the current implementation and feedback from the wild:My plan is for that to be a separate intent and process. Happy to chat offline about that.2) Feature-Policy - used to delegate hints to 3rd parties - is about to change name and structure. How/when will this be addressed?Client Hints should be another FP feature, so the migration path should be similar to other features.+Ian Clelland - thoughts?
It would be helpful if there was a clear specification for this work, as well as a better justification and assessment of implications in practice.
Specifications
The list of documents related to Privacy Sandbox, under which this work is conducted, are listed on the Privacy Sandbox home page.
From this page the privacy budget comments are described on a repository maintained by Brad Lassey. https://github.com/bslassey/privacy-budget
There is a WICG document on UA Client-Hints which is a general introduction. This document references an unofficial draft document hosted by the W3C titled Client Hints Infrastructure. This in turn references an IETF Experiment which expires in December 2020. There are also references to multiple other documents. These documents seem to change on a weekly basis.
Where use cases are negatively impacted by the change these are not adequately mitigated for. For example, device vendors active install base monitoring is not considered, whilst browser vendors are.
The combined effect of these multiple documents is to make it hard for anyone not fully engaged in the work to follow what is supposed to be the outcome and how success will be measured. Even if as a developer I were fully engaged I would find it impossible to work out what I am supposed to be developing.
Given the potential to break the web for 4,000,000,000 people, and the huge global effort involved in switching from an established standard for interoperability please could Chromium present a clear specification which describes the modification and the expected outcomes? In the past I have found Google documentation clear and easy to follow.
Justification and Implications
The reference material supporting the change has been discussed within a very small group of engineers, from very large companies who can afford to spend the time working on such subjects, without considering a broader set of stakeholder considerations or participants. One referenced example is the TAG design review which lacks diversity of interests and considerations. A wider group of representatives are needed.
Such discussions do not consider the possibility people should be free to decide who they trust, or the need to design solutions to verify their trust choices have been respected. Discussions to date approach the subject of privacy from the position of browser vendors being the most trusted party and as such should adopt a paternalistic approach to all matters of trust between people and others.
I believe the W3C and IETF are becoming aware of this imbalance and are encouraging greater participation and engagement. Indeed, my small business has joined the W3C and is actively participating in W3C groups. As an example, I’m leading the development of success criteria to improve web advertising and have written an update on these subjects for my company 51Degrees.
Industry commentators are recognising the current direction of thinking will result in a two-tier internet. One where people are anonymous and denied access to services and information. The other where people are forced to surrender their personal information in return for services and information that is funded via advertising. The “middle ground” enjoyed by many today will be removed. The perverse effect will be that privacy engineers will have made people less private in practice.
How can someone be both for this outcome AND support net neutrality?
There are also unresolved matters related to abuse of dominant market position. Google can collect personal data about people to use as they wish because they control the Android OS, Search, YouTube, and Maps among others. Google control the sign-up process. Users have no choice other than to accept Google’s terms if they wish to use Android mobile devices. Such terms, and their default bias, enable Google to collect and use information that is denied to an increasing number of other parties by work such as this. As Google services are now considered essential to society this work will attract the attention of regulators.
For at least these reasons the justification for this works must be enhanced and a clear, version controlled specification created so that those impacted can understand and explain what is happening.
It would be helpful if there was a clear specification for this work
Thank you. Are these the only two documents that are relevant to this work? How are changes to one reflected in the other?
Hi Yoav,Just wanted to clarify part of your message:
> First, the phase of the project that reduces the amount of available information in the traditional User-Agent string to browser (e.g. “Chrome”), major browser version, and mobile/desktop, has been deferred until at least 2021 to provide additional time for the ecosystem to evaluate the new UA-CH capabilities.
Will the delay also include freezing the UA string (i.e. will the current UA string be up-to-date until some change in 2021).
Thanks and really appreciate your work!
James
Hey all,As part of our close monitoring of the COVID-19 impact on the web ecosystem and our recent announcements about changes to Chrome’s upcoming release schedule, I wanted to share an update on the upcoming User-Agent Client Hints project.
First, the phase of the project that reduces the amount of available information in the traditional User-Agent string to browser (e.g. “Chrome”), major browser version, and mobile/desktop, has been deferred until at least 2021 to provide additional time for the ecosystem to evaluate the new UA-CH capabilities.
Second, the initial phase of the UA-CH project which introduces new Client Hints focused on providing similar informational capabilities to the traditional User-Agent HTTP header string, is now expected to be ready to ship for Chrome version 84 which is scheduled to be released to stable on July 14th. This release is intended to allow for developers to experiment and provide feedback. This introduction is entirely additive, with no expected impact on existing systems.
While work on UA-CH continues, we don’t currently know in what ways or for how long COVID-19 will impact the web ecosystem’s ability to test and implement support for this change. We will continue to monitor the situation and revisit this topic as the situation evolves.
Thanks,
Yoav
--
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJwB4XZvmBBGb_RK_onn_whey-n4HoaYhRunPhJ-ksHsoR9R4w%40mail.gmail.com.
When will the user agent freeze take effect?
Will any browser updates after the freeze simply not be reflected in the version numbers in the UA post-freeze, such that post-freeze the UA will no longer necessarily reflect the user’s browser version?
--Bret LoweryOn Jul 27, 2020, at 3:31 PM, Aaron Tagliaboschi <aaro...@chromium.org> wrote:--On Mon, Jul 27, 2020, 15:19 James Nghiem <james...@berkeley.edu> wrote:Hi Yoav,Just wanted to clarify part of your message:
> First, the phase of the project that reduces the amount of available information in the traditional User-Agent string to browser (e.g. “Chrome”), major browser version, and mobile/desktop, has been deferred until at least 2021 to provide additional time for the ecosystem to evaluate the new UA-CH capabilities.
Will the delay also include freezing the UA string (i.e. will the current UA string be up-to-date until some change in 2021).That is correctThanks and really appreciate your work!
JamesOn Thursday, June 18, 2020 at 2:01:37 PM UTC-7 Aaron Tagliaboschi wrote:Thank you. Are these the only two documents that are relevant to this work? How are changes to one reflected in the other?There are currently three related but technically independent proposed specs:
- the HTTP Client Hints spec deals with the actual client hint mechanics; that is to say the advertisement mechanism (the Accept-CH header and the preferences store).
- the Client Hint Infrastructure spec discusses how browsers decide when to store CH preferences and when to send headers. This document is actually the proposed changes to the HTML, Fetch, and Permission Policy (nee Feature Policy specs)
- the User Agent Client Hints spec lays out the specifics if the user agent set of client hints
They're generally laid out so there's no duplicate or redundant information, so they reference each other.
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJwB4XZvmBBGb_RK_onn_whey-n4HoaYhRunPhJ-ksHsoR9R4w%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/8D3F4527-BCCC-457D-903F-A79BF2A1DC47%40atlantacounselingsolutions.com.
Contact emails
yoav...@chromium.org, aaro...@chromium.org
Summary
We want to freeze and unify (but not remove) the User Agent string in HTTP requests as well as in `navigator.userAgent`
Motivation
The User-Agent string is an abundant source of passive fingerprinting information about our users. It contains many details about the user’s browser and device as well as many lies ("Mozilla/5.0", anyone?) that were or are needed for compatibility purposes, as servers grew reliant on bad User Agent sniffing.
On top of those privacy issues, User-Agent sniffing is an abundant source of compatibility issues, in particular for minority browsers, resulting in browsers lying about themselves (generally or to specific sites), and sites (including Google properties) being broken in some browsers for no good reason.
The above abuse makes it desirable to freeze the UA string and replace it with a better mechanism. There have been past attempts at UA string freezing from the Safari team, but without an alternative way to perform UA based content-negotiation, they had to be partially reverted.
The User Agent Client Hints (UA-CH) feature provides an alternate source for the information the User-Agent string provides, both in its request header form as well as its JS API one.
Its main advantages are:
It provides the required information only when the server requests it, over secure connections, making any fingerprinting that relies on it be active fingerprinting, which enables such use to be audited, as well as acted-upon by the browser (e.g. in a future implementation of the Privacy Budget).
It provides the information in small increments, so servers are only exposed to the information they need and request, rather than being exposed to the full gamut of the UA string even if they are just trying to figure out one detail about the browser. (e.g. brand and major version)
Since it provides the information via dedicated fields, it enables better ergonomics and makes it less likely for servers to get it wrong and cause compatibility issues.
And finally, starting fresh will enable us to drop a lot of the legacy baggage that the UA string carries (“Mozilla/5.0”, “like Gecko”, “like KHTML”, etc) going forward.
Once UA-CH ships as an alternative means for browser-specific content adaptation, we would like to freeze the User-Agent string.
We propose to deprecate at M81 (starting to emit console warnings in pages that read that string in JS), freeze its version information at M83, and unify strings of different devices at M85. See detailed freezing plan below.
This timeline provides 3 months for developers to move to the new mechanism for their future browser and OS version needs, and 6 months for more sophisticated OS or device specific targeting.
Freezing plan
Different parts of the UA string have different compatibility implications.
Some parts of it, such as the browser version and the OS version, can be frozen without any backwards compatibility implications. Values that worked in the past will continue to work in the future.
Other parts, such as the model (for mobile devices) and the OS platform, can have implications on sites that tailor their UI to the underlying OS or that target a very specific model in their layout. Such sites will need to migrate to use UA-CH.
As such we are planning to freeze the parts that are amenable to freezing fairly early, and gradually unify the rest.
(*) For the mobile value, we may split it further based on common device dimensions, as a one-time exercise, to reduce the compatibility risk of unification.
Interoperability and Compatibility Risk
The compatibility risk varies at different stages.
For the freezing planned for M83, the compatibility risk is low. Existing UA sniffing code will continue to work as expected. It is only future UA sniffing code that will need to change and use the UA client hints instead.
For the unification planned for M85, the compatibility risk is medium. Some sites can modify their responses based on the OS and device model, and those sites will have to change their UA sniffing code to use UA-CH. We expect such sites to be well maintained (otherwise, how can they keep up with OS UI and device model changes?). Therefore, having 4 releases to modify their code seems sufficient.
In the long term, we expect this change to improve compatibility, as UA sniffing based on UA-CH is bound to be more reliable than the current status quo.
As for interoperability, we have other vendors on board with UA freezing, but not necessarily with the UA Client Hints mechanism, that is supposed to replace it. That can create a tricky situation, where developers would need to rely on the User-Agent string for some browsers and on UA-CH for others.
Edge: Public support
Firefox: Public support for freezing the UA string - “freezing the User Agent string without any client hints—seems worth-prototyping”
Safari: Shipped to some extent. Safari has attempted to completely freeze the UA string in the past, without providing an alternative mechanism. That got a lot of pushback, which resulted in somewhat reverting that decision. Nowadays, their UA string seems frozen, other than updates to the OS version and the browser major version.
Alternative implementation suggestion for web developers
For many (most?) uses of UA sniffing today, a better tool for the job would be to use feature detection. Where feature detection fails developers, UA Client Hints are the right path forward.
Potential deployment hurdles compared to status quo:
Third party services that rely on the UA string would need to convince the sites that include them to delegate that information to them using Feature Policy.
The opt-in based mechanism of Client Hints currently suffers from the fact that on the very-first view, the browser have not yet received the opt-in. That means that requiring specific UA information (e.g. device model, platform) on the very-first navigation request may incur a delay. We are exploring options to enable the opt-in to happen further down the stack to avoid that delay.
Usage information from UseCounter
If we were going to remove the User-Agent string outright, the answer to that would’ve been “a lot!!!!111”.
Given that we are planning to freeze it, we expect the “removal” to be mostly backwards compatible, barring cases of specific OS or device adaptation.
Client-side UA sniffing use counters show it’s being used by ~90% of sites. But again, since we’re talking about freezing, it should not break most uses.
Entry on the feature dashboard
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/281c9887-4679-4b84-aef0-dd65ad660628n%40chromium.org.
Just wanted to follow up on this since I haven't seen or been able to find any updates to the timeline! Do we have any specific dates on when we're expecting UACH to roll out?
A good question.
There are a lot of outstanding issues registered at W3C.
Issues · WICG/ua-client-hints (github.com)
A lot of feedback has been provided. My
organization does not consider CH or UACH are ready for deployment as this
proposal increases the data overhead associated with HTTP/Web, reduces
performance, enhances information asymmetries, does not address the problems it
was intending to resolve and is founded on flawed assumptions. There are also wider questions in relation to "Privacy Sandbox" of which this is part that are awaiting decisions from higher authorities.
I’d like to see a W3C Working Group created and the W3C Process followed before deployment so that Google can demonstrate to the world that genuine consensus was achieved prior to a de facto standard being created that damages the web.
Regards,
James Rosewell (51Degrees)
Just wanted to follow up on this since I haven't seen or been able to find any updates to the timeline! Do we have any specific dates on when we're expecting UACH to roll out?On Friday, January 29, 2021 at 9:16:39 AM UTC-8 Thomas Ziegelbecker wrote:
Ah I see it in the release now. A few more questions:
As part of the Chrome 89 rollout, I'm assuming that the UA will also be frozen, correct?
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/a609b857-7b59-4d3b-b5a1-bd21d435e2e6n%40chromium.org.
Google and CMA reached an agreement on 4th February 2022 concerning Privacy Sandbox and Topics. The agreement requires Google to train staff in Annex 3, and paragraph 4.119 states, “Google has committed to instruct its staff and agents not to make claims to other market players that contradict the commitments, and to provide training to its relevant staff and agents to ensure that they are aware of the requirements of the Final Commitments”.
Paragraph 21 of the
agreement states “During the standstill period, the CMA may notify Google
that competition law concerns remain such that the Purpose of the Commitments
will not be achieved. Google will work with the CMA without delay to seek to
resolve concerns raised and address comments made by the CMA with a view to
achieving the Purpose of the Commitments. Google will inform the CMA of how it
has responded to those comments”.
The standstill
period is defined in the agreement as is User-Agent reduction.
Will Google representatives confirm they have approval from the CMA to progress this proposal? If not then please can they confirm that the proposal is now delayed until such approval is provided?
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6e72d610-9978-4d7a-862a-5830edbf6b81n%40chromium.org.
Hey all!
Looks like I'm jumping on this train a little bit late, but still probably I’m not the last person on earth that is in the field of the web and only recently stumbled this change. After reading all this thread carefully I still feel that not everything was resolved so I’m posting it.
Some of the reason that precise device detection is applied is to serve the best content for the users. Something that was mentioned in the thread previously. There’re multiple reasons for it that I see:
Experimenting lately with the UA-CH implementation for the websites, mostly the Critical-CH as high entropy CH are required to make the above decision on the “first hit”. The conclusion is that the proposed Critical-CH change is far from ideal, even harmful. The specs basically tell:
1. Browser sends a request to the server with a low entropy CH, e.g. “Sec-CH-UA-Mobile”.You all probably know the above, and some problems were raised already in this thread but I’ll mention them again as some weren’t answered:
1. Two requests – doubles the time till user receives the content – and there’re multiple researches that show how badly the load time affects the conversion.The alternative solution that was discussed by developers is to use redirects that are sent from the server for more data. Basically, the scheme stays as the above but the server sends 307, or other redirect code, with Accept-CH, so after the redirect the client adds the missing CH. The discussed drawback was that there could be an infinite loop if it’s implemented wrongly.
Benefits of this approach is that the server doesn’t waste time on full page render, the response size is lower and that the server is sure that the browser will make the second request.
Tried to optimize the first approach, with Critical-CH, to save as much as possible on the first request, the response that was 200 with empty page (don’t do it on prod!). But it is not future proof. Where browser may not send second request at all. Also guessing that browser supports UA-CH is not defined – just heuristic that “Sec-CH-UA-Mobile” is set, and the second guess is that one of the high entropy is missing is by maintaining on server side list of all HE-CH and comparing it to the request. This list is also constantly changing so this is also a not well-defined part of the specification.
Also
tested the Critical-CH together with 307. So, even if the guess that browser doesn’t
support CH is wrong, the browser will still follow the redirect and render the
required page to the user. For browsers that support CH the expected behavior
was to just make a second request and ignore the response. And here was a
surprise, there was no second request to the page, the browser just followed
the redirect. This behavior was mentioned only in the source code thread, and not
in the specs:
https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/client_hints/README.md
“The Critical-CH retry mechanism is implemented as content::CriticalClientHintsThrottle and all of the important logic is in WillProcessResponse. When a retry situation is found (and the redirected_ flag isn’t set) the header is stored, the new hints are stored as normal and the request is “restarted” (i.e. the response is changed to an internal redirect to the same location, which is also what DevTools sees).”
This message is already got long, and there’re more of non-answered questions about CH that deserve a separate post. But my main point is that there’s no defined way how to get the required high entropy client hints without harming the user and the web site owner.
Regards
Hey all!
1. To provide the best UX based on the device – so mobile gets mobile web sites, desktop gets desktop experience and tablet is something in the gray zone – that’s where the use of the exact model is required, and based on device size, CPU capabilities and other properties the correct UI is provided.
Looks like I'm jumping on this train a little bit late, but still probably I’m not the last person on earth that is in the field of the web and only recently stumbled this change. After reading all this thread carefully I still feel that not everything was resolved so I’m posting it.
Some of the reason that precise device detection is applied is to serve the best content for the users. Something that was mentioned in the thread previously. There’re multiple reasons for it that I see:2. Another requirement is to get full version of the browser and preferably the OS versions. As an example, if you work with video streaming, HLS, WebRTC and etc. they tend to break pretty often and could be fixed or broke again even between minor versions. And sometimes OS itself is a reason for it. So, the combination of these is required to allow/disallow some features and apply workarounds.
3. BI for detecting anomalies in specific version. You usually don’t need user’s fingerprint if the user is registered and logged in. But to be able to support the above 2 points and make other per device decisions the shortened UA is not enough.Experimenting lately with the UA-CH implementation for the websites, mostly the Critical-CH as high entropy CH are required to make the above decision on the “first hit”. The conclusion is that the proposed Critical-CH change is far from ideal, even harmful. The specs basically tell:
1. Browser sends a request to the server with a low entropy CH, e.g. “Sec-CH-UA-Mobile”.
2. Server sends in respond Critical-CH with list of high entropy CH, together with a full web page.
3. Browser detects new high entropy CH that are required and makes another request now with required CH
4. Server sends back a page that is adapted to the full UA-CHYou all probably know the above, and some problems were raised already in this thread but I’ll mention them again as some weren’t answered:
1. Two requests – doubles the time till user receives the content – and there’re multiple researches that show how badly the load time affects the conversion.
2. The server renders the full page on first request and it’s being omitted: More time on meaningless respond sending, more traffic, more time on server creating the response – think of server going to DB – useless load on all layers. More power consumption…
3. How the server knows that the client will send a second request or not? Maybe the browser doesn’t support at all client hints then the server needs to prepare the whole page and not optimize internal BL. The other case is that browser supports CH but won’t send them anyway (if in the future user will opt out from some CH, if it will be available, or the privacy budget is empty).
4. BL should be rewritten as now user makes sometimes 2 requests.
The alternative solution that was discussed by developers is to use redirects that are sent from the server for more data. Basically, the scheme stays as the above but the server sends 307, or other redirect code, with Accept-CH, so after the redirect the client adds the missing CH. The discussed drawback was that there could be an infinite loop if it’s implemented wrongly.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/575ebed2-697a-4169-84d4-5dce9a433fd7n%40chromium.org.
Hi Kirill,Thanks for your feedback.On Thu, Apr 21, 2022 at 2:16 PM Kirill Kandalov <kirill....@gmail.com> wrote:Hey all!
1. To provide the best UX based on the device – so mobile gets mobile web sites, desktop gets desktop experience and tablet is something in the gray zone – that’s where the use of the exact model is required, and based on device size, CPU capabilities and other properties the correct UI is provided.
Looks like I'm jumping on this train a little bit late, but still probably I’m not the last person on earth that is in the field of the web and only recently stumbled this change. After reading all this thread carefully I still feel that not everything was resolved so I’m posting it.
Some of the reason that precise device detection is applied is to serve the best content for the users. Something that was mentioned in the thread previously. There’re multiple reasons for it that I see:2. Another requirement is to get full version of the browser and preferably the OS versions. As an example, if you work with video streaming, HLS, WebRTC and etc. they tend to break pretty often and could be fixed or broke again even between minor versions. And sometimes OS itself is a reason for it. So, the combination of these is required to allow/disallow some features and apply workarounds.
3. BI for detecting anomalies in specific version. You usually don’t need user’s fingerprint if the user is registered and logged in. But to be able to support the above 2 points and make other per device decisions the shortened UA is not enough.Experimenting lately with the UA-CH implementation for the websites, mostly the Critical-CH as high entropy CH are required to make the above decision on the “first hit”. The conclusion is that the proposed Critical-CH change is far from ideal, even harmful. The specs basically tell:
1. Browser sends a request to the server with a low entropy CH, e.g. “Sec-CH-UA-Mobile”.
2. Server sends in respond Critical-CH with list of high entropy CH, together with a full web page.
3. Browser detects new high entropy CH that are required and makes another request now with required CH
4. Server sends back a page that is adapted to the full UA-CHYou all probably know the above, and some problems were raised already in this thread but I’ll mention them again as some weren’t answered:
1. Two requests – doubles the time till user receives the content – and there’re multiple researches that show how badly the load time affects the conversion.
2. The server renders the full page on first request and it’s being omitted: More time on meaningless respond sending, more traffic, more time on server creating the response – think of server going to DB – useless load on all layers. More power consumption…
3. How the server knows that the client will send a second request or not? Maybe the browser doesn’t support at all client hints then the server needs to prepare the whole page and not optimize internal BL. The other case is that browser supports CH but won’t send them anyway (if in the future user will opt out from some CH, if it will be available, or the privacy budget is empty).That's correct; if the browser doesn't support client hints, the Critical-CH response header won't have any effect. There are also some situations in which client hints won't get sent, for example: (1) insecure request (e.g. not https), (2) the request is coming from a subresource or embedded iframe and the top-level frame's permissions policy didn't delegate, (3) Javascript is disabled, (4) UA string is overridden
4. BL should be rewritten as now user makes sometimes 2 requests.That's correct in that Critical-CH will result in 2 network round trips. There are two mitigations in place for this, with varying degrees of availability:(1) Use the ACCEPT_CH frame in the ALPS extension, so that the requested client hints are known at connection establishment time and thus, won't require an extra round trip. This is currently only implemented in BoringSSL (not yet in OpenSSL): https://chromestatus.com/feature/5555544540577792
(2) We recently launched a feature where the Accept-CH storage persists beyond browser restarts, so hopefully the Critical-CH header should take effect less (only the first navigation ever, as opposed to the first navigation after the browser restarts or user session otherwise ends): https://bugs.chromium.org/p/chromium/issues/detail?id=1296339. So hopefully this should improve client hints coverage such that restarts via Critical-CH are needed far less.
If you encounter any issues or have further feedback, please do file an open issue at crbug.new using the Blink>Network>ClientHints component.
The alternative solution that was discussed by developers is to use redirects that are sent from the server for more data. Basically, the scheme stays as the above but the server sends 307, or other redirect code, with Accept-CH, so after the redirect the client adds the missing CH. The discussed drawback was that there could be an infinite loop if it’s implemented wrongly.
Do you mean the 307 hits the same page in the redirect, but this time it includes the client hints? That's effectively what the CriticalClientHintsThrottle is doing now. And there are safeguards in place in it to prevent infinite redirects: https://source.chromium.org/chromium/chromium/src/+/main:content/browser/client_hints/critical_client_hints_throttle.cc;l=91-92
Hi,Thanks for the quick reply!On Monday, April 25, 2022 at 6:34:56 PM UTC+3 abe...@google.com wrote:Hi Kirill,Thanks for your feedback.On Thu, Apr 21, 2022 at 2:16 PM Kirill Kandalov <kirill....@gmail.com> wrote:Hey all!
1. To provide the best UX based on the device – so mobile gets mobile web sites, desktop gets desktop experience and tablet is something in the gray zone – that’s where the use of the exact model is required, and based on device size, CPU capabilities and other properties the correct UI is provided.
Looks like I'm jumping on this train a little bit late, but still probably I’m not the last person on earth that is in the field of the web and only recently stumbled this change. After reading all this thread carefully I still feel that not everything was resolved so I’m posting it.
Some of the reason that precise device detection is applied is to serve the best content for the users. Something that was mentioned in the thread previously. There’re multiple reasons for it that I see:2. Another requirement is to get full version of the browser and preferably the OS versions. As an example, if you work with video streaming, HLS, WebRTC and etc. they tend to break pretty often and could be fixed or broke again even between minor versions. And sometimes OS itself is a reason for it. So, the combination of these is required to allow/disallow some features and apply workarounds.
3. BI for detecting anomalies in specific version. You usually don’t need user’s fingerprint if the user is registered and logged in. But to be able to support the above 2 points and make other per device decisions the shortened UA is not enough.Experimenting lately with the UA-CH implementation for the websites, mostly the Critical-CH as high entropy CH are required to make the above decision on the “first hit”. The conclusion is that the proposed Critical-CH change is far from ideal, even harmful. The specs basically tell:
1. Browser sends a request to the server with a low entropy CH, e.g. “Sec-CH-UA-Mobile”.
2. Server sends in respond Critical-CH with list of high entropy CH, together with a full web page.
3. Browser detects new high entropy CH that are required and makes another request now with required CH
4. Server sends back a page that is adapted to the full UA-CHYou all probably know the above, and some problems were raised already in this thread but I’ll mention them again as some weren’t answered:
1. Two requests – doubles the time till user receives the content – and there’re multiple researches that show how badly the load time affects the conversion.
2. The server renders the full page on first request and it’s being omitted: More time on meaningless respond sending, more traffic, more time on server creating the response – think of server going to DB – useless load on all layers. More power consumption…
3. How the server knows that the client will send a second request or not? Maybe the browser doesn’t support at all client hints then the server needs to prepare the whole page and not optimize internal BL. The other case is that browser supports CH but won’t send them anyway (if in the future user will opt out from some CH, if it will be available, or the privacy budget is empty).That's correct; if the browser doesn't support client hints, the Critical-CH response header won't have any effect. There are also some situations in which client hints won't get sent, for example: (1) insecure request (e.g. not https), (2) the request is coming from a subresource or embedded iframe and the top-level frame's permissions policy didn't delegate, (3) Javascript is disabled, (4) UA string is overriddenSo my point is that we need to know reliably in the first request on the server side if the second request is a possibility at all.
If a client would A. list supported client hints in the first request B. If the condition for second request are met (HTTPS, not iframe with invalid permission policy etc) only then the server would know how to respond correctly.
BTW if you raised the iframe topic. Is it enough for the page from the parent domain to set permission policy on the iframe without stating the target domain that this policy is given for? I'm seeing a very common scenario where parent domain A, embeds iframe with some Ad network from domain B, and when the page is rendered, the iframe, from domain B, redirects itself to the ad from domain C. That's how ad network works. And the site owner of domain A has no clue of domain C.
I read the spec, including https://datatracker.ietf.org/doc/html/draft-davidben-http-client-hint-reliability-02
4. BL should be rewritten as now user makes sometimes 2 requests.That's correct in that Critical-CH will result in 2 network round trips. There are two mitigations in place for this, with varying degrees of availability:(1) Use the ACCEPT_CH frame in the ALPS extension, so that the requested client hints are known at connection establishment time and thus, won't require an extra round trip. This is currently only implemented in BoringSSL (not yet in OpenSSL): https://chromestatus.com/feature/5555544540577792It'll take time till web servers will support it (I didn't find any mention of IIS supporting this). Also the 3rd party LBs, like AWS ALB, don't share their work plan in this direction. So it's currently couldn't be used in production.
(2) We recently launched a feature where the Accept-CH storage persists beyond browser restarts, so hopefully the Critical-CH header should take effect less (only the first navigation ever, as opposed to the first navigation after the browser restarts or user session otherwise ends): https://bugs.chromium.org/p/chromium/issues/detail?id=1296339. So hopefully this should improve client hints coverage such that restarts via Critical-CH are needed far less.Unfortunately I get access denied on this topic. From your description it doesn't solve the first visit of the user, whose load time we need to optimize. Will the Accept-CH persists between different incognito visits to the same domain? If the answer is that it will not persist then it's also an use case that is not covered by the proposal.
If you encounter any issues or have further feedback, please do file an open issue at crbug.new using the Blink>Network>ClientHints component.About CH in Chrome Dev Tools built-in device lists. They don't use CH at all, should I open it as a bug in the same tracker? It does allow to add custom devices based on CH. But I would expect that if CH is being promoted then the debugging tools would be ready for it too. Currently need to explain everyone how to manually add new devices and not to use the current list.
The alternative solution that was discussed by developers is to use redirects that are sent from the server for more data. Basically, the scheme stays as the above but the server sends 307, or other redirect code, with Accept-CH, so after the redirect the client adds the missing CH. The discussed drawback was that there could be an infinite loop if it’s implemented wrongly.
Do you mean the 307 hits the same page in the redirect, but this time it includes the client hints? That's effectively what the CriticalClientHintsThrottle is doing now. And there are safeguards in place in it to prevent infinite redirects: https://source.chromium.org/chromium/chromium/src/+/main:content/browser/client_hints/critical_client_hints_throttle.cc;l=91-92Almost, with 307 response the browsers that don't support CH will do the redirect. It's different from 200 OK response with Critical-CH and empty body, in such case Chrome will do another request, and other browsers would be stuck with empty page.
If a client would A. list supported client hints in the first request B. If the condition for second request are met (HTTPS, not iframe with invalid permission policy etc) only then the server would know how to respond correctly.The main idea behind client hints is to turn the user agent information into an active retrieval surface (the site has to explicitly request it). So on the very first communication with that site, it's not possible for the browser to know ahead of time. User agents could potentially create an off-band registration mechanism that would seed a file that the browser loads up, but management and update of that would be a big challenge and not entirely reliable on its own.
BTW if you raised the iframe topic. Is it enough for the page from the parent domain to set permission policy on the iframe without stating the target domain that this policy is given for? I'm seeing a very common scenario where parent domain A, embeds iframe with some Ad network from domain B, and when the page is rendered, the iframe, from domain B, redirects itself to the ad from domain C. That's how ad network works. And the site owner of domain A has no clue of domain C.The permissions policy does require specifying which origins the permissions are applied to: https://w3c.github.io/webappsec-permissions-policy/#examples
I read the spec, including https://datatracker.ietf.org/doc/html/draft-davidben-http-client-hint-reliability-02
4. BL should be rewritten as now user makes sometimes 2 requests.That's correct in that Critical-CH will result in 2 network round trips. There are two mitigations in place for this, with varying degrees of availability:(1) Use the ACCEPT_CH frame in the ALPS extension, so that the requested client hints are known at connection establishment time and thus, won't require an extra round trip. This is currently only implemented in BoringSSL (not yet in OpenSSL): https://chromestatus.com/feature/5555544540577792It'll take time till web servers will support it (I didn't find any mention of IIS supporting this). Also the 3rd party LBs, like AWS ALB, don't share their work plan in this direction. So it's currently couldn't be used in production.Agree, it'll be a while before this gets large scale adoption.(2) We recently launched a feature where the Accept-CH storage persists beyond browser restarts, so hopefully the Critical-CH header should take effect less (only the first navigation ever, as opposed to the first navigation after the browser restarts or user session otherwise ends): https://bugs.chromium.org/p/chromium/issues/detail?id=1296339. So hopefully this should improve client hints coverage such that restarts via Critical-CH are needed far less.Unfortunately I get access denied on this topic. From your description it doesn't solve the first visit of the user, whose load time we need to optimize. Will the Accept-CH persists between different incognito visits to the same domain? If the answer is that it will not persist then it's also an use case that is not covered by the proposal.Sorry, the bug should be open to you now. Correct, the very first visit by the user agent to that origin would require the extra round-trip if the Critical-CH header is specified.If you encounter any issues or have further feedback, please do file an open issue at crbug.new using the Blink>Network>ClientHints component.About CH in Chrome Dev Tools built-in device lists. They don't use CH at all, should I open it as a bug in the same tracker? It does allow to add custom devices based on CH. But I would expect that if CH is being promoted then the debugging tools would be ready for it too. Currently need to explain everyone how to manually add new devices and not to use the current list.Yes, please open any requests or bugs you find to the same component in crbug.The alternative solution that was discussed by developers is to use redirects that are sent from the server for more data. Basically, the scheme stays as the above but the server sends 307, or other redirect code, with Accept-CH, so after the redirect the client adds the missing CH. The discussed drawback was that there could be an infinite loop if it’s implemented wrongly.
Do you mean the 307 hits the same page in the redirect, but this time it includes the client hints? That's effectively what the CriticalClientHintsThrottle is doing now. And there are safeguards in place in it to prevent infinite redirects: https://source.chromium.org/chromium/chromium/src/+/main:content/browser/client_hints/critical_client_hints_throttle.cc;l=91-92Almost, with 307 response the browsers that don't support CH will do the redirect. It's different from 200 OK response with Critical-CH and empty body, in such case Chrome will do another request, and other browsers would be stuck with empty page.Are you saying that if the site sends Critical-CH and a 200 OK response, browsers that don't support client hints would show an empty page? Why wouldn't the browser just ignore the Critical-CH header if it doesn't support client hints? I might not be following exactly..
Dear Google,
Adding Matthew Hancox and David Verroken in their role as Monitoring Trustee of Google’s commitments with the CMA.
Google make the following statements in their May 2022 report in relation to this Intent to Deprecate and Freeze notice.
API / Technology |
Feedback Theme (Ranked by Prevalence) |
Questions and Concerns Summary |
Chrome Response |
User Agent Reduction |
Performance |
There are concerns about the latency of getting hints via Critical-CH (on the first page load). |
Chrome is investigating ways to improve performance. |
User-Agent Reduction / User-Agent Client Hints |
Anti-Fraud / Anti-Abuse concerns |
Having as much information as possible is important when debugging certain types of attacks, including Denial of Service. Losing some info from the UA string may pose challenges. |
Chrome is in discussions and evaluating ways to maintain privacy while providing sufficient information that will be useful for debugging. |
User Agent Reduction |
Confusion around OT setup |
Multiple Origin Trial participants recommended improving documentation with examples of how to enroll in the Origin Trial. |
The Reduced UA Origin Trial is ending, but Chrome intends to improve the instructions for the Deprecation Trial (including making the example demo more prominent). |
User Agent Reduction |
Concern about values of specific hint |
Questions around if the Sec-CH-UA-Model is the same as < deviceModel> in the User-Agent string. |
Sec-CH-UA-Model is the same as <deviceModel> in the User-Agent string. Chrome will try to make this more clear in future documentation. |
User-Agent Reduction |
Concern about enrolling in Deprecation Trial |
Questions around h ow to enroll a large number of domains into the Deprecation Trial. |
Chrome has considered centralized approaches when designing the Deprecation Trial, but Chrome believes the existing Origin Trial is the best option as it gives all control to developers ( since they can choose to s end the header or not). |
User-Agent Client Hints |
Concerns around prescriptive nature of UA-CH |
There is a concern that UA-CH is overly prescriptive when compared to the flexibility the User-Agent header offers, as de ned by rfc7231. |
Chrome sees the prescriptive nature of UA-CH headers as an important improvement over the flexibility of the UA string, both from the point of view of eventual cross-browser interoperability and user privacy protection (by preventing arbitrary additions of high-entropy identifiers). However the issue remains open in case others also share this concern and would like to provide feedback. |
User-Agent Client Hints |
Concerns that the API is being used to block certain browsers |
Concern that a site is using the API to look for “Google Chrome” or “Microsoft Edge” and blocking all other browsers. |
The concept of a brand list was designed to handle this case - a browser can send “Google Chrome” in addition to their own brands. |
User-Agent Client Hints |
Request for a method to enumerate all supported hints |
Interest in having a programmatic way to know all supported hints for a browser. |
Chrome is evaluating the feature request. |
User-Agent Reduction / User-Agent Client Hints |
Anti-Fraud / Anti-Abuse concerns |
Client hints are not available on first load for HTTP1 |
One of the Client Hints Reliability APIs (ACCEPT_CH) is only available over HTTP2 and H TTP3. For servers who are still served over HTTP1, they will need to rely solely on Critical-CH. |
User-Agent Reduction |
Impact on Chrome for Android |
Questions on how this impacts Chrome on Android in particular. |
UA Reduction as well as UA-CH will ship on Chrome on Android, in addition to Desktop. For Chrome on Android, the changes will only take place in “Phase 6”, currently scheduled for Chrome 110. |
Gnatcatcher + User-Agent Reduction |
Reducing signals for anti-fraud |
Anti-fraud impact of concurrently reducing IP and U A access. |
Expecting Willful IP Blindness anti-fraud policy stipulations (to allow use of I P for anti-fraud use cases) will resolve defensibility concerns around IP proxying. |
Latency
Google acknowledged Privacy Sandbox is part of their “Ad Systems” in the commitments to the CMA. User Agent Reduction and User Agent Client Hints are part of Privacy Sandbox.
Any delay in retrieving the UACH values will delay the population of the Open RTB Structured User Agent (SUA) data thus delaying the request for advertising. Those websites that are bundled with the web browser via defaults, or are well-known and visited frequently, are likely to benefit over those that are visited for the first time. Therefore the latency issue is material to the impact on advertising and the economics of the Open Web.
Complexity
Any increase in complexity associated with obtaining information (for example the permissions policy) will be easier for companies with more engineers to implement than companies with fewer engineers. The web became the web in part due to simplicity over alternatives. As companies with more engineers tend to be larger, complexity benefits the largest companies in markets, and becomes a “tax” for smaller companies.
Other Analysis of Report
Movement for an Open Web (MOW) provide further analysis of the full first quarter report here including the lack of information on training of Google employees concerning their obligations.
Need to Pause
There is now sufficient justification for Google to pause the deprecation of the User Agent as currently planned to enable the following to be achieved.
If Google are not minded to adopt the above please can you provide your justification or an alternative remedy? Given the current timeline advertised for reduction and deprecation the industry needs this guidance in advance of the next quarterly report in August 2022.
Regards,
James Rosewell
From: 'James Rosewell - 51Degrees' via blink-dev <blin...@chromium.org>
Sent: 15 January 2020 18:51
To: blink-dev <blin...@chromium.org>
Cc: foo...@chromium.org; aaro...@chromium.org; yoav...@chromium.org
Subject: Re: [blink-dev] Intent to Deprecate and Freeze: The User-Agent string
Hi Yoav,
Whilst the change may be sensible from an engineering perspective - the User-Agent string is not efficient - it's going to be a breaking change for many industries and services that don't operate to browser provider "dog year" timescales.
Any technology being "retired" or regulatory change would typically be acompanied with a consultaiton period and two years notice. Consider GDPR or mobile networks.
Here's some examples:
1. Programmatic advertising. It's many players - including Google - have spent years developing message formats that have the User-Agent embedded in them. Upgrading them to client hints will require a major version change. The last major version change to AdCom / OpenRTB was released in November 2019 after many years consultation.
2. Analytics solutions will need to rush through changes to support client hints. Their users will need to migrate their deployments otherwise false assumptions will be formed wasting effort and causing confusion.
3. Performance improvements that require instant knowledge of the user agent to minimise data and improve render time for certain devices would be compromised due to the handshake. Consider people living in India where average device profiles are very different to western europe or north america.
4. Any implementation that removes the informaiton provided from the majority, whilst enabling Google due to it's size, influence or breadth of services (play store, android, search), to become the defacto single source of information about browser, operating system and device model usage globally risks being anti competitive. The implementaiton details don't appear to be clear enough to form a conclusion on this.
If the core problem is "we are broadcasting a lot of information about our users, in clear text, to all servers" then a staged approach might be to strengthen the warning around non SSL secure web sites and thrid parties to increase users control and awareness. Such a change could be accompanied with user experience monitoring (I opt in) and the severity of the problem better understood. Adding an SSL certificate is a change many web site operators have already made or are planning to make.
I'm interested in the subject as my business (51Degrees) provides device detection services utilising User-Agent for web traffic. TAC and app keys are used for non web. We've come a long way from "sniffing" using machine learning and other techniques to support analytics, optimisation and problem diagnosis. Switching to client hints helps us from an engineering perspective. Rolling out the change to client hints - in parallel with User-Agent - to gather the extra evidence fields is a substantial change.
Regards,
James
--
You received this message because you are subscribed to a topic in the Google Groups "blink-dev" group.
To unsubscribe from this topic, visit
https://groups.google.com/a/chromium.org/d/topic/blink-dev/-2JIRNMWJ7s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
blink-dev+...@chromium.org.
To view this discussion on the web visit
https://groups.google.com/a/chromium.org/d/msgid/blink-dev/9c0ada91-754d-48f5-a76d-f0a9fc5363a5%40chromium.org.
On 10th and 11th January 2023, I accessed the top 10,000 websites accessed from the UK[1] using a domestic internet connection and the Chrome web browser. I recorded the following information from the response.
1. The presence of programmatic advertising.
2. Enablement of User Agent Client Hints (UA-CH) via the presence of the Accept-CH HTTP header.
Of the 10,000 websites accessed 4,392 utilise advertising of some form. Of these 46 have enabled UA-CH.
~1% of the top 4,392 advertising funded websites accessed from the UK are ready for User Agent Reduction as at 10th/11th January 2023.
From 1st February Demand Side Platforms (DSPs) that compete with Google will no longer receive the information they need to identify the model of device and inform their fraud detection algorithms from the top 99% of the websites that incorporate advertising of some kind and are accessed from the UK.
The spend from advertisers that would have previously been directed towards these websites and DSPs will likely be directed elsewhere including to Google’s competing advertising services. Thus the User Agent Reduction change will directly benefit Google financially to the detriment of competitors until these websites have implemented the necessary replacement technology - User Agent Client Hints. More work is needed to inform websites that incorporate advertising about the User Agent Reduction change prior to making further User Agent Reduction changes.
[1] A list provided by SEMRush on 10th January was used to establish the top 10k websites visited from the UK. https://www.semrush.com/analytics/ranks/rank/?db=uk