Intent to Deprecate & Remove: Android build number in user-agent identification

[Thiemo Nagel]

# Primary eng (and PM?) emails

tna...@chromium.org, to...@chromium.org, shai...@google.com

# Summary

Remove the Android build number (e.g. “NJH47F” or “OPM4.171019.021.D1”) from the user-agent identification, i.e. the User-Agent header and navigator.userAgent. – Apply to embedders but postpone changing WebView until the Android Compatibility Definition Document (CDD) has been updated which currently mandates the status quo.

# Motivation

The build number may be used for exploit targeting and on some devices (e.g. Pixel phones) the build number is a relevant fingerprinting surface.

Also to bring Chrome closer in line with RFC 7231 section 5.5.3 which stipulates with regards to the User-Agent header: “[...] SHOULD limit generated product identifiers to what is necessary to identify the product [...] SHOULD NOT generate a User-Agent field containing needlessly fine-grained detail [...]”

# Interoperability and Compatibility Risk

The format of the user-agent strings varies between browsers and consumers of the string often turn to heuristics to parse it which can be brittle at times. Removing a token from the string only seems to incur a low risk, though. To mitigate that risk, a kill switch is to be implemented which is intended to be used only in the face of widespread breakage.

Firefox doesn’t include OS build numbers.

Safari Mobile does include the iOS build number as part of the “Mobile” token. (Chrome on iOS emulates Safari and should probably keep doing so to prevent compatibility issues. iOS is outside the scope of this intent, though, as it’s not a Blink platform.)

Edge doesn’t include the Windows build number. (Neither does Chrome on Windows.)

# Alternative implementation suggestion for web developers

None. Developers that really need access to the OS build number probably shouldn’t target the web platform but write native code instead.

# Usage information from UseCounter

Header usage cannot be measured and there is no UseCounter data for navigator.userAgent. Likely most servers parse the user-agent string in one way or another but the build number field probably doesn’t see a lot of use (apart from fingerprinting).

# Entry on the feature dashboard

https://www.chromestatus.com/feature/4558585463832576

# Requesting approval to remove too?

Yes.

--

Thiemo Nagel

Software Engineer

Google Germany GmbH, Erika-Mann-Straße 33, 80686 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

[Yoav Weiss]

Removing privacy sensitive data from the UA string makes sense. 

On Mon, Jul 2, 2018 at 7:43 PM Thiemo Nagel <tna...@chromium.org> wrote:

# Primary eng (and PM?) emails

tna...@chromium.org, to...@chromium.org, shai...@google.com

# Summary

Remove the Android build number (e.g. “NJH47F” or “OPM4.171019.021.D1”) from the user-agent identification, i.e. the User-Agent header and navigator.userAgent. – Apply to embedders but postpone changing WebView until the Android Compatibility Definition Document (CDD) has been updated which currently mandates the status quo.

# Motivation

The build number may be used for exploit targeting and on some devices (e.g. Pixel phones) the build number is a relevant fingerprinting surface.

Is the "exploit targeting" issue restricted? I cannot access it... If it is, is there a non-sensitive writeup that can be shared on the subject? Can you expand on the fingerprinting concerns?

 

Also to bring Chrome closer in line with RFC 7231 section 5.5.3 which stipulates with regards to the User-Agent header: “[...] SHOULD limit generated product identifiers to what is necessary to identify the product [...] SHOULD NOT generate a User-Agent field containing needlessly fine-grained detail [...]”

# Interoperability and Compatibility Risk

The format of the user-agent strings varies between browsers and consumers of the string often turn to heuristics to parse it which can be brittle at times. Removing a token from the string only seems to incur a low risk, though. To mitigate that risk, a kill switch is to be implemented which is intended to be used only in the face of widespread breakage.

Firefox doesn’t include OS build numbers.

Safari Mobile does include the iOS build number as part of the “Mobile” token. (Chrome on iOS emulates Safari and should probably keep doing so to prevent compatibility issues. iOS is outside the scope of this intent, though, as it’s not a Blink platform.)

Edge doesn’t include the Windows build number. (Neither does Chrome on Windows.)

# Alternative implementation suggestion for web developers

None. Developers that really need access to the OS build number probably shouldn’t target the web platform but write native code instead.

Are there any known legitimate use-cases? Working around bugs, etc? I'm assuming those are all covered by the browser version (rather than the OS), but wondering if there are such workarounds I'm not aware of

 

# Usage information from UseCounter

Header usage cannot be measured and there is no UseCounter data for navigator.userAgent. Likely most servers parse the user-agent string in one way or another but the build number field probably doesn’t see a lot of use (apart from fingerprinting).

# Entry on the feature dashboard

https://www.chromestatus.com/feature/4558585463832576

# Requesting approval to remove too?

Yes.

--

Thiemo Nagel

Software Engineer

Google Germany GmbH, Erika-Mann-Straße 33, 80686 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAC%3DO2%2BRD2DtDhq%2B9%3DLdtaH%3DhcNns8PHazHY4m1VoGhvAV8_SfA%40mail.gmail.com.

[Mike West]

LGTM1.

Folks targeting specific feature sets can reasonably rely upon Chrome's version number to give them the detail they want to use server-side. The Android build number falls pretty clearly on the side of unnecessary information, and removing is seems quite reasonable to me.

-mike

On Tue, Jul 3, 2018 at 8:34 AM Yoav Weiss <yo...@yoav.ws> wrote:

Removing privacy sensitive data from the UA string makes sense. 

On Mon, Jul 2, 2018 at 7:43 PM Thiemo Nagel <tna...@chromium.org> wrote:

# Primary eng (and PM?) emails

tna...@chromium.org, to...@chromium.org, shai...@google.com

# Summary

Remove the Android build number (e.g. “NJH47F” or “OPM4.171019.021.D1”) from the user-agent identification, i.e. the User-Agent header and navigator.userAgent. – Apply to embedders but postpone changing WebView until the Android Compatibility Definition Document (CDD) has been updated which currently mandates the status quo.

# Motivation

The build number may be used for exploit targeting and on some devices (e.g. Pixel phones) the build number is a relevant fingerprinting surface.

Is the "exploit targeting" issue restricted? I cannot access it... If it is, is there a non-sensitive writeup that can be shared on the subject?

The issue is restricted. It boils down to what you'd expect: knowing the exact build information for a given requestor makes it possible to target just those users with an attack that doesn't expose the attacker to broad scrutiny. Stagefright is one earlier example. I don't know of any floating around at the moment, but who knows what projectzero will cook up next?

 

Can you expand on the fingerprinting concerns?

They seem straightforward? More bits in the UA string === more entropy to build a fingerprint on top of.

[Yoav Weiss]

So just to better understand what this intent proposes: Looking at Android UA string documentation, current strings are marked as

Mozilla/5.0 (Linux; <Android Version>; <Build Tag etc.>) AppleWebKit/<WebKit Rev>(KHTML, like Gecko) Chrome/<Chrome Rev> Mobile Safari/<WebKit Rev>

We're talking about removing the "Build Tag etc" piece, right? Does that mean that the device hardware name will also be removed?

If so, that is something that has a legitimate use-case for: figuring out the device viewport dimensions and DPR, for image compression purposes.

There's also a shipped replacement for that use-case: Client Hints.

[Mike West]

On Tue, Jul 3, 2018 at 10:10 AM Yoav Weiss <yo...@yoav.ws> wrote:

So just to better understand what this intent proposes: Looking at Android UA string documentation, current strings are marked as

Mozilla/5.0 (Linux; <Android Version>; <Build Tag etc.>) AppleWebKit/<WebKit Rev>(KHTML, like Gecko) Chrome/<Chrome Rev> Mobile Safari/<WebKit Rev>

We're talking about removing the "Build Tag etc" piece, right? Does that mean that the device hardware name will also be removed?

If so, that is something that has a legitimate use-case for: figuring out the device viewport dimensions and DPR, for image compression purposes.

There's also a shipped replacement for that use-case: Client Hints.

As you note, Client Hints addresses exactly this use case. UA sniffing is both unnecessary and unreliable. I'd suggest that removing the build information from the UA string will provide incentives to move to that better-supported option.

[PhistucK]

Are Client Hints sent on the first request to a website? If so, they are indeed a good alternative, but if not, then this use case is missing (whether it is worth it or not is another question).

If the build tag is the subject here and it also includes the device information, then the name of this intent ("build number") is a bit misleading since the scope might be larger (I do not know whether the actual build number/ID alone can identify a device, but the entire build tag probably can).

It always looked weird to me that Chrome exposes so much information on the device in the user agent string, so I think this is a good idea.

☆PhistucK

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3DdxL6mYYUk6inxm9Lgbf%3D4KuamZDTibW6diHOwFE7kdAQ%40mail.gmail.com.

[Daniel Bratell]

I see nothing about deprecating so I assume the request is to remove the code directly. As this string is parsed server side it is hard to know how common it is, but some have had success looking in github. Both for server side scripts and for javascript looking at the navigator.useragent string so I would recommend you to take a look there.

I agree that it's a good thing removing this information from the user agent string, but I'm not sure what the risks are. User Agent parsing is traditionally hacky[1] and we can be sure some script somewhere will break. At the very least we can give web developers some time to comment.

/Daniel

[1] When Opera went from version 9 to 10, some user agent parsers started treating it as Opera 1 since they had just extracted the first digit. The solution: Opera had to stay forever young at version 9.80 in the user agent string.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CABc02_L_7SUmsbThwXU5%3DoM9ZkRMzHaOv5f8tiWR6xZJkaMgpA%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[Torne (Richard Coles)]

An example would be helpful here: a current useragent for an out-of-the-box Pixel looks like

"Mozilla/5.0 (Linux; Android 8.1.0; Pixel Build/OPM4.171019.021.D1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.98 Mobile Safari/537.36"

This proposal would remove "Build/OPM4.171019.021.D1", leaving:

"Mozilla/5.0 (Linux; Android 8.1.0; Pixel) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.98 Mobile Safari/537.36"

We are *not* currently proposing removing the device model from the UA.

[Chris Harrelson]

LGTM2

I agree that the reasons of security and privacy outweigh concerns about compat in this case, especially since there should be no good reason to target at this level of specificity. In addition, it is very hard to know exactly what might break with this change, and it would be impossible to even provide a devtools warning message to help developers migrate their code.

Please come back to this thread if you see any real sites broken by this change.

Please also update the chromestatus entry with the example you gave, to help developers understand the change should they encounter it.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAEV-rjdmd1LM7tKqbcq%3DHA0iswurib0G9draJUMN8KA69n8e2w%40mail.gmail.com.

[Thiemo Nagel]

Hey Daniel,

 

I see nothing about deprecating so I assume the request is to remove the code directly.

Yes. As Chris wrote, it's hard to come up with a targeted deprecation notice. Probably a majority of sites parse the user agent string for some thing or another, but I'd expect very few sites making (legit) use of the build number. My plan would be to roll out via the natural release channel progression, and to pause the rollout in case there's reports of relevant breakage. Do you have any thoughts about ways to reach out to developers?

(I've updated chromestatus with Torne's samples.)

[Thiemo Nagel]

I've updated chromestatus:

* Confirmed that Mobile Safari has frozen their UA string and doesn't report a real build number anymore (starting with iOS 11.3).

* Created a non-restricted tracking bug. (Sorry for linking to a restricted bug in the first place.)

[Daniel Bratell]

Ah, so the most valuable information will still be there. That reduces the risk it will break anything. Still, it's risky so be open to feedback about it breaking sites.

LGTM3

/Daniel

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAC%3DO2%2BTCSzXA0ngL9CavO%2BUXMqbwxPRCbCJMH4ZEMcGAbiR60Q%40mail.gmail.com.

[Thiemo Nagel]

Courtesy of mgalonsky, this is now live on clank canary (M-70). :)

[Thiemo Nagel]

To close the loop: After the change being life for ~6 weeks I found ~20 mentions in news/blogs but no reports of breakage.

[Daniel Bratell]

I know of one problem: Facebook started sending a low resolution logo at times. Probably a server side script that tried to figure out device capabilities from the user agent that failed. 

/Daniel

--

/* Opera Software, Linköping, Sweden: CET (UTC+1) */

[Thiemo Nagel]

Thanks Daniel! Do you have any pointers? Has the problem been fixed?

[Daniel Bratell]

I don't know, but it was detected just ~2 weeks ago so it may still be the case. The symptom is that the logotype is slightly blurry and a used might very well think that is intentional.

To quote an internal bug "Tried in opera desktop with devtools. Old user agent with Nexus 5X viewport size gives us the higher resolution image. New user agent gives us a lower resolution image. Adding Build/10.7.A.228 to the new user agent makes facebook give us a higher resolution image.". It was also reproducible in Chrome. I assumed that Facebook would notice and fix themselves so I didn't think more about it.

/Daniel