Remove browser and OS architecture from Firefox's User-Agent string?

[Chris Peterson]

Hello, UA string fans! I propose that Firefox's UA string on Windows and
Linux omit the browser and OS architectures to reduce UA fingerprinting
entropy (and save a few header bytes).

I have tentative thumbs up from some webcompat and privacy people, but
as there is no official module owner for User-Agent string at this time,
I wanted to get some wider feedback here.

The primary use case for including OS architecture (Win32, Win64) and
version in the UA string is to allow software download sites to offer
compatible software. A second use case I've heard is for websites to
workaround platform bugs or serve a page design to match the OS theme,
but I have not seen real examples of this.

The UA string on Windows and Linux currently reveals both the browser
and OS architectures:

* 32-bit Firefox on 32-bit OS: "Windows", "Linux i686" or armv7l
* 32-bit Firefox on 64-bit OS: "WOW64", "Linux i686 on x86_64"
* 64-bit Firefox on 64-bit OS: "Win64", "Linux x86_64" or aarch64

I propose that Win64 and WOW64 use the unadorned Windows UA already used
by Firefox on x86 and AArch64 Windows:

< "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
Firefox/66.0"
> "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"

And that Linux omit the OS architecture entirely (like Firefox on
Android or always spoof "i686" if an architecture token is needed for UA
parsing webcompat):

< "Mozilla/5.0 (X11; Linux i686 on x86_64; rv:66.0) Gecko/20100101
Firefox/66.0"
> "Mozilla/5.0 (X11; Linux; rv:66.0) Gecko/20100101 Firefox/66.0"

If a software download site doesn't see "Win64" or "WOW64" in the UA
string, it might serve a Win32 x86 executable to a Win64 OS user. The
x86 executable would still be compatible, just not optimal. (Firefox and
Chrome don't have this problem because they ship smart stub installers.)
Linux users are unlikely to download much software from sites that sniff
their OS architecture, so omitting it should be safe.

If a site *really* wants to detect the client's browser or OS
architecture, it can use Flash APIs or detect known differences between
Firefox's 32-bit and 64-bit JavaScript Math functions [1].

If we think allowing software download sites to sniff and serve Win64
executables is important, we can include the OS architecture but omit
the browser architecture (i.e. report "Win64" instead of "WOW64"). Sites
don't need to know whether the user is running a x86 or x86_64 browser
on their Win64 OS.

I propose no change to the macOS UA string at this time. Removing
"Intel" now would not reduce any fingerprinting entropy (all modern Macs
are x86_64) and might risk confusing some UA string parsers. If AArch64
MacBooks become a real platform, I propose we then remove "Intel" so
x86_64 and AArch64 macOS would have the same UA string:

< "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101
Firefox/66.0"
> "Mozilla/5.0 (Macintosh; Mac OS X 10.14; rv:66.0) Gecko/20100101
Firefox/66.0".

Here is a spreadsheet comparing UA strings of different browser and OS
architectures:

https://docs.google.com/spreadsheets/d/1I--o6uYWUkBw05IP964Ee2aZCf67P9E3TxpuDawH4_I/edit#gid=0


[1] https://github.com/ghacksuserjs/ghacks-user.js/issues/657

[j.j.]

Am Freitag, 10. Mai 2019 22:37:51 UTC+2 schrieb Chris Peterson:
> Hello, UA string fans! I propose that Firefox's UA string on Windows and
> Linux omit the browser and OS architectures to reduce UA fingerprinting
> entropy (and save a few header bytes).

...

> < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
> Firefox/66.0"
> > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"

Note that "navigator.oscpu" returns "Windows NT 6.1; Win64; x64" or similar. This needs to change then.

[Henri Sivonen]

On Fri, May 10, 2019 at 11:40 PM Chris Peterson <cpet...@mozilla.com> wrote:
> I propose that Win64 and WOW64 use the unadorned Windows UA already used
> by Firefox on x86 and AArch64 Windows:
>
> < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
> Firefox/66.0"
> > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"

Would there be significant downsides to hard-coding the Windows
version to "10.0" in order to put Windows 7 and 8.x users in the same
anonymity set with Windows 10 users?

(We could still publish statistics of Windows version adoption at
https://data.firefox.com/dashboard/hardware )

> And that Linux omit the OS architecture entirely (like Firefox on
> Android or always spoof "i686" if an architecture token is needed for UA
> parsing webcompat):

Do we have any anecdata of the Web compat impact of not having
anything between "Linux" and the next semicolon? Is there any evidence
that " i686" would be a better single filler for everyone than "
x86_64" if something is needed there for Web compat?

Do we have indications if "Linux" is needed for Web compat? According
to https://docs.google.com/spreadsheets/d/1I--o6uYWUkBw05IP964Ee2aZCf67P9E3TxpuDawH4_I/edit#gid=0
FreeBSD currently does not say "Linux". (Chrome on Chrome OS does not
say Linux, either, but does say "X11; ".) That is, could "X11; " alone
be sufficient for Web compat? (I'm happy to see that running Firefox
in Wayland mode still says "X11; ". Let's keep it that way!)

Do we have an idea if distros would counteract Mozilla and restore the
CPU architecture if we removed it? Previous evidence suggests that
distros are willing to split the anonymity set for self-promotional
reasons by adding "; Ubuntu" or "; Fedora". Is there a similar distro
interest in exposing the CPU architecture?

https://docs.google.com/spreadsheets/d/1I--o6uYWUkBw05IP964Ee2aZCf67P9E3TxpuDawH4_I/edit#gid=0
suggests making Firefox on FreeBSD say "Linux". Are there indications
that the self-promotion interests of FreeBSD wouldn't override privacy
or Web compat benefits of saying "Linux"?

> I propose no change to the macOS UA string at this time. Removing
> "Intel" now would not reduce any fingerprinting entropy (all modern Macs
> are x86_64) and might risk confusing some UA string parsers. If AArch64
> MacBooks become a real platform, I propose we then remove "Intel" so
> x86_64 and AArch64 macOS would have the same UA string:
>
> < "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101
> Firefox/66.0"
> > "Mozilla/5.0 (Macintosh; Mac OS X 10.14; rv:66.0) Gecko/20100101
> Firefox/66.0".

Or they could have the same UA string by Aarch64 saying "Intel"...

Meanwhile, could we make the system version number "10.14" (or
whatever is latest at a given point in time) regardless of actual
version number to put all macOS users in the same anonymity set?
(Curiously, despite Apple's privacy efforts, Safari exposes the third
component of the OS version number. Also, it uses underscores instead
of periods as the separator.)

> Here is a spreadsheet comparing UA strings of different browser and OS
> architectures:
>
> https://docs.google.com/spreadsheets/d/1I--o6uYWUkBw05IP964Ee2aZCf67P9E3TxpuDawH4_I/edit#gid=0

The reference there to
https://bugzilla.mozilla.org/show_bug.cgi?id=1169772 about exposing
_some_ Android version number for Web compat says the reason not to
make Firefox claim the same Android version for all users regardless
of actual system version is that doing so would require bumping the
version later:
https://bugzilla.mozilla.org/show_bug.cgi?id=1169772#c36

It seems that for privacy reasons, we should claim the latest Android
version for everyone even if it means introducing the recurring task
of incrementing the number annually or so.

--
Henri Sivonen
hsiv...@mozilla.com

[Chris Peterson]

On 5/11/2019 4:11 AM, j.j. wrote:
>> < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
>> Firefox/66.0"
>> > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"
> Note that "navigator.oscpu" returns "Windows NT 6.1; Win64; x64" or similar. This needs to change then.
>

Yes. navigator.oscpu and the UA string share some common code, so they
would both be fixed to match 32-bit Windows.

btw, navigator.platform has already been fixed to return "Win32" for
Win32, WOW64, and Win64 Firefox builds (in bug 1472618).

navigator.platform returns:

Browser | Win32 OS | Win64 OS
-----------+----------+---------
IE11 | "Win32" | "Win64"
Chrome | "Win32" | "Win32"
Edge | "Win32" | "Win32" (changed to match Chrome)
Firefox 62 | "Win32" | "Win64"
Firefox 63 | "Win32" | "Win32" (changed in bug 1472618)

[L. David Baron]

On Monday 2019-05-13 16:14 -0700, Chris Peterson wrote:
> On 5/11/2019 4:11 AM, j.j. wrote:
> > > < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
> > > Firefox/66.0"
> > > > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"
> > Note that "navigator.oscpu" returns "Windows NT 6.1; Win64; x64" or similar. This needs to change then.
> >
>
> Yes. navigator.oscpu and the UA string share some common code, so they would
> both be fixed to match 32-bit Windows.

I think it might be worth considering letting them diverge.

I'm skeptical of the idea that we can remove the ability to detect
things like 32-bit versus 64-bit from the overall fingerprinting
surface. It seems like these should be detectable through things
like performance characteristics, if not through behavior
differences (like a Math one you mentioned earlier in the thread).
Likewise for some of the other differences here -- although I'd be
interested to see an argument that we actually can prevent them from
being detected.

However, there's another distinction worth considering, which is
passive fingerprinting versus active fingerprinting. The UA string
allows passive fingerprinting -- fingerprinting that isn't possible
to detect by looking at the HTML, CSS, and JS that was sent over the
wire. The attack surface for passive fingerprinting is small enough
that it seems like something that we can reasonably work to reduce.
Given the set of APIs already on the web, it's not clear whether we
can prevent users from being identified through active
fingerprinting without breaking significant web functionality.

So I think there's may be value in removing these distinctions from
the User-Agent header we send over HTTP even if they're still
accessible from Javascript (and useful there for sites offering
downloads).

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla https://www.mozilla.org/ 𝄂
Before I built a wall I'd ask to know
What I was walling in or walling out,
And to whom I was like to give offense.
- Robert Frost, Mending Wall (1914)

[Tom Ritter]

On Tue, May 14, 2019 at 4:26 PM L. David Baron <dba...@dbaron.org> wrote:
> So I think there's may be value in removing these distinctions from
> the User-Agent header we send over HTTP even if they're still
> accessible from Javascript (and useful there for sites offering
> downloads).

While I would prefer to remove the distinction everywhere; Tor Browser
came to this same conclusion. They send the same OS for all platforms
in the HTTP Header (partially because Web Logs so commonly record this
and partially in principal) but they report the correct OS via
javascript (because we found issues with compatibility (primarily
keyboard shortcuts in things like gdocs.)

-tom

[j.j.]

Am Dienstag, 14. Mai 2019 18:26:49 UTC+2 schrieb L. David Baron:
> On Monday 2019-05-13 16:14 -0700, Chris Peterson wrote:
> > On 5/11/2019 4:11 AM, j.j. wrote:
> > > > < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
> > > > Firefox/66.0"
> > > > > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"
> > > Note that "navigator.oscpu" returns "Windows NT 6.1; Win64; x64" or similar. This needs to change then.
> > >
> >
> > Yes. navigator.oscpu and the UA string share some common code, so they would
> > both be fixed to match 32-bit Windows.
>
> I think it might be worth considering letting them diverge.

Having more js information in "navigator.oscpu" than in "navigator.userAgent" could animate web developers using oscpu while in an ideal world it should be dropped completely.
It's Mozilla only and endet up in the HTML spec (Gecko navigator compatibility mode) only for web compat concerns (browser detection).
(BTW, chancing oscpu string requires a spec change too)

[Mike Taylor]

That's interesting -- do you have links to bugs?

thanks,

--
Mike Taylor
Web Compat, Mozilla

[Tom Ritter]

In reverse chronological order:
https://bugzilla.mozilla.org/show_bug.cgi?id=1509829 is where I
uplifted Tor's behavior
https://trac.torproject.org/projects/tor/ticket/26146 is where Tor
decided on the behavior to split header and navigator
https://bugzilla.mozilla.org/show_bug.cgi?id=1404608 is where we
decided we needed to not spoof the OS at all in header or navigator
https://bugzilla.mozilla.org/show_bug.cgi?id=1405810 is where we
discovered the modifier key shortcut problem

[Chris Peterson]

On 5/11/2019 8:59 AM, Henri Sivonen wrote:
>>> "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"
>
> Would there be significant downsides to hard-coding the Windows
> version to "10.0" in order to put Windows 7 and 8.x users in the same
> anonymity set with Windows 10 users?

...
> Meanwhile, could we make the system version number "10.14" (or
> whatever is latest at a given point in time) regardless of actual
> version number to put all macOS users in the same anonymity set?
> (Curiously, despite Apple's privacy efforts, Safari exposes the third
> component of the OS version number. Also, it uses underscores instead
> of periods as the separator.)

Firefox spoofs the latest OS versions when the
privacy.resistFingerprinting pref is enabled (e.g. in Tor). I think
always spoofing the OS version is worth considering as a follow up task.
I'd like to limit the scope of this initial proposal just to CPU
architecture.

> It seems that for privacy reasons, we should claim the latest Android
> version for everyone even if it means introducing the recurring task
> of incrementing the number annually or so.

We've already bumped privacy.resistFingerprinting's spoofed OS versions
for new ESR versions, so this wouldn't be a burden:

https://bugzilla.mozilla.org/show_bug.cgi?id=1511434

> Do we have indications if "Linux" is needed for Web compat? According
> to https://docs.google.com/spreadsheets/d/1I--o6uYWUkBw05IP964Ee2aZCf67P9E3TxpuDawH4_I/edit#gid=0
> FreeBSD currently does not say "Linux". (Chrome on Chrome OS does not
> say Linux, either, but does say "X11; ".) That is, could "X11; " alone
> be sufficient for Web compat? (I'm happy to see that running Firefox
> in Wayland mode still says "X11; ". Let's keep it that way!)

I don't know. Trimming the OS is worth considering as a follow up task.

> Do we have an idea if distros would counteract Mozilla and restore the
> CPU architecture if we removed it? Previous evidence suggests that
> distros are willing to split the anonymity set for self-promotional
> reasons by adding "; Ubuntu" or "; Fedora". Is there a similar distro
> interest in exposing the CPU architecture?

I can try to contact some distro representatives and ask about them
exposing the CPU architecture.

> https://docs.google.com/spreadsheets/d/1I--o6uYWUkBw05IP964Ee2aZCf67P9E3TxpuDawH4_I/edit#gid=0
> suggests making Firefox on FreeBSD say "Linux". Are there indications
> that the self-promotion interests of FreeBSD wouldn't override privacy
> or Web compat benefits of saying "Linux"?

We can probably delegate that decision to the FreeBSD developers. I
shouldn't have included it because it's beyond the scope of my proposal
to remove CPU architecture.

>> I propose no change to the macOS UA string at this time. Removing
>> "Intel" now would not reduce any fingerprinting entropy (all modern Macs
>> are x86_64) and might risk confusing some UA string parsers. If AArch64
>> MacBooks become a real platform, I propose we then remove "Intel" so
>> x86_64 and AArch64 macOS would have the same UA string:
>>
>> < "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101
>> Firefox/66.0"
>> > "Mozilla/5.0 (Macintosh; Mac OS X 10.14; rv:66.0) Gecko/20100101
>> Firefox/66.0".
>
> Or they could have the same UA string by Aarch64 saying "Intel"...

I see that iOS Safari's UA reports "CPU iPhone" where macOS Safari
reports "Intel Mac OS X". Presumably Apple thought a placeholder UA
token (like "CPU") was needed instead of just "iPhone". I assume Apple
would use the same "CPU" placeholder for Safari on a hypothetical
AArch64 macOS. In the meantime, there's little value in removing "Intel".

[Chris Peterson]

Distinguishing between active and passive fingerprinting is a good point.

Gecko could omit platform architecture from the User-Agent header but
keep it in the navigator.userAgent API. Is returning two different UA
strings (like Tor) preferable to keeping the Gecko-specific
navigator.oscpu API?

I'm not convinced of a legitimate use case for web content to simply be
handed the platform architecture on Linux, Android, or macOS. Web
content could still try to detect performance characteristics, but the
detection methods might not work reliably on all CPU models,
architectures, and operating systems. For example, the ghacks detection
testing [1] could distinguish Win32 and Win64 Firefox builds, but not
Win32 or Win64 OS.

The HTML spec says [1] Gecko's navigator.oscpu may return an empty
string or "a string representing the platform". We could limit
navigator.oscpu to return:

* empty string "" on Linux, Android, and macOS
* "Windows NT 10.0" for Win32 Firefox (like today)
* "Windows NT 10.0; Win64; x64" for Win64 (like today) and WOW64 Firefox
(instead of "Windows NT 10.0; WOW64")

and still adhere to the HTML spec. We could then return the same UA
string in the User-Agent header and navigator.userAgent API while still
giving websites an easy way to distinguish Win32 and Win64 OS using
navigator.oscpu.


[1] https://github.com/ghacksuserjs/ghacks-user.js/issues/657
[2]
https://html.spec.whatwg.org/multipage/system-state.html#dom-navigator-oscpu

[Chris Peterson]

On 5/14/2019 9:53 AM, Tom Ritter wrote:

[Randell Jesup]

>On Fri, May 10, 2019 at 11:40 PM Chris Peterson <cpet...@mozilla.com> wrote:
>> I propose that Win64 and WOW64 use the unadorned Windows UA already used
>> by Firefox on x86 and AArch64 Windows:
>>
>> < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
>> Firefox/66.0"
>> > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"
>
>Would there be significant downsides to hard-coding the Windows
>version to "10.0" in order to put Windows 7 and 8.x users in the same
>anonymity set with Windows 10 users?
>
>(We could still publish statistics of Windows version adoption at
>https://data.firefox.com/dashboard/hardware )

I wonder if any sites distributing windows executables might key off the
OS version to default to the correct exe for your version of windows?

If we do this, one would presume that such sites let you select
alternate versions to download already. And if not, they could, but
they might not bother/notice.

--
Randell Jesup, Mozilla Corp
remove "news" for personal email

[Henri Sivonen]

On Tue, May 21, 2019 at 12:40 AM Randell Jesup <rjesu...@jesup.org> wrote:
>
> >On Fri, May 10, 2019 at 11:40 PM Chris Peterson <cpet...@mozilla.com> wrote:
> >> I propose that Win64 and WOW64 use the unadorned Windows UA already used
> >> by Firefox on x86 and AArch64 Windows:
> >>
> >> < "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101
> >> Firefox/66.0"
> >> > "Mozilla/5.0 (Windows NT 10.0; rv:66.0) Gecko/20100101 Firefox/66.0"
> >
> >Would there be significant downsides to hard-coding the Windows
> >version to "10.0" in order to put Windows 7 and 8.x users in the same
> >anonymity set with Windows 10 users?
> >
> >(We could still publish statistics of Windows version adoption at
> >https://data.firefox.com/dashboard/hardware )
>
> I wonder if any sites distributing windows executables might key off the
> OS version to default to the correct exe for your version of windows?

Are there known examples of apps like that? AFAICT, even Skype.com
provides the Windows 7 -compatible .exe as the download even with a
Windows 10 UA string, and you need to know to go to the Store if you
want the Windows 10-only version.

--
Henri Sivonen
hsiv...@mozilla.com

[Mike Taylor]

On 5/21/19 12:45 AM, Henri Sivonen wrote:
> On Tue, May 21, 2019 at 12:40 AM Randell Jesup <rjesu...@jesup.org> wrote:

>> I wonder if any sites distributing windows executables might key off the
>> OS version to default to the correct exe for your version of windows?
>

> Are there known examples of apps like that? AFAICT, even Skype.com
> provides the Windows 7 -compatible .exe as the download even with a
> Windows 10 UA string, and you need to know to go to the Store if you
> want the Windows 10-only version.

A while back I tried to hit up a bunch of software download sites and
wasn't able to find anything to answer this question. At best, I think
the answer is maybe? I wonder if PI could help us with this type of thing.