Cisco's OpenH264 binary blob is downloaded without prompting the user

[Luke Bratch]

Hi

Bug 1100304 [1] regarding the Cisco OpenH264 binary blob being
downloaded without user prompt has been closed and dismissed. People
interested in pursuing the matter have been directed to this list.

Most issues have been pointed out in the Bugzilla entry already, but to
summarise for this list:

1. An unverified binary blob being downloaded and marked as executable
is not OK, *especially without the user being informed and/or asked*.

2. The initial Mozilla blog post [2] on the subject stated that the blob
would be downloaded "when needed", whereas in fact it happens
automatically upon browser load.

3. There isn't any way to verify the blob by any documented reproducible
build method that I am aware of.

4. The download even happens in safe mode.

5. The behaviour of point 2 means that running "cfx run" (for instance
when developing extensions) means that the blob is downloaded over and
over again, consuming significant network resources.

6. The preferences to disable the downloading are numerous (there seem
to be at least three required) and hidden in about:config.

7. The only way to set those preferences by default requires local
administrative access. (And may disappear in the future!? [3])

Please note - I'm not after a point-by-point debate regarding the above
points, I only listed them all to convey the breadth of issues to people
who haven't encountered the bug report yet.

What I'm really after is some reassurance that Mozilla does recognise
the issues introduced by this binary blob. I refer to both the ethical
issues (of embracing H.264 and of distributing untrusted binaries to
Firefox users) and the practical issues (of prompting/informing users in
advance and offering an easier way to disable).

In the same vein, my understanding of Mozilla's stance on plugins has
been muddied by the introduction of GMP. The introduction of both
plugin blocklists and automatic Click to Play were signs to me that
Mozilla were moving away from plugins, but this new plugin framework
seems like a regression back towards embracing plugins.

I accept that Bugzilla is not a productive place to continue the debate,
so any input from this list would be appreciated.

Thanks
Luke Bratch


[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1100304
[2]
https://blog.mozilla.org/blog/2013/10/30/video-interoperability-on-the-web-gets-a-boost-from-ciscos-h-264-codec/
[3]
https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/A_brief_guide_to_Mozilla_preferences#Changing_defaults
_______________________________________________
firefox-dev mailing list
firef...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev

[Gervase Markham]

On 23/12/14 12:54, Luke Bratch wrote:
> 1. An unverified binary blob being downloaded and marked as executable
> is not OK, *especially without the user being informed and/or asked*.

What exactly do you mean by "unverified"?

I very much hope that there are mechanisms by which the blob is
confirmed to be a genuine one (either by hash or signing or pinned
HTTPS). If there are not, that's a bug. Can anyone confirm either way?

The aim is to provide reproducible (deterministic) builds, so that it
will no longer be unverified that the blob you get with the magic Cisco
patent license pixie dust is identical to the one you can build yourself.

This was part of the original promise of how this would work, so I hope
it has not been forgotten.

I can't immediately find a bug number for this work: does anyone have one?

> 2. The initial Mozilla blog post [2] on the subject stated that the blob
> would be downloaded "when needed", whereas in fact it happens
> automatically upon browser load.

Would you be significantly happier if it were downloaded the first time
a user hit a WebRTC-using page?

> 4. The download even happens in safe mode.

Hmm. That does sound like a bug, or at least arguably a wrong behaviour.
Would you care to file it?

> 5. The behaviour of point 2 means that running "cfx run" (for instance
> when developing extensions) means that the blob is downloaded over and
> over again, consuming significant network resources.

That also sounds like a bug which should be filed.

> 6. The preferences to disable the downloading are numerous (there seem
> to be at least three required) and hidden in about:config.

Do you have more specific information?

> 7. The only way to set those preferences by default requires local
> administrative access. (And may disappear in the future!? [3])

The proper method used by admins isn't marked as "may be removed".

> Please note - I'm not after a point-by-point debate regarding the above
> points, I only listed them all to convey the breadth of issues to people
> who haven't encountered the bug report yet.

But one bug report cannot deal with such a breadth of issues, and a bug
report which says "this has problems, back it out" will quite rightly be
WONTFIXed. If the individual issues concern you (and some certainly
should) then please file bugs on them. If your actual aim is to get
Mozilla's decision to support H.264 reversed, then I'm afraid that's not
going to happen. The market forces which led to this sad decision are
still exactly the same.

> What I'm really after is some reassurance that Mozilla does recognise
> the issues introduced by this binary blob. I refer to both the ethical
> issues (of embracing H.264 and of distributing untrusted binaries to
> Firefox users) and the practical issues (of prompting/informing users in
> advance and offering an easier way to disable).

You don't think the acres of anguished debate at the time the decision
was taken are evidence enough that Mozilla is not particularly happy
about its need to support H.264? (I don't think "embrace" is a fair word.)

It also depends what you mean by "untrusted" binaries. People who
download Firefox have to trust Mozilla. Now, if there is opportunity for
the OpenH264 blob to be interfered with between our build servers and
you, that's clearly a bug which should be fixed. If not, then it's as
trusted as the Firefox binary the user downloaded in the first place.

If you actually mean "not open source", that's a different question with
a different answer. The OpenH264 code is open source. Unless your
position is that no-one should ever use any patented technology. We
don't like software patents, but we live in a world where they exist.

> In the same vein, my understanding of Mozilla's stance on plugins has
> been muddied by the introduction of GMP. The introduction of both
> plugin blocklists and automatic Click to Play were signs to me that
> Mozilla were moving away from plugins, but this new plugin framework
> seems like a regression back towards embracing plugins.

You use the word "embrace" a lot. :-) Mozilla is trying very hard to
eliminate generic NPAPI plugins (as is Chrome). We have had to replace
them with something much more specific in two cases - OpenH264 and EME
CDMs. Both of these cases are, everyone agrees, non-ideal. This is not
the same, though, as "embracing plugins".

Gerv

[Gervase Markham]

On 27/12/14 14:41, Richard Z wrote:
> has been filled before and closed:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1100304

That bug is titled "Cisco's OpenH264 binary blob is downloaded without
prompting the user". That is not the same as a bug which says "Make
OpenH264 builds deterministic and reproducible", which is what I was
asking about.

> Even after the download happened it might be a good idea to have it allowed
> only for some pages as the WebRTC functionality may cause substantial bandwidth
> use

...which may occur even when not using H.264...

> and there may be possibly privacy abuses waiting to happen if someone
> happens to enable your micro and webcam remotely.

Which is why Firefox asks first. This potential risk is also nothing to
do with H264.

>>> 4. The download even happens in safe mode.
>>
>> Hmm. That does sound like a bug, or at least arguably a wrong behaviour.
>> Would you care to file it?
>

> Has been filled
> https://bugzilla.mozilla.org/show_bug.cgi?id=1105990

Great, thank you.

>>> 6. The preferences to disable the downloading are numerous (there seem
>>> to be at least three required) and hidden in about:config.
>>
>> Do you have more specific information?
>

> you need to tweak
> media.gmp-gmpopenh264.{autoupdate,enabled,provider.enabled}

So just setting media.gmp-gmpopenh264.provider.enabled to false doesn't
stop the download? That sounds on the face of it like a bug too.

> In addition to those issues listed above, the binary is currently downloaded
> over plain http (https://bugzilla.mozilla.org/show_bug.cgi?id=1102531).

dveditz writes:

"3) after the download we validate the hash and the filesize match
before doing anything"

So it seems that there is no risk of tampering here. HTTPS may be
desirable for other reasons, but not because there is a risk of the
binary being altered in transit.

Also, as Cisco host the binary, changing to HTTPS is not directly in our
control.

> Also, downloading over "http" on an open WIFI gives everyone else on the
> network information about your device (OS, Ffx version) that many of us would
> rather not disclose so easily - without sligthest user intervention.

If your device makes absolutely no HTTP requests whatsoever during a
browsing session, that argument may have merit. I'd be interested in
data on what proportion of sessions in the wild actually meet those
criteria...

> However, even Firefox should be more cautious about the license. Afaics the
> main Firefox product has a different license than this plugin - should not
> the user be prompted about this and asked to accept or deny this different
> license? it might happen that users would violate some of the complicated
> terms of use by accident.

The license of the plugin is displayed inside Firefox after it has been
downloaded. We don't believe it's necessary to present it to the user up
front, otherwise we would have done so.

[Gervase Markham]

On 27/12/14 19:12, Richard Z wrote:
> not having the code downloaded at all is an additional layer of security
> on top of asking. I do use different profiles for different kind of web
> activities and trying to have minimal set of plugins for each site.

Firefox has the ability to use your camera and microphone whether or not
the H.264 codec is downloaded. You gain no protection against this risk
from not having it.

> I prefer trusted package managers and want the choice.

Cisco have indicated that they are willing to package the codec in
whatever ways the community finds useful. If you want it a different
way, talk to them. We don't have the ability to distribute it in other
packages, because we aren't distributing it.

[»Q«]

On Tue, 23 Dec 2014 12:54:16 +0000, Luke Bratch wrote:

> 1. An unverified binary blob being downloaded and marked as executable
> is not OK, *especially without the user being informed and/or asked*.

Also, about:rights and about:license are still correct in a technical
sense, but each leaves the impression that using Firefox still means
using only free software, which is no longer true for most users.

[Richard Z]

On Sat, Dec 27, 2014 at 11:42:32AM +0000, Gervase Markham wrote:
> On 23/12/14 12:54, Luke Bratch wrote:
> > 1. An unverified binary blob being downloaded and marked as executable
> > is not OK, *especially without the user being informed and/or asked*.
>
> What exactly do you mean by "unverified"?
>
> I very much hope that there are mechanisms by which the blob is
> confirmed to be a genuine one (either by hash or signing or pinned
> HTTPS). If there are not, that's a bug. Can anyone confirm either way?
>
> The aim is to provide reproducible (deterministic) builds, so that it
> will no longer be unverified that the blob you get with the magic Cisco
> patent license pixie dust is identical to the one you can build yourself.
>
> This was part of the original promise of how this would work, so I hope
> it has not been forgotten.
>
> I can't immediately find a bug number for this work: does anyone have one?

has been filled before and closed:
https://bugzilla.mozilla.org/show_bug.cgi?id=1100304

> > 2. The initial Mozilla blog post [2] on the subject stated that the blob
> > would be downloaded "when needed", whereas in fact it happens
> > automatically upon browser load.
>
> Would you be significantly happier if it were downloaded the first time
> a user hit a WebRTC-using page?

definitely. It should behave just like other plugins - say plugin required,
download yes/no.

Even after the download happened it might be a good idea to have it allowed
only for some pages as the WebRTC functionality may cause substantial bandwidth

use and there may be possibly privacy abuses waiting to happen if someone
happens to enable your micro and webcam remotely. Bugs or malware doing similar
things did repeatedly happen.
This might be more a thing for NoScript or similar though.

>
> > 4. The download even happens in safe mode.
>
> Hmm. That does sound like a bug, or at least arguably a wrong behaviour.
> Would you care to file it?

Has been filled
https://bugzilla.mozilla.org/show_bug.cgi?id=1105990

> > 5. The behaviour of point 2 means that running "cfx run" (for instance
> > when developing extensions) means that the blob is downloaded over and
> > over again, consuming significant network resources.
>
> That also sounds like a bug which should be filed.

this is not a separate bug imho - rather a consequence of the already previously
mentioned https://bugzilla.mozilla.org/show_bug.cgi?id=1100304

>
> > 6. The preferences to disable the downloading are numerous (there seem
> > to be at least three required) and hidden in about:config.
>
> Do you have more specific information?

you need to tweak
media.gmp-gmpopenh264.{autoupdate,enabled,provider.enabled}

Because the download happens immediately after start (with a fresh profile)
you have very little chance to edit about:config to do this.

Some distributions (at least Fedora) now ship disabling all of these,
they have to do it as the binary blob is incompatible with their license.

As a consequence it is rather cumbersome for end users to enable the WebRTC
plugin.

> > 7. The only way to set those preferences by default requires local
> > administrative access. (And may disappear in the future!? [3])
>
> The proper method used by admins isn't marked as "may be removed".
>
> > Please note - I'm not after a point-by-point debate regarding the above
> > points, I only listed them all to convey the breadth of issues to people
> > who haven't encountered the bug report yet.
>
> But one bug report cannot deal with such a breadth of issues, and a bug
> report which says "this has problems, back it out" will quite rightly be
> WONTFIXed. If the individual issues concern you (and some certainly
> should) then please file bugs on them. If your actual aim is to get
> Mozilla's decision to support H.264 reversed, then I'm afraid that's not
> going to happen. The market forces which led to this sad decision are
> still exactly the same.

nope, personally I would be happy to use H.264 for various things. But
quite a few things about the way how the thing is auto-downloaded need
considerable tweeking and a people are unhappy that this is not regarded
important enough.
People are also unhappy finding out about this by looking at wireshark
or similar.. you will see plenty of complaints if you google around and
definitely gives bad publicity.

In addition to those issues listed above, the binary is currently downloaded
over plain http (https://bugzilla.mozilla.org/show_bug.cgi?id=1102531).

While I am sincerely hoping that the fingerprinting will catch all resulting
abuse attempts we know how brittle that mesh of ad hoc security is compared
to pgp verified downloads of typical Linux distributions.

Also, downloading over "http" on an open WIFI gives everyone else on the
network information about your device (OS, Ffx version) that many of us would
rather not disclose so easily - without sligthest user intervention.

> It also depends what you mean by "untrusted" binaries. People who
> download Firefox have to trust Mozilla. Now, if there is opportunity for
> the OpenH264 blob to be interfered with between our build servers and
> you, that's clearly a bug which should be fixed. If not, then it's as
> trusted as the Firefox binary the user downloaded in the first place.

People who download Firefox from Fedora or Debian have to trust Fedora or
Debian. Now they have to trust two additional instances - and additionally
live with the uncertainties of somewhat messy security downloading over
http.

> If you actually mean "not open source", that's a different question with
> a different answer. The OpenH264 code is open source. Unless your
> position is that no-one should ever use any patented technology. We
> don't like software patents, but we live in a world where they exist.

In Fedora we could probably outsource that plugin to rpmfusion.

However, even Firefox should be more cautious about the license. Afaics the
main Firefox product has a different license than this plugin - should not
the user be prompted about this and asked to accept or deny this different
license? it might happen that users would violate some of the complicated
terms of use by accident.

Richard

---
Name and OpenPGP keys available from pgp key servers

[Richard Z]

On Sat, Dec 27, 2014 at 03:34:01PM +0000, Gervase Markham wrote:

> > and there may be possibly privacy abuses waiting to happen if someone
> > happens to enable your micro and webcam remotely.
>
> Which is why Firefox asks first. This potential risk is also nothing to
> do with H264.

not having the code downloaded at all is an additional layer of security
on top of asking. I do use different profiles for different kind of web
activities and trying to have minimal set of plugins for each site.

> >>> 6. The preferences to disable the downloading are numerous (there seem
> >>> to be at least three required) and hidden in about:config.
> >>
> >> Do you have more specific information?
> >
> > you need to tweak
> > media.gmp-gmpopenh264.{autoupdate,enabled,provider.enabled}
>
> So just setting media.gmp-gmpopenh264.provider.enabled to false doesn't
> stop the download? That sounds on the face of it like a bug too.

Here is a somewhat related bug
https://bugzilla.mozilla.org/show_bug.cgi?id=1044268

> > In addition to those issues listed above, the binary is currently downloaded
> > over plain http (https://bugzilla.mozilla.org/show_bug.cgi?id=1102531).
>
> dveditz writes:
>
> "3) after the download we validate the hash and the filesize match
> before doing anything"
>
> So it seems that there is no risk of tampering here. HTTPS may be
> desirable for other reasons, but not because there is a risk of the
> binary being altered in transit.
>
> Also, as Cisco host the binary, changing to HTTPS is not directly in our
> control.

This is diverging a bit from the main issue - that the download should be
on demand, but I will bite in anyway.

Filesize and hash are downloaded from https://aus4.mozilla.org afaics. Https
had so many holes in the last two years like the annual production of Swiss
cheese. For good reasons the certificate of aus4.mozilla.org is hard-pinned
because https certificate validation is fundamentally untrusted.
Judging by the other glitches (ignores safe-mode, needs 3 prefs instead of one)
it looks a bit hot-stitched to me.

I prefer trusted package managers and want the choice.

> > Also, downloading over "http" on an open WIFI gives everyone else on the
> > network information about your device (OS, Ffx version) that many of us would
> > rather not disclose so easily - without sligthest user intervention.
>
> If your device makes absolutely no HTTP requests whatsoever during a
> browsing session, that argument may have merit. I'd be interested in
> data on what proportion of sessions in the wild actually meet those
> criteria...

also consider the proportion of users who are trying to deliberately obscure
their browser fingerprint by one of the many UA switching extensions.

Richard

---
Name and OpenPGP keys available from pgp key servers

[bobbuun]

While it's on-topic, is there any reason the plug-in doesn't have the
option to be set as "click to activate" like all the others can be? It's
only "always" and "never".

-D Putman

[Gervase Markham]

On 26/12/14 20:54, »Q« wrote:
> Also, about:rights and about:license are still correct in a technical
> sense, but each leaves the impression that using Firefox still means
> using only free software, which is no longer true for most users.

I don't agree; I think OpenH264 is free software according to both the
OSD and the FSF definitions. The patent situation is deeply regrettable,
but it doesn't make the software non-free. x264 is free software.

(Also, on a pragmatic point about "most", it remains to be seen how
often users will use WebRTC with H.264 video.)

Gerv

[Gervase Markham]

On 27/12/14 15:34, Gervase Markham wrote:
> That bug is titled "Cisco's OpenH264 binary blob is downloaded without
> prompting the user". That is not the same as a bug which says "Make
> OpenH264 builds deterministic and reproducible", which is what I was
> asking about.

To add to this thread: there is now a specific bug on this:
https://bugzilla.mozilla.org/show_bug.cgi?id=1115874

[Georg Fritzsche]

On 28 Dec 2014, at 18:02, bobbuun <bob...@hotmail.com> wrote:

> While it's on-topic, is there any reason the plug-in doesn't have the option to be set as "click to activate" like all the others can be? It's only "always" and "never”.

“Ask to activate” is not meaningful for these plugins - they are not used in a context where they automatically/unexpectedly activate like NPAPI plugins are.
In the case of OpenH264, it is activated specifically for some WebRTC calls.

Georg

[»Q«]

On Mon, 29 Dec 2014 09:56:07 +0000
Gervase Markham <gerv-4eJtQOnF...@public.gmane.org> wrote:

> On 26/12/14 20:54, »Q« wrote:
> > Also, about:rights and about:license are still correct in a
> > technical sense, but each leaves the impression that using Firefox
> > still means using only free software, which is no longer true for
> > most users.
>
> I don't agree; I think OpenH264 is free software according to both the
> OSD and the FSF definitions. The patent situation is deeply
> regrettable, but it doesn't make the software non-free. x264 is free
> software.

You're right. Sorry for the noise. (I think I must have been confusing
OpenH264 with a very different thing, the encrypted media extension DRM
stuff.)

> (Also, on a pragmatic point about "most", it remains to be seen how
> often users will use WebRTC with H.264 video.)

Yes. I was very unclear; I meant most users, whether they use Hello or
not, will have the codec downloaded and installed.

[Manuel Reimer]

On 12/27/2014 12:42 PM, Gervase Markham wrote:
> It also depends what you mean by "untrusted" binaries.

In my case this would mean: Binaries built from trusted sources by
trusted "build systems".

How are the OpenH264 binaries built and which way do they go? So far you
communicated this nowhere.

Are the binaries compiled by Cisco or by Mozilla build servers?

IMHO the best and most secure way would be to have them built by the
Mozilla build servers, then keep a checksum and send the binary to Cisco
for distribution. This way it's easy to verify that nothing was
added/modified between "Mozilla build servers" and "end user".

In my opinion it would be even better to allow Linux distributions to
*optionally* build the files by themselves. If a distributor chooses to
do so and to install the file to a global place into the system, then
the whole "download from Cisco magic" should be disabled. This wouldn't
be more problematic than shipping tools like ffmpeg.

Manuel

[Gavin Sharp]

Andreas clarified this on his blog:
http://andreasgal.com/2014/10/14/openh264-now-in-firefox/#comment-6620

Mozilla builds the binary and then later verifies its hash before using it.

Gavin

[Richard Z]

On Fri, Jan 02, 2015 at 02:53:30PM -0500, Gavin Sharp wrote:
> Andreas clarified this on his blog:
> http://andreasgal.com/2014/10/14/openh264-now-in-firefox/#comment-6620
>
> Mozilla builds the binary and then later verifies its hash before using it.

this verification could be a lot stronger. Currently the plain sha512
checksum is fetched via https. If that brittle SSL security layer breaks
and someone does a successfull MITM he can install any binary he wants.

Without an additional PGP verification the binary is untrusted.

Richard

---
Name and OpenPGP keys available from pgp key servers

[Benjamin Smedberg]

On 12/23/2014 7:54 AM, Luke Bratch wrote:
> 6. The preferences to disable the downloading are numerous (there seem
> to be at least three required) and hidden in about:config.

I don't think this is correct. There is one UI point for disabling both
the download and activation of OpenH264 in the addon manager, which is
to switch it from "Always Activate" to "Never Activate" in the plugins pane.

Under the hood and if you are running automated tests, this maps to the
single preference media.gmp-gmpopenh264.enabled;false

There are other prefs such as media.gmp-gmpopenh264.provider.enabled but
those are not meant to be user-settable and exist to hide OpenH264 in
the addon manager for applications that don't support it (such as
Thunderbird), since the addon manager code is shared.

--BDS

[Richard Z]

On Sun, Jan 04, 2015 at 03:02:48PM -0500, Benjamin Smedberg wrote:
>
> On 12/23/2014 7:54 AM, Luke Bratch wrote:
> >6. The preferences to disable the downloading are numerous (there seem
> >to be at least three required) and hidden in about:config.
>
> I don't think this is correct. There is one UI point for disabling both the
> download and activation of OpenH264 in the addon manager, which is to switch
> it from "Always Activate" to "Never Activate" in the plugins pane.

> Under the hood and if you are running automated tests, this maps to the
> single preference media.gmp-gmpopenh264.enabled;false

clearly not sufficient with my firefox 34.

this works:
pref("media.gmp-gmpopenh264.provider.enabled",false);
pref("media.gmp-gmpopenh264.autoupdate",false);
pref("media.gmp-gmpopenh264.enabled",false);

Richard

---
Name and OpenPGP keys available from pgp key servers

[Benjamin Smedberg]

On 1/4/2015 5:35 PM, Richard Z wrote:
>
>> Under the hood and if you are running automated tests, this maps to the
>> single preference media.gmp-gmpopenh264.enabled;false
> clearly not sufficient with my firefox 34.
>
> this works:
> pref("media.gmp-gmpopenh264.provider.enabled",false);
> pref("media.gmp-gmpopenh264.autoupdate",false);
> pref("media.gmp-gmpopenh264.enabled",false);

I filed bug 1117765 for this. provider.enabled is definitely incorrect,
since it will cause OpenH264 to disappear completely from the addon
manager which is incorrect. .enabled=false should be sufficient to
disable initial download and update.

--BDS