Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

hardware-accelerated audio/video decoding in Gecko (bug 714408)

14,178 views
Skip to first unread message

Andreas Gal

unread,
Mar 12, 2012, 3:28:04 AM3/12/12
to dev-platform

I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).

Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).

Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).

I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.

Let me know if you have any questions.

Andreas

Asa Dotzler

unread,
Mar 12, 2012, 4:02:31 AM3/12/12
to
On 3/12/2012 12:28 AM, Andreas Gal wrote:
> I don't think this bug significantly changes our position on open
> video. We will continue to promote and support open codecs, but when
> and where existing codecs are already installed and licensed on
> devices we will make use of them in order to provide people with the
> best possible experience.

For the desktop, this could be problematic. While Windows 7 does provide
access to an already installed and licensed h.264 codec, Windows XP does
not. If we simply enable system codecs where available, how will Web
developers know when they can and cannot count on system codecs?

I'm not opposing this move, I wonder though how we can go "half way" and
not follow-up with shipping h.264 on Windows XP and any other platforms
where it isn't pre-installed. It seems like "Gecko is Gecko" gets pretty
broken if half of our users have a codec and the other half doesn't.

- A

Andreas Gal

unread,
Mar 12, 2012, 4:16:18 AM3/12/12
to Asa Dotzler, dev-pl...@lists.mozilla.org

Desktop is a more complex equation, thats why we are attacking mobile first. On mobile the field is pretty homogeneous, with Gecko being the only browser not supporting the common formats.

For desktop one possibility is to expose MPAPI (Media Plugin API, I couldn't think of a better name for now) via NPAPI. Third parties could then provide additional audio/video decoders as a plugin. Flash, for example, contains all relevant decoders and could hook into MPAPI to provide this capability on Windows XP. Third parties could even offer plugins for sale to cover licensing cost. These are all suboptimal solutions for the 40% of our users base on Windows XP, but so is not helping the other 60% play the media they want, with the codecs they already have on their system.

Andreas
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

ger...@info-care.com.pt

unread,
Mar 12, 2012, 8:54:49 AM3/12/12
to dev-platform
I wonder if this means small appliances like the Raspberry Pi (which has hw decoding H.264 - Broadcom BCM2835) or the SolidRun CuBox (Marvell ARMADA 500) could find themselves more and more useful!

Doug Turner

unread,
Mar 12, 2012, 9:00:44 AM3/12/12
to Andreas Gal, dev-platform
This is great. We really need this on Android where many mobile sites only use h.264 video.

Doug


On Mar 12, 2012, at 12:28 AM, Andreas Gal wrote:

>
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Jeremie Patonnier

unread,
Mar 12, 2012, 9:05:05 AM3/12/12
to
On Mar 12, 4:02 am, Asa Dotzler <a...@mozilla.org> wrote:
> On 3/12/2012 12:28 AM, Andreas Gal wrote:
> For the desktop, this could be problematic. While Windows 7 does provide
> access to an already installed and licensed h.264 codec, Windows XP does
> not. If we simply enable system codecs where available, how will Web
> developers know when they can and cannot count on system codecs?

Well, I don't think this is a problem. Web developer already must take
care about this if they want to provide a cross browser experience to
their user (by providing both H264 and WEBM encoded videos).

As per the HTML5 spec, the "source" element provide all the necessary
fallback required to deal with that issue.

So by doing the following, web developper already deal with the video
codecs insanity on the Web :

<video>
<source src="foo.mp4" type="video/mp4">
<source src="foo.webm" type="video/webm">
Any other fallback for browsers that do not understand the video
tag.
</video>

So even if Firefox can decode the mp4 video due to hardware
acceleration on some plateforme, it does change nothing in a web
developer point of view. therefor, IMHO, this feature does not improve
nothing for web dev, but can make a difference for users (saving life
battery on laptop for exemple. This is a nice to have feature for
users and it change nothing for developer.

Cheers
Jérémie

Joe Drew

unread,
Mar 12, 2012, 10:29:14 AM3/12/12
to dev-platform

On 2012-03-12 9:00 AM, Doug Turner wrote:
> This is great. We really need this on Android where many mobile sites only use h.264 video.

I don't see why this matters. At the time we shipped Theora (and WebM),
most _desktop_ sites only supported h.264 (mostly through Flash).

I am very concerned that, by supporting system codecs, we're simply
capitulating on Free codecs. If, as we believe, mobile is the future,
and we support h.264 or other non-free codecs on mobile, then the future
of codecs is non-free.

joe

Joe Drew

unread,
Mar 12, 2012, 10:32:58 AM3/12/12
to dev-platform
On 2012-03-12 3:28 AM, Andreas Gal wrote:
> There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

Further to my earlier message: there is exactly as much justification
stopping our users from using system decoders on mobile as there is on
desktop. If we want to support non-free formats via system codecs, we
should make that our official plan for Firefox on all platforms. Doing
so via the back door, which is what I perceive this as, is wrong.

joe

Blake Winton

unread,
Mar 12, 2012, 10:40:31 AM3/12/12
to
On 12-03-12 10:32 , Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop.

I see a slight difference in justification, in that 40% of our desktop
users (Windows XP) don't get an h.264 codec from their system, so
enabling system codecs on desktop won't help nearly as much of our
userbase, and will fragment the desktop version of gecko, and make it
harder for web developers to test their sites.

> If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.

I can see how you would feel that way, but I don't think that we're
trying to do this via the back door (otherwise Andreas wouldn't have
posted in a public forum ;). Regardless, I agree that we should make it
our official plan on all platforms (or not), if only to keep our
position consistent.

Later,
Blake.

Doug Turner

unread,
Mar 12, 2012, 10:47:45 AM3/12/12
to Joe Drew, dev-platform

On Mar 12, 2012, at 7:29 AM, Joe Drew wrote:

>
> On 2012-03-12 9:00 AM, Doug Turner wrote:
>> This is great. We really need this on Android where many mobile sites only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM), most _desktop_ sites only supported h.264 (mostly through Flash).

Why what doesn't matter?

Like… when I use Firefox on Android, I really want video to *just* work. A few months ago, I did a quick look at popular video sites that Firefox for Android gets served. There were a lot of sites that returned mobile content but their video tags only listed h.264. For example, both DailyMotion.com and Vimeo.com fit into this category.

As I understand it, what are we are doing it basically waiting it. We are hoping that the sites transcode to WebM. Sites are slowing doing this for Desktop where FF has a significant marketshare (YouTube committed to this, but I don't think it has completed its conversion yet). Mobile is a bit different. We do not have any significant marketshare and leverage is hard. Sites will know only have to do the conversation for Firefox desktop, but realized that Firefox for Android exists and they should serve mobile content with both h.264 and WebM hurts Firefox for Android more that it helps. This is a long bet. And I think we need to make Firefox for Android not have signifiant deficiencies from the default browser.

Using the hardware decoder that is present - and paid for - on the device will help make Firefox for Android's that much more usable.

> I am very concerned that, by supporting system codecs, we're simply capitulating on Free codecs. If, as we believe, mobile is the future, and we support h.264 or other non-free codecs on mobile, then the future of codecs is non-free.


I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.

Regards,
Doug

Ted Mielczarek

unread,
Mar 12, 2012, 10:57:06 AM3/12/12
to Doug Turner, Joe Drew, dev-platform
On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner <do...@mozilla.com> wrote:
> I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.

This may be true, but can you say with a straight face that supporting
non-free codecs on mobile (including on our *own* platform, B2G) isn't
tantamount to giving up on free codecs for the web? It seems pretty
obvious to me that we will lose any moral high ground we hold by doing
this.

Now, it may be that we have already lost this war. If that's the case,
we should just admit it and move on, and not sabotage ourselves by
shipping support only on mobile platforms.

-Ted

Joe Drew

unread,
Mar 12, 2012, 10:59:21 AM3/12/12
to dev-platform
On 2012-03-12 10:40 AM, Blake Winton wrote:
> I see a slight difference in justification, in that 40% of our desktop
> users (Windows XP) don't get an h.264 codec from their system, so
> enabling system codecs on desktop won't help nearly as much of our
> userbase, and will fragment the desktop version of gecko, and make it
> harder for web developers to test their sites.

There isn't really a difference between "mobile gecko" and "desktop
gecko" - or, at least, there shouldn't be. We produce one product for
many platforms. Stated another way, we support one product (the web)
everywhere.

joe

Blake Winton

unread,
Mar 12, 2012, 11:42:49 AM3/12/12
to
I believe there is from a web developer's point of view. You need
fairly different layouts for mobile to account for the smaller screens,
and larger pointing devices (fingers vs. cursor). You also expect
different performance characteristics, and likely different capabilities
(i.e. is Flash installed, or is there an h.264 encoder).

We might use the same platform to support the two different markets, but
that doesn't make the markets the same. :)

Later,
Blake.

Doug Turner

unread,
Mar 12, 2012, 11:52:31 AM3/12/12
to Ted Mielczarek, Joe Drew, dev-platform
I just don't know. To me, I just want software to work. If a site wants to send me h.264, great. If it wants to send webm, great.

Doug

John Volikas

unread,
Mar 12, 2012, 11:55:56 AM3/12/12
to dev-platform
If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

Andreas Gal

unread,
Mar 12, 2012, 12:16:18 PM3/12/12
to Ted Mielczarek, Joe Drew, Doug Turner, dev-platform

On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:

> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner <do...@mozilla.com> wrote:
>> I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>
> This may be true, but can you say with a straight face that supporting
> non-free codecs on mobile (including on our *own* platform, B2G) isn't
> tantamount to giving up on free codecs for the web? It seems pretty
> obvious to me that we will lose any moral high ground we hold by doing
> this.
>
> Now, it may be that we have already lost this war. If that's the case,
> we should just admit it and move on, and not sabotage ourselves by
> shipping support only on mobile platforms.

I do believe this war is lost. Just look around. Almost none of the content users want to watch is available in WebM. The only reason desktop is usable is because of Flash, a proprietary plugin, playing video for us (in H.264, mostly). Even Google, supposedly a proponent of open codecs, never fully converted youtube and never dropped H.264 from Chrome. Taking a principled (I would at this point prefer 'stubborn' I think) stance on H.264 won't change reality. It just hurts us and our users.

Firefox got to the point where we are on desktop today by embracing reality. In the early days we started supporting IE-isms like document.all that were god awfully ugly and non-standard. But it was needed for compatibility so we can give people a usable web experience. The web uses H.264. Thats an unpleasant fact, but its a fact. We have to support it whether we like it or not, so we can be around for the next round and continue to influence the web for the better.

Andreas

>
> -Ted

Andreas Gal

unread,
Mar 12, 2012, 12:18:45 PM3/12/12
to John Volikas, dev-pl...@lists.mozilla.org

> If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

Yes, sorry if that wasn't clear from the original email but thats the intent. Anything the system has decoders for we should pass through and allow to work, including MP3 and AAC.

Andreas

Joe Drew

unread,
Mar 12, 2012, 12:23:31 PM3/12/12
to dev-platform
On 2012-03-12 12:16 PM, Andreas Gal wrote:
> I do believe this war is lost.

Please re-frame your discussion, in that case, as enabling platform
codecs on all platforms. Doing it only on mobile makes no sense.

joe

smaug

unread,
Mar 12, 2012, 12:24:30 PM3/12/12
to Doug Turner, Ted Mielczarek, Joe Drew
On 03/12/2012 05:52 PM, Doug Turner wrote:
> I just don't know. To me, I just want software to work.

But we need to think more than just the current state of the web. We
really need to think about the future and the web as a platform.
Do we want to prevent fully open source and free implementations
(including hw) to be able to render everything in the internet?


-Olli

Andreas Gal

unread,
Mar 12, 2012, 12:27:15 PM3/12/12
to Joe Drew, dev-platform
I think the discussion is framed correctly. I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet. Once I do, I will come back and propose it.

Andreas

>
> joe

Andreas Gal

unread,
Mar 12, 2012, 12:30:36 PM3/12/12
to smaug, Joe Drew, Ted Mielczarek, dev-pl...@lists.mozilla.org

On Mar 12, 2012, at 9:24 AM, smaug wrote:

> On 03/12/2012 05:52 PM, Doug Turner wrote:
>> I just don't know. To me, I just want software to work.
>
> But we need to think more than just the current state of the web. We
> really need to think about the future and the web as a platform.
> Do we want to prevent fully open source and free implementations
> (including hw) to be able to render everything in the internet?

Yes, we do. H.264 isn't the end of the codec story. Higher compression ratio successors to H.264 are already being developed. We need to proactively push on open codecs in that space so for the next cycle of this we have a working, free alternative ready from the start. I want to be around to fight that battle, instead of dying on the H.264 hill in a lost war.

Andreas

>
>
> -Olli
>
>
>> If a site wants to send me h.264, great. If it wants to send webm, great.
>>
>> Doug
>>
>> On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:
>>
>>> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner<do...@mozilla.com> wrote:
>>>> I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>>>
>>> This may be true, but can you say with a straight face that supporting
>>> non-free codecs on mobile (including on our *own* platform, B2G) isn't
>>> tantamount to giving up on free codecs for the web? It seems pretty
>>> obvious to me that we will lose any moral high ground we hold by doing
>>> this.
>>>
>>> Now, it may be that we have already lost this war. If that's the case,
>>> we should just admit it and move on, and not sabotage ourselves by
>>> shipping support only on mobile platforms.
>>>
>>> -Ted
>>
>

L. David Baron

unread,
Mar 12, 2012, 12:39:22 PM3/12/12
to Andreas Gal, dev-pl...@lists.mozilla.org
On Monday 2012-03-12 00:28 -0700, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get
> review for it. It adds hardware-accelerated audio/video decoding
> support to Gecko using system decoders already present on the
> system. Android, for example, ships by default with a number of
> decoders, and in particular for such mobile devices we really have
> to use these hardware-accelerated decoders for good battery life
> (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we
> will add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present
> on the system, including H.264 and MP3. There is really no
> justification to stop our users from using system decoders already
> on the device, so we will not filter any formats.

I disagree with this for two reasons. Joe has already stated the
first (that it fundamentally disagrees with our open codec story).

But the second is that if we are going to support system decoders,
we should be choosing which ones rather than just supporting all
codecs that the system supports. (Our motivation for doing this is
to expose a particular one or two codecs, not to expose all codecs.
So we should expose those and not everything.) I believe this is
preferable for three reasons:

(1) A more coherent story for Web developers: Having a smaller set
of codecs on the Web presents a more coherent story to Web
developers and doesn't encourage them to waste time using codecs
(by offering support on some platforms) that aren't widely
supported and that we don't actually think should become a part of
the Web.

(2) A smaller set of codecs required for future Web platform
implementations: Supporting "all" system codecs risks the
possibility that we'll add a larger number of codecs to the set of
codecs that are required to build a new implementation of the Web
platform. This increases the costs of doing so and reduces future
competition.

(3) Smaller security exposure: Each additional codec
implementation we expose adds additional security exposure; we
shouldn't expose codecs that we don't think Web authors should be
using.

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla http://www.mozilla.org/ 𝄂

Andreas Gal

unread,
Mar 12, 2012, 12:46:21 PM3/12/12
to L. David Baron, dev-pl...@lists.mozilla.org
> But the second is that if we are going to support system decoders,
> we should be choosing which ones rather than just supporting all
> codecs that the system supports. (Our motivation for doing this is
> to expose a particular one or two codecs, not to expose all codecs.
> So we should expose those and not everything.) I believe this is
> preferable for three reasons:

This is a fair argument, and its technically what the patch does. We specify a list of codecs we allow to pass through. What my original email meant was really "we will not filter for political reasons". In practice, for exactly the reason you listed, we want pick the relevant codecs users want and expose those explicitly for simple technical reasons. H.264/AVC, AAC and MP3 is probably what we want there, but I am happy to hear alternative proposals. Pending this discussion, the patch currently only allows H.264 to pass through, but we already have a 1-liner in the bug to add MP3.

Andreas

Ralph Giles

unread,
Mar 12, 2012, 1:31:11 PM3/12/12
to
On 12/03/12 09:27 AM, Andreas Gal wrote:

> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.

If we're going to compromise our principles on this issue, we should
license a proprietary decoder implementation and ship it with our
official desktop builds. That way we have

a) consistent support on all platforms, including Windows XP;

b) a smaller maintenance burden than calling different backends on each
platform;

c) smaller increase in security footprint;

d) better resistance to DRM implementation pressure than a pass-through
scheme offers.

That would be the honest way to support non-free codecs on the Web. They
cost money, and current licensing is incompatible with free software
implementation. We should be as transparent about that state of affairs
as we can.

-r

Ralph Giles

unread,
Mar 12, 2012, 1:32:45 PM3/12/12
to
On 12/03/12 05:54 AM, ger...@info-care.com.pt wrote:

> I wonder if this means small appliances like the Raspberry Pi (which has hw decoding H.264 - Broadcom BCM2835) or the SolidRun CuBox (Marvell ARMADA 500) could find themselves more and more useful!

I think you'd find the Raspberry Pi's GPU is perfectly capable of
decoding WebM (and Ogg Theora), although Broadcom hasn't marketed an
implementation of those formats.

-r

Justin Dolske

unread,
Mar 12, 2012, 1:34:49 PM3/12/12
to
On 3/12/12 7:29 AM, Joe Drew wrote:
\
>> This is great. We really need this on Android where many mobile sites
>> only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM),
> most _desktop_ sites only supported h.264 (mostly through Flash).

To put it another way:

We had these exact same discussions back when we first shipped Theora,
including the issue of H264 on mobile (then with Fennec-on-Nokia).

Rather that just rehashing the same arguments, I'd like to ask what's
changed such that we need to reopen this issue?

Justin

Doug Turner

unread,
Mar 12, 2012, 1:40:30 PM3/12/12
to Justin Dolske, dev-pl...@lists.mozilla.org
I guess nothing has changed. The market didn't adopt any new codec. h.264 video content is what continues to be used everywhere and our browser just falls down there.

Doug

Daniel Cater

unread,
Mar 12, 2012, 1:44:04 PM3/12/12
to dev-platform
On Monday, March 12, 2012 7:28:04 AM UTC, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Isn't this another issue which goes beyond the B2G or Fennec module owners and needs buy-in from core Gecko module owners?

In particular, given http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.html and http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I expect Robert, Chris Pearce, Chris Double and others to be included in this discussion.

With more browsers, sites and hardware manufacturers working on existing or upcoming WebM / VP8 support I would be very disappointed to see Mozilla cave on this issue.

Daniel Cater

unread,
Mar 12, 2012, 1:44:04 PM3/12/12
to mozilla.de...@googlegroups.com, dev-platform

Mike Hommey

unread,
Mar 12, 2012, 1:48:13 PM3/12/12
to Doug Turner, dev-pl...@lists.mozilla.org, Justin Dolske
On Mon, Mar 12, 2012 at 10:40:30AM -0700, Doug Turner wrote:
>
> On Mar 12, 2012, at 10:34 AM, Justin Dolske wrote:
>
> I guess nothing has changed. The market didn't adopt any new codec.
> h.264 video content is what continues to be used everywhere and our
> browser just falls down there.

I guess we should feel responsible for this sad state of affairs. How do
we expect people to use open formats when we don't do much to make them
easy to produce? (and here "we" is much broader than Mozilla)

Mike

Daniel Cater

unread,
Mar 12, 2012, 1:57:34 PM3/12/12
to Justin Dolske, dev-pl...@lists.mozilla.org
http://en.wikipedia.org/wiki/WebM_Project shows a fair few supporters of WebM including Opera, Chrome, Firefox, Android, Adobe, Nvidia, AMD, ARM, Broadcom, Qualcomm, Texas Instruments, Nvidia, Rockchip, YouTube, Skype and Logitech.

Also Google had pledged to remove H.264 support from Chrome but hasn't yet done so: http://blog.chromium.org/2011/01/html-video-codec-support-in-chrome.html

Daniel Cater

unread,
Mar 12, 2012, 1:57:34 PM3/12/12
to mozilla.de...@googlegroups.com, dev-pl...@lists.mozilla.org, Justin Dolske
On Monday, March 12, 2012 5:40:30 PM UTC, Doug Turner wrote:

Ms2ger

unread,
Mar 12, 2012, 2:28:09 PM3/12/12
to Andreas Gal
On 03/12/2012 08:28 AM, Andreas Gal wrote:
>
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.

Did you deliberately write the subject of this thread in such a way that
it is easy to miss that you are trying to reopen the issue of supporting
patent-encumbered codecs, against the policy we have had ever since the
video element was first implemented, and have you considered the effect
that would have on Mozilla's credibility?

Thanks,
Ms2ger

Joe Drew

unread,
Mar 12, 2012, 2:30:01 PM3/12/12
to dev-platform


On 2012-03-12 12:31 PM, Doug Turner wrote:
> I understand that Olli. My assertion is that if our mobile offering has deficiencies -- like not being able to play the video tag when the default can -- we will not be relevant. Users simply do not care about the codec. They care if it works or if it doesn't.

Our current desktop product has these same "deficiencies."

joe

Doug Turner

unread,
Mar 12, 2012, 2:35:10 PM3/12/12
to Joe Drew, dev-platform
But it has users! Sites are a lot more likely to test their code against FF desktop, then Firefox for Android.

Philip Chee

unread,
Mar 12, 2012, 2:37:31 PM3/12/12
to
On Mon, 12 Mar 2012 09:16:18 -0700, Andreas Gal wrote:

> video for us (in H.264, mostly). Even Google, supposedly a proponent
> of open codecs, never fully converted youtube.

Data point. Speaking as the current author of Flashblock. I have been
getting increasing number of requests from users to block html5 video.
The main driver of these requests appears to be Youtube. So Youtube may
not be fully converted to html5/video/webm - yet - but it certainly
looks like Google is making a significant effort.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.

Andreas Gal

unread,
Mar 12, 2012, 2:44:33 PM3/12/12
to Daniel Cater, Justin Dolske, dev-pl...@lists.mozilla.org, mozilla.de...@googlegroups.com
Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.

We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

Andreas

Sent from Mobile.

Ralph Giles

unread,
Mar 12, 2012, 2:49:51 PM3/12/12
to
On 12/03/12 10:48 AM, Mike Hommey wrote:

> I guess we should feel responsible for this sad state of affairs. How do
> we expect people to use open formats when we don't do much to make them
> easy to produce? (and here "we" is much broader than Mozilla)

I think many of us hoped Google, as a larger organization and curators
of the WebM project, would provide leadership here. In retrospect that
was a mistake; this matters more to Mozilla than it does to Google.

But a number of us have been working on improving the situation. I've
been pushing hard on getting a WebM stream on Air Mozilla since we
dropped the Theora option in the fall. We made this a priority this
year, hiring a David Richards to implement WebM support in icecast, for
example.

But there's certainly more we could do, especially with evangelism.
YouTube mostly works with HTML5 WebM now, but almost every day a Vimeo
iframe tells me I need to use a different browser. We should be reaching
out them and asking them to support the Open Web.

Another point to be aware of is that this battle is not just about the
html video element. The WebRTC project (web video chat) is also trying
to choose a baseline codec. The way the IETF draft is currently written,
the WebM video codec (VP8) will be chosen as mandatory to implement
unless a royalty free profile of h.264 is made available before March
15th (that's *this week*). This is a greener field and free codecs are
making better progress there in part because of the line Mozilla has
held elsewhere.

-r

Andreas Gal

unread,
Mar 12, 2012, 3:06:24 PM3/12/12
to Ralph Giles, dev-pl...@lists.mozilla.org
Even if a license grant is made for H.264 for WebRTC the need for this patch still holds. On anything with a battery we have to use hardware decoders. Those will always be buried somewhere in hardware with some kernel pieces in addition. We will never ship with those bits, and we have to use whats on the system, which really takes us back to where this argument started.

Sent from Mobile.

Jeff Walden

unread,
Mar 12, 2012, 3:14:14 PM3/12/12
to
On 03/12/2012 09:30 AM, Andreas Gal wrote:
>> Do we want to prevent fully open source and free implementations
>> (including hw) to be able to render everything in the internet?
>
> Yes, we do.

This is worth highlighting. And highlighting again. And again. And again.

The web platform should be based on open standards. I disagree profoundly with moves that take us in the opposite direction.

Jeff

Daniel Veditz

unread,
Mar 12, 2012, 3:39:56 PM3/12/12
to Jeff Walden
Unfortunate quote juxtaposition in Andreas's post made his post hard
to understand, but he's saying the opposite of what it appears in
this shortened version. "Yes, we do." was a response to the first
sentence in that paragraph: "But we need to think more than just the
current state of the web."

His point was that if we remain inflexible and go down with the WebM
ship we will not be around to have any influence in the next battle
over h265.

Hopefully he's learned his lesson about careful quote trimming :-)

-Dan Veditz

Jeff Walden

unread,
Mar 12, 2012, 3:51:04 PM3/12/12
to
On 03/12/2012 12:39 PM, Daniel Veditz wrote:
> His point was that if we remain inflexible and go down with the WebM
> ship we will not be around to have any influence in the next battle
> over h265.

I think if we yield on this point, there's little reason for others to assume we wouldn't yield on that point as well, honestly.

Whether the particular quoting/juxtaposition was intended or not, I think the position when it comes down to brass tacks would be that we effectively would be preventing free/open source implementations from rendering the Internet. Intent or lack thereof aside doesn't seem much more than a matter of cosmetics to me.

Jeff

Christopher Blizzard

unread,
Mar 12, 2012, 4:15:34 PM3/12/12
to Andreas Gal, Joe Drew, Doug Turner, Ted Mielczarek, dev-platform
On 3/12/2012 9:16 AM, Andreas Gal wrote:
> On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:
>
>> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner<do...@mozilla.com> wrote:
>>> I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>> This may be true, but can you say with a straight face that supporting
>> non-free codecs on mobile (including on our *own* platform, B2G) isn't
>> tantamount to giving up on free codecs for the web? It seems pretty
>> obvious to me that we will lose any moral high ground we hold by doing
>> this.
>>
>> Now, it may be that we have already lost this war. If that's the case,
>> we should just admit it and move on, and not sabotage ourselves by
>> shipping support only on mobile platforms.
> I do believe this war is lost. Just look around. Almost none of the content users want to watch is available in WebM. The only reason desktop is usable is because of Flash, a proprietary plugin, playing video for us (in H.264, mostly). Even Google, supposedly a proponent of open codecs, never fully converted youtube and never dropped H.264 from Chrome. Taking a principled (I would at this point prefer 'stubborn' I think) stance on H.264 won't change reality. It just hurts us and our users.

Just to back this up, I keep talking to people building sites and there
are only a couple of organizations that are willing to embrace WebM
because it's the right thing to do. Transcoding & hosting costs are
huge.* Beyond that I've not really run into anyone who wants to do
WebM. It's just seen as a cost that Firefox is incurring on web developers.

I think that if Google had switched off H.264 support in their browser
soon after they said they would (well over a year ago) then the shift
might have taken place. But that window is gone. It's time to move on,
listen to our users and developers and fight a different battle.

--Chris

* I talked with an organization that has a lot of video a couple of
weeks ago. Actual quote: "If we wanted to transcode we could, but that
would be a full month of time if we used our entire cluster, not to
mention the storage costs at scale. We just can't afford to do that."

Christopher Blizzard

unread,
Mar 12, 2012, 4:20:56 PM3/12/12
to Joe Drew, dev-platform
On 3/12/2012 7:32 AM, Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
>
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop. If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.

I agree we should make that the plan. But we should start with Mobile
where the codecs are more widely available and move from there. We're
somewhat resource constrained here.

--Chris

Christopher Blizzard

unread,
Mar 12, 2012, 4:23:36 PM3/12/12
to Andreas Gal, L. David Baron, dev-pl...@lists.mozilla.org
I agree with both of you on this. We want a subset of system codecs in
order to have a coherent story for web developers. MP3, AAC, H.264/AVC
is the list I care about, starting with mobile and moving out from there.

--Chris

Christopher Blizzard

unread,
Mar 12, 2012, 4:24:51 PM3/12/12
to Justin Dolske, dev-pl...@lists.mozilla.org
We've only seen format adoption on youtube. Basically everyone else
uses H.264 & Flash. There are occasional exceptions, but it's not
getting better with time.

--Chris

Christopher Blizzard

unread,
Mar 12, 2012, 4:26:22 PM3/12/12
to John Volikas, dev-pl...@lists.mozilla.org
On 3/12/2012 8:55 AM, John Volikas wrote:
> If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for<audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

I agree.

--Chris

Christopher Blizzard

unread,
Mar 12, 2012, 4:35:44 PM3/12/12
to Ms2ger, dev-pl...@lists.mozilla.org
On 3/12/2012 11:28 AM, Ms2ger wrote:
>
> Did you deliberately write the subject of this thread in such a way
> that it is easy to miss that you are trying to reopen the issue of
> supporting patent-encumbered codecs, against the policy we have had
> ever since the video element was first implemented, and have you
> considered the effect that would have on Mozilla's credibility?

I'm not going to speak to Andreas' original message since I didn't write
it. But I would say to the issue of credibility that right now many web
developers feel like Firefox isn't serving their needs since they want
us to support H.264. So from the outside-in perspective we're seen as a
problem, not providing a solution.

I'd love to stand up and support open codecs. (We still will!) I think
it's a shame that MPEG LA & Micrsoft FUD combined with Google's
undermining of their own effort has gotten us to the place where are
today, but here we are. This is reality and we've been trying to change
it for a couple of years on our own but we haven't had much success.

So I'll flip this around: what would you do in this case?

--Chris

Ms2ger

unread,
Mar 12, 2012, 4:48:01 PM3/12/12
to
Support open codecs in the one way we can: refuse to support
patent-encumbered codecs. Once we support them, we have no leverage left
to argue against them, and our credibility in standards organizations
will be reduced even more–after all, if we give up on making sure it is
possible for anybody, anywhere, to build a browser that is compatible
with the web, without requiring payment for that, why, then, would we be
able to take a strong stance on other issues? Why would other browser
vendors not be correct to believe that we will give in eventually, and
that our arguments need not be taken into consideration?

The argument that this change would only affect those of our users who
already have a patent-encumbered codec installed is, of course,
short-sighted. Once Firefox on recent Windows versions supports H.264,
*all* browsers will be required to support it as well, because authors
will no longer see a reason to use an open codec.

For these reasons, I believe we should use the one weapon we have in
this battle-field, and that is continuing to refuse to support H.264.

HTH
Ms2ger

Justin Lebar

unread,
Mar 12, 2012, 4:51:55 PM3/12/12
to Christopher Blizzard, Ms2ger, dev-pl...@lists.mozilla.org
> So I'll flip this around: what would you do in this case?

I feel like our PR on this topic has been pretty non-existent,
particularly if we consider the degree to which most of us don't want
to add support for proprietary codecs.

If I were in marketing (thank goodness I'm not!), I'd publicly call on
Google to fulfill its promises of old. I'd communicate through
official channels why we don't want to support H.264, MP3, etc, and
why we think Google is harming the web. I'd explore legal options,
like whether Google could address the issue of submarine patents by
promising to in some way legally support WebM users. And I might set
a public deadline -- if Google doesn't un-support H.264 by date X,
then we'll start supporting system H.264 and MP3 codecs.

Maybe these are bad ideas. But I feel like we're going down without a
real fight, and that makes me sad. Surely we're not totally helpless
here.

-Justin

On Mon, Mar 12, 2012 at 4:35 PM, Christopher Blizzard
<bliz...@mozilla.com> wrote:
> On 3/12/2012 11:28 AM, Ms2ger wrote:
>>
>>
>> Did you deliberately write the subject of this thread in such a way that
>> it is easy to miss that you are trying to reopen the issue of supporting
>> patent-encumbered codecs, against the policy we have had ever since the
>> video element was first implemented, and have you considered the effect that
>> would have on Mozilla's credibility?
>
>
> I'm not going to speak to Andreas' original message since I didn't write it.
>  But I would say to the issue of credibility that right now many web
> developers feel like Firefox isn't serving their needs since they want us to
> support H.264.  So from the outside-in perspective we're seen as a problem,
> not providing a solution.
>
> I'd love to stand up and support open codecs.  (We still will!)  I think
> it's a shame that MPEG LA & Micrsoft FUD combined with Google's undermining
> of their own effort has gotten us to the place where are today, but here we
> are.  This is reality and we've been trying to change it for a couple of
> years on our own but we haven't had much success.
>
> So I'll flip this around: what would you do in this case?
>
> --Chris

Justin Dolske

unread,
Mar 12, 2012, 4:54:18 PM3/12/12
to
On 3/12/12 1:24 PM, Christopher Blizzard wrote:

>> Rather that just rehashing the same arguments, I'd like to ask what's
>> changed such that we need to reopen this issue?
>
> We've only seen format adoption on youtube. Basically everyone else uses
> H.264 & Flash. There are occasional exceptions, but it's not getting
> better with time.

Who is "everyone"?

I think it _has_ gotten better with time... When we started H.264 was
everywhere and there was no other option. Now YouTube is available in
WebM (no small feat!), Bing uses Theora for video backgrounds on their
front page, and now even the IE Blog is touting IE10 with WebM.
[http://blogs.msdn.com/b/ie/archive/2012/02/29/windows-consumer-preview-the-fifth-ie10-platform-preview.aspx]

The state of HTML5 video started off from a bad place, and to be fair
still isn't in a good place. So reassessing Mozilla stance is not
unreasonable.

But I think if Mozilla is going to do an about-face on open video
standards (and it _is_ an about-face), then there should be some serious
discussion about it. Certainly more than than a few terse words saying
it's hopeless and obvious. That makes it sound more like a half-hearted
notification of a decision that's already final.

At the very least, it needs to be explained enough so the community can
understand the change. We spent a lot of time and made a lot of blog
posts about why H.264 was bad for the web. Leaving those who advocated
for us suddenly high-and-dry doesn't feel like the right thing to do.

Justin

Robert O'Callahan

unread,
Mar 12, 2012, 5:10:34 PM3/12/12
to Daniel Cater, dev-platform, mozilla.de...@googlegroups.com
On Mon, Mar 12, 2012 at 5:44 PM, Daniel Cater <djc...@gmail.com> wrote:

> In particular, given
> http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.htmland
> http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I
> expect Robert, Chris Pearce, Chris Double and others to be included in this
> discussion.
>

Hi.

With more browsers, sites and hardware manufacturers working on existing or
> upcoming WebM / VP8 support I would be very disappointed to see Mozilla
> cave on this issue.
>

Which browsers? I have not heard that significant browsers that don't
already support WebM (Safari, IE) are planning to add WebM support.

Which sites? I have not heard of major sites that don't already support
WebM (almost everyone except Youtube) planning to add WebM support.

There's a bit more hardware support emerging, but nothing that affects the
equation for Web sites.

I agree with almost everything Andreas and Chris Blizzard have written on
this thread, except I'm not happy about its timing or the way it has been
presented.

Rob
--
“You have heard that it was said, ‘Love your neighbor and hate your enemy.’
But I tell you, love your enemies and pray for those who persecute you,
that you may be children of your Father in heaven. ... If you love those
who love you, what reward will you get? Are not even the tax collectors
doing that? And if you greet only your own people, what are you doing more
than others?" [Matthew 5:43-47]

Robert O'Callahan

unread,
Mar 12, 2012, 5:11:50 PM3/12/12
to Justin Dolske, dev-pl...@lists.mozilla.org
On Mon, Mar 12, 2012 at 8:54 PM, Justin Dolske <dol...@mozilla.com> wrote:

> But I think if Mozilla is going to do an about-face on open video
> standards (and it _is_ an about-face), then there should be some serious
> discussion about it. Certainly more than than a few terse words saying it's
> hopeless and obvious. That makes it sound more like a half-hearted
> notification of a decision that's already final.
>
> At the very least, it needs to be explained enough so the community can
> understand the change. We spent a lot of time and made a lot of blog posts
> about why H.264 was bad for the web. Leaving those who advocated for us
> suddenly high-and-dry doesn't feel like the right thing to do.


Beyond that very least, what procedure would you have us follow?

Dao

unread,
Mar 12, 2012, 5:26:15 PM3/12/12
to
On 12.03.2012 22:10, Robert O'Callahan wrote:
> Which sites? I have not heard of major sites that don't already support
> WebM (almost everyone except Youtube) planning to add WebM support.

Why would sites make their plans public?

German public television started serving WebM recently:
http://www.tagesschau.de/multimedia/video/video1078792.html

Robert O'Callahan

unread,
Mar 12, 2012, 5:30:33 PM3/12/12
to Justin Lebar, Ms2ger, Christopher Blizzard, dev-pl...@lists.mozilla.org
On Mon, Mar 12, 2012 at 8:51 PM, Justin Lebar <justin...@gmail.com>wrote:

> If I were in marketing (thank goodness I'm not!), I'd publicly call on
> Google to fulfill its promises of old. I'd communicate through
> official channels why we don't want to support H.264, MP3, etc, and
> why we think Google is harming the web.


It would be unfair to single out Google as harming the Web. They've done
more for open video codecs than most other industry players have done (i.e,
from "nothing" to "actively opposing").

I'd explore legal options,
> like whether Google could address the issue of submarine patents by
> promising to in some way legally support WebM users.


Only Google can do that. We talked to them about it, and they haven't done
it.

And I might set
> a public deadline -- if Google doesn't un-support H.264 by date X,
> then we'll start supporting system H.264 and MP3 codecs.
>

That might be a good PR tactic to cover our butts, but realistically I
don't think it will move them. We've been begging them in private for some
time.

Maybe these are bad ideas. But I feel like we're going down without a
> real fight, and that makes me sad. Surely we're not totally helpless
> here.
>

We have fought. We, alone of all major browsers (sorry Opera desktop), have
held out against supporting patent-encumbered codecs for a long time. I
feel it's grossly unfair to our efforts to describe that as "not a real
fight".

We held the line in the hope that the industry would follow, and that
Google would do a lot to improve and support WebM, especially removing
H.264 support from Chrome. So we've held the line, and watched, and waited,
and personally I am extremely disappointed by the results.

Gijs Kruitbosch

unread,
Mar 12, 2012, 5:34:57 PM3/12/12
to
I have to say that having read the entire thread I am left confused. Two of the
main arguments I heard for 'giving up' the H.264 battle were:

1) No new browsers/sites are moving to WebM
2) On mobile we need hardware implementation support, so we need H.264

Then there was an alternative message about giving in here so we'll have a voice
to yell over the next fight. I personally think that could go both ways (no
users -> no voice vs. you gave up last time, so you'll cave later on this one, too).

But, back to those two main reasons. Several other messages in the thread,
people have mentioned that IE10 will support / supports WebM (where it did not
before), that 'most' video content sites support it, and that the broadcom
chipset which is being targeted also supports WebM and Theora decoding, it just
hasn't been marketed.

That makes it sound like neither (1) nor (2) are good reasons. I'm imagining
that the counter-claims are somehow wrong or incomplete or that there are other
reasons that I'm missing, but I haven't seen those expressed. Can someone help
me understand?

~ Gijs

Chris Double

unread,
Mar 12, 2012, 5:35:46 PM3/12/12
to Daniel Cater, dev-platform, mozilla.de...@googlegroups.com
On Tue, Mar 13, 2012 at 6:44 AM, Daniel Cater <djc...@gmail.com> wrote:
>
> In particular, given http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.html and http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I expect Robert, Chris Pearce, Chris Double and others to be included in this discussion.

Unfortunately what used to be the video team doesn't really have much
say in the direction of how video is supported in Firefox these days.
We usually find out late in the piece what the plans are and what is
going to be implemented and we get tasked with implementing it. This
is, I assume, because much of video support is sensitive or tied up
with NDA's and the like so we don't find out until decisions are
mostly made. Note this is my opinion only, not necessarily the opinion
of the others you've asked for input from.

Chris.
--
http://www.bluishcoder.co.nz

Dao

unread,
Mar 12, 2012, 5:39:46 PM3/12/12
to
On 12.03.2012 22:34, Gijs Kruitbosch wrote:
> people have mentioned that IE10 will support / supports WebM
> (where it did not before),

IE10 doesn't and won't support WebM. The IEBlog site embedded a WebM
video.
http://blogs.msdn.com/b/ie/archive/2012/02/29/windows-consumer-preview-the-fifth-ie10-platform-preview.aspx

Chris Pearce

unread,
Mar 12, 2012, 5:55:48 PM3/12/12
to dev-pl...@lists.mozilla.org
On 13/03/2012 10:30 a.m., Robert O'Callahan wrote:
> We held the line in the hope that the industry would follow, and that
> Google would do a lot to improve and support WebM, especially removing
> H.264 support from Chrome. So we've held the line, and watched, and waited,
> and personally I am extremely disappointed by the results.

+1. And we have bled and lost support and of the early adopters and
created fostered ill-will in the webdev community because of this issue.

If Google were to announce that they were to drop support of H.264 in
Chrome tomorrow, we'd still have IE and Safari holding out (most people
won't bother to install the required WebM plugins for said browsers). I
don't think Safari ever would about face, and there's only a remote
chance IE would change.

On 13/03/2012 6:31 a.m., Ralph Giles wrote:
> On 12/03/12 09:27 AM, Andreas Gal wrote:
>
>> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.
> If we're going to compromise our principles on this issue, we should
> license a proprietary decoder implementation and ship it with our
> official desktop builds.

I agree with Ralph: the best solution for our users is to suck it up and
pay the H.264/AAC/MP3 license fees and ship libav/ffmpeg on desktop, and
ship hardware decoders on mobile. If necessary we can patch security
holes in libav, so we're not vulnerable to the black-box problem we get
with system codecs on desktop, and we can offer H.264 support to the 40%
of our users on XP - something that IE can't do.


On 13/03/2012 7:44 a.m., Andreas Gal wrote:
> Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.

In my darker moments I've wondered if this was a deliberate strategy by
Chrome to make us continue to hold an untenable position and continue to
bleed users.

Having said that, it would be a shame if YouTube turned on WebM by
default and Chrome dropped H.264 support just as we agreed to pickup
H.264/MP3 support on desktop. We should reach out to Google through
offical channels, and try to get them to publicly commit to the above
before we make a decision.

> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals.

We owe our success to the users who use our product (Firefox), not to
the downstream users of our code. And we owe our continued success to
retaining said users.

Regards,
Chris P.

Philipp Wagner

unread,
Mar 12, 2012, 6:04:45 PM3/12/12
to
Replying on top because it doesn't fit to a specific quote in the thread:

Have we actually done everything in our power to use the available
hardware for efficient WebM decoding? I don't mean special decoder chips
(which are not available in today's phones AFAIK), but things like the
NEON extensions for ARM etc.?

Philipp

Am 12.03.2012 08:28, schrieb Andreas Gal:
>
> I want to land bug 714408 on mozilla-central as soon as I get review
> for it. It adds hardware-accelerated audio/video decoding support to
> Gecko using system decoders already present on the system. Android,
> for example, ships by default with a number of decoders, and in
> particular for such mobile devices we really have to use these
> hardware-accelerated decoders for good battery life (and
> performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will
> add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present on
> the system, including H.264 and MP3. There is really no justification
> to stop our users from using system decoders already on the device,
> so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI,
> which is currently internal to Gecko, but we might expose it via
> NPAPI to external decoder providers as well. MPAPI helps us deal with
> incompatibilities in systems decoders (without risking Gecko not
> starting up which would happen if we try to link against the system
> decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser
> and fully integrates with our rendering pipeline (CSS and all). On
> Android we might have to add a second video path using overlays which
> would only work with a small subset of CSS since extracting video
> frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open
> video. We will continue to promote and support open codecs, but when
> and where existing codecs are already installed and licensed on
> devices we will make use of them in order to provide people with the
> best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Ralph Giles

unread,
Mar 12, 2012, 6:33:34 PM3/12/12
to Philipp Wagner, dev-pl...@lists.mozilla.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon 12 Mar 2012 03:04:45 PM PDT, Philipp Wagner wrote:

> Have we actually done everything in our power to use the available
> hardware for efficient WebM decoding? I don't mean special decoder
> chips (which are not available in today's phones AFAIK), but things
> like the NEON extensions for ARM etc.?

We use NEON when it's available, yes, and Tim Terriberry has
contributed many optimizations upstream.

Special decoder chips are an interesting story. NVidia's page on the
Tegra 2[1] advertises VP8 support, but there doesn't seem to be a
driver for it on the Samsung Tablets or the other mobile devices we've
tried. TI demonstrated 1080p VP8 decoding on their OMAP chips 18 months
ago[2]; this is the chip that's in the Samsung Galaxy SII we're using
to develop boot to gecko. And yet, it's unclear if Android is using it.
TI's Ubuntu PPA for the OMAP4 supports VP6 and VP7...but not VP8.

So there *is* hardware decoder support in shipped high end devices, it
just doesn't seem to be hooked up.

[1] http://www.nvidia.com/object/tegra-2.html
[2]
http://blog.webmproject.org/2010/10/demo-of-webm-running-on-ti-omap-4.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPXnm+AAoJEEcAD3uxRB3v5l8IAJS4/Y78WSc26iwx/VqajyVY
U5KNnF/A9Cd+eLFpUPdBFfTwWw7ZEDxsd3AvfDvFckenjoXc6cxPJNydJ+jk0ZDw
RJO5dXkxeOtNsH6+P83jLjliy5uYZfvbGDsVEDnN12RPB4WfRx9O2eyIdqd8UacN
vt3XMU/5BlFjmeUOtRqEaTHZNV28kfwT4RiQLwok6SqB5Vgme5fgXnLOdvbPPUpG
Qi58UNRHpnpO1Kfz1BzOlB3C8ia7jbN+W9c2Da+q1x7+9ZEwBvbfpAio1W6qqcEQ
msJllU7JytB+nNwejaQ0aT+wVK3QihfLrKb//Ha8uaMi9bomnSxkGHxff8PiEuc=
=cxS7
-----END PGP SIGNATURE-----

Philipp Wagner

unread,
Mar 12, 2012, 7:01:18 PM3/12/12
to
Am 12.03.2012 23:33, schrieb Ralph Giles:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mon 12 Mar 2012 03:04:45 PM PDT, Philipp Wagner wrote:
>
>> Have we actually done everything in our power to use the available
>> hardware for efficient WebM decoding? I don't mean special decoder
>> chips (which are not available in today's phones AFAIK), but things
>> like the NEON extensions for ARM etc.?
>
> We use NEON when it's available, yes, and Tim Terriberry has
> contributed many optimizations upstream.
>
> Special decoder chips are an interesting story. NVidia's page on the
> Tegra 2[1] advertises VP8 support, but there doesn't seem to be a
> driver for it on the Samsung Tablets or the other mobile devices we've
> tried. TI demonstrated 1080p VP8 decoding on their OMAP chips 18 months
> ago[2]; this is the chip that's in the Samsung Galaxy SII we're using
> to develop boot to gecko. And yet, it's unclear if Android is using it.
> TI's Ubuntu PPA for the OMAP4 supports VP6 and VP7...but not VP8.
>
> So there *is* hardware decoder support in shipped high end devices, it
> just doesn't seem to be hooked up.

That's good to know. So for B2G those hardware decoders would be usable?
Has somebody looked into that?

Philipp

Justin Dolske

unread,
Mar 12, 2012, 7:24:44 PM3/12/12
to
On 3/12/12 2:11 PM, Robert O'Callahan wrote:

>> At the very least, it needs to be explained enough so the community can
>> understand the change. We spent a lot of time and made a lot of blog posts
>> about why H.264 was bad for the web. Leaving those who advocated for us
>> suddenly high-and-dry doesn't feel like the right thing to do.
>
> Beyond that very least, what procedure would you have us follow?

That's good enough for me: discuss, decide, communicate. I just wanted
to leave room for the possibility of someone wanting more. :)

Justin

Justin Dolske

unread,
Mar 12, 2012, 7:31:57 PM3/12/12
to
On 3/12/12 2:39 PM, Dao wrote:
> On 12.03.2012 22:34, Gijs Kruitbosch wrote:
>> people have mentioned that IE10 will support / supports WebM
>> (where it did not before),
>
> IE10 doesn't and won't support WebM. The IEBlog site embedded a WebM
> video.

Yeah, I was just pointing out that even though IE10 may not support it,
serving such content on Bing and Microsoft blogs is a different matter.

On that note, ISTR Microsoft saying that had no intention of supporting
Theora/WebM. But I think that was before the above happened, as well as
their acquisition of Skype (which is using V8?), and before they found
out that H.264 patents were a headache for Windows Phone
(http://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/02/22/google-please-don-t-kill-video-on-the-web.aspx?Redirected=true)

I wonder what the odds are of them reevaluating support for WebM in a
future IE release. Hmm.

Justin

Justin Dolske

unread,
Mar 12, 2012, 7:33:25 PM3/12/12
to
Yeah, supporting one without the other is probably not going to be useful.

Though I wonder how well HW-accelerated H.264 + enscripten'd AAC-in-JS
would work... :)

Justin

Robert O'Callahan

unread,
Mar 12, 2012, 9:04:47 PM3/12/12
to Ms2ger, dev-pl...@lists.mozilla.org
On Mon, Mar 12, 2012 at 8:48 PM, Ms2ger <ms2...@gmail.com> wrote:

> Support open codecs in the one way we can: refuse to support
> patent-encumbered codecs. Once we support them, we have no leverage left to
> argue against them, and our credibility in standards organizations will be
> reduced even more–after all, if we give up on making sure it is possible
> for anybody, anywhere, to build a browser that is compatible with the web,
> without requiring payment for that, why, then, would we be able to take a
> strong stance on other issues? Why would other browser vendors not be
> correct to believe that we will give in eventually, and that our arguments
> need not be taken into consideration?
>

I don't think that compromising or conceding defeat on some issue means we
no longer have credibility in general. We don't have to appear infallible
and invincible to have leverage. People have to know that we'll fight long
and hard for what we think is right, which we've done here.

For these reasons, I believe we should use the one weapon we have in this
> battle-field, and that is continuing to refuse to support H.264.
>

I don't think it's working. With Google effectively abandoning us by not
removing H.264 from Chrome*, and Adobe's support for WebM never
materializing, I believe our stance is doing little more than hurting Web
developers and Firefox users.

* Actually this characterization is unfair to us. Google never even joined
us.

Robert O'Callahan

unread,
Mar 12, 2012, 9:11:50 PM3/12/12
to Ralph Giles, dev-pl...@lists.mozilla.org, Philipp Wagner
On Mon, Mar 12, 2012 at 10:33 PM, Ralph Giles <gi...@mozilla.com> wrote:

> Special decoder chips are an interesting story. NVidia's page on the
> Tegra 2[1] advertises VP8 support, but there doesn't seem to be a
> driver for it on the Samsung Tablets or the other mobile devices we've
> tried. TI demonstrated 1080p VP8 decoding on their OMAP chips 18 months
> ago[2]; this is the chip that's in the Samsung Galaxy SII we're using
> to develop boot to gecko. And yet, it's unclear if Android is using it.
>

>From discussions with Google people, I'm pretty sure it's not.

The fact that Android doesn't even use what WebM hardware support there is
is one aspect of Google's general apathy towards WebM.

Ragavan Srinivasan

unread,
Mar 12, 2012, 9:16:24 PM3/12/12
to

Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review
> for it. It adds hardware-accelerated audio/video decoding support to
> Gecko using system decoders already present on the system. Android,
> for example, ships by default with a number of decoders, and in
> particular for such mobile devices we really have to use these
> hardware-accelerated decoders for good battery life (and
> performance).

I realize I'm coming in pretty late, but for what it's worth, I am
absolutely thrilled this is happening from an Apps product perspective.
This is going to go a long way towards supporting audio/video apps and
ensuring users have a great experience.


> Initially this will be enabled on Gonk (B2G). In a few weeks we will
> add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present on
> the system, including H.264 and MP3. There is really no justification
> to stop our users from using system decoders already on the device,
> so we will not filter any formats.

Who is responsible for the Android support (B2G team or someone else?)
and what versions of Android are we planning on supporting? I realize
it's early, but for Apps, Android is also pretty high on the list, so
I'm curious as to when this will happen as well.

Thanks again for making this happen!

Regards,
Ragavan

Robert O'Callahan

unread,
Mar 12, 2012, 9:17:18 PM3/12/12
to Dao, dev-pl...@lists.mozilla.org
On Mon, Mar 12, 2012 at 9:26 PM, Dao <d...@design-noir.de> wrote:

> On 12.03.2012 22:10, Robert O'Callahan wrote:
>
>> Which sites? I have not heard of major sites that don't already support
>> WebM (almost everyone except Youtube) planning to add WebM support.
>>
>
> Why would sites make their plans public?
>

They might not, but we talk to people as well.

German public television started serving WebM recently:
> http://www.tagesschau.de/**multimedia/video/video1078792.**html<http://www.tagesschau.de/multimedia/video/video1078792.html>


I didn't know that, and that's great, but I'm afraid it's only a drop in
the bucket.

Asa Dotzler

unread,
Mar 12, 2012, 9:54:54 PM3/12/12
to
On 3/12/2012 2:55 PM, Chris Pearce wrote:
> Having said that, it would be a shame if YouTube turned on WebM by
> default and Chrome dropped H.264 support just as we agreed to pickup
> H.264/MP3 support on desktop. We should reach out to Google through
> offical channels, and try to get them to publicly commit to the above
> before we make a decision.

We have been reaching out to them for months and months. It hasn't worked.

- A

Asa Dotzler

unread,
Mar 12, 2012, 9:58:18 PM3/12/12
to
On 3/12/2012 11:44 AM, Andreas Gal wrote:
> Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.
>
> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

Actually, we can and should license decoders and ship those where
platforms don't have built-in support. Not doing so means abandoning
about half of our users today.

- A

Asa Dotzler

unread,
Mar 12, 2012, 10:01:50 PM3/12/12
to
On 3/12/2012 7:32 AM, Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
>
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop. If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.
>
> joe

Yes. Absolutely. h.264+AAC is the de facto standard for the <video>
element and if we're going to support it, we should support it everywhere.

- A

Asa Dotzler

unread,
Mar 12, 2012, 10:02:46 PM3/12/12
to
On 3/12/2012 7:40 AM, Blake Winton wrote:
> On 12-03-12 10:32 , Joe Drew wrote:
>> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>>> There is really no justification to stop our users from using system
>>> decoders already on the device, so we will not filter any formats.
>> Further to my earlier message: there is exactly as much justification
>> stopping our users from using system decoders on mobile as there is on
>> desktop.
>
> I see a slight difference in justification, in that 40% of our desktop
> users (Windows XP) don't get an h.264 codec from their system, so
> enabling system codecs on desktop won't help nearly as much of our
> userbase, and will fragment the desktop version of gecko, and make it
> harder for web developers to test their sites.

Which is why we have to license and ship the decoders on Windows XP and
Linux.

- A

Asa Dotzler

unread,
Mar 12, 2012, 10:08:25 PM3/12/12
to
On 3/12/2012 7:47 AM, Doug Turner wrote:

> As I understand it, what are we are doing it basically waiting it.
> We are hoping that the sites transcode to WebM. Sites are slowing
> doing this for Desktop where FF has a significant marketshare
> (YouTube committed to this, but I don't think it has completed its
> conversion yet). Mobile is a bit different. We do not have any
> significant marketshare and leverage is hard. Sites will know only
> have to do the conversation for Firefox desktop, but realized that
> Firefox for Android exists and they should serve mobile content with
> both h.264 and WebM hurts Firefox for Android more that it helps.
> This is a long bet. And I think we need to make Firefox for Android
> not have signifiant deficiencies from the default browser.

We're not just waiting. We're talking with major video sites and they're
saying "no" to WebM. The costs of transcoding huge libraries just
doesn't make sense to them.

Firefox on Desktop is experiencing these same 'significant deficiencies"
and the tide is not turning in any appreciable way. All that's happening
while we wait is that Web developers are embracing other browsers and
their primary targets.

What I fear people aren't getting here is that Gecko is the _only_
mainstream browser that doesn't support h.264. We lost. It's not like
we're at some tipping point and it's 3 of 4 browsers on the side of
royalty free codecs with the forth about to agree. Things have tipped
the other way and to not realize that and continue to hold out for a
change that will not happen does little but cost us users and developer
mindshare.

It's time to bite that particular bullet and deliver h.264+AAC (and
probably mp3) in Firefox -- across all platforms and devices.

- A

Justin Lebar

unread,
Mar 12, 2012, 10:09:12 PM3/12/12
to Asa Dotzler, dev-pl...@lists.mozilla.org
> Which is why we have to license and ship the decoders on Windows XP and
> Linux.

Maybe this is a dumb question, but how will this affect Linux distros
which build Firefox but call it TemperatureAnimal? Will they be able
to use the decoders we've licensed?

Asa Dotzler

unread,
Mar 12, 2012, 10:13:21 PM3/12/12
to
On 3/12/2012 12:14 PM, Jeff Walden wrote:
> On 03/12/2012 09:30 AM, Andreas Gal wrote:
>>> Do we want to prevent fully open source and free implementations
>>> (including hw) to be able to render everything in the internet?
>>
>> Yes, we do.
>
> This is worth highlighting. And highlighting again. And again. And
> again.
>
> The web platform should be based on open standards. I disagree
> profoundly with moves that take us in the opposite direction.
>
> Jeff

We've got a rich history of stepping back from religious purity on
standards to deliver something that just works. I don't think our
implementing document.all prevented us from saying no and holding the
line on plenty of other crazy DOM features.

- A

Andreas Gal

unread,
Mar 12, 2012, 10:10:50 PM3/12/12
to Asa Dotzler, dev-pl...@lists.mozilla.org
Nobody said we will abandon anyone. I am confident we will find a way that doesn't require us to ship encumbered codecs. We can create an opportunity for third parties to plug the shrinking (but still large) WinXP gap. Everywhere else the solution is obvious.

Sent from Mobile.
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

Asa Dotzler

unread,
Mar 12, 2012, 10:18:13 PM3/12/12
to
On 3/12/2012 10:31 AM, Ralph Giles wrote:
> On 12/03/12 09:27 AM, Andreas Gal wrote:
>
>> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.
>
> If we're going to compromise our principles on this issue, we should
> license a proprietary decoder implementation and ship it with our
> official desktop builds. That way we have
>
> a) consistent support on all platforms, including Windows XP;
>
> b) a smaller maintenance burden than calling different backends on each
> platform;
>
> c) smaller increase in security footprint;
>
> d) better resistance to DRM implementation pressure than a pass-through
> scheme offers.
>
> That would be the honest way to support non-free codecs on the Web. They
> cost money, and current licensing is incompatible with free software
> implementation. We should be as transparent about that state of affairs
> as we can.


Thank you Ralph for laying this out. I agree with it 100%. We should
license and ship these decoders. If there's a major cost difference (
don't know the economics very well) then I'd be fine with just shipping
them where they're missing (more code burden) and relying on system
stuff where it already exists. This will ensure that Web developers have
a clear target and that users get a consistent behavior. That's good for
the Web and good for our users.

- A

Asa Dotzler

unread,
Mar 12, 2012, 10:24:43 PM3/12/12
to
I don't think so. Ideally, Ubuntu and other distros would foot the bill
for a codec on the system (maybe we could help them with that) and then
we'd have a solution that was accessible to a larger set of application
developers.

Or maybe we could do it in such a way that Firefox deposited the codec
into the system where it could be used by any other app. That would
still require, I think, users getting Firefox from us and not from a
distro.

Then again, I could be totally wrong.

- A

Chris Pearce

unread,
Mar 12, 2012, 10:25:07 PM3/12/12
to dev-pl...@lists.mozilla.org
On 13/03/2012 3:09 p.m., Justin Lebar wrote:
>> Which is why we have to license and ship the decoders on Windows XP and
>> Linux.
> Maybe this is a dumb question, but how will this affect Linux distros
> which build Firefox but call it TemperatureAnimal? Will they be able
> to use the decoders we've licensed?
>

Probably not. Could we create a plugin (such as Andreas is suggesting in
bug 714408) which works in a generic Gecko runtime, purchase licenses
for use in a "generic Gecko runtime", and bundle that with Firefox? Then
any downstream Gecko user can either bundle that (if it's legal?) or
have their app/users download that and install it so they use the
encumbered codecs?

Chris P.

Asa Dotzler

unread,
Mar 12, 2012, 10:26:56 PM3/12/12
to
On 3/12/2012 7:10 PM, Andreas Gal wrote:
> Nobody said we will abandon anyone. I am confident we will find a way that doesn't require us to ship encumbered codecs. We can create an opportunity for third parties to plug the shrinking (but still large) WinXP gap. Everywhere else the solution is obvious.

OK. Sounds good to me. I can't imagine why someone else would pay the
licensing fees to deliver the codec to Firefox users for free, (unless
it was Adobe or someone already paying those fees) but I'll take your
word for it.

- A

Chris Pearce

unread,
Mar 12, 2012, 10:28:41 PM3/12/12
to dev-pl...@lists.mozilla.org
On 13/03/2012 3:18 p.m., Asa Dotzler wrote:
> We should license and ship these decoders. If there's a major cost
> difference ( don't know the economics very well) then I'd be fine with
> just shipping them where they're missing (more code burden) and
> relying on system stuff where it already exists.

If the cost isn't too great, licensing and shipping decoders would be my
preferred solution, since it's by far the quickest path to market, has
the smallest maintenance overhead, and easiest way to ensure consistent
cross platform behaviour.


Chris P.



Asa Dotzler

unread,
Mar 12, 2012, 10:34:35 PM3/12/12
to
I think it's only single digit millions per year. Not outrageous.

- A

therealbr...@gmail.com

unread,
Mar 12, 2012, 10:38:59 PM3/12/12
to mozilla.de...@googlegroups.com, dev-pl...@lists.mozilla.org
On Monday, March 12, 2012 2:55:48 PM UTC-7, Chris Pearce wrote:
> On 13/03/2012 10:30 a.m., Robert O'Callahan wrote:
> > We held the line in the hope that the industry would follow, and that
> > Google would do a lot to improve and support WebM, especially removing
> > H.264 support from Chrome. So we've held the line, and watched, and waited,
> > and personally I am extremely disappointed by the results.
>
> +1. And we have bled and lost support and of the early adopters and
> created fostered ill-will in the webdev community because of this issue.

Right, and Google Chrome has not.

Google may not have intended to punk us into hurting ourselves (I don't think they did) but they certainly have wisely avoided hurting Chrome's market share by turning off H.264 decoding from <video>.

Even now, they could IMHO well afford to drop H.264 support from <video> on *desktop* Chrome, because they have best-in-class Flash integration on desktop, and because authors typically write <object> fallback inside <video>. They have win/win now, and win later should they some day drop H.264 from <video>.

Think about this, you who are claiming Mozilla did not hold out or pay a high enough price.

Think in particular about your own vaunted purity. I argue you are hiding behind Mother Adobe's skirts right now, since Firefox users absolutely rely on Flash to play H.264 video embedded in <video> fallback <object> elements. So please don't sermonize or lecture -- take a realistic view of the entire fallback logic chain, and of Firefox's current acute dependency on Flash.

If we rely on Flash to play H.264 now, how pure are we? Compared to using a system decoder, not any more pure, in my view.


> If Google were to announce that they were to drop support of H.264 in
> Chrome tomorrow, we'd still have IE and Safari holding out (most people
> won't bother to install the required WebM plugins for said browsers). I
> don't think Safari ever would about face, and there's only a remote
> chance IE would change.

Again, Chrome on desktop would be most ready to drop H.264 from <video> because they have practically custom-forked Flash into best-of-breed fallback.

Chrome on Android 4? I doubt they'll drop H.264 from video. Stock browser on all Android? No way.

> I agree with Ralph: the best solution for our users is to suck it up and
> pay the H.264/AAC/MP3 license fees and ship libav/ffmpeg on desktop, and
> ship hardware decoders on mobile. If necessary we can patch security
> holes in libav, so we're not vulnerable to the black-box problem we get
> with system codecs on desktop, and we can offer H.264 support to the 40%
> of our users on XP - something that IE can't do.

We should proceed carefully. Mozilla will not pay for an H.264 license that does not benefit downstreams. I personally do not believe we should pay any such rent. It's better for us to ride on an OS decoder if possible (we ride on Flash now, see above).

OS dependencies are ugly but browsers including Mozilla and Firefox have taken them since day 1 of the modern (Netscape and beyond) web.

> On 13/03/2012 7:44 a.m., Andreas Gal wrote:
> > Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.
>
> In my darker moments I've wondered if this was a deliberate strategy by
> Chrome to make us continue to hold an untenable position and continue to
> bleed users.

I don't think Google did anything specifically to hurt us (we did, rather), but they clearly are helping themselves by not turning off H.264 in Chrome, and they're not turning it off in Android stock browser either. And on desktop they have invested so much in Flash that they could afford to turn off H.264 from <video>, but so what? Money talks.

> Having said that, it would be a shame if YouTube turned on WebM by
> default and Chrome dropped H.264 support just as we agreed to pickup
> H.264/MP3 support on desktop. We should reach out to Google through
> offical channels, and try to get them to publicly commit to the above
> before we make a decision.

We can reassess. Right now and this calendar year, the die is cast.

/be
Message has been deleted

Nicholas Nethercote

unread,
Mar 12, 2012, 11:16:25 PM3/12/12
to rob...@ocallahan.org, dev-platform, Daniel Cater, mozilla.de...@googlegroups.com
On Mon, Mar 12, 2012 at 2:10 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
>
> I agree with almost everything Andreas and Chris Blizzard have written on
> this thread, except I'm not happy about its timing or the way it has been
> presented.

Exactly. A change in a highly controversial policy position shouldn't
happen like this. (I disagree strongly with the notion that mobile
and desktop can be treated separately on this issue.)

Nick

Andreas Gal

unread,
Mar 12, 2012, 11:22:34 PM3/12/12
to Nicholas Nethercote, mozilla.de...@googlegroups.com, dev-platform, rob...@ocallahan.org, Daniel Cater
What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?

Andreas

>
> Nick

Joe Drew

unread,
Mar 12, 2012, 11:32:47 PM3/12/12
to Asa Dotzler, dev-pl...@lists.mozilla.org
Please don't conflate our purity (or lack thereof) on standards with our patent purity. This thread, despite its misleading subject line, is about shipping something that makes a basic primitive on the web unimplementable in a free stack in order to benefit our users.

document.all was easily and freely implemented by one and all, and had the great side effect of helping our users use the real web. h.264 and AAC only do the latter.
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Joe Drew

unread,
Mar 12, 2012, 11:35:41 PM3/12/12
to Andreas Gal, Nicholas Nethercote, Daniel Cater, dev-platform, rob...@ocallahan.org, mozilla.de...@googlegroups.com
A) hiding behind a subject line that nobody would object to; that is, not calling out the explicit and blatant change in policy it represents.
B) pretending that this affects only mobile. The same arguments apply to desktop.

Luckily the conversation has been steered in the right direction.

Sent from my mobile
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Andreas Gal <g...@mozilla.com> wrote:


On Mar 12, 2012, at 11:16 PM, Nicholas Nethercote wrote:

> On Mon, Mar 12, 2012 at 2:10 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
>>
>> I agree with almost everything Andreas and Chris Blizzard have written on
>> this thread, except I'm not happy about its timing or the way it has been
>> presented.
>
> Exactly. A change in a highly controversial policy position shouldn't
> happen like this. (I disagree strongly with the notion that mobile
> and desktop can be treated separately on this issue.)

What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?

Andreas

>
> Nick

Andreas Gal

unread,
Mar 12, 2012, 11:44:09 PM3/12/12
to Joe Drew, dev-platform, Daniel Cater, Nicholas Nethercote, rob...@ocallahan.org, mozilla.de...@googlegroups.com

On Mar 12, 2012, at 11:35 PM, Joe Drew wrote:

> A) hiding behind a subject line that nobody would object to; that is, not calling out the explicit and blatant change in policy it represents.

Calling a post to dev-platform about this "hiding" is simply absurd. This is the longest thread on dev-platform in months. Its on various news sites. Nobody is at risk of missing this. The bug has been in the open for weeks. A large number of people have been following along even before I posted about this. I am sorry you don't agree with the content of the post, but I suggest we focus on technical arguments instead of arguing over titles of emails.

> B) pretending that this affects only mobile. The same arguments apply to desktop.

Nobody pretends this only affects mobile. Please go back and read the post and the bug. We have a solution for mobile in hand. We want to land that. Then we want to focus on desktop.

Andreas

>
> Luckily the conversation has been steered in the right direction.
>
> Sent from my mobile
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>
> Andreas Gal <g...@mozilla.com> wrote:
>
> On Mar 12, 2012, at 11:16 PM, Nicholas Nethercote wrote:
>
> > On Mon, Mar 12, 2012 at 2:10 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> >>
> >> I agree with almost everything Andreas and Chris Blizzard have written on
> >> this thread, except I'm not happy about its timing or the way it has been
> >> presented.
> >
> > Exactly. A change in a highly controversial policy position shouldn't
> > happen like this. (I disagree strongly with the notion that mobile
> > and desktop can be treated separately on this issue.)
>
> What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?
>
> Andreas
>
> >
> > Nick
> >
>

Asa Dotzler

unread,
Mar 12, 2012, 11:46:13 PM3/12/12
to
On 3/12/2012 8:32 PM, Joe Drew wrote:
> Please don't conflate our purity (or lack thereof) on standards with
> our patent purity. This thread, despite its misleading subject line,
> is about shipping something that makes a basic primitive on the web
> unimplementable in a free stack in order to benefit our users.

I think this is about both. I think future fights on codecs will
definitely be about both. But I take your point. My point isn't so much
concerned with that distinction though.

Our shipping or not shipping h.264 doesn't make the Web unimplementable
in a free stack. The dominance of h.264 video on the Web -- something
our 3 years of Open Video support has not changed or even slowed down,
is what makes the Web unimplementable in a free stack.

What we ship is not always the same thing as "the Web." We can influence
the Web some of the time. We might even be able to kill a bad idea
single-handedly some of the time. But sometimes the Web decides to go
where we don't want it and there's nothing we can do to stop it. I
believe this is one of those cases and what we do now by not supporting
the Web, as it is, is to lose developers and users.

- A

Brendan Eich

unread,
Mar 12, 2012, 11:47:07 PM3/12/12
to
On Mar 12, 8:35 pm, Joe Drew <j...@mozilla.com> wrote:
> A) hiding behind a subject line that nobody would object to; that is, not calling out the explicit and blatant change in policy it represents.

What policy? Unencumbered-only-or-bust? Not my policy, nor mitchell's.

FWIW shaver now concedes this is the realistic course for mobile.

> B) pretending that this affects only mobile. The same arguments apply to desktop.

Wrong. Flash fallback on desktop may be the better (still OS-
dependent, at bottom) course. This is about pragmatics, not high do-or-
die principle.

/be

Nicholas Nethercote

unread,
Mar 13, 2012, 12:02:14 AM3/13/12
to Andreas Gal, mozilla.de...@googlegroups.com, dev-platform, rob...@ocallahan.org, Daniel Cater
On Mon, Mar 12, 2012 at 8:22 PM, Andreas Gal <g...@mozilla.com> wrote:
>>
>> Exactly.  A change in a highly controversial policy position shouldn't
>> happen like this.  (I disagree strongly with the notion that mobile
>> and desktop can be treated separately on this issue.)
>
> What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?

This is a discussion to reach consensus? Ok, good!

Your original post sounded to me like you were saying that this
decision was a done deal, particularly the "I don't think this bug
significantly changes our position on open video" bit. Apologies if I
misinterpreted it.

Nick

Andreas Gal

unread,
Mar 13, 2012, 12:17:54 AM3/13/12
to Nicholas Nethercote, mozilla.de...@googlegroups.com, dev-platform, rob...@ocallahan.org, Daniel Cater
"I want to land bug 714408 on mozilla-central as soon as I get review for it."

I am expressing my desire to do something, and I am asking for feedback about it. I am not even a peer for the video code (first time contributor in fact). The module owners (including Roc and Brendan) and peers will make the necessary decisions once the dust settles (including granting review or not). We have a solid project governance structure. Have faith in it.

Andreas

>
> Nick

Henri Sivonen

unread,
Mar 13, 2012, 2:39:11 AM3/13/12
to dev-pl...@lists.mozilla.org
On Mon, Mar 12, 2012 at 8:44 PM, Andreas Gal <andre...@gmail.com> wrote:
> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

I think the crucial line that's about to be crossed is exposing
functionality that cannot be implemented royalty-free to the Web. (We
already expose features that are implemented using royalty-bearing
technologies--anything that calls a Windows or Mac OS X API--to the
Web, but that's OK as long as it's also possible to implement those
features on top of a royalty-free OS (Linux).)

*If* we cross that line by exposing H.264 to the Web in a Mozilla
product (it's already exposed by everyone else--including by Opera in
their mobile and TV products--we are very alone with our current
position) here's how I think we should do it:

I think it's very important to have one Web platform and to have Gecko
expose the same feature set everywhere. Moreover, if we expose H.264
to the Web (regardless of whether we ship an H.264 implementation), we
will have reneged on our principle not to ship non-RF codecs. After
that, Web authors will not bother to support us with WebM on *any*
platform and will wait for us to expose H.264 to the Web everywhere.
(Many Web authors are in that mode already.)

Therefore, if/when we expose H.264 to the Web in some Mozilla product,
we should have it ready across the product line--including Firefox on
Windows XP and Firefox on desktop Linux.

I think Mozilla Corporation should have two products: Firefox and
"Mozilla MPEG Components" (the latter name is just a placeholder, of
course). The source code for both would be available under Open Source
licenses. However, while anyone can take the Firefox source, compile
it and distribute it royalty-free under a different name, it would be
acknowledged that distributing a self-compiled copy of Mozilla MPEG
Components probably requires licenses from MPEG-LA and Via Licensing
in some countries.

Mozilla MPEG Components would include the Android 4.0 software
implementations of H.264 and LC-AAC (or maybe libav implementations).
(I think the discussion about whether to support MP3 is a separate
discussion. I don't know if the MP4 container is RF, so I don't know
which side of the Firefox/Mozilla MPEG Components divide it should
go.) Mozilla MPEG Components would expose its functionality through an
API exactly or roughly like your MPAPI proposal. Mozilla MPEG
Components would compile into a shared library.

Firefox would look for a library that quacks like Mozilla MPEG
Components. If found, Firefox would report video/mp4 with the right
codecs parameter as supported via canPlayType and play
H.264/LC-AAC/MP4. If not found, Firefox would behave like today.

I think Mozilla Corporation should take the patent licenses at the cap
price where number of units shipped doesn't matter and ship
productized binaries of Mozilla MPEG Components for all platforms that
Mozilla ships Firefox for. Also, I think it would be good to ship
Mozilla MPEG Components for platforms like FreeBSD and Solaris where
there are non-Mozilla supported builds of the Firefox codebase
available.

Mozilla-shipped Firefox releases would come with a bundled Mozilla
MPEG Components release, so users of Mozilla-shipped Firefox wouldn't
have to go through any special steps to get H.264/AAC support. Parties
who self-compile the Firefox codebase (either under the Firefox name
or some ElementAnimal name) could use a copy of Mozilla MPEG
Components binaries shipped by Mozilla Corporation for free. If they
wanted to, they could compile Mozilla MPEG Components themselves and
get patent licenses on their own. Even non-Gecko products would be
welcome to use Mozilla MPEG Components as shipped by Mozilla
Corporation for free, so if Opera wanted to use it on desktop, I think
Mozilla should let them. If someone wanted to write a Gstreamer
plug-in that wrapped Mozilla MPEG Components for use in other Linux
apps, that would be cool too.

Basically, I think Mozilla needs to pay for the patent licenses in
order to deal with Windows XP and not leave functionality of Firefox
on XP up to a third party. Once the patent licenses are sunk cost, I
think it makes sense to hack the MPEG-LA scheme to the maximum extent
possible and give a codec-lib-only product away for free to everyone.

Then, once that baseline software functionality is available on all
platforms that Gecko runs on, I think it would make sense to use
system decoders instead of Mozilla MPEG Components where available.
But I think using system decoders where available as the first step
would be bad as it would fragment the feature set Web authors can
expect of Firefox.

- -

Some tangential points:
* In order to avoid fragmenting the platform exposed to Web sites, I
think we should still control the set of video formats exposed to the
Web and shouldn't expose random system codecs even when we technically
could.
* The licensing regime for MP3 is even more unpleasant than the
regime for H.264 and AAC, so I think MP3 requires a separate
discussion and decision. It's also worth noting that it's not too long
until the MP3 spec turns 20 years old, so it might make sense to wait
until then.
* On the face of things, it seems that using the Android 4.0 software
implementations of H.264 and AAC would be a better match for code
that's designed to support Android hardware acceleration, too. Also,
for a component of this nature, Apache License 2.0 might be nicer than
LGPL 2.1. However, there might be technical reasons to prefer libav's
decoders. I'm not competent to judge that.
* Conceding to a patent toll booth once sets a terrible precedent.
Already at the W3C some people are using the H.264 precedent as an
excuse to introduce DRM schemes that not only are patent-encumbered
but are also trade secret-encumbered (the operation of H.264 is not a
secret) and anti-circumvention-encumbered (decoding H.264 is not an
anti-circumvention-law-invoking activity).

--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/

Mike Hommey

unread,
Mar 13, 2012, 3:48:15 AM3/13/12
to dev-pl...@lists.mozilla.org
On Mon, Mar 12, 2012 at 07:38:59PM -0700, therealbr...@gmail.com
wrote:
> We should proceed carefully. Mozilla will not pay for an H.264 license
> that does not benefit downstreams. I personally do not believe we
> should pay any such rent. It's better for us to ride on an OS decoder
> if possible (we ride on Flash now, see above).

We could actually ride on Flash when an OS decoder is not present. We
could certainly build our own .swf <video> player for those.
As for Linux, I'm not sure to what extent h264 decoding is available to
them by default. I know Debian has it, now. I guess Ubuntu has, too.

Mike

Brendan Eich

unread,
Mar 13, 2012, 3:56:10 AM3/13/12
to
On Mar 13, 12:48 am, Mike Hommey <m...@glandium.org> wrote:
> On Mon, Mar 12, 2012 at 07:38:59PM -0700, therealbrendane...@gmail.com
> wrote:
>
> > We should proceed carefully. Mozilla will not pay for an H.264 license
> > that does not benefit downstreams. I personally do not believe we
> > should pay any such rent. It's better for us to ride on an OS decoder
> > if possible (we ride on Flash now, see above).
>
> We could actually ride on Flash when an OS decoder is not present. We
> could certainly build our own .swf <video> player for those.

Yes, we discussed this a few years back -- it's quite doable.

I agree with Henri's point that we should unify the web-author-facing
API, not just do <object>-in-<video> fallback on some platforms (but
that is standard operating procedure). I do not agree we must pay the
rentiers their shakedown fees.

/be

Ralph Giles

unread,
Mar 13, 2012, 4:27:36 AM3/13/12
to Asa Dotzler, dev-pl...@lists.mozilla.org
On 12-03-12 8:46 PM, Asa Dotzler wrote:

> our 3 years of Open Video support has not changed or even slowed down,
> is what makes the Web unimplementable in a free stack.

I disagree that we haven't slowed down open Video Adoption, but that's
not an experiment we get to do. It's more accurate to just say we're not
happy with the results.

If we land this patch, and do the other things, we should be clear that
we've been forced to by adoption patterns, and that non-free codecs
still hurt the Open Web.

-r

Chris Double

unread,
Mar 13, 2012, 5:35:57 AM3/13/12
to dev-pl...@lists.mozilla.org
On Tue, Mar 13, 2012 at 8:56 PM, Brendan Eich
<therealbr...@gmail.com> wrote:
>> We could actually ride on Flash when an OS decoder is not present. We
>> could certainly build our own .swf <video> player for those.
>
> Yes, we discussed this a few years back -- it's quite doable.

I had a prototype at one point that did this and had discussions with
Adobe. It required at the time changes to their flash player due to us
requiring control of the downloaded stream (for browser caching, etc)
and/or cross origin changes - I'd have to look up the exact reasons it
stalled.

Chris.
--
http://www.bluishcoder.co.nz

Henri Sivonen

unread,
Mar 13, 2012, 5:54:25 AM3/13/12
to dev-pl...@lists.mozilla.org
On Tue, Mar 13, 2012 at 9:48 AM, Mike Hommey <m...@glandium.org> wrote:
> As for Linux, I'm not sure to what extent h264 decoding is available to
> them by default. I know Debian has it, now.

By default? Really? That sounds *very* un-Debian.

> I guess Ubuntu has, too.

I just did two fresh Ubuntu installs (64-bit) in a VM.

By default, H.264, MP3 or AAC support or Flash Player is not
installed. Once you need support, install wizards pop up for these.

In the system installer, there's an unchecked-by-default checkbox for
"third-party sofware" some of which is closed source. It talks about
Flash Player and a Fraunhofer-licensed Fluendo MP3 codec. If you check
this box, you get Flash Player, Fluendo MP3 codec (presumably; I
didn't verify) and the "bad", "ugly" and "ffmpeg" Gstreamer plug-in
sets (which were not mentioned in the installer UI).

The "bad", "ugly" and "ffmpeg" Gstreamer plug-in sets are marked as
not supported by Canonical. Canonical is listed as an MPEG-LA H.264
pool licensee. I don't know how the legal side of this works. The
"check your laws" legal warning is now gone from the install wizards.

Alternatively, H.264 and AAC support for Gstreamer is available for
$34.95 as part of Fluendo Complete Playback Pack from the Ubuntu
Software Center.
It is loading more messages.
0 new messages