hardware-accelerated audio/video decoding in Gecko (bug 714408)

[Andreas Gal]

I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).

Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).

Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).

I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.

Let me know if you have any questions.

Andreas

[Asa Dotzler]

On 3/12/2012 12:28 AM, Andreas Gal wrote:
> I don't think this bug significantly changes our position on open
> video. We will continue to promote and support open codecs, but when
> and where existing codecs are already installed and licensed on
> devices we will make use of them in order to provide people with the
> best possible experience.

For the desktop, this could be problematic. While Windows 7 does provide
access to an already installed and licensed h.264 codec, Windows XP does
not. If we simply enable system codecs where available, how will Web
developers know when they can and cannot count on system codecs?

I'm not opposing this move, I wonder though how we can go "half way" and
not follow-up with shipping h.264 on Windows XP and any other platforms
where it isn't pre-installed. It seems like "Gecko is Gecko" gets pretty
broken if half of our users have a codec and the other half doesn't.

- A

[Andreas Gal]

Desktop is a more complex equation, thats why we are attacking mobile first. On mobile the field is pretty homogeneous, with Gecko being the only browser not supporting the common formats.

For desktop one possibility is to expose MPAPI (Media Plugin API, I couldn't think of a better name for now) via NPAPI. Third parties could then provide additional audio/video decoders as a plugin. Flash, for example, contains all relevant decoders and could hook into MPAPI to provide this capability on Windows XP. Third parties could even offer plugins for sale to cover licensing cost. These are all suboptimal solutions for the 40% of our users base on Windows XP, but so is not helping the other 60% play the media they want, with the codecs they already have on their system.

Andreas

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[ger...@info-care.com.pt]

I wonder if this means small appliances like the Raspberry Pi (which has hw decoding H.264 - Broadcom BCM2835) or the SolidRun CuBox (Marvell ARMADA 500) could find themselves more and more useful!

[Doug Turner]

This is great. We really need this on Android where many mobile sites only use h.264 video.

Doug

On Mar 12, 2012, at 12:28 AM, Andreas Gal wrote:

>
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

[Jeremie Patonnier]

On Mar 12, 4:02 am, Asa Dotzler <a...@mozilla.org> wrote:
> On 3/12/2012 12:28 AM, Andreas Gal wrote:
> For the desktop, this could be problematic. While Windows 7 does provide
> access to an already installed and licensed h.264 codec, Windows XP does
> not. If we simply enable system codecs where available, how will Web
> developers know when they can and cannot count on system codecs?

Well, I don't think this is a problem. Web developer already must take
care about this if they want to provide a cross browser experience to
their user (by providing both H264 and WEBM encoded videos).

As per the HTML5 spec, the "source" element provide all the necessary
fallback required to deal with that issue.

So by doing the following, web developper already deal with the video
codecs insanity on the Web :

<video>
<source src="foo.mp4" type="video/mp4">
<source src="foo.webm" type="video/webm">
Any other fallback for browsers that do not understand the video
tag.
</video>

So even if Firefox can decode the mp4 video due to hardware
acceleration on some plateforme, it does change nothing in a web
developer point of view. therefor, IMHO, this feature does not improve
nothing for web dev, but can make a difference for users (saving life
battery on laptop for exemple. This is a nice to have feature for
users and it change nothing for developer.

Cheers
Jérémie

[Joe Drew]

On 2012-03-12 9:00 AM, Doug Turner wrote:
> This is great. We really need this on Android where many mobile sites only use h.264 video.

I don't see why this matters. At the time we shipped Theora (and WebM),
most _desktop_ sites only supported h.264 (mostly through Flash).

I am very concerned that, by supporting system codecs, we're simply
capitulating on Free codecs. If, as we believe, mobile is the future,
and we support h.264 or other non-free codecs on mobile, then the future
of codecs is non-free.

joe

[Joe Drew]

On 2012-03-12 3:28 AM, Andreas Gal wrote:
> There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

Further to my earlier message: there is exactly as much justification
stopping our users from using system decoders on mobile as there is on
desktop. If we want to support non-free formats via system codecs, we
should make that our official plan for Firefox on all platforms. Doing
so via the back door, which is what I perceive this as, is wrong.

joe

[Blake Winton]

On 12-03-12 10:32 , Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop.

I see a slight difference in justification, in that 40% of our desktop
users (Windows XP) don't get an h.264 codec from their system, so
enabling system codecs on desktop won't help nearly as much of our
userbase, and will fragment the desktop version of gecko, and make it
harder for web developers to test their sites.

> If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.

I can see how you would feel that way, but I don't think that we're
trying to do this via the back door (otherwise Andreas wouldn't have
posted in a public forum ;). Regardless, I agree that we should make it
our official plan on all platforms (or not), if only to keep our
position consistent.

Later,
Blake.

[Doug Turner]

On Mar 12, 2012, at 7:29 AM, Joe Drew wrote:

>
> On 2012-03-12 9:00 AM, Doug Turner wrote:
>> This is great. We really need this on Android where many mobile sites only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM), most _desktop_ sites only supported h.264 (mostly through Flash).

Why what doesn't matter?

Like… when I use Firefox on Android, I really want video to *just* work. A few months ago, I did a quick look at popular video sites that Firefox for Android gets served. There were a lot of sites that returned mobile content but their video tags only listed h.264. For example, both DailyMotion.com and Vimeo.com fit into this category.

As I understand it, what are we are doing it basically waiting it. We are hoping that the sites transcode to WebM. Sites are slowing doing this for Desktop where FF has a significant marketshare (YouTube committed to this, but I don't think it has completed its conversion yet). Mobile is a bit different. We do not have any significant marketshare and leverage is hard. Sites will know only have to do the conversation for Firefox desktop, but realized that Firefox for Android exists and they should serve mobile content with both h.264 and WebM hurts Firefox for Android more that it helps. This is a long bet. And I think we need to make Firefox for Android not have signifiant deficiencies from the default browser.

Using the hardware decoder that is present - and paid for - on the device will help make Firefox for Android's that much more usable.

> I am very concerned that, by supporting system codecs, we're simply capitulating on Free codecs. If, as we believe, mobile is the future, and we support h.264 or other non-free codecs on mobile, then the future of codecs is non-free.

I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.

Regards,
Doug

[Ted Mielczarek]

On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner <do...@mozilla.com> wrote:
> I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.

This may be true, but can you say with a straight face that supporting
non-free codecs on mobile (including on our *own* platform, B2G) isn't
tantamount to giving up on free codecs for the web? It seems pretty
obvious to me that we will lose any moral high ground we hold by doing
this.

Now, it may be that we have already lost this war. If that's the case,
we should just admit it and move on, and not sabotage ourselves by
shipping support only on mobile platforms.

-Ted

[Joe Drew]

On 2012-03-12 10:40 AM, Blake Winton wrote:
> I see a slight difference in justification, in that 40% of our desktop
> users (Windows XP) don't get an h.264 codec from their system, so
> enabling system codecs on desktop won't help nearly as much of our
> userbase, and will fragment the desktop version of gecko, and make it
> harder for web developers to test their sites.

There isn't really a difference between "mobile gecko" and "desktop
gecko" - or, at least, there shouldn't be. We produce one product for
many platforms. Stated another way, we support one product (the web)
everywhere.

joe

[Blake Winton]

I believe there is from a web developer's point of view. You need
fairly different layouts for mobile to account for the smaller screens,
and larger pointing devices (fingers vs. cursor). You also expect
different performance characteristics, and likely different capabilities
(i.e. is Flash installed, or is there an h.264 encoder).

We might use the same platform to support the two different markets, but
that doesn't make the markets the same. :)

Later,
Blake.

[Doug Turner]

I just don't know. To me, I just want software to work. If a site wants to send me h.264, great. If it wants to send webm, great.

Doug

[John Volikas]

If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

[Andreas Gal]

On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:

> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner <do...@mozilla.com> wrote:
>> I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>
> This may be true, but can you say with a straight face that supporting
> non-free codecs on mobile (including on our *own* platform, B2G) isn't
> tantamount to giving up on free codecs for the web? It seems pretty
> obvious to me that we will lose any moral high ground we hold by doing
> this.
>
> Now, it may be that we have already lost this war. If that's the case,
> we should just admit it and move on, and not sabotage ourselves by
> shipping support only on mobile platforms.

I do believe this war is lost. Just look around. Almost none of the content users want to watch is available in WebM. The only reason desktop is usable is because of Flash, a proprietary plugin, playing video for us (in H.264, mostly). Even Google, supposedly a proponent of open codecs, never fully converted youtube and never dropped H.264 from Chrome. Taking a principled (I would at this point prefer 'stubborn' I think) stance on H.264 won't change reality. It just hurts us and our users.

Firefox got to the point where we are on desktop today by embracing reality. In the early days we started supporting IE-isms like document.all that were god awfully ugly and non-standard. But it was needed for compatibility so we can give people a usable web experience. The web uses H.264. Thats an unpleasant fact, but its a fact. We have to support it whether we like it or not, so we can be around for the next round and continue to influence the web for the better.

Andreas

>
> -Ted

[Andreas Gal]

> If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

Yes, sorry if that wasn't clear from the original email but thats the intent. Anything the system has decoders for we should pass through and allow to work, including MP3 and AAC.

Andreas

[Joe Drew]

On 2012-03-12 12:16 PM, Andreas Gal wrote:
> I do believe this war is lost.

Please re-frame your discussion, in that case, as enabling platform
codecs on all platforms. Doing it only on mobile makes no sense.

joe

[smaug]

On 03/12/2012 05:52 PM, Doug Turner wrote:
> I just don't know. To me, I just want software to work.

But we need to think more than just the current state of the web. We
really need to think about the future and the web as a platform.
Do we want to prevent fully open source and free implementations
(including hw) to be able to render everything in the internet?


-Olli

[Andreas Gal]

I think the discussion is framed correctly. I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet. Once I do, I will come back and propose it.

Andreas

>
> joe

[Andreas Gal]

On Mar 12, 2012, at 9:24 AM, smaug wrote:

> On 03/12/2012 05:52 PM, Doug Turner wrote:
>> I just don't know. To me, I just want software to work.
>
> But we need to think more than just the current state of the web. We
> really need to think about the future and the web as a platform.
> Do we want to prevent fully open source and free implementations
> (including hw) to be able to render everything in the internet?

Yes, we do. H.264 isn't the end of the codec story. Higher compression ratio successors to H.264 are already being developed. We need to proactively push on open codecs in that space so for the next cycle of this we have a working, free alternative ready from the start. I want to be around to fight that battle, instead of dying on the H.264 hill in a lost war.

Andreas

>
>
> -Olli
>
>
>> If a site wants to send me h.264, great. If it wants to send webm, great.
>>
>> Doug
>>
>> On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:
>>
>>> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner<do...@mozilla.com> wrote:
>>>> I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>>>
>>> This may be true, but can you say with a straight face that supporting
>>> non-free codecs on mobile (including on our *own* platform, B2G) isn't
>>> tantamount to giving up on free codecs for the web? It seems pretty
>>> obvious to me that we will lose any moral high ground we hold by doing
>>> this.
>>>
>>> Now, it may be that we have already lost this war. If that's the case,
>>> we should just admit it and move on, and not sabotage ourselves by
>>> shipping support only on mobile platforms.
>>>
>>> -Ted
>>
>

[L. David Baron]

On Monday 2012-03-12 00:28 -0700, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get
> review for it. It adds hardware-accelerated audio/video decoding
> support to Gecko using system decoders already present on the
> system. Android, for example, ships by default with a number of
> decoders, and in particular for such mobile devices we really have
> to use these hardware-accelerated decoders for good battery life
> (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we
> will add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present
> on the system, including H.264 and MP3. There is really no
> justification to stop our users from using system decoders already
> on the device, so we will not filter any formats.

I disagree with this for two reasons. Joe has already stated the
first (that it fundamentally disagrees with our open codec story).

But the second is that if we are going to support system decoders,
we should be choosing which ones rather than just supporting all
codecs that the system supports. (Our motivation for doing this is
to expose a particular one or two codecs, not to expose all codecs.
So we should expose those and not everything.) I believe this is
preferable for three reasons:

(1) A more coherent story for Web developers: Having a smaller set
of codecs on the Web presents a more coherent story to Web
developers and doesn't encourage them to waste time using codecs
(by offering support on some platforms) that aren't widely
supported and that we don't actually think should become a part of
the Web.

(2) A smaller set of codecs required for future Web platform
implementations: Supporting "all" system codecs risks the
possibility that we'll add a larger number of codecs to the set of
codecs that are required to build a new implementation of the Web
platform. This increases the costs of doing so and reduces future
competition.

(3) Smaller security exposure: Each additional codec
implementation we expose adds additional security exposure; we
shouldn't expose codecs that we don't think Web authors should be
using.

-David

--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla http://www.mozilla.org/ 𝄂

[Andreas Gal]

> But the second is that if we are going to support system decoders,
> we should be choosing which ones rather than just supporting all
> codecs that the system supports. (Our motivation for doing this is
> to expose a particular one or two codecs, not to expose all codecs.
> So we should expose those and not everything.) I believe this is
> preferable for three reasons:

This is a fair argument, and its technically what the patch does. We specify a list of codecs we allow to pass through. What my original email meant was really "we will not filter for political reasons". In practice, for exactly the reason you listed, we want pick the relevant codecs users want and expose those explicitly for simple technical reasons. H.264/AVC, AAC and MP3 is probably what we want there, but I am happy to hear alternative proposals. Pending this discussion, the patch currently only allows H.264 to pass through, but we already have a 1-liner in the bug to add MP3.

Andreas

[Ralph Giles]

On 12/03/12 09:27 AM, Andreas Gal wrote:

> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.

If we're going to compromise our principles on this issue, we should
license a proprietary decoder implementation and ship it with our
official desktop builds. That way we have

a) consistent support on all platforms, including Windows XP;

b) a smaller maintenance burden than calling different backends on each
platform;

c) smaller increase in security footprint;

d) better resistance to DRM implementation pressure than a pass-through
scheme offers.

That would be the honest way to support non-free codecs on the Web. They
cost money, and current licensing is incompatible with free software
implementation. We should be as transparent about that state of affairs
as we can.

-r

[Ralph Giles]

On 12/03/12 05:54 AM, ger...@info-care.com.pt wrote:

> I wonder if this means small appliances like the Raspberry Pi (which has hw decoding H.264 - Broadcom BCM2835) or the SolidRun CuBox (Marvell ARMADA 500) could find themselves more and more useful!

I think you'd find the Raspberry Pi's GPU is perfectly capable of
decoding WebM (and Ogg Theora), although Broadcom hasn't marketed an
implementation of those formats.

-r

[Justin Dolske]

On 3/12/12 7:29 AM, Joe Drew wrote:
\
>> This is great. We really need this on Android where many mobile sites
>> only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM),
> most _desktop_ sites only supported h.264 (mostly through Flash).

To put it another way:

We had these exact same discussions back when we first shipped Theora,
including the issue of H264 on mobile (then with Fennec-on-Nokia).

Rather that just rehashing the same arguments, I'd like to ask what's
changed such that we need to reopen this issue?

Justin

[Doug Turner]

I guess nothing has changed. The market didn't adopt any new codec. h.264 video content is what continues to be used everywhere and our browser just falls down there.

Doug

[Daniel Cater]

On Monday, March 12, 2012 7:28:04 AM UTC, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>

> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Isn't this another issue which goes beyond the B2G or Fennec module owners and needs buy-in from core Gecko module owners?

In particular, given http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.html and http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I expect Robert, Chris Pearce, Chris Double and others to be included in this discussion.

With more browsers, sites and hardware manufacturers working on existing or upcoming WebM / VP8 support I would be very disappointed to see Mozilla cave on this issue.

[Daniel Cater]

[Mike Hommey]

On Mon, Mar 12, 2012 at 10:40:30AM -0700, Doug Turner wrote:
>
> On Mar 12, 2012, at 10:34 AM, Justin Dolske wrote:
>

> I guess nothing has changed. The market didn't adopt any new codec.
> h.264 video content is what continues to be used everywhere and our
> browser just falls down there.

I guess we should feel responsible for this sad state of affairs. How do
we expect people to use open formats when we don't do much to make them
easy to produce? (and here "we" is much broader than Mozilla)

Mike

[Daniel Cater]

http://en.wikipedia.org/wiki/WebM_Project shows a fair few supporters of WebM including Opera, Chrome, Firefox, Android, Adobe, Nvidia, AMD, ARM, Broadcom, Qualcomm, Texas Instruments, Nvidia, Rockchip, YouTube, Skype and Logitech.

Also Google had pledged to remove H.264 support from Chrome but hasn't yet done so: http://blog.chromium.org/2011/01/html-video-codec-support-in-chrome.html

[Daniel Cater]

On Monday, March 12, 2012 5:40:30 PM UTC, Doug Turner wrote:

[Ms2ger]

On 03/12/2012 08:28 AM, Andreas Gal wrote:
>
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.

Did you deliberately write the subject of this thread in such a way that
it is easy to miss that you are trying to reopen the issue of supporting
patent-encumbered codecs, against the policy we have had ever since the
video element was first implemented, and have you considered the effect
that would have on Mozilla's credibility?

Thanks,
Ms2ger

[Joe Drew]

On 2012-03-12 12:31 PM, Doug Turner wrote:
> I understand that Olli. My assertion is that if our mobile offering has deficiencies -- like not being able to play the video tag when the default can -- we will not be relevant. Users simply do not care about the codec. They care if it works or if it doesn't.

Our current desktop product has these same "deficiencies."

joe

[Doug Turner]

But it has users! Sites are a lot more likely to test their code against FF desktop, then Firefox for Android.

[Philip Chee]

On Mon, 12 Mar 2012 09:16:18 -0700, Andreas Gal wrote:

> video for us (in H.264, mostly). Even Google, supposedly a proponent

> of open codecs, never fully converted youtube.

Data point. Speaking as the current author of Flashblock. I have been
getting increasing number of requests from users to block html5 video.
The main driver of these requests appears to be Youtube. So Youtube may
not be fully converted to html5/video/webm - yet - but it certainly
looks like Google is making a significant effort.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.

[Andreas Gal]

Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.

We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

Andreas

Sent from Mobile.

[Ralph Giles]

On 12/03/12 10:48 AM, Mike Hommey wrote:

> I guess we should feel responsible for this sad state of affairs. How do
> we expect people to use open formats when we don't do much to make them
> easy to produce? (and here "we" is much broader than Mozilla)

I think many of us hoped Google, as a larger organization and curators
of the WebM project, would provide leadership here. In retrospect that
was a mistake; this matters more to Mozilla than it does to Google.

But a number of us have been working on improving the situation. I've
been pushing hard on getting a WebM stream on Air Mozilla since we
dropped the Theora option in the fall. We made this a priority this
year, hiring a David Richards to implement WebM support in icecast, for
example.

But there's certainly more we could do, especially with evangelism.
YouTube mostly works with HTML5 WebM now, but almost every day a Vimeo
iframe tells me I need to use a different browser. We should be reaching
out them and asking them to support the Open Web.

Another point to be aware of is that this battle is not just about the
html video element. The WebRTC project (web video chat) is also trying
to choose a baseline codec. The way the IETF draft is currently written,
the WebM video codec (VP8) will be chosen as mandatory to implement
unless a royalty free profile of h.264 is made available before March
15th (that's *this week*). This is a greener field and free codecs are
making better progress there in part because of the line Mozilla has
held elsewhere.

-r

[Andreas Gal]

Even if a license grant is made for H.264 for WebRTC the need for this patch still holds. On anything with a battery we have to use hardware decoders. Those will always be buried somewhere in hardware with some kernel pieces in addition. We will never ship with those bits, and we have to use whats on the system, which really takes us back to where this argument started.

Sent from Mobile.

[Jeff Walden]

On 03/12/2012 09:30 AM, Andreas Gal wrote:
>> Do we want to prevent fully open source and free implementations
>> (including hw) to be able to render everything in the internet?
>
> Yes, we do.

This is worth highlighting. And highlighting again. And again. And again.

The web platform should be based on open standards. I disagree profoundly with moves that take us in the opposite direction.

Jeff

[Daniel Veditz]

Unfortunate quote juxtaposition in Andreas's post made his post hard
to understand, but he's saying the opposite of what it appears in
this shortened version. "Yes, we do." was a response to the first
sentence in that paragraph: "But we need to think more than just the

current state of the web."

His point was that if we remain inflexible and go down with the WebM
ship we will not be around to have any influence in the next battle
over h265.

Hopefully he's learned his lesson about careful quote trimming :-)

-Dan Veditz

[Jeff Walden]

On 03/12/2012 12:39 PM, Daniel Veditz wrote:
> His point was that if we remain inflexible and go down with the WebM
> ship we will not be around to have any influence in the next battle
> over h265.

I think if we yield on this point, there's little reason for others to assume we wouldn't yield on that point as well, honestly.

Whether the particular quoting/juxtaposition was intended or not, I think the position when it comes down to brass tacks would be that we effectively would be preventing free/open source implementations from rendering the Internet. Intent or lack thereof aside doesn't seem much more than a matter of cosmetics to me.

Jeff

[Christopher Blizzard]

On 3/12/2012 9:16 AM, Andreas Gal wrote:
> On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:
>
>> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner<do...@mozilla.com> wrote:
>>> I do not think that is the case. I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>> This may be true, but can you say with a straight face that supporting
>> non-free codecs on mobile (including on our *own* platform, B2G) isn't
>> tantamount to giving up on free codecs for the web? It seems pretty
>> obvious to me that we will lose any moral high ground we hold by doing
>> this.
>>
>> Now, it may be that we have already lost this war. If that's the case,
>> we should just admit it and move on, and not sabotage ourselves by
>> shipping support only on mobile platforms.

> I do believe this war is lost. Just look around. Almost none of the content users want to watch is available in WebM. The only reason desktop is usable is because of Flash, a proprietary plugin, playing video for us (in H.264, mostly). Even Google, supposedly a proponent of open codecs, never fully converted youtube and never dropped H.264 from Chrome. Taking a principled (I would at this point prefer 'stubborn' I think) stance on H.264 won't change reality. It just hurts us and our users.

Just to back this up, I keep talking to people building sites and there
are only a couple of organizations that are willing to embrace WebM
because it's the right thing to do. Transcoding & hosting costs are
huge.* Beyond that I've not really run into anyone who wants to do
WebM. It's just seen as a cost that Firefox is incurring on web developers.

I think that if Google had switched off H.264 support in their browser
soon after they said they would (well over a year ago) then the shift
might have taken place. But that window is gone. It's time to move on,
listen to our users and developers and fight a different battle.

--Chris

* I talked with an organization that has a lot of video a couple of
weeks ago. Actual quote: "If we wanted to transcode we could, but that
would be a full month of time if we used our entire cluster, not to
mention the storage costs at scale. We just can't afford to do that."

[Christopher Blizzard]

On 3/12/2012 7:32 AM, Joe Drew wrote:

> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
>

> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop. If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.

I agree we should make that the plan. But we should start with Mobile
where the codecs are more widely available and move from there. We're
somewhat resource constrained here.

--Chris

[Christopher Blizzard]

I agree with both of you on this. We want a subset of system codecs in
order to have a coherent story for web developers. MP3, AAC, H.264/AVC
is the list I care about, starting with mobile and moving out from there.

--Chris

[Christopher Blizzard]

We've only seen format adoption on youtube. Basically everyone else
uses H.264 & Flash. There are occasional exceptions, but it's not
getting better with time.

--Chris

[Christopher Blizzard]

On 3/12/2012 8:55 AM, John Volikas wrote:
> If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for<audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

I agree.

--Chris

[Christopher Blizzard]

On 3/12/2012 11:28 AM, Ms2ger wrote:
>
> Did you deliberately write the subject of this thread in such a way
> that it is easy to miss that you are trying to reopen the issue of
> supporting patent-encumbered codecs, against the policy we have had
> ever since the video element was first implemented, and have you
> considered the effect that would have on Mozilla's credibility?

I'm not going to speak to Andreas' original message since I didn't write
it. But I would say to the issue of credibility that right now many web
developers feel like Firefox isn't serving their needs since they want
us to support H.264. So from the outside-in perspective we're seen as a
problem, not providing a solution.

I'd love to stand up and support open codecs. (We still will!) I think
it's a shame that MPEG LA & Micrsoft FUD combined with Google's
undermining of their own effort has gotten us to the place where are
today, but here we are. This is reality and we've been trying to change
it for a couple of years on our own but we haven't had much success.

So I'll flip this around: what would you do in this case?

--Chris

[Ms2ger]

Support open codecs in the one way we can: refuse to support
patent-encumbered codecs. Once we support them, we have no leverage left
to argue against them, and our credibility in standards organizations
will be reduced even more–after all, if we give up on making sure it is
possible for anybody, anywhere, to build a browser that is compatible
with the web, without requiring payment for that, why, then, would we be
able to take a strong stance on other issues? Why would other browser
vendors not be correct to believe that we will give in eventually, and
that our arguments need not be taken into consideration?

The argument that this change would only affect those of our users who
already have a patent-encumbered codec installed is, of course,
short-sighted. Once Firefox on recent Windows versions supports H.264,
*all* browsers will be required to support it as well, because authors
will no longer see a reason to use an open codec.

For these reasons, I believe we should use the one weapon we have in
this battle-field, and that is continuing to refuse to support H.264.

HTH
Ms2ger