hardware-accelerated audio/video decoding in Gecko (bug 714408)

Showing 1-272 of 272 messages
hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 12:28 AM

I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).

Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).

Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).

I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.

Let me know if you have any questions.

Andreas
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 1:02 AM
On 3/12/2012 12:28 AM, Andreas Gal wrote:
> I don't think this bug significantly changes our position on open
> video. We will continue to promote and support open codecs, but when
> and where existing codecs are already installed and licensed on
> devices we will make use of them in order to provide people with the
> best possible experience.

For the desktop, this could be problematic. While Windows 7 does provide
access to an already installed and licensed h.264 codec, Windows XP does
not. If we simply enable system codecs where available, how will Web
developers know when they can and cannot count on system codecs?

I'm not opposing this move, I wonder though how we can go "half way" and
not follow-up with shipping h.264 on Windows XP and any other platforms
where it isn't pre-installed. It seems like "Gecko is Gecko" gets pretty
broken if half of our users have a codec and the other half doesn't.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 1:16 AM

Desktop is a more complex equation, thats why we are attacking mobile first. On mobile the field is pretty homogeneous, with Gecko being the only browser not supporting the common formats.

For desktop one possibility is to expose MPAPI (Media Plugin API, I couldn't think of a better name for now) via NPAPI. Third parties could then provide additional audio/video decoders as a plugin. Flash, for example, contains all relevant decoders and could hook into MPAPI to provide this capability on Windows XP. Third parties could even offer plugins for sale to cover licensing cost. These are all suboptimal solutions for the 40% of our users base on Windows XP, but so is not helping the other 60% play the media they want, with the codecs they already have on their system.

Andreas
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gerardo Lisboa 3/12/12 5:54 AM
I wonder if this means small appliances like the Raspberry Pi (which has hw decoding H.264 - Broadcom BCM2835) or the SolidRun CuBox (Marvell ARMADA 500) could find themselves more and more useful!
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Doug Turner 3/12/12 6:00 AM
This is great.  We really need this on Android where many mobile sites only use h.264 video.

Doug


On Mar 12, 2012, at 12:28 AM, Andreas Gal wrote:

>
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Jeremie Patonnier 3/12/12 6:05 AM
On Mar 12, 4:02 am, Asa Dotzler <a...@mozilla.org> wrote:
> On 3/12/2012 12:28 AM, Andreas Gal wrote:
> For the desktop, this could be problematic. While Windows 7 does provide
> access to an already installed and licensed h.264 codec, Windows XP does
> not. If we simply enable system codecs where available, how will Web
> developers know when they can and cannot count on system codecs?

Well, I don't think this is a problem. Web developer already must take
care about this if they want to provide a cross browser experience to
their user (by providing both H264 and WEBM encoded videos).

As per the HTML5 spec, the "source" element provide all the necessary
fallback required to deal with that issue.

So by doing the following, web developper already deal with the video
codecs insanity on the Web :

<video>
    <source src="foo.mp4" type="video/mp4">
    <source src="foo.webm" type="video/webm">
    Any other fallback for browsers that do not understand the video
tag.
</video>

So even if Firefox can decode the mp4 video due to hardware
acceleration on some plateforme, it does change nothing in a web
developer point of view. therefor, IMHO, this feature does not improve
nothing for web dev, but can make a difference for users (saving life
battery on laptop for exemple. This is a nice to have feature for
users and it change nothing for developer.

Cheers
Jérémie
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 7:29 AM

On 2012-03-12 9:00 AM, Doug Turner wrote:
> This is great.  We really need this on Android where many mobile sites only use h.264 video.

I don't see why this matters. At the time we shipped Theora (and WebM),
most _desktop_ sites only supported h.264 (mostly through Flash).

I am very concerned that, by supporting system codecs, we're simply
capitulating on Free codecs. If, as we believe, mobile is the future,
and we support h.264 or other non-free codecs on mobile, then the future
of codecs is non-free.

joe
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 7:32 AM
On 2012-03-12 3:28 AM, Andreas Gal wrote:
> There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

Further to my earlier message: there is exactly as much justification
stopping our users from using system decoders on mobile as there is on
desktop. If we want to support non-free formats via system codecs, we
should make that our official plan for Firefox on all platforms. Doing
so via the back door, which is what I perceive this as, is wrong.

joe
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Blake Winton 3/12/12 7:40 AM
On 12-03-12 10:32 , Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop.

I see a slight difference in justification, in that 40% of our desktop
users (Windows XP) don't get an h.264 codec from their system, so
enabling system codecs on desktop won't help nearly as much of our
userbase, and will fragment the desktop version of gecko, and make it
harder for web developers to test their sites.

> If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.

I can see how you would feel that way, but I don't think that we're
trying to do this via the back door (otherwise Andreas wouldn't have
posted in a public forum ;).  Regardless, I agree that we should make it
our official plan on all platforms (or not), if only to keep our
position consistent.

Later,
Blake.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Doug Turner 3/12/12 7:47 AM

On Mar 12, 2012, at 7:29 AM, Joe Drew wrote:

>
> On 2012-03-12 9:00 AM, Doug Turner wrote:
>> This is great.  We really need this on Android where many mobile sites only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM), most _desktop_ sites only supported h.264 (mostly through Flash).

Why what doesn't matter?  

Like… when I use Firefox on Android, I really want video to *just* work.  A few months ago, I did a quick look at popular video sites that Firefox for Android gets served.  There were a lot of sites that returned mobile content but their video tags only listed h.264.  For example, both DailyMotion.com and Vimeo.com fit into this category.

As I understand it, what are we are doing it basically waiting it.  We are hoping that the sites transcode to WebM.  Sites are slowing doing this for Desktop where FF has a significant marketshare (YouTube committed to this, but I don't think it has completed its conversion yet).  Mobile is a bit different.  We do not have any significant marketshare and leverage is hard.  Sites will know only have to do the conversation for Firefox desktop, but realized that Firefox for Android exists and they should serve mobile content with both h.264 and WebM hurts Firefox for Android more that it helps.  This is a long bet.  And I think we need to make Firefox for Android not have signifiant deficiencies from the default browser.

Using the hardware decoder that is present - and paid for - on the device will help make Firefox for Android's that much more usable.

> I am very concerned that, by supporting system codecs, we're simply capitulating on Free codecs. If, as we believe, mobile is the future, and we support h.264 or other non-free codecs on mobile, then the future of codecs is non-free.


I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.

Regards,
Doug

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ted Mielczarek 3/12/12 7:57 AM
On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner <do...@mozilla.com> wrote:
> I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.

This may be true, but can you say with a straight face that supporting
non-free codecs on mobile (including on our *own* platform, B2G) isn't
tantamount to giving up on free codecs for the web? It seems pretty
obvious to me that we will lose any moral high ground we hold by doing
this.

Now, it may be that we have already lost this war. If that's the case,
we should just admit it and move on, and not sabotage ourselves by
shipping support only on mobile platforms.

-Ted
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 7:59 AM
On 2012-03-12 10:40 AM, Blake Winton wrote:
> I see a slight difference in justification, in that 40% of our desktop
> users (Windows XP) don't get an h.264 codec from their system, so
> enabling system codecs on desktop won't help nearly as much of our
> userbase, and will fragment the desktop version of gecko, and make it
> harder for web developers to test their sites.

There isn't really a difference between "mobile gecko" and "desktop
gecko" - or, at least, there shouldn't be. We produce one product for
many platforms. Stated another way, we support one product (the web)
everywhere.

joe
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Blake Winton 3/12/12 8:42 AM
I believe there is from a web developer's point of view.  You need
fairly different layouts for mobile to account for the smaller screens,
and larger pointing devices (fingers vs. cursor).  You also expect
different performance characteristics, and likely different capabilities
(i.e. is Flash installed, or is there an h.264 encoder).

We might use the same platform to support the two different markets, but
that doesn't make the markets the same.  :)

Later,
Blake.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Doug Turner 3/12/12 8:52 AM
I just don't know.  To me, I just want software to work.  If a site wants to send me h.264, great.  If it wants to send webm, great.

Doug
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) ferongr 3/12/12 8:55 AM
If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 9:16 AM

On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:

> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner <do...@mozilla.com> wrote:
>> I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>
> This may be true, but can you say with a straight face that supporting
> non-free codecs on mobile (including on our *own* platform, B2G) isn't
> tantamount to giving up on free codecs for the web? It seems pretty
> obvious to me that we will lose any moral high ground we hold by doing
> this.
>
> Now, it may be that we have already lost this war. If that's the case,
> we should just admit it and move on, and not sabotage ourselves by
> shipping support only on mobile platforms.

I do believe this war is lost. Just look around. Almost none of the content users want to watch is available in WebM. The only reason desktop is usable is because of Flash, a proprietary plugin, playing video for us (in H.264, mostly). Even Google, supposedly a proponent of open codecs, never fully converted youtube and never dropped H.264 from Chrome. Taking a principled (I would at this point prefer 'stubborn' I think) stance on H.264 won't change reality. It just hurts us and our users.

Firefox got to the point where we are on desktop today by embracing reality. In the early days we started supporting IE-isms like document.all that were god awfully ugly and non-standard. But it was needed for compatibility so we can give people a usable web experience. The web uses H.264. Thats an unpleasant fact, but its a fact. We have to support it whether we like it or not, so we can be around for the next round and continue to influence the web for the better.

Andreas

>
> -Ted
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 9:18 AM

> If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

Yes, sorry if that wasn't clear from the original email but thats the intent. Anything the system has decoders for we should pass through and allow to work, including MP3 and AAC.

Andreas

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 9:23 AM
On 2012-03-12 12:16 PM, Andreas Gal wrote:
> I do believe this war is lost.

Please re-frame your discussion, in that case, as enabling platform
codecs on all platforms. Doing it only on mobile makes no sense.

joe
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) smaug 3/12/12 9:24 AM
On 03/12/2012 05:52 PM, Doug Turner wrote:
> I just don't know.  To me, I just want software to work.

But we need to think more than just the current state of the web. We
really need to think about the future and the web as a platform.
Do we want to prevent fully open source and free implementations
(including hw) to be able to render everything in the internet?


-Olli
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 9:27 AM
I think the discussion is framed correctly. I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet. Once I do, I will come back and propose it.

Andreas

>
> joe
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 9:30 AM

On Mar 12, 2012, at 9:24 AM, smaug wrote:

> On 03/12/2012 05:52 PM, Doug Turner wrote:
>> I just don't know.  To me, I just want software to work.
>
> But we need to think more than just the current state of the web. We
> really need to think about the future and the web as a platform.
> Do we want to prevent fully open source and free implementations
> (including hw) to be able to render everything in the internet?

Yes, we do. H.264 isn't the end of the codec story. Higher compression ratio successors to H.264 are already being developed. We need to proactively push on open codecs in that space so for the next cycle of this we have a working, free alternative ready from the start. I want to be around to fight that battle, instead of dying on the H.264 hill in a lost war.

Andreas

>
>
> -Olli
>
>
>> If a site wants to send me h.264, great.  If it wants to send webm, great.
>>
>> Doug
>>
>> On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:
>>
>>> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner<do...@mozilla.com>  wrote:
>>>> I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>>>
>>> This may be true, but can you say with a straight face that supporting
>>> non-free codecs on mobile (including on our *own* platform, B2G) isn't
>>> tantamount to giving up on free codecs for the web? It seems pretty
>>> obvious to me that we will lose any moral high ground we hold by doing
>>> this.
>>>
>>> Now, it may be that we have already lost this war. If that's the case,
>>> we should just admit it and move on, and not sabotage ourselves by
>>> shipping support only on mobile platforms.
>>>
>>> -Ted
>>
>
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) L. David Baron 3/12/12 9:39 AM
On Monday 2012-03-12 00:28 -0700, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get
> review for it. It adds hardware-accelerated audio/video decoding
> support to Gecko using system decoders already present on the
> system. Android, for example, ships by default with a number of
> decoders, and in particular for such mobile devices we really have
> to use these hardware-accelerated decoders for good battery life
> (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we
> will add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present
> on the system, including H.264 and MP3. There is really no
> justification to stop our users from using system decoders already
> on the device, so we will not filter any formats.

I disagree with this for two reasons.  Joe has already stated the
first (that it fundamentally disagrees with our open codec story).

But the second is that if we are going to support system decoders,
we should be choosing which ones rather than just supporting all
codecs that the system supports.  (Our motivation for doing this is
to expose a particular one or two codecs, not to expose all codecs.
So we should expose those and not everything.)  I believe this is
preferable for three reasons:

 (1) A more coherent story for Web developers:  Having a smaller set
 of codecs on the Web presents a more coherent story to Web
 developers and doesn't encourage them to waste time using codecs
 (by offering support on some platforms) that aren't widely
 supported and that we don't actually think should become a part of
 the Web.

 (2) A smaller set of codecs required for future Web platform
 implementations:  Supporting "all" system codecs risks the
 possibility that we'll add a larger number of codecs to the set of
 codecs that are required to build a new implementation of the Web
 platform.  This increases the costs of doing so and reduces future
 competition.

 (3) Smaller security exposure:  Each additional codec
 implementation we expose adds additional security exposure; we
 shouldn't expose codecs that we don't think Web authors should be
 using.

-David

--
𝄞   L. David Baron                         http://dbaron.org/   𝄂
𝄢   Mozilla                           http://www.mozilla.org/   𝄂
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 9:46 AM
> But the second is that if we are going to support system decoders,
> we should be choosing which ones rather than just supporting all
> codecs that the system supports.  (Our motivation for doing this is
> to expose a particular one or two codecs, not to expose all codecs.
> So we should expose those and not everything.)  I believe this is
> preferable for three reasons:

This is a fair argument, and its technically what the patch does. We specify a list of codecs we allow to pass through. What my original email meant was really "we will not filter for political reasons". In practice, for exactly the reason you listed, we want pick the relevant codecs users want and expose those explicitly for simple technical reasons. H.264/AVC, AAC and MP3 is probably what we want there, but I am happy to hear alternative proposals. Pending this discussion, the patch currently only allows H.264 to pass through, but we already have a 1-liner in the bug to add MP3.

Andreas
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 3/12/12 10:31 AM
On 12/03/12 09:27 AM, Andreas Gal wrote:

> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.

If we're going to compromise our principles on this issue, we should
license a proprietary decoder implementation and ship it with our
official desktop builds. That way we have

a) consistent support on all platforms, including Windows XP;

b) a smaller maintenance burden than calling different backends on each
platform;

c) smaller increase in security footprint;

d) better resistance to DRM implementation pressure than a pass-through
scheme offers.

That would be the honest way to support non-free codecs on the Web. They
cost money, and current licensing is incompatible with free software
implementation. We should be as transparent about that state of affairs
as we can.

 -r
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 3/12/12 10:32 AM
On 12/03/12 05:54 AM, ger...@info-care.com.pt wrote:

> I wonder if this means small appliances like the Raspberry Pi (which has hw decoding H.264 - Broadcom BCM2835) or the SolidRun CuBox (Marvell ARMADA 500) could find themselves more and more useful!

I think you'd find the Raspberry Pi's GPU is perfectly capable of
decoding WebM (and Ogg Theora), although Broadcom hasn't marketed an
implementation of those formats.

 -r

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/12/12 10:34 AM
On 3/12/12 7:29 AM, Joe Drew wrote:
\
>> This is great. We really need this on Android where many mobile sites
>> only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM),
> most _desktop_ sites only supported h.264 (mostly through Flash).

To put it another way:

We had these exact same discussions back when we first shipped Theora,
including the issue of H264 on mobile (then with Fennec-on-Nokia).

Rather that just rehashing the same arguments, I'd like to ask what's
changed such that we need to reopen this issue?

Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Doug Turner 3/12/12 10:40 AM
I guess nothing has changed.  The market didn't adopt any new codec.  h.264 video content is what continues to be used everywhere and our browser just falls down there.

Doug
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Daniel Cater 3/12/12 10:44 AM
On Monday, March 12, 2012 7:28:04 AM UTC, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Isn't this another issue which goes beyond the B2G or Fennec module owners and needs buy-in from core Gecko module owners?

In particular, given http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.html and http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I expect Robert, Chris Pearce, Chris Double and others to be included in this discussion.

With more browsers, sites and hardware manufacturers working on existing or upcoming WebM / VP8 support I would be very disappointed to see Mozilla cave on this issue.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Daniel Cater 3/12/12 10:44 AM
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Mike Hommey 3/12/12 10:48 AM
I guess we should feel responsible for this sad state of affairs. How do
we expect people to use open formats when we don't do much to make them
easy to produce? (and here "we" is much broader than Mozilla)

Mike
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Daniel Cater 3/12/12 10:57 AM
http://en.wikipedia.org/wiki/WebM_Project shows a fair few supporters of WebM including Opera, Chrome, Firefox, Android, Adobe, Nvidia, AMD, ARM, Broadcom, Qualcomm, Texas Instruments, Nvidia, Rockchip, YouTube, Skype and Logitech.

Also Google had pledged to remove H.264 support from Chrome but hasn't yet done so: http://blog.chromium.org/2011/01/html-video-codec-support-in-chrome.html
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Daniel Cater 3/12/12 10:57 AM
On Monday, March 12, 2012 5:40:30 PM UTC, Doug Turner wrote:
Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Ms2ger 3/12/12 11:28 AM
On 03/12/2012 08:28 AM, Andreas Gal wrote:
>
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.

Did you deliberately write the subject of this thread in such a way that
it is easy to miss that you are trying to reopen the issue of supporting
patent-encumbered codecs, against the policy we have had ever since the
video element was first implemented, and have you considered the effect
that would have on Mozilla's credibility?

Thanks,
Ms2ger
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 11:30 AM


On 2012-03-12 12:31 PM, Doug Turner wrote:
> I understand that Olli.  My assertion is that if our mobile offering has deficiencies -- like not being able to play the video tag when the default can -- we will not be relevant.  Users simply do not care about the codec.  They care if it works or if it doesn't.

Our current desktop product has these same "deficiencies."

joe
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Doug Turner 3/12/12 11:35 AM
But it has users!  Sites are a lot more likely to test their code against FF desktop, then Firefox for Android.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Philip Chee 3/12/12 11:37 AM
On Mon, 12 Mar 2012 09:16:18 -0700, Andreas Gal wrote:

> video for us (in H.264, mostly). Even Google, supposedly a proponent
> of open codecs, never fully converted youtube.

Data point. Speaking as the current author of Flashblock. I have been
getting increasing number of requests from users to block html5 video.
The main driver of these requests appears to be Youtube. So Youtube may
not be fully converted to html5/video/webm - yet - but it certainly
looks like Google is making a significant effort.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 11:44 AM
Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.

We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

Andreas

Sent from Mobile.
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 3/12/12 11:49 AM
On 12/03/12 10:48 AM, Mike Hommey wrote:

> I guess we should feel responsible for this sad state of affairs. How do
> we expect people to use open formats when we don't do much to make them
> easy to produce? (and here "we" is much broader than Mozilla)

I think many of us hoped Google, as a larger organization and curators
of the WebM project, would provide leadership here. In retrospect that
was a mistake; this matters more to Mozilla than it does to Google.

But a number of us have been working on improving the situation. I've
been pushing hard on getting a WebM stream on Air Mozilla since we
dropped the Theora option in the fall. We made this a priority this
year, hiring a David Richards to implement WebM support in icecast, for
example.

But there's certainly more we could do, especially with evangelism.
YouTube mostly works with HTML5 WebM now, but almost every day a Vimeo
iframe tells me I need to use a different browser. We should be reaching
out them and asking them to support the Open Web.

Another point to be aware of is that this battle is not just about the
html video element. The WebRTC project (web video chat) is also trying
to choose a baseline codec. The way the IETF draft is currently written,
the WebM video codec (VP8) will be chosen as mandatory to implement
unless a royalty free profile of h.264 is made available before March
15th (that's *this week*). This is a greener field and free codecs are
making better progress there in part because of the line Mozilla has
held elsewhere.

 -r
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 12:06 PM
Even if a license grant is made for H.264 for WebRTC the need for this patch still holds. On anything with a battery we have to use hardware decoders. Those will always be buried somewhere in hardware with some kernel pieces in addition. We will never ship with those bits, and we have to use whats on the system, which really takes us back to where this argument started.

Sent from Mobile.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Jeff Walden 3/12/12 12:14 PM
On 03/12/2012 09:30 AM, Andreas Gal wrote:
>> Do we want to prevent fully open source and free implementations
>> (including hw) to be able to render everything in the internet?
>
> Yes, we do.

This is worth highlighting.  And highlighting again.  And again.  And again.

The web platform should be based on open standards.  I disagree profoundly with moves that take us in the opposite direction.

Jeff
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Daniel Veditz 3/12/12 12:39 PM
Unfortunate quote juxtaposition in Andreas's post made his post hard
to understand, but he's saying the opposite of what it appears in
this shortened version. "Yes, we do." was a response to the first
sentence in that paragraph: "But we need to think more than just the
current state of the web."

His point was that if we remain inflexible and go down with the WebM
ship we will not be around to have any influence in the next battle
over h265.

Hopefully he's learned his lesson about careful quote trimming :-)

-Dan Veditz
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Jeff Walden 3/12/12 12:51 PM
On 03/12/2012 12:39 PM, Daniel Veditz wrote:
> His point was that if we remain inflexible and go down with the WebM
> ship we will not be around to have any influence in the next battle
> over h265.

I think if we yield on this point, there's little reason for others to assume we wouldn't yield on that point as well, honestly.

Whether the particular quoting/juxtaposition was intended or not, I think the position when it comes down to brass tacks would be that we effectively would be preventing free/open source implementations from rendering the Internet.  Intent or lack thereof aside doesn't seem much more than a matter of cosmetics to me.

Jeff
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) christophe...@gmail.com 3/12/12 1:15 PM
On 3/12/2012 9:16 AM, Andreas Gal wrote:
> On Mar 12, 2012, at 7:57 AM, Ted Mielczarek wrote:
>
>> On Mon, Mar 12, 2012 at 10:47 AM, Doug Turner<do...@mozilla.com>  wrote:
>>> I do not think that is the case.  I just think taking a hard line on free codec on mobile isn't worth the cost to Firefox for Android.
>> This may be true, but can you say with a straight face that supporting
>> non-free codecs on mobile (including on our *own* platform, B2G) isn't
>> tantamount to giving up on free codecs for the web? It seems pretty
>> obvious to me that we will lose any moral high ground we hold by doing
>> this.
>>
>> Now, it may be that we have already lost this war. If that's the case,
>> we should just admit it and move on, and not sabotage ourselves by
>> shipping support only on mobile platforms.
> I do believe this war is lost. Just look around. Almost none of the content users want to watch is available in WebM. The only reason desktop is usable is because of Flash, a proprietary plugin, playing video for us (in H.264, mostly). Even Google, supposedly a proponent of open codecs, never fully converted youtube and never dropped H.264 from Chrome. Taking a principled (I would at this point prefer 'stubborn' I think) stance on H.264 won't change reality. It just hurts us and our users.

Just to back this up, I keep talking to people building sites and there
are only a couple of organizations that are willing to embrace WebM
because it's the right thing to do.  Transcoding & hosting costs are
huge.*  Beyond that I've not really run into anyone who wants to do
WebM.  It's just seen as a cost that Firefox is incurring on web developers.

I think that if Google had switched off H.264 support in their browser
soon after they said they would (well over a year ago) then the shift
might have taken place.  But that window is gone.  It's time to move on,
listen to our users and developers and fight a different battle.

--Chris

* I talked with an organization that has a lot of video a couple of
weeks ago.  Actual quote: "If we wanted to transcode we could, but that
would be a full month of time if we used our entire cluster, not to
mention the storage costs at scale.  We just can't afford to do that."

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) christophe...@gmail.com 3/12/12 1:20 PM
On 3/12/2012 7:32 AM, Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
>
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop. If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.

I agree we should make that the plan.  But we should start with Mobile
where the codecs are more widely available and move from there.  We're
somewhat resource constrained here.

--Chris
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) christophe...@gmail.com 3/12/12 1:23 PM
On 3/12/2012 9:46 AM, Andreas Gal wrote:
>> But the second is that if we are going to support system decoders,
>> we should be choosing which ones rather than just supporting all
>> codecs that the system supports.  (Our motivation for doing this is
>> to expose a particular one or two codecs, not to expose all codecs.
>> So we should expose those and not everything.)  I believe this is
>> preferable for three reasons:
> This is a fair argument, and its technically what the patch does. We specify a list of codecs we allow to pass through. What my original email meant was really "we will not filter for political reasons". In practice, for exactly the reason you listed, we want pick the relevant codecs users want and expose those explicitly for simple technical reasons. H.264/AVC, AAC and MP3 is probably what we want there, but I am happy to hear alternative proposals. Pending this discussion, the patch currently only allows H.264 to pass through, but we already have a 1-liner in the bug to add MP3.
>
> Andreas
>

I agree with both of you on this.  We want a subset of system codecs in
order to have a coherent story for web developers.  MP3, AAC, H.264/AVC
is the list I care about, starting with mobile and moving out from there.

--Chris
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) christophe...@gmail.com 3/12/12 1:24 PM
We've only seen format adoption on youtube.  Basically everyone else
uses H.264 & Flash.  There are occasional exceptions, but it's not
getting better with time.

--Chris
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) christophe...@gmail.com 3/12/12 1:26 PM
On 3/12/2012 8:55 AM, John Volikas wrote:
> If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for<audio>  elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.

I agree.

--Chris
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) christophe...@gmail.com 3/12/12 1:35 PM
On 3/12/2012 11:28 AM, Ms2ger wrote:
>
> Did you deliberately write the subject of this thread in such a way
> that it is easy to miss that you are trying to reopen the issue of
> supporting patent-encumbered codecs, against the policy we have had
> ever since the video element was first implemented, and have you
> considered the effect that would have on Mozilla's credibility?

I'm not going to speak to Andreas' original message since I didn't write
it.  But I would say to the issue of credibility that right now many web
developers feel like Firefox isn't serving their needs since they want
us to support H.264.  So from the outside-in perspective we're seen as a
problem, not providing a solution.

I'd love to stand up and support open codecs.  (We still will!)  I think
it's a shame that MPEG LA & Micrsoft FUD combined with Google's
undermining of their own effort has gotten us to the place where are
today, but here we are.  This is reality and we've been trying to change
it for a couple of years on our own but we haven't had much success.

So I'll flip this around: what would you do in this case?

--Chris
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Ms2ger 3/12/12 1:48 PM
Support open codecs in the one way we can: refuse to support
patent-encumbered codecs. Once we support them, we have no leverage left
to argue against them, and our credibility in standards organizations
will be reduced even more–after all, if we give up on making sure it is
possible for anybody, anywhere, to build a browser that is compatible
with the web, without requiring payment for that, why, then, would we be
able to take a strong stance on other issues? Why would other browser
vendors not be correct to believe that we will give in eventually, and
that our arguments need not be taken into consideration?

The argument that this change would only affect those of our users who
already have a patent-encumbered codec installed is, of course,
short-sighted. Once Firefox on recent Windows versions supports H.264,
*all* browsers will be required to support it as well, because authors
will no longer see a reason to use an open codec.

For these reasons, I believe we should use the one weapon we have in
this battle-field, and that is continuing to refuse to support H.264.

HTH
Ms2ger
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Justin Lebar 3/12/12 1:51 PM
> So I'll flip this around: what would you do in this case?

I feel like our PR on this topic has been pretty non-existent,
particularly if we consider the degree to which most of us don't want
to add support for proprietary codecs.

If I were in marketing (thank goodness I'm not!), I'd publicly call on
Google to fulfill its promises of old.  I'd communicate through
official channels why we don't want to support H.264, MP3, etc, and
why we think Google is harming the web.  I'd explore legal options,
like whether Google could address the issue of submarine patents by
promising to in some way legally support WebM users.  And I might set
a public deadline -- if Google doesn't un-support H.264 by date X,
then we'll start supporting system H.264 and MP3 codecs.

Maybe these are bad ideas.  But I feel like we're going down without a
real fight, and that makes me sad.  Surely we're not totally helpless
here.

-Justin

On Mon, Mar 12, 2012 at 4:35 PM, Christopher Blizzard
<bliz...@mozilla.com> wrote:
> On 3/12/2012 11:28 AM, Ms2ger wrote:
>>
>>
>> Did you deliberately write the subject of this thread in such a way that
>> it is easy to miss that you are trying to reopen the issue of supporting
>> patent-encumbered codecs, against the policy we have had ever since the
>> video element was first implemented, and have you considered the effect that
>> would have on Mozilla's credibility?
>
>
> I'm not going to speak to Andreas' original message since I didn't write it.
>  But I would say to the issue of credibility that right now many web
> developers feel like Firefox isn't serving their needs since they want us to
> support H.264.  So from the outside-in perspective we're seen as a problem,
> not providing a solution.
>
> I'd love to stand up and support open codecs.  (We still will!)  I think
> it's a shame that MPEG LA & Micrsoft FUD combined with Google's undermining
> of their own effort has gotten us to the place where are today, but here we
> are.  This is reality and we've been trying to change it for a couple of
> years on our own but we haven't had much success.
>
> So I'll flip this around: what would you do in this case?
>
> --Chris
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/12/12 1:54 PM
On 3/12/12 1:24 PM, Christopher Blizzard wrote:

>> Rather that just rehashing the same arguments, I'd like to ask what's
>> changed such that we need to reopen this issue?
>
> We've only seen format adoption on youtube. Basically everyone else uses
> H.264 & Flash. There are occasional exceptions, but it's not getting
> better with time.

Who is "everyone"?

I think it _has_ gotten better with time... When we started H.264 was
everywhere and there was no other option. Now YouTube is available in
WebM (no small feat!), Bing uses Theora for video backgrounds on their
front page, and now even the IE Blog is touting IE10 with WebM.
[http://blogs.msdn.com/b/ie/archive/2012/02/29/windows-consumer-preview-the-fifth-ie10-platform-preview.aspx]

The state of HTML5 video started off from a bad place, and to be fair
still isn't in a good place. So reassessing Mozilla stance is not
unreasonable.

But I think if Mozilla is going to do an about-face on open video
standards (and it _is_ an about-face), then there should be some serious
discussion about it. Certainly more than than a few terse words saying
it's hopeless and obvious. That makes it sound more like a half-hearted
notification of a decision that's already final.

At the very least, it needs to be explained enough so the community can
understand the change. We spent a lot of time and made a lot of blog
posts about why H.264 was bad for the web. Leaving those who advocated
for us suddenly high-and-dry doesn't feel like the right thing to do.

Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/12/12 2:10 PM
On Mon, Mar 12, 2012 at 5:44 PM, Daniel Cater <djc...@gmail.com> wrote:

> In particular, given
> http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.htmland
> http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I
> expect Robert, Chris Pearce, Chris Double and others to be included in this
> discussion.
>

Hi.

With more browsers, sites and hardware manufacturers working on existing or
> upcoming WebM / VP8 support I would be very disappointed to see Mozilla
> cave on this issue.
>

Which browsers? I have not heard that significant browsers that don't
already support WebM (Safari, IE) are planning to add WebM support.

Which sites? I have not heard of major sites that don't already support
WebM (almost everyone except Youtube) planning to add WebM support.

There's a bit more hardware support emerging, but nothing that affects the
equation for Web sites.

I agree with almost everything Andreas and Chris Blizzard have written on
this thread, except I'm not happy about its timing or the way it has been
presented.

Rob
--
“You have heard that it was said, ‘Love your neighbor and hate your enemy.’
But I tell you, love your enemies and pray for those who persecute you,
that you may be children of your Father in heaven. ... If you love those
who love you, what reward will you get? Are not even the tax collectors
doing that? And if you greet only your own people, what are you doing more
than others?" [Matthew 5:43-47]
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/12/12 2:11 PM
On Mon, Mar 12, 2012 at 8:54 PM, Justin Dolske <dol...@mozilla.com> wrote:

> But I think if Mozilla is going to do an about-face on open video
> standards (and it _is_ an about-face), then there should be some serious
> discussion about it. Certainly more than than a few terse words saying it's
> hopeless and obvious. That makes it sound more like a half-hearted
> notification of a decision that's already final.
>
> At the very least, it needs to be explained enough so the community can
> understand the change. We spent a lot of time and made a lot of blog posts
> about why H.264 was bad for the web. Leaving those who advocated for us
> suddenly high-and-dry doesn't feel like the right thing to do.


Beyond that very least, what procedure would you have us follow?
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Dao 3/12/12 2:26 PM
On 12.03.2012 22:10, Robert O'Callahan wrote:
> Which sites? I have not heard of major sites that don't already support
> WebM (almost everyone except Youtube) planning to add WebM support.

Why would sites make their plans public?

German public television started serving WebM recently:
http://www.tagesschau.de/multimedia/video/video1078792.html
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Robert O'Callahan 3/12/12 2:30 PM
On Mon, Mar 12, 2012 at 8:51 PM, Justin Lebar <justin...@gmail.com>wrote:

> If I were in marketing (thank goodness I'm not!), I'd publicly call on
> Google to fulfill its promises of old.  I'd communicate through
> official channels why we don't want to support H.264, MP3, etc, and
> why we think Google is harming the web.


It would be unfair to single out Google as harming the Web. They've done
more for open video codecs than most other industry players have done (i.e,
from "nothing" to "actively opposing").

I'd explore legal options,
> like whether Google could address the issue of submarine patents by
> promising to in some way legally support WebM users.


Only Google can do that. We talked to them about it, and they haven't done
it.

And I might set
> a public deadline -- if Google doesn't un-support H.264 by date X,
> then we'll start supporting system H.264 and MP3 codecs.
>

That might be a good PR tactic to cover our butts, but realistically I
don't think it will move them. We've been begging them in private for some
time.

Maybe these are bad ideas.  But I feel like we're going down without a
> real fight, and that makes me sad.  Surely we're not totally helpless
> here.
>

We have fought. We, alone of all major browsers (sorry Opera desktop), have
held out against supporting patent-encumbered codecs for a long time. I
feel it's grossly unfair to our efforts to describe that as "not a real
fight".

We held the line in the hope that the industry would follow, and that
Google would do a lot to improve and support WebM, especially removing
H.264 support from Chrome. So we've held the line, and watched, and waited,
and personally I am extremely disappointed by the results.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gijs Kruitbosch 3/12/12 2:34 PM
I have to say that having read the entire thread I am left confused. Two of the
main arguments I heard for 'giving up' the H.264 battle were:

1) No new browsers/sites are moving to WebM
2) On mobile we need hardware implementation support, so we need H.264

Then there was an alternative message about giving in here so we'll have a voice
to yell over the next fight. I personally think that could go both ways (no
users -> no voice vs. you gave up last time, so you'll cave later on this one, too).

But, back to those two main reasons. Several other messages in the thread,
people have mentioned that IE10 will support / supports WebM (where it did not
before), that 'most' video content sites support it, and that the broadcom
chipset which is being targeted also supports WebM and Theora decoding, it just
hasn't been marketed.

That makes it sound like neither (1) nor (2) are good reasons. I'm imagining
that the counter-claims are somehow wrong or incomplete or that there are other
reasons that I'm missing, but I haven't seen those expressed. Can someone help
me understand?

~ Gijs
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Double 3/12/12 2:35 PM
On Tue, Mar 13, 2012 at 6:44 AM, Daniel Cater <djc...@gmail.com> wrote:
>
> In particular, given http://robert.ocallahan.org/2009/06/directshow-and-platform-media_23.html and http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html I expect Robert, Chris Pearce, Chris Double and others to be included in this discussion.

Unfortunately what used to be the video team doesn't really have much
say in the direction of how video is supported in Firefox these days.
We usually find out late in the piece what the plans are and what is
going to be implemented and we get tasked with implementing it. This
is, I assume, because much of video support is sensitive or tied up
with NDA's and the like so we don't find out until decisions are
mostly made. Note this is my opinion only, not necessarily the opinion
of the others you've asked for input from.

Chris.
--
http://www.bluishcoder.co.nz
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Dao 3/12/12 2:39 PM
On 12.03.2012 22:34, Gijs Kruitbosch wrote:
> people have mentioned that IE10 will support / supports WebM
> (where it did not before),

IE10 doesn't and won't support WebM. The IEBlog site embedded a WebM
video.
http://blogs.msdn.com/b/ie/archive/2012/02/29/windows-consumer-preview-the-fifth-ie10-platform-preview.aspx
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Chris Pearce 3/12/12 2:55 PM
On 13/03/2012 10:30 a.m., Robert O'Callahan wrote:
> We held the line in the hope that the industry would follow, and that
> Google would do a lot to improve and support WebM, especially removing
> H.264 support from Chrome. So we've held the line, and watched, and waited,
> and personally I am extremely disappointed by the results.

+1. And we have bled and lost support and of the early adopters and
created fostered ill-will in the webdev community because of this issue.

If Google were to announce that they were to drop support of H.264 in
Chrome tomorrow, we'd still have IE and Safari holding out (most people
won't bother to install the required WebM plugins for said browsers). I
don't think Safari ever would about face, and there's only a remote
chance IE would change.

On 13/03/2012 6:31 a.m., Ralph Giles wrote:
> On 12/03/12 09:27 AM, Andreas Gal wrote:
>
>> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.
> If we're going to compromise our principles on this issue, we should
> license a proprietary decoder implementation and ship it with our
> official desktop builds.

I agree with Ralph: the best solution for our users is to suck it up and
pay the H.264/AAC/MP3 license fees and ship libav/ffmpeg on desktop, and
ship hardware decoders on mobile. If necessary we can patch security
holes in libav, so we're not vulnerable to the black-box problem we get
with system codecs on desktop, and we can offer H.264 support to the 40%
of our users on XP - something that IE can't do.


On 13/03/2012 7:44 a.m., Andreas Gal wrote:
> Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.

In my darker moments I've wondered if this was a deliberate strategy by
Chrome to make us continue to hold an untenable position and continue to
bleed users.

Having said that, it would be a shame if YouTube turned on WebM by
default and Chrome dropped H.264 support just as we agreed to pickup
H.264/MP3 support on desktop. We should reach out to Google through
offical channels, and try to get them to publicly commit to the above
before we make a decision.

> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals.

We owe our success to the users who use our product (Firefox), not to
the downstream users of our code. And we owe our continued success to
retaining said users.

Regards,
Chris P.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Philipp Wagner 3/12/12 3:04 PM
Replying on top because it doesn't fit to a specific quote in the thread:

Have we actually done everything in our power to use the available
hardware for efficient WebM decoding? I don't mean special decoder chips
(which are not available in today's phones AFAIK), but things like the
NEON extensions for ARM etc.?

Philipp

Am 12.03.2012 08:28, schrieb Andreas Gal:
>
> I want to land bug 714408 on mozilla-central as soon as I get review
> for it. It adds hardware-accelerated audio/video decoding support to
> Gecko using system decoders already present on the system. Android,
> for example, ships by default with a number of decoders, and in
> particular for such mobile devices we really have to use these
> hardware-accelerated decoders for good battery life (and
> performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will
> add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present on
> the system, including H.264 and MP3. There is really no justification
> to stop our users from using system decoders already on the device,
> so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI,
> which is currently internal to Gecko, but we might expose it via
> NPAPI to external decoder providers as well. MPAPI helps us deal with
> incompatibilities in systems decoders (without risking Gecko not
> starting up which would happen if we try to link against the system
> decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser
> and fully integrates with our rendering pipeline (CSS and all). On
> Android we might have to add a second video path using overlays which
> would only work with a small subset of CSS since extracting video
> frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open
> video. We will continue to promote and support open codecs, but when
> and where existing codecs are already installed and licensed on
> devices we will make use of them in order to provide people with the
> best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 3/12/12 3:33 PM
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon 12 Mar 2012 03:04:45 PM PDT, Philipp Wagner wrote:

> Have we actually done everything in our power to use the available
> hardware for efficient WebM decoding? I don't mean special decoder
> chips (which are not available in today's phones AFAIK), but things
> like the NEON extensions for ARM etc.?

We use NEON when it's available, yes, and Tim Terriberry has
contributed many optimizations upstream.

Special decoder chips are an interesting story. NVidia's page on the
Tegra 2[1] advertises VP8 support, but there doesn't seem to be a
driver for it on the Samsung Tablets or the other mobile devices we've
tried. TI demonstrated 1080p VP8 decoding on their OMAP chips 18 months
ago[2]; this is the  chip that's in the Samsung Galaxy SII we're using
to develop boot to gecko. And yet, it's unclear if Android is using it.
TI's Ubuntu PPA for the OMAP4 supports VP6 and VP7...but not VP8.

So there *is* hardware decoder support in shipped high end devices, it
just doesn't seem to be hooked up.

[1] http://www.nvidia.com/object/tegra-2.html
[2]
http://blog.webmproject.org/2010/10/demo-of-webm-running-on-ti-omap-4.html
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPXnm+AAoJEEcAD3uxRB3v5l8IAJS4/Y78WSc26iwx/VqajyVY
U5KNnF/A9Cd+eLFpUPdBFfTwWw7ZEDxsd3AvfDvFckenjoXc6cxPJNydJ+jk0ZDw
RJO5dXkxeOtNsH6+P83jLjliy5uYZfvbGDsVEDnN12RPB4WfRx9O2eyIdqd8UacN
vt3XMU/5BlFjmeUOtRqEaTHZNV28kfwT4RiQLwok6SqB5Vgme5fgXnLOdvbPPUpG
Qi58UNRHpnpO1Kfz1BzOlB3C8ia7jbN+W9c2Da+q1x7+9ZEwBvbfpAio1W6qqcEQ
msJllU7JytB+nNwejaQ0aT+wVK3QihfLrKb//Ha8uaMi9bomnSxkGHxff8PiEuc=
=cxS7
-----END PGP SIGNATURE-----

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Philipp Wagner 3/12/12 4:01 PM
Am 12.03.2012 23:33, schrieb Ralph Giles:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Mon 12 Mar 2012 03:04:45 PM PDT, Philipp Wagner wrote:
>
>> Have we actually done everything in our power to use the available
>> hardware for efficient WebM decoding? I don't mean special decoder
>> chips (which are not available in today's phones AFAIK), but things
>> like the NEON extensions for ARM etc.?
>
> We use NEON when it's available, yes, and Tim Terriberry has
> contributed many optimizations upstream.
>
> Special decoder chips are an interesting story. NVidia's page on the
> Tegra 2[1] advertises VP8 support, but there doesn't seem to be a
> driver for it on the Samsung Tablets or the other mobile devices we've
> tried. TI demonstrated 1080p VP8 decoding on their OMAP chips 18 months
> ago[2]; this is the  chip that's in the Samsung Galaxy SII we're using
> to develop boot to gecko. And yet, it's unclear if Android is using it.
> TI's Ubuntu PPA for the OMAP4 supports VP6 and VP7...but not VP8.
>
> So there *is* hardware decoder support in shipped high end devices, it
> just doesn't seem to be hooked up.

That's good to know. So for B2G those hardware decoders would be usable?
Has somebody looked into that?

Philipp
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/12/12 4:24 PM
On 3/12/12 2:11 PM, Robert O'Callahan wrote:

>> At the very least, it needs to be explained enough so the community can
>> understand the change. We spent a lot of time and made a lot of blog posts
>> about why H.264 was bad for the web. Leaving those who advocated for us
>> suddenly high-and-dry doesn't feel like the right thing to do.
>
> Beyond that very least, what procedure would you have us follow?

That's good enough for me: discuss, decide, communicate. I just wanted
to leave room for the possibility of someone wanting more. :)

Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/12/12 4:31 PM
On 3/12/12 2:39 PM, Dao wrote:
> On 12.03.2012 22:34, Gijs Kruitbosch wrote:
>> people have mentioned that IE10 will support / supports WebM
>> (where it did not before),
>
> IE10 doesn't and won't support WebM. The IEBlog site embedded a WebM
> video.

Yeah, I was just pointing out that even though IE10 may not support it,
serving such content on Bing and Microsoft blogs is a different matter.

On that note, ISTR Microsoft saying that had no intention of supporting
Theora/WebM. But I think that was before the above happened, as well as
their acquisition of Skype (which is using V8?), and before they found
out that H.264 patents were a headache for Windows Phone
(http://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/02/22/google-please-don-t-kill-video-on-the-web.aspx?Redirected=true)

I wonder what the odds are of them reevaluating support for WebM in a
future IE release. Hmm.

Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/12/12 4:33 PM
Yeah, supporting one without the other is probably not going to be useful.

Though I wonder how well HW-accelerated H.264 + enscripten'd AAC-in-JS
would work... :)

Justin
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Robert O'Callahan 3/12/12 6:04 PM
On Mon, Mar 12, 2012 at 8:48 PM, Ms2ger <ms2...@gmail.com> wrote:

> Support open codecs in the one way we can: refuse to support
> patent-encumbered codecs. Once we support them, we have no leverage left to
> argue against them, and our credibility in standards organizations will be
> reduced even more–after all, if we give up on making sure it is possible
> for anybody, anywhere, to build a browser that is compatible with the web,
> without requiring payment for that, why, then, would we be able to take a
> strong stance on other issues? Why would other browser vendors not be
> correct to believe that we will give in eventually, and that our arguments
> need not be taken into consideration?
>

I don't think that compromising or conceding defeat on some issue means we
no longer have credibility in general. We don't have to appear infallible
and invincible to have leverage. People have to know that we'll fight long
and hard for what we think is right, which we've done here.

For these reasons, I believe we should use the one weapon we have in this
> battle-field, and that is continuing to refuse to support H.264.
>

I don't think it's working. With Google effectively abandoning us by not
removing H.264 from Chrome*, and Adobe's support for WebM never
materializing, I believe our stance is doing little more than hurting Web
developers and Firefox users.

* Actually this characterization is unfair to us. Google never even joined
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/12/12 6:11 PM
On Mon, Mar 12, 2012 at 10:33 PM, Ralph Giles <gi...@mozilla.com> wrote:

> Special decoder chips are an interesting story. NVidia's page on the
> Tegra 2[1] advertises VP8 support, but there doesn't seem to be a
> driver for it on the Samsung Tablets or the other mobile devices we've
> tried. TI demonstrated 1080p VP8 decoding on their OMAP chips 18 months
> ago[2]; this is the  chip that's in the Samsung Galaxy SII we're using
> to develop boot to gecko. And yet, it's unclear if Android is using it.
>

>From discussions with Google people, I'm pretty sure it's not.

The fact that Android doesn't even use what WebM hardware support there is
is one aspect of Google's general apathy towards WebM.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ragavan Srinivasan 3/12/12 6:16 PM

Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review
> for it. It adds hardware-accelerated audio/video decoding support to
> Gecko using system decoders already present on the system. Android,
> for example, ships by default with a number of decoders, and in
> particular for such mobile devices we really have to use these
> hardware-accelerated decoders for good battery life (and
> performance).

I realize I'm coming in pretty late, but for what it's worth, I am
absolutely thrilled this is happening from an Apps product perspective.
This is going to go a long way towards supporting audio/video apps and
ensuring users have a great experience.


> Initially this will be enabled on Gonk (B2G). In a few weeks we will
> add support for Android as well. We will support decoding any
> video/audio format that is supported by existing decoders present on
> the system, including H.264 and MP3. There is really no justification
> to stop our users from using system decoders already on the device,
> so we will not filter any formats.

Who is responsible for the Android support (B2G team or someone else?)
and what versions of Android are we planning on supporting? I realize
it's early, but for Apps, Android is also pretty high on the list, so
I'm curious as to when this will happen as well.

Thanks again for making this happen!

Regards,
Ragavan
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/12/12 6:17 PM
On Mon, Mar 12, 2012 at 9:26 PM, Dao <d...@design-noir.de> wrote:

> On 12.03.2012 22:10, Robert O'Callahan wrote:
>
>> Which sites? I have not heard of major sites that don't already support
>> WebM (almost everyone except Youtube) planning to add WebM support.
>>
>
> Why would sites make their plans public?
>

They might not, but we talk to people as well.

German public television started serving WebM recently:
> http://www.tagesschau.de/**multimedia/video/video1078792.**html<http://www.tagesschau.de/multimedia/video/video1078792.html>


I didn't know that, and that's great, but I'm afraid it's only a drop in
the bucket.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Asa Dotzler 3/12/12 6:54 PM
On 3/12/2012 2:55 PM, Chris Pearce wrote:
> Having said that, it would be a shame if YouTube turned on WebM by
> default and Chrome dropped H.264 support just as we agreed to pickup
> H.264/MP3 support on desktop. We should reach out to Google through
> offical channels, and try to get them to publicly commit to the above
> before we make a decision.

We have been reaching out to them for months and months. It hasn't worked.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 6:58 PM
On 3/12/2012 11:44 AM, Andreas Gal wrote:
> Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.
>
> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

Actually, we can and should license decoders and ship those where
platforms don't have built-in support. Not doing so means abandoning
about half of our users today.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:01 PM
On 3/12/2012 7:32 AM, Joe Drew wrote:
> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>> There is really no justification to stop our users from using system
>> decoders already on the device, so we will not filter any formats.
>
> Further to my earlier message: there is exactly as much justification
> stopping our users from using system decoders on mobile as there is on
> desktop. If we want to support non-free formats via system codecs, we
> should make that our official plan for Firefox on all platforms. Doing
> so via the back door, which is what I perceive this as, is wrong.
>
> joe

Yes. Absolutely. h.264+AAC is the de facto standard for the <video>
element and if we're going to support it, we should support it everywhere.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:02 PM
On 3/12/2012 7:40 AM, Blake Winton wrote:
> On 12-03-12 10:32 , Joe Drew wrote:
>> On 2012-03-12 3:28 AM, Andreas Gal wrote:
>>> There is really no justification to stop our users from using system
>>> decoders already on the device, so we will not filter any formats.
>> Further to my earlier message: there is exactly as much justification
>> stopping our users from using system decoders on mobile as there is on
>> desktop.
>
> I see a slight difference in justification, in that 40% of our desktop
> users (Windows XP) don't get an h.264 codec from their system, so
> enabling system codecs on desktop won't help nearly as much of our
> userbase, and will fragment the desktop version of gecko, and make it
> harder for web developers to test their sites.

Which is why we have to license and ship the decoders on Windows XP and
Linux.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:08 PM
On 3/12/2012 7:47 AM, Doug Turner wrote:

> As I understand it, what are we are doing it basically waiting it.
> We are hoping that the sites transcode to WebM.  Sites are slowing
> doing this for Desktop where FF has a significant marketshare
> (YouTube committed to this, but I don't think it has completed its
> conversion yet).  Mobile is a bit different.  We do not have any
> significant marketshare and leverage is hard.  Sites will know only
> have to do the conversation for Firefox desktop, but realized that
> Firefox for Android exists and they should serve mobile content with
> both h.264 and WebM hurts Firefox for Android more that it helps.
> This is a long bet.  And I think we need to make Firefox for Android
> not have signifiant deficiencies from the default browser.

We're not just waiting. We're talking with major video sites and they're
saying "no" to WebM.  The costs of transcoding huge libraries just
doesn't make sense to them.

Firefox on Desktop is experiencing these same 'significant deficiencies"
and the tide is not turning in any appreciable way. All that's happening
while we wait is that Web developers are embracing other browsers and
their primary targets.

What I fear people aren't getting here is that Gecko is the _only_
mainstream browser that doesn't support h.264. We lost. It's not like
we're at some tipping point and it's 3 of 4 browsers on the side of
royalty free codecs with the forth about to agree. Things have tipped
the other way and to not realize that and continue to hold out for a
change that will not happen does little but cost us users and developer
mindshare.

It's time to bite that particular bullet and deliver h.264+AAC (and
probably mp3) in Firefox -- across all platforms and devices.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Lebar 3/12/12 7:09 PM
> Which is why we have to license and ship the decoders on Windows XP and
> Linux.

Maybe this is a dumb question, but how will this affect Linux distros
which build Firefox but call it TemperatureAnimal?  Will they be able
to use the decoders we've licensed?
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:13 PM
On 3/12/2012 12:14 PM, Jeff Walden wrote:
> On 03/12/2012 09:30 AM, Andreas Gal wrote:
>>> Do we want to prevent fully open source and free implementations
>>> (including hw) to be able to render everything in the internet?
>>
>> Yes, we do.
>
> This is worth highlighting.  And highlighting again.  And again.  And
> again.
>
> The web platform should be based on open standards.  I disagree
> profoundly with moves that take us in the opposite direction.
>
> Jeff

We've got a rich history of stepping back from religious purity on
standards to deliver something that just works. I don't think our
implementing document.all prevented us from saying no and holding the
line on plenty of other crazy DOM features.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 7:10 PM
Nobody said we will abandon anyone. I am confident we will find a way that doesn't require us to ship encumbered codecs. We can create an opportunity for third parties to plug the shrinking (but still large) WinXP gap. Everywhere else the solution is obvious.

Sent from Mobile.
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:18 PM
On 3/12/2012 10:31 AM, Ralph Giles wrote:
> On 12/03/12 09:27 AM, Andreas Gal wrote:
>
>> I have a solution for mobile, and I am focusing on mobile for now. I would like to solve desktop as well, but there I don't have a working solution for the 40% of XP users--yet.
>
> If we're going to compromise our principles on this issue, we should
> license a proprietary decoder implementation and ship it with our
> official desktop builds. That way we have
>
> a) consistent support on all platforms, including Windows XP;
>
> b) a smaller maintenance burden than calling different backends on each
> platform;
>
> c) smaller increase in security footprint;
>
> d) better resistance to DRM implementation pressure than a pass-through
> scheme offers.
>
> That would be the honest way to support non-free codecs on the Web. They
> cost money, and current licensing is incompatible with free software
> implementation. We should be as transparent about that state of affairs
> as we can.


Thank you Ralph for laying this out. I agree with it 100%. We should
license and ship these decoders. If there's a major cost difference (
don't know the economics very well) then I'd be fine with just shipping
them where they're missing (more code burden) and relying on system
stuff where it already exists. This will ensure that Web developers have
a clear target and that users get a consistent behavior. That's good for
the Web and good for our users.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:24 PM
I don't think so.  Ideally, Ubuntu and other distros would foot the bill
for a codec on the system (maybe we could help them with that) and then
we'd have a solution that was accessible to a larger set of application
developers.

Or maybe we could do it in such a way that Firefox deposited the codec
into the system where it could be used by any other app. That would
still require, I think, users getting Firefox from us and not from a
distro.

Then again, I could be totally wrong.

- A

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Pearce 3/12/12 7:25 PM
On 13/03/2012 3:09 p.m., Justin Lebar wrote:
>> Which is why we have to license and ship the decoders on Windows XP and
>> Linux.
> Maybe this is a dumb question, but how will this affect Linux distros
> which build Firefox but call it TemperatureAnimal?  Will they be able
> to use the decoders we've licensed?
>

Probably not. Could we create a plugin (such as Andreas is suggesting in
bug 714408) which works in a generic Gecko runtime, purchase licenses
for use in a "generic Gecko runtime", and bundle that with Firefox? Then
any downstream Gecko user can either bundle that (if it's legal?) or
have their app/users download that and install it so they use the
encumbered codecs?

Chris P.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:26 PM
On 3/12/2012 7:10 PM, Andreas Gal wrote:
> Nobody said we will abandon anyone. I am confident we will find a way that doesn't require us to ship encumbered codecs. We can create an opportunity for third parties to plug the shrinking (but still large) WinXP gap. Everywhere else the solution is obvious.

OK. Sounds good to me. I can't imagine why someone else would pay the
licensing fees to deliver the codec to Firefox users for free, (unless
it was Adobe or someone already paying those fees) but I'll take your
word for it.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Pearce 3/12/12 7:28 PM
On 13/03/2012 3:18 p.m., Asa Dotzler wrote:
> We should license and ship these decoders. If there's a major cost
> difference ( don't know the economics very well) then I'd be fine with
> just shipping them where they're missing (more code burden) and
> relying on system stuff where it already exists.

If the cost isn't too great, licensing and shipping decoders would be my
preferred solution, since it's by far the quickest path to market, has
the smallest maintenance overhead, and easiest way to ensure consistent
cross platform behaviour.


Chris P.



Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 7:34 PM
I think it's only single digit millions per year. Not outrageous.

- A

Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) therealbr...@gmail.com 3/12/12 7:38 PM
On Monday, March 12, 2012 2:55:48 PM UTC-7, Chris Pearce wrote:
> On 13/03/2012 10:30 a.m., Robert O'Callahan wrote:
> > We held the line in the hope that the industry would follow, and that
> > Google would do a lot to improve and support WebM, especially removing
> > H.264 support from Chrome. So we've held the line, and watched, and waited,
> > and personally I am extremely disappointed by the results.
>
> +1. And we have bled and lost support and of the early adopters and
> created fostered ill-will in the webdev community because of this issue.

Right, and Google Chrome has not.

Google may not have intended to punk us into hurting ourselves (I don't think they did) but they certainly have wisely avoided hurting Chrome's market share by turning off H.264 decoding from <video>.

Even now, they could IMHO well afford to drop H.264 support from <video> on *desktop* Chrome, because they have best-in-class Flash integration on desktop, and because authors typically write <object> fallback inside <video>. They have win/win now, and win later should they some day drop H.264 from <video>.

Think about this, you who are claiming Mozilla did not hold out or pay a high enough price.

Think in particular about your own vaunted purity. I argue you are hiding behind Mother Adobe's skirts right now, since Firefox users absolutely rely on Flash to play H.264 video embedded in <video> fallback <object> elements. So please don't sermonize or lecture -- take a realistic view of the entire fallback logic chain, and of Firefox's current acute dependency on Flash.

If we rely on Flash to play H.264 now, how pure are we? Compared to using a system decoder, not any more pure, in my view.


> If Google were to announce that they were to drop support of H.264 in
> Chrome tomorrow, we'd still have IE and Safari holding out (most people
> won't bother to install the required WebM plugins for said browsers). I
> don't think Safari ever would about face, and there's only a remote
> chance IE would change.

Again, Chrome on desktop would be most ready to drop H.264 from <video> because they have practically custom-forked Flash into best-of-breed fallback.

Chrome on Android 4? I doubt they'll drop H.264 from video. Stock browser on all Android? No way.

> I agree with Ralph: the best solution for our users is to suck it up and
> pay the H.264/AAC/MP3 license fees and ship libav/ffmpeg on desktop, and
> ship hardware decoders on mobile. If necessary we can patch security
> holes in libav, so we're not vulnerable to the black-box problem we get
> with system codecs on desktop, and we can offer H.264 support to the 40%
> of our users on XP - something that IE can't do.

We should proceed carefully. Mozilla will not pay for an H.264 license that does not benefit downstreams. I personally do not believe we should pay any such rent. It's better for us to ride on an OS decoder if possible (we ride on Flash now, see above).

OS dependencies are ugly but browsers including Mozilla and Firefox have taken them since day 1 of the modern (Netscape and beyond) web.

> On 13/03/2012 7:44 a.m., Andreas Gal wrote:
> > Google pledged many things they didn't follow through with and our users and our project are paying the price. H.264 wont go away. Holding out just a little longer buys us exactly nothing.
>
> In my darker moments I've wondered if this was a deliberate strategy by
> Chrome to make us continue to hold an untenable position and continue to
> bleed users.

I don't think Google did anything specifically to hurt us (we did, rather), but they clearly are helping themselves by not turning off H.264 in Chrome, and they're not turning it off in Android stock browser either. And on desktop they have invested so much in Flash that they could afford to turn off H.264 from <video>, but so what? Money talks.

> Having said that, it would be a shame if YouTube turned on WebM by
> default and Chrome dropped H.264 support just as we agreed to pickup
> H.264/MP3 support on desktop. We should reach out to Google through
> offical channels, and try to get them to publicly commit to the above
> before we make a decision.

We can reassess. Right now and this calendar year, the die is cast.

/be
unk...@googlegroups.com 3/12/12 7:38 PM <This message has been deleted.>
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Nicholas Nethercote 3/12/12 8:16 PM
On Mon, Mar 12, 2012 at 2:10 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
>
> I agree with almost everything Andreas and Chris Blizzard have written on
> this thread, except I'm not happy about its timing or the way it has been
> presented.

Exactly.  A change in a highly controversial policy position shouldn't
happen like this.  (I disagree strongly with the notion that mobile
and desktop can be treated separately on this issue.)

Nick
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 8:22 PM
What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?

Andreas

>
> Nick
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 8:32 PM
Please don't conflate our purity (or lack thereof) on standards with our patent purity. This thread, despite its misleading subject line, is about shipping something that makes a basic primitive on the web unimplementable in a free stack in order to benefit our users.

document.all was easily and freely implemented by one and all, and had the great side effect of helping our users use the real web. h.264 and AAC only do the latter.
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joe Drew 3/12/12 8:35 PM
A) hiding behind a subject line that nobody would object to; that is, not calling out the explicit and blatant change in policy it represents.
B) pretending that this affects only mobile. The same arguments apply to desktop.

Luckily the conversation has been steered in the right direction.

Sent from my mobile
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Andreas Gal <g...@mozilla.com> wrote:


On Mar 12, 2012, at 11:16 PM, Nicholas Nethercote wrote:

> On Mon, Mar 12, 2012 at 2:10 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
>>
>> I agree with almost everything Andreas and Chris Blizzard have written on
>> this thread, except I'm not happy about its timing or the way it has been
>> presented.
>
> Exactly. A change in a highly controversial policy position shouldn't
> happen like this. (I disagree strongly with the notion that mobile
> and desktop can be treated separately on this issue.)

What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?

Andreas

>
> Nick
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 8:44 PM

On Mar 12, 2012, at 11:35 PM, Joe Drew wrote:

> A) hiding behind a subject line that nobody would object to; that is, not calling out the explicit and blatant change in policy it represents.

Calling a post to dev-platform about this "hiding" is simply absurd. This is the longest thread on dev-platform in months. Its on various news sites. Nobody is at risk of missing this. The bug has been in the open for weeks. A large number of people have been following along even before I posted about this. I am sorry you don't agree with the content of the post, but I suggest we focus on technical arguments instead of arguing over titles of emails.

> B) pretending that this affects only mobile. The same arguments apply to desktop.

Nobody pretends this only affects mobile. Please go back and read the post and the bug. We have a solution for mobile in hand. We want to land that. Then we want to focus on desktop.

Andreas

>
> Luckily the conversation has been steered in the right direction.
>
> Sent from my mobile
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>
> Andreas Gal <g...@mozilla.com> wrote:
>
> On Mar 12, 2012, at 11:16 PM, Nicholas Nethercote wrote:
>
> > On Mon, Mar 12, 2012 at 2:10 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> >>
> >> I agree with almost everything Andreas and Chris Blizzard have written on
> >> this thread, except I'm not happy about its timing or the way it has been
> >> presented.
> >
> > Exactly.  A change in a highly controversial policy position shouldn't
> > happen like this.  (I disagree strongly with the notion that mobile
> > and desktop can be treated separately on this issue.)
>
> What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?
>
> Andreas
>
> >
> > Nick
> >
>
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/12/12 8:46 PM
On 3/12/2012 8:32 PM, Joe Drew wrote:
> Please don't conflate our purity (or lack thereof) on standards with
> our patent purity. This thread, despite its misleading subject line,
> is about shipping something that makes a basic primitive on the web
> unimplementable in a free stack in order to benefit our users.

I think this is about both. I think future fights on codecs will
definitely be about both. But I take your point. My point isn't so much
concerned with that distinction though.

Our shipping or not shipping h.264 doesn't make the Web unimplementable
in a free stack. The dominance of h.264 video on the Web -- something
our 3 years of Open Video support has not changed or even slowed down,
is what makes the Web unimplementable in a free stack.

What we ship is not always the same thing as "the Web." We can influence
the Web some of the time. We might even be able to kill a bad idea
single-handedly some of the time. But sometimes the Web decides to go
where we don't want it and there's nothing we can do to stop it. I
believe this is one of those cases and what we do now by not supporting
the Web, as it is, is to lose developers and users.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/12/12 8:47 PM
On Mar 12, 8:35 pm, Joe Drew <j...@mozilla.com> wrote:
> A) hiding behind a subject line that nobody would object to; that is, not calling out the explicit and blatant change in policy it represents.

What policy? Unencumbered-only-or-bust? Not my policy, nor mitchell's.

FWIW shaver now concedes this is the realistic course for mobile.

> B) pretending that this affects only mobile. The same arguments apply to desktop.

Wrong. Flash fallback on desktop may be the better (still OS-
dependent, at bottom) course. This is about pragmatics, not high do-or-
die principle.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Nicholas Nethercote 3/12/12 9:02 PM
On Mon, Mar 12, 2012 at 8:22 PM, Andreas Gal <g...@mozilla.com> wrote:
>>
>> Exactly.  A change in a highly controversial policy position shouldn't
>> happen like this.  (I disagree strongly with the notion that mobile
>> and desktop can be treated separately on this issue.)
>
> What exactly do you disagree with? The 'like this' is a public discussion on dev-platform with the goal of reaching a consensus. What other process would you prefer?

This is a discussion to reach consensus?  Ok, good!

Your original post sounded to me like you were saying that this
decision was a done deal, particularly the "I don't think this bug
significantly changes our position on open video" bit.  Apologies if I
misinterpreted it.

Nick
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/12/12 9:17 PM
"I want to land bug 714408 on mozilla-central as soon as I get review for it."

I am expressing my desire to do something, and I am asking for feedback about it. I am not even a peer for the video code (first time contributor in fact). The module owners (including Roc and Brendan) and peers will make the necessary decisions once the dust settles (including granting review or not). We have a solid project governance structure. Have faith in it.

Andreas

>
> Nick

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/12/12 11:39 PM
On Mon, Mar 12, 2012 at 8:44 PM, Andreas Gal <andre...@gmail.com> wrote:
> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.

I think the crucial line that's about to be crossed is exposing
functionality that cannot be implemented royalty-free to the Web. (We
already expose features that are implemented using royalty-bearing
technologies--anything that calls a Windows or Mac OS X API--to the
Web, but that's OK as long as it's also possible to implement those
features on top of a royalty-free OS (Linux).)

*If* we cross that line by exposing H.264 to the Web in a Mozilla
product (it's already exposed by everyone else--including by Opera in
their mobile and TV products--we are very alone with our current
position) here's how I think we should do it:

I think it's very important to have one Web platform and to have Gecko
expose the same feature set everywhere. Moreover, if we expose H.264
to the Web (regardless of whether we ship an H.264 implementation), we
will have reneged on our principle not to ship non-RF codecs. After
that, Web authors will not bother to support us with WebM on *any*
platform and will wait for us to expose H.264 to the Web everywhere.
(Many Web authors are in that mode already.)

Therefore, if/when we expose H.264 to the Web in some Mozilla product,
we should have it ready across the product line--including Firefox on
Windows XP and Firefox on desktop Linux.

I think Mozilla Corporation should have two products: Firefox and
"Mozilla MPEG Components" (the latter name is just a placeholder, of
course). The source code for both would be available under Open Source
licenses. However, while anyone can take the Firefox source, compile
it and distribute it royalty-free under a different name, it would be
acknowledged that distributing a self-compiled copy of Mozilla MPEG
Components probably requires licenses from MPEG-LA and Via Licensing
in some countries.

Mozilla MPEG Components would include the Android 4.0 software
implementations of H.264 and LC-AAC (or maybe libav implementations).
(I think the discussion about whether to support MP3 is a separate
discussion. I don't know if the MP4 container is RF, so I don't know
which side of the Firefox/Mozilla MPEG Components divide it should
go.) Mozilla MPEG Components would expose its functionality through an
API exactly or roughly like your MPAPI proposal. Mozilla MPEG
Components would compile into a shared library.

Firefox would look for a library that quacks like Mozilla MPEG
Components. If found, Firefox would report video/mp4 with the right
codecs parameter as supported via canPlayType and play
H.264/LC-AAC/MP4. If not found, Firefox would behave like today.

I think Mozilla Corporation should take the patent licenses at the cap
price where number of units shipped doesn't matter and ship
productized binaries of Mozilla MPEG Components for all platforms that
Mozilla ships Firefox for. Also, I think it would be good to ship
Mozilla MPEG Components for platforms like FreeBSD and Solaris where
there are non-Mozilla supported builds of the Firefox codebase
available.

Mozilla-shipped Firefox releases would come with a bundled Mozilla
MPEG Components release, so users of Mozilla-shipped Firefox wouldn't
have to go through any special steps to get H.264/AAC support. Parties
who self-compile the Firefox codebase (either under the Firefox name
or some ElementAnimal name) could use a copy of Mozilla MPEG
Components binaries shipped by Mozilla Corporation for free. If they
wanted to, they could compile Mozilla MPEG Components themselves and
get patent licenses on their own. Even non-Gecko products would be
welcome to use Mozilla MPEG Components as shipped by Mozilla
Corporation for free, so if Opera wanted to use it on desktop, I think
Mozilla should let them. If someone wanted to write a Gstreamer
plug-in that wrapped Mozilla MPEG Components for use in other Linux
apps, that would be cool too.

Basically, I think Mozilla needs to pay for the patent licenses in
order to deal with Windows XP and not leave functionality of Firefox
on XP up to a third party. Once the patent licenses are sunk cost, I
think it makes sense to hack the MPEG-LA scheme to the maximum extent
possible and give a codec-lib-only product away for free to everyone.

Then, once that baseline software functionality is available on all
platforms that Gecko runs on, I think it would make sense to use
system decoders instead of Mozilla MPEG Components where available.
But I think using system decoders where available as the first step
would be bad as it would fragment the feature set Web authors can
expect of Firefox.

- -

Some tangential points:
 * In order to avoid fragmenting the platform exposed to Web sites, I
think we should still control the set of video formats exposed to the
Web and shouldn't expose random system codecs even when we technically
could.
 * The licensing regime for MP3 is even more unpleasant than the
regime for H.264 and AAC, so I think MP3 requires a separate
discussion and decision. It's also worth noting that it's not too long
until the MP3 spec turns 20 years old, so it might make sense to wait
until then.
 * On the face of things, it seems that using the Android 4.0 software
implementations of H.264 and AAC would be a better match for code
that's designed to support Android hardware acceleration, too. Also,
for a component of this nature, Apache License 2.0 might be nicer than
LGPL 2.1. However, there might be technical reasons to prefer libav's
decoders. I'm not competent to judge that.
 * Conceding to a patent toll booth once sets a terrible precedent.
Already at the W3C some people are using the H.264 precedent as an
excuse to introduce DRM schemes that not only are patent-encumbered
but are also trade secret-encumbered (the operation of H.264 is not a
secret) and anti-circumvention-encumbered (decoding H.264 is not an
anti-circumvention-law-invoking activity).

--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Mike Hommey 3/13/12 12:48 AM
On Mon, Mar 12, 2012 at 07:38:59PM -0700, therealbr...@gmail.com
wrote:
> We should proceed carefully. Mozilla will not pay for an H.264 license
> that does not benefit downstreams. I personally do not believe we
> should pay any such rent. It's better for us to ride on an OS decoder
> if possible (we ride on Flash now, see above).

We could actually ride on Flash when an OS decoder is not present. We
could certainly build our own .swf <video> player for those.
As for Linux, I'm not sure to what extent h264 decoding is available to
them by default. I know Debian has it, now. I guess Ubuntu has, too.

Mike
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Brendan Eich 3/13/12 12:56 AM
On Mar 13, 12:48 am, Mike Hommey <m...@glandium.org> wrote:
> On Mon, Mar 12, 2012 at 07:38:59PM -0700, therealbrendane...@gmail.com
> wrote:
>
> > We should proceed carefully. Mozilla will not pay for an H.264 license
> > that does not benefit downstreams. I personally do not believe we
> > should pay any such rent. It's better for us to ride on an OS decoder
> > if possible (we ride on Flash now, see above).
>
> We could actually ride on Flash when an OS decoder is not present. We
> could certainly build our own .swf <video> player for those.

Yes, we discussed this a few years back -- it's quite doable.

I agree with Henri's point that we should unify the web-author-facing
API, not just do <object>-in-<video> fallback on some platforms (but
that is standard operating procedure). I do not agree we must pay the
rentiers their shakedown fees.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 3/13/12 1:27 AM
On 12-03-12 8:46 PM, Asa Dotzler wrote:

> our 3 years of Open Video support has not changed or even slowed down,
> is what makes the Web unimplementable in a free stack.

I disagree that we haven't slowed down open Video Adoption, but that's
not an experiment we get to do. It's more accurate to just say we're not
happy with the results.

If we land this patch, and do the other things, we should be clear that
we've been forced to by adoption patterns, and that non-free codecs
still hurt the Open Web.

  -r
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Chris Double 3/13/12 2:35 AM
On Tue, Mar 13, 2012 at 8:56 PM, Brendan Eich
<therealbr...@gmail.com> wrote:
>> We could actually ride on Flash when an OS decoder is not present. We
>> could certainly build our own .swf <video> player for those.
>
> Yes, we discussed this a few years back -- it's quite doable.

I had a prototype at one point that did this and had discussions with
Adobe. It required at the time changes to their flash player due to us
requiring control of the downloaded stream (for browser caching, etc)
and/or cross origin changes - I'd have to look up the exact reasons it
stalled.

Chris.
--
http://www.bluishcoder.co.nz
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Henri Sivonen 3/13/12 2:54 AM
On Tue, Mar 13, 2012 at 9:48 AM, Mike Hommey <m...@glandium.org> wrote:
> As for Linux, I'm not sure to what extent h264 decoding is available to
> them by default. I know Debian has it, now.

By default? Really? That sounds *very* un-Debian.

> I guess Ubuntu has, too.

I just did two fresh Ubuntu installs (64-bit) in a VM.

By default, H.264, MP3 or AAC support or Flash Player is not
installed. Once you need support, install wizards pop up for these.

In the system installer, there's an unchecked-by-default checkbox for
"third-party sofware" some of which is closed source. It talks about
Flash Player and a Fraunhofer-licensed Fluendo MP3 codec. If you check
this box, you get Flash Player, Fluendo MP3 codec (presumably; I
didn't verify) and the "bad", "ugly" and "ffmpeg" Gstreamer plug-in
sets (which were not mentioned in the installer UI).

The  "bad", "ugly" and "ffmpeg" Gstreamer plug-in sets are marked as
not supported by Canonical. Canonical is listed as an MPEG-LA H.264
pool licensee. I don't know how the legal side of this works. The
"check your laws" legal warning is now gone from the install wizards.

Alternatively, H.264 and AAC support for Gstreamer is available for
$34.95 as part of Fluendo Complete Playback Pack from the Ubuntu
Software Center.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) yves.va...@gmail.com 3/13/12 3:17 AM
Le lundi 12 mars 2012 08:28:04 UTC+1, Andreas Gal a écrit :
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

This all sounds reasonable and pragmatic to me, I'd love to see h264, mp3 and others… supported where the OS already licensed it, especially on Mobile.
Opera Mobile is doing it already, their solution is perfect and I think this should be the way to go.
Especially since downloading an mp3 in Firefox just launches it in any installed external player, there is no benefit in doing this when you could just re-use what is existing in the OS, the user experience would be much better.

On a side note: I think the codec war isn't even the subject here, we can continue to raise awareness, communicate and develop technologies such as WebM. PNG is a good example of how long it takes and eventually developers will see their benefits.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Mike Hommey 3/13/12 3:30 AM
On Tue, Mar 13, 2012 at 11:54:25AM +0200, Henri Sivonen wrote:
> On Tue, Mar 13, 2012 at 9:48 AM, Mike Hommey <m...@glandium.org> wrote:
> > As for Linux, I'm not sure to what extent h264 decoding is available to
> > them by default. I know Debian has it, now.
>
> By default? Really? That sounds *very* un-Debian.

Debian's ffmpeg/libav supports h264 and aac out of the box, so all you need is
that.

Mike
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) nuke...@mozilla-hispano.org 3/13/12 3:43 AM
Hi,

I'm worried we have to re-evaluate this.

I think time ago we all agreed about how H264 wasn't good for the web and how based on our values we had to support Theora/Vorbis/Webm formats. What happened with that?

I'm OK if H264 support is done via third party plugin, as Java or Flash do.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) nuke...@mozilla-hispano.org 3/13/12 3:43 AM
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/13/12 3:53 AM
On Tue, Mar 13, 2012 at 12:43 PM,  <nuke...@mozilla-hispano.org> wrote:
> I think time ago we all agreed about how H264 wasn't good for the web and how based on our values we had to support Theora/Vorbis/Webm formats. What happened with that?

Web authors by and large didn't agree with our values to such a degree
as to actually serve WebM or Ogg to Firefox a significant proportion
of times when users encounter video. Chrome didn't drop H.264 to make
authors pay more attention.

> I'm OK if H264 support is done via third party plugin, as Java or Flash do.

Keeping the yucky stuff in third-party code doesn't really help keep
it off the Web.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) nuke...@mozilla-hispano.org 3/13/12 3:59 AM
El martes 13 de marzo de 2012 11:53:52 UTC+1, Henri Sivonen  escribió:
> Web authors by and large didn't agree with our values to such a degree
> as to actually serve WebM or Ogg to Firefox a significant proportion
> of times when users encounter video. Chrome didn't drop H.264 to make
> authors pay more attention.

If we start supporting things we agree that are bad for the web, what makes us different from others vendors/browsers?

Regards.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) nuke...@mozilla-hispano.org 3/13/12 3:59 AM
El martes 13 de marzo de 2012 11:53:52 UTC+1, Henri Sivonen  escribió:
> Web authors by and large didn't agree with our values to such a degree
> as to actually serve WebM or Ogg to Firefox a significant proportion
> of times when users encounter video. Chrome didn't drop H.264 to make
> authors pay more attention.

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 3/13/12 4:08 AM
On 13/03/12 06:39, Henri Sivonen wrote:
> Basically, I think Mozilla needs to pay for the patent licenses in
> order to deal with Windows XP and not leave functionality of Firefox
> on XP up to a third party. Once the patent licenses are sunk cost, I
> think it makes sense to hack the MPEG-LA scheme to the maximum extent
> possible and give a codec-lib-only product away for free to everyone.

All of what Henri said, but particularly this.

cdouble suggests the "fallback to Flash" option requires changes to the
Flash player, and we aren't going to get anything out of Adobe right
now. So to support XP, we need to ship codecs. And if we are going to
ship codecs, we should use and ship our own codecs everywhere where it
doesn't lead to big performance or battery issues - for consistency, for
control, and to give others the ability to leverage them and avoid the
patent tax.

Gerv
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 3/13/12 4:33 AM
On 12/03/12 07:28, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review
> for it. It adds hardware-accelerated audio/video decoding support to
> Gecko using system decoders already present on the system.

Given that doing this requires us changing our current policy on
supporting non-RF video formats[0], the public discussion on doing so
should not be rushed just because the code to implement one possible
outcome has got to the "review" stage.

I'm not opposed in principle to changing that policy - as others have
said, Mozilla combines principles with pragmatics. A browser with no
users would have no leverage. I do think, though,  we would have been
better to have the whole debate after March 15th. If I were from MPEG-LA
and reading this thread, I would see little incentive to make baseline
H.264 RF for WebRTC any more. (But perhaps I don't have the whole
picture there.)

Anyway, if we are going to change the policy, we need to figure out what
the "next best" approach is. A "we need to do this on mobile this week,
so let's do it and then figure out the next step" approach seems
unlikely to me to end up in the best place. Even if we don't write the
code, I think we should agree on which support:

- for which exact codecs (audio and video)
- on which platforms
- using which technical means and which libraries (ours or system)
- paying which amounts of legalized extortion
- exposing which APIs to content and to other system apps
- using what manner of shipping (separate product/shared libraries)

before shipping support anywhere. Various options have been presented in
this thread; I don't think turning it into a strategy of record should
take too long.

Ideally, "how we interact with DRM" should also be on that list,
although perhaps that's a discussion which is too big in scope to
include for now. We need at least to bear in mind possible future
developments there.

In addressing the above questions, I very much like Henri's plan
(further up/down the thread).

Gerv

[0] Stated several times publicly by Roc, e.g. here:
http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Cameron Kaiser 3/13/12 6:56 AM
> > Maybe this is a dumb question, but how will this affect Linux distros
> > which build Firefox but call it TemperatureAnimal?  Will they be able
> > to use the decoders we've licensed?
>
> I don't think so.  Ideally, Ubuntu and other distros would foot the bill
> for a codec on the system (maybe we could help them with that) and then
> we'd have a solution that was accessible to a larger set of application
> developers.

Speaking as a TemperatureAnimal (well, VersionAnimal: TenFourFox), I'm
a little distressed at the legal exposure we could end up taking if we
shipped something like that.

At least on OS X, I'd feel a lot better if such an implementation
leveraged QuickTime.

We ship an add-on called the QuickTime Enabler that enables the
browser to launch a window for H.264 video playback. It's kludgey and
it's not integrated in the same way other HTML5 video is, but it
works, and it's not our decoder so we don't owe anybody anything for
it. (The trick we exploit doesn't work on QT X, though, but that's
fine for a PowerPC audience.)

Cameron Kaiser
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/13/12 7:20 AM
On Tue, Mar 13, 2012 at 3:56 PM, Cameron Kaiser <cka...@floodgap.com> wrote:
> At least on OS X, I'd feel a lot better if such an implementation
> leveraged QuickTime.

Does QuickTime provide low-level enough APIs to fit in Gecko's video
architecture? On the OS X versions Firefox runs on? On the OS X
versions TenFourFox runs on?
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) benjamin....@gmail.com 3/13/12 7:53 AM
On Monday, March 12, 2012 5:55:48 PM UTC-4, Chris Pearce wrote:
> On 13/03/2012 10:30 a.m., Robert O'Callahan wrote:
> > We held the line in the hope that the industry would follow, and that
> > Google would do a lot to improve and support WebM, especially removing
> > H.264 support from Chrome. So we've held the line, and watched, and waited,
> > and personally I am extremely disappointed by the results.
>
> +1. And we have bled and lost support and of the early adopters and
> created fostered ill-will in the webdev community because of this issue.

Two points:

1. Perhaps Mozilla has fostered ill-will in certain communities, but Mozilla's refusal to support H.264 has also generated tremendous goodwill in others.  I think anyone who has attended any of the Open Video Conference events has encountered some of them.

2. I think a lot of us are disappointed with the WebM adoption curve so far, but there's more going on than meets the eye.  Real silicon on the market now has WebM acceleration built in and supported.  If Mozilla chose the Marvell Armada 1500, Rockchip RK2918, or Nvidia Tegra 3 CPUs for its B2G device, it could ship with fast, low-wattage hardware WebM tomorrow.  Once this generation of chips becomes the mobile mainstream, the codec market will look very different.

> If Google were to announce that they were to drop support of H.264 in
> Chrome tomorrow, we'd still have IE and Safari holding out (most people
> won't bother to install the required WebM plugins for said browsers).

IMHO Google can easily achieve saturation of these plugins.  Will they?  Don't ask me!
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Boris Zbarsky 3/13/12 8:10 AM
If nothing else, the fact that we don't support those things until we
exhaust all possible alternatives...

-Boris

Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) benjamin....@gmail.com 3/13/12 7:53 AM
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) ferongr 3/12/12 8:55 AM
On Monday, March 12, 2012 9:28:04 AM UTC+2, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

If you (Mozilla) are going to compromise from your original position on non RF codecs you might as well go all out and also enable MP3 and AAC support for <audio> elements if the relevant decoders/filters are available. h264 videos commonly have AAC and HE-AAC audio tracks for the most part.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) yves.va...@gmail.com 3/13/12 3:17 AM
Le lundi 12 mars 2012 08:28:04 UTC+1, Andreas Gal a écrit :
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Ted Mielczarek 3/13/12 8:21 AM
On Tue, Mar 13, 2012 at 10:53 AM,  <benjamin....@gmail.com> wrote:
> 2. I think a lot of us are disappointed with the WebM adoption curve so far, but there's more going on than meets the eye.  Real silicon on the market now has WebM acceleration built in and supported.  If Mozilla chose the Marvell Armada 1500, Rockchip RK2918, or Nvidia Tegra 3 CPUs for its B2G device, it could ship with fast, low-wattage hardware WebM tomorrow.  Once this generation of chips becomes the mobile mainstream, the codec market will look very different.

That's a lot of ifs, and it doesn't change the supply side of the
equation (video content available on the web).

>> If Google were to announce that they were to drop support of H.264 in
>> Chrome tomorrow, we'd still have IE and Safari holding out (most people
>> won't bother to install the required WebM plugins for said browsers).
>
> IMHO Google can easily achieve saturation of these plugins.  Will they?  Don't ask me!

I think it's pretty evident that we shouldn't rely on Google changing
anything to get us the outcome we desire at this point. They have
certainly contributed a lot to the cause (purchasing On2 and
freely-licensing VP8, contributing to WebM, making YouTube content
available in WebM), but I see no evidence that they're going to change
the support of h.264 in Chrome or Android.

-Ted
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Cameron Kaiser 3/13/12 8:37 AM
> > At least on OS X, I'd feel a lot better if such an implementation
> > leveraged QuickTime.
>
> Does QuickTime provide low-level enough APIs to fit in Gecko's video
> architecture? On the OS X versions Firefox runs on? On the OS X
> versions TenFourFox runs on?

QT7 (I don't know about QTKit or QT X) has ways of extracting frames
out of a QuickTime movie, and from there you can turn that into a
CGImage and do whatever you need to do with the bitmap information.
Q&A 1443 comes to mind: https://developer.apple.com/library/mac/#qa/qa1443/_index.html

The 10.4Fx QuickTime Enabler is even simpler, however. It creates a
QuickTime playlist for the content (which was dropped in QT X) and
uses ctypes and LaunchServices to hand that to QuickTime Player.app,
which then plays the video outside of the browser. Since it's
QuickTime Player that does the loadin'n'encodin', I figure this is
pretty MPEG-LA-proof. It wouldn't be suitable for Firefox generally,
however, but it's good for Power Macs because it's much less grunt to
play the video in a separate window and not have to composite it.

You can try it yourself, but it won't work on 10.6 or 10.7, and I
don't know if it works on Intel 10.5 (it should, however):
http://tenfourfox.googlecode.com/files/tenfourfox-quicktime-enabler-alpha114.xpi

The reason I bring this up is that having an MPEG-LA license is all
well and good for Firefox, but I hesitate on what that means for
SeaMonkey, let alone little builders like us. The QTE is a way for us
to serve our users and not get in trouble. I would be worried about
our own licensing situation if we had to fall back on ffmpeg.

Cameron Kaiser
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/13/12 9:36 AM
Also, on a legal side, to people who ask to bundle Firefox with open
source H264 codec, please check if it's legally possible. I remember
an article (OSnews ?) that explained that once you're a MPEG-LA
licencee you have to use the official proprietay codec.
But i may be wrong.
Thanks
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ben Adida 3/13/12 9:36 AM
On 3/12/12 9:16 AM, Andreas Gal wrote:
>
> I do believe this war is lost.

This seems to be the key point.

We should be careful of arguments that *only* say "users and web
developers want this!" Of course, we would be foolish to ignore our
users and developers, but following their immediate desires blindly can
lead us to a closed web. The patent holders are smart: the first hit is
always free. We are uniquely positioned to look further into the future
and shape the Web.

So we took a position on this for the last few years: we will fight for
the future of the Web by using our market power (and, we thought,
Google's) to force change. I was super proud of that position. It's one
of the reasons I joined Mozilla. I think it was the right fight.

A year later, it's possible we need to give up this fight. Maybe the war
is indeed lost. Do we have data on that, not just anecdotes? Are we sure
about the trend lines? Can we push on Google one last time, with a deadline?

If all signs point in the direction Andreas is stating, then I agree
that a retreat on this battle, combined with a clear resolve to better
fight the next fight, is the right thing to do.

But I don't think we should do this based on "users wants this" alone.
"Users wants this" is not enough to protect the Open Web, and we could
find ourselves in a few years with "Users don't want this, but now it's
too late to change the game."

-Ben
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/13/12 9:33 AM
Hi,

I'm just a french Firefox user and supporter.

I was very pround of Mozilla decision to only support free codecs
(WebM, Ogg Theora/vorbis) and i understand that relying on Flash
(which is already dominant - at least on desktop) is less bad than
relying on a patent encumbered H.264 codec.

Therefore i'm a bit sad to read that discussion, since WebM had some
achievements, for instance Opera and Chrome support, hardware
acceleration in some SoC, and of course WebM in YouTube which is not
exactly a small player.

I know that Mozilla makes great efforts to support WebM but i would
like to be sure that everything has been tempted before seing Mozilla
embracing H264 :

1°) What are exact (not supposed) Google plans about dropping H264
within Chrome ?
2°) What are exact Microsoft plans concerning WebM looking that issue
https://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/02/22/google-please-don-t-kill-video-on-the-web.aspx?Redirected=true
pointed by Justin Dolske ?
3°) Is it - yes or no - possible to decode VP8 with Rockchip RK29xx,
Nvidia Tegra 2 & 3,  Texas Instrument OMAP 4 ? I'm a bit suprised to
read that Mozilla would still be investigating that question, i
thought that B2G project and Firefox4Android would has raise these
questions earlier ?

Because already having Youtube playing WebM videos and several ARM Soc
that can decode VP8 with low power consumption is not exactly a
hopeless situation !

When Apple decided to drop Flash on mobile (whatever its reasons) the
context was not such different. What would make H264 adoption so
urgent for Mozilla ?

Imagine that B2G and Firefox4Android can use hardware to decode WebM
efficiently and YouTube which already plays WebM, i think it could add
pressure i.e. to Vimeo to support WebM since firefox user would use
YouTube and not Vimeo.

To sum up my thougts, investigate hardware WebM support on Rockchip
RK29xx, Nvidia Tegra 2 & 3,  Texas Instrument OMAP 4 with Android and
B2G seems like a priority before to decide anything. Also, ask Google
and Microsoft what are their plans for their browser.

Thanks for your past work and keep on rocking !


unk...@googlegroups.com 3/13/12 9:39 AM <This message has been deleted.>
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/13/12 9:44 AM
On 3/13/2012 4:33 AM, Gervase Markham wrote: > Given that doing this
requires us changing our current policy on
> supporting non-RF video formats[0],

> [0] Stated several times publicly by Roc, e.g. here:
> http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html

I have stupid question. How is ROC's position on non-RF formats "policy"
and not simply "the position of a module owner"?

I looked all over mozilla (newsgroups, sites, blogs, etc) and couldn't
find any Mozilla Policy on this issue.

If it is simply the position of some of the leaders in our community,
does that automatically make it a Mozilla Policy?

I'm not asserting with 100% certaintly that it isn't Mozilla Policy, but
from what I've been able find in documentation, it's not.

- A


Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/13/12 9:51 AM

On 3/13/2012 9:33 AM, antistress wrote:

> I know that Mozilla makes great efforts to support WebM but i would
> like to be sure that everything has been tempted before seing Mozilla
> embracing H264 :
>
> 1°) What are exact (not supposed) Google plans about dropping H264
> within Chrome ?

You'd have to ask Google that. We've asked them and they're not saying.

> 2°) What are exact Microsoft plans concerning WebM looking that issue
> https://blogs.technet.com/b/microsoft_on_the_issues/archive/2012/02/22/google-please-don-t-kill-video-on-the-web.aspx?Redirected=true
> pointed by Justin Dolske ?

Microsoft has said publicly that they have no plans to ship WebM on
Windows but that IE would support it if the codec were properly
installed into Windows.

> 3°) Is it - yes or no - possible to decode VP8 with Rockchip RK29xx,
> Nvidia Tegra 2 & 3,  Texas Instrument OMAP 4 ? I'm a bit suprised to
> read that Mozilla would still be investigating that question, i
> thought that B2G project and Firefox4Android would has raise these
> questions earlier ?

Hardware decoders have not made sufficient progress into the market. See
details elsewhere in this thread.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/13/12 9:46 AM
The "war is lost" statement was misphrased. The word is "battle", not
"war". The war goes on. WebRTC is a new battle where unencumbered
formats should prevail. There are other battle fronts.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Andreas Gal 3/13/12 11:31 AM
Very true. We should move the focus to providing an open replacement for the upcoming H.265 standard from the start so history won't repeat itself.

Andreas

>
> /be
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Jet Villegas 3/13/12 11:37 AM
I would have used "hardware-enabled" to describe the codecs we're talking about here. "hardware-accelerated" implies that software codecs are OK and they really are not. Battery life is the one limiting factor that isn't getting better fast enough. The Flash Player team has made it policy to not add software-only codecs as they've been skewered for power consumption. We should really take the pragmatic approach here and not use up all the device's juice when the consumer already paid for video hardware.

-- Jet
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/13/12 11:47 AM
Also, I think it's important to note that fighting the battle wasn't
wasted effort. WebM is in much better shape because of Mozilla's
efforts. Not only is WebM in better shape, but I think it actually
proves that open codecs can compete. It didn't win, but it demonstrated
viability and it may yet go on to claim a critical role in WebRTC.

Finally, I think there's something important in our having taken that
stance. We've demonstrated that we don't default to "what's easy". We
may not be able to win every battle, but we don't shy away from fighting
the good fight.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/13/12 11:51 AM
On Tue, Mar 13, 2012 at 6:36 PM, antistress <thibaut...@gmail.com> wrote:
> Also, on a legal side, to people who ask to bundle Firefox with open
> source H264 codec, please check if it's legally possible.

Google bundles the ffmpeg H.264 decoder with Chrome. The decoder is
under the LGPL. See
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-June/020064.html

Google supplies an Open Source (Apache License 2.0) implementation of
H.264 as part of Android. It's highly probable that vendors that ship
it have licensed additional patents from elsewhere on top of the
copyright and patent grants Google makes under Apache License 2.0.

> I remember
> an article (OSnews ?) that explained that once you're a MPEG-LA
> licencee you have to use the official proprietay codec.

There's no "official proprietary" H.264 *implementation*. The whole
point of the standard is to enable independent interoperable
implementations. There are plenty of different H.264 implementations.
Which one you use isn't important as far as MPEG-LA is concerned.

IANAL of course.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Randell Jesup 3/13/12 12:02 PM
Andreas Gal <andre...@gmail.com> wrote
>Even if a license grant is made for H.264 for WebRTC the need for this
>patch still holds. On anything with a battery we have to use hardware
>decoders. Those will always be buried somewhere in hardware with some
>kernel pieces in addition. We will never ship with those bits, and we
>have to use whats on the system, which really takes us back to where
>this argument started.

The argument about power is one I've made to the google people in
WebRTC, and their response was that the radio and display nowadays eat
more power than video encode and decode, even without hardware assist.
I think the number (off-the-cuff) quoted was 2W for display/radio, and
max 0.5W for the CPU.  Note that the CPU probably can't hit the same
encode resolution/frame-rate that HW can, however - but for WebRTC from
a mobile device that's probably ok.  Decode is much simpler.

--
Randell Jesup, Mozilla Corp
remove ".news" for personal email
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Hubert Figuière 3/13/12 12:12 PM
On 13/03/12 02:39 AM, Henri Sivonen wrote:
> On Mon, Mar 12, 2012 at 8:44 PM, Andreas Gal <andre...@gmail.com> wrote:
>> We can't license decodes and ship those. That would bar others from shipping Firefox without entering into the same deals. All we are talking about is using existing accelerated decoders that already are licensed and available on the system.
>
> I think the crucial line that's about to be crossed is exposing
> functionality that cannot be implemented royalty-free to the Web. (We
> already expose features that are implemented using royalty-bearing
> technologies--anything that calls a Windows or Mac OS X API--to the
> Web, but that's OK as long as it's also possible to implement those
> features on top of a royalty-free OS (Linux).)
>
> *If* we cross that line by exposing H.264 to the Web in a Mozilla
> product (it's already exposed by everyone else--including by Opera in
> their mobile and TV products--we are very alone with our current
> position) here's how I think we should do it:

[...]

Henri,

I have to say that your implementation proposal is by far the best I
have seen and I believe it strike the right balance.

Also I have to add, even that makes me sad to say it, that if we
implement h264 on Mobile (and I do believe this is something that we
need), we are *required* to do so on Desktop as to not fragment our own
web platform. It will have a cost. A monetary cost AND a cost on freedom.


Cheers,

Hub
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Boris Zbarsky 3/13/12 12:18 PM
On 3/13/12 11:33 AM, antistress wrote:
> Therefore i'm a bit sad to read that discussion

Just to be clear, I don't think a single person involved in this
discussion is _happy_.  The current situations sucks.  A lot.

> 1°) What are exact (not supposed) Google plans about dropping H264
> within Chrome ?

We don't know, and they won't tell us.  There have been past promises to
tell us something by some point, and those promises have not been
honored as far as I can tell.

If you know some way to get Google's exact plans, please do tell.  :(

> When Apple decided to drop Flash on mobile (whatever its reasons) the
> context was not such different. What would make H264 adoption so
> urgent for Mozilla ?

One difference is that people really had no option other than iOS if
they wanted devices like the iPhone or iPad for a while.  On the other
hand, if people want a browser that does pretty much what Firefox does
but also supports H.264 they have several perfectly viable choices.

> Imagine that B2G and Firefox4Android can use hardware to decode WebM
> efficiently and YouTube which already plays WebM, i think it could add
> pressure i.e. to Vimeo to support WebM since firefox user would use
> YouTube and not Vimeo.

I think Vimeo is betting on Firefox4Android and B2G never having enough
market share to make this a real issue.  And if enough web video is not
playable on them, then not playing it may be enough to prevent them ever
gaining such market share...

> To sum up my thougts, investigate hardware WebM support on Rockchip
> RK29xx, Nvidia Tegra 2&  3,  Texas Instrument OMAP 4 with Android and
> B2G seems like a priority before to decide anything.

I think this has been done, no?

> Also, ask Google and Microsoft what are their plans for their browser.

Also been done...  Multiple times, by multiple people, in multiple
venues, over the span of a year now.  We know what Microsoft's plans are
(see Asa's reply).  Google is not talking for the most part.

-Boris
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Pearce 3/13/12 12:23 PM
On 14/03/2012 3:20 a.m., Henri Sivonen wrote:
> On Tue, Mar 13, 2012 at 3:56 PM, Cameron Kaiser<cka...@floodgap.com>  wrote:
>> At least on OS X, I'd feel a lot better if such an implementation
>> leveraged QuickTime.
>
> Does QuickTime provide low-level enough APIs to fit in Gecko's video
> architecture? On the OS X versions Firefox runs on? On the OS X
> versions TenFourFox runs on?
>


Matthew Gregan will know OSX media frameworks better than I as he worked
on the original Quicktime HTML5 backend we developed before we decided
not to use platform codecs (bug 435298), but IIRC it took significant
bashing to get QuickTime to accept data downloaded by our network stack.
IIRC that was on OSX 10.5, and on at least one of 10.6 and 10.7 we'd
have to use a different API to get playback support, possibly both, so
we'd need 2 maybe 3 backends to cover OSX using platform codecs.

On Windows 7 we can use Windows Media Foundations' H.264 decoder. This
may allow us to take advantage of any specialist video decoding hardware
the machine has.

On Windows Vista we can only use Windows Media Foundations' H.264
decoder but only through it's "SourceReader" interface. I think that
would be adequate for our needs (I thought this was the easiest way to
implement playback on Win7 last time I looked at this; it may not be the
optimal way). I thought this was only available on Vista SP2, but I
can't find a reference for that, so maybe I misremembered.

As mentioned elsewhere, Windows XP includes no H.264 decoder.

Linux we can use GStreamer, or libav if it's installed on system.

One point I forgot to mention, re: us shipping libav/ffmpeg on Windows;
libav won't compile with MSVC. It uses C99 which isn't supported by MSVC
[1]. So we'd need to find a way around that if we were to pay for a
license and ship libav. Last I heard Chrome committed a libav binary
compiled with mingw into their tree and just link against with that on
Windows.


Chris P.

[1] http://libav.org/faq.html#Is-Microsoft-Visual-C_002b_002b-supported_003f
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Peterson 3/13/12 12:28 PM
On 3/12/12 7:01 PM, Asa Dotzler wrote:
> Yes. Absolutely. h.264+AAC is the de facto standard for the <video>
> element and if we're going to support it, we should support it everywhere.

I agree. Supporting H.264 for the <video> element allows content
providers to "level up" from proprietary Flash video to HTML5 video.

H.264 may be patent-encumbered, but at least HTML5 video is an open
framework that gives content providers a roadmap for unifying their
production pipelines and adopting open codecs in the future.


chris peterson

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert Kaiser 3/13/12 12:50 PM
Andreas Gal schrieb:
> I don't think this bug significantly changes our position on open video.

I think enough discussion in here has proven that not to be true - in
fact, we'll remove the last incentive for web authors to actually use
open video at all, IMHO.

I find this very sad, but the whole story on video on the web is sad so
far in my eyes. Maybe that will change in the future.

What I wonder is if I as a user will be able to easily turn off non-open
codecs on my Firefox or B2G installations (in the latter case, I might
actually want to differentiate web content from local content, though,
just like my mobile devices do now as well)?
I have no problems if I "suffer" personally from not being able to play
all web content, but I have a strong interest in not using
patent-encumbered formats when I don't have to.

All that said, I need to re-evaluate if this new direction is in
conflict with my involvement as a Mozilla Rep with an event part of
FSFE's Document Freedom Day which is to promote open standards. Not sure
if I can reasonably hand out Mozilla swag there if we are not standing
as firmly on open standards wrt video as we have been in the past.

Robert Kaiser
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert Kaiser 3/13/12 12:52 PM
Henri Sivonen schrieb:
> I think Mozilla Corporation should have two products: Firefox and
> "Mozilla MPEG Components"

The latter should be a pre-installed add-on (like testpilot on beta)
that can easily be disabled by users who prefer to not use
patent-encumbered formats.

Robert Kaiser
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Kise S. 3/13/12 1:53 PM
On Monday, March 12, 2012 10:28:04 AM UTC+3, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas



On Monday, March 12, 2012 10:28:04 AM UTC+3, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas



On Monday, March 12, 2012 10:28:04 AM UTC+3, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas



On Monday, March 12, 2012 10:28:04 AM UTC+3, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

I am a developer of one big video sites in arabic our close to 1tb video library are in h264 there is no chance in hell that i or the website owner going to re-encode our full library to another codec which is not even fully supported, so for firefox we use flash if the user doesn't have flash then they are sol and your stance on only supporting RL codecs bleeding you dry, you don't have to support it or like it, you can put warning and discourage the use of non RL codecs, but give the users the chance to use the codecs if they feel like it, otherwise firefox market will shrink even farther.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 1:23 PM
Here are my $0.02 on this topic ...

Whether or not WebM has a chance to become the #1 video format for the web obviously depends much on what Google is doing, and whether they are willing to fulfill their commitment to remove H.264 from Chrome, as announced in January 2011. This is the only potential game changer that I currently see, as long as Chrome along with IE and Safari support H.264, I don't think anything big will change towards WebM's favor.

So I think it's hard to make a good decision as long as Google doesn't make their plans clear. Which is why I believe that the best thing to do for now is to put pressure on Google, also publicly, to finally say whether or not they still plan to remove H.264 support from Chrome, and if yes, when.

If they don't (or don't reply/react at all), there is no use to keep resisting, then the fight would be lost.

If however there is a positive reply from Google, Mozilla shouldn't go forward supporting H.264.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) John Jensen 3/13/12 2:28 PM
> Right now and this calendar year, the die is cast.

Or, said another way, "the market has spoken". 80% of video content, as
surveyed by a web video indexing company in December 2011, is served via
H.264. This proportion has increased every time they've measured it:

http://blog.mefeedia.com/html5-dec-2011

While, as Henri has noted [1], the market share of *browsers* that
support WebM has come close to surpassing that of H.264, that, with the
growth of mobile and tablet platforms, seems unlikely to actually
happen. Even if it does, content providers are as of now unwilling to
switch.

[1] http://hsivonen.iki.fi/webm-share/

--
John Jensen
Web Technology Competitive Analyst
Mozilla Corporation
+1 604 218 0400
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Double 3/13/12 2:44 PM
On Wed, Mar 14, 2012 at 3:20 AM, Henri Sivonen <hsiv...@iki.fi> wrote:
> Does QuickTime provide low-level enough APIs to fit in Gecko's video
> architecture? On the OS X versions Firefox runs on? On the OS X
> versions TenFourFox runs on?

We were working on a quicktime backend at one point in bug 435298:

<https://bugzilla.mozilla.org/show_bug.cgi?id=435298>

Matthew Gregan developed it. This was when we were working on OS
decoding support via gstreamer, directshow and quicktime.

Chris.
--
http://www.bluishcoder.co.nz
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Mike Hommey 3/13/12 3:00 PM
On Tue, Mar 13, 2012 at 02:28:04PM -0700, John Jensen wrote:
> >Right now and this calendar year, the die is cast.
>
> Or, said another way, "the market has spoken". 80% of video content,
> as surveyed by a web video indexing company in December 2011, is
> served via H.264. This proportion has increased every time they've
> measured it:
>
> http://blog.mefeedia.com/html5-dec-2011

That should come to no surprise. About everything produces H264 now
(phones, digital cameras...). That's where the battle was lost, really.

Mike
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) bawjaws 3/13/12 3:20 PM
I'd like to propose that, even if platform codecs are used to provide
H.264 and AAC for the video tag, Firefox should still blacklist AAC
for the audio tag and instead support MP3 and Vorbis.

This topic is complicated enough that I'm not suggesting I know for
certain what the right final decision on this topic is, but it seems
it deserves at least some consideration as a separate issue from video
codec support.

Some relevant info:

MP3 is patent encumbered, but depending on what you read, the decoding
patents should expire sometime between the end of 2012 and 2020. I'd
love it if Mozilla could help locate a definitive answer on what that
end date might be, including actively seeking both legal and technical
workarounds to the remaining patents (decode and encode) if it could
bring that date forward.

As far as I'm aware there is no browser, certainly no popular browser,
that supports AAC audio but not MP3. This, along with its continued
success in personal audio arena, makes MP3 a valid lowest-common-
demoninator format. For many uses e.g. music or  voice narration of a
slideshow MP3 is more than capable. For anyone who requires greater
technical capabilities e.g. looping sound effects for games, the costs
and effort of transcoding to AAC and Vorbis are worlds away from
having to transcode entire video libraries.

There is a popular, high-quality, open source mp3 encoder, as is there
for Vorbis. The same cannot be said for AAC.

Fluendo seems able to provide an MP3 decoder at no cost to the Linux
end user, which Ubuntu for example optionally package, for use by any
Gstreamer enabled apps and have done for several years now. I believe
they pay for this right, but don't know the details.

Henri Sivonen in this discussion claims mp3 licensing is even worse
than AAC. I thought I'd heard the opposite from an x264 developer.
Either way, it's a problem that disappears with the patent expiry and
(given the Fluendo decoder and proprietary platform support) not
something that will bother users in the meantime.

From a a purely technical point of view, including the code and paying
licence fees for AAC but not using it in the audio tag seems perverse.
But this whole discussion is about Mozilla doing what is best for the
open web. I think a strong case can be made for intentionally leaving
out AAC support for the audio tag and that doing so, even after
implementing it for video, gives Mozilla an interesting hook to talk
about why its (proposed) support for H.264 video is merely a strategic
concession in a wider struggle for an open web.

regards,

dave
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) justi...@gmail.com 3/13/12 3:24 PM
This can't happen fast enough. I hate having to store multiple copies of the same video and audio files. Just make is simple. If the user want to watch a video any browser that refuses to show it for any non-technical reason is a bad browser.

I can already tell you *I* don't target or use Firefox on a daily basis any more because the majority of video/audio sites I go to visit dont work. Their developers don't want to deal with transcoding and storing multipule copies any more than the rest of us.

If you don't want to provide a codec because of a believe or philosophical stance fine, your right. But if its already installed why not use it? I suppose if your stance not to, but in the end more and more people are leaving Firefox as a result.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/13/12 4:02 PM
On 3/13/2012 1:23 PM, mks....@gmail.com wrote: > Here are my $0.02 on
This sounds nice, but, IMO, the time for Google to do something was a
year ago. Even if Google committed today to dropping h.264 from Chrome
desktop, the world has changed enough in the last year that I don't
think even Chrome desktop's position can have much influence one way or
another today.

The momentum was there a year ago. Maybe it was even still there 8
months ago. I don't think it's there today and Chrome alone probably
can't restore that momentum. The rise of Mobile and of iPad and the
changing landscape for Adobe including streaming server support for
h.264 for <video> element (with no VP8 anywhere in sight) and the
massive growth of Windows 7 compared to a year ago where h.264 comes
with the system and IE -- too much has changed for the worse. Turning
off h.264 in Chrome doesn't feel to me like it could change the course
enough.

If Google said that they were not only dropping h.264 from Chrome
Desktop, Android OS, and Chrome for Android; and making 100% of the
YouTube catalog available in WebM immediately; and they offered up
serious improvements to the encoders; and Adobe said it was bundling
WebM in flash  all of that today or on a date certain in the very near
term, I might feel differently.

- A



Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/13/12 4:23 PM
Could we imagine another - intermediate - solution ?

In former Mozilla plans, WebM was the goal and Flash was only used to
help the transition to be done.

Could we imagine that Mozilla keep that goal (promoting WebM), and
make use of H264 only to help the transition on mobile (where Flash is
about to disapear) ?
I mean : if the SoC doesn't support WebM acceleration, then allow
H264. But if the SoC does support WebM acceleration, then don't allow
H264.

That way :
- B2G could be launched with hardware decoding (low power consumption)
on every SoC (using WebM when it's possible as 1st choice, H264 in the
other case)
- Mozilla would keep on pressuring to promote WebM

Thoughts ?
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/13/12 4:44 PM
On Mar 13, 4:23 pm, antistress <thibaut.beth...@gmail.com> wrote:
> Could we imagine another - intermediate - solution ?
>
> In former Mozilla plans, WebM was the goal and Flash was only used to
> help the transition to be done.
>
> Could we imagine that Mozilla keep that goal (promoting WebM), and
> make use of H264 only to help the transition on mobile (where Flash is
> about to disapear) ?
> I mean : if the SoC doesn't support WebM acceleration, then allow
> H264. But if the SoC does support WebM acceleration, then don't allow
> H264.

This is a recipe for non-interoperation. Content producers are not
making WebM available. That's the problem.

> That way :
> - B2G could be launched with hardware decoding (low power consumption)
> on every SoC (using WebM when it's possible as 1st choice, H264 in the
> other case)
> - Mozilla would keep on pressuring to promote WebM

What pressure?

You have to realize we have very little mobile market share right now.
As Asa points out, Apple has moved the needle hard, and Android is not
going WebM-only or even WebM-over-H.264. We're emtering the game late
with few chips.

B2G is our big bet, it's a good one. But it won't even get on phones
if it doesn't do H.264 video. No Flash fallback, remember.

The game theory is merciless. Google might have helped a year ago.
Might not have been enough then, but they didn't remove H.264 <video>
support, and if they had their Flash fallback is best-of-breed. The
real issue (ignoring Apple and Microsoft) is Android stock browser
(Chrome on ICS is just new enough to exclude from consideration of the
last year). Android is not going to push WebM at the price of breaking
H.264 content -- no way. I can safely write that without speaking for
Google.

The pressure to promote WebM was needed from a bigger player than
Mozilla, and it was needed a year ago. Not now, and not desktop-only
(which is all Chrome could offer in terms of market power, until ICS
takes off mor).

It might not have worked then, even with Google on-side. Now, with
just Mozilla going it alone, all we do is kill our mobile initiatives
in order to appear pure (while relying on impure Flash fallback on
desktop). That does not serve our mission or users.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) brad.o...@gmail.com 3/13/12 4:35 PM
Allowing system decoders would be nice for open source UNIX OS's. As long as there is an option for using FFmpeg and *NOT* GStreamer.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 6:18 PM
On Wednesday, March 14, 2012 12:02:13 AM UTC+1, Asa Dotzler wrote:
> Even if Google committed today to dropping h.264 from Chrome
> desktop, the world has changed enough in the last year that I don't
> think even Chrome desktop's position can have much influence one way or
> another today.

I believe if Google drops H.264 support from Chrome (even if only the desktop version), no web author could seriously ignore WebM. Chrome + Firefox + Opera make a huge share of the market with tendency on the rise. I think that could change the game towards WebM. I agree that this should happen sooner than later.

YouTube already offers the vast majority of videos as WebM, and at least if Flash is disabled in the browser (not sure, but perhaps even if enabled) it delivers WebM by default, since not so long ago. So YouTube is almost there, and to expect that YouTube will indeed deliver HTML5 video using WebM by default in the near future doesn't seem unrealistic to me.

Google supporting H.264 in Chrome is IMO the by far biggest factor that's holding WebM back. And maybe even Microsoft would reconsider adding WebM support to IE if Chrome no longer supported H.264. They may just need one more incentive to finally do it, and Chrome dropping H.264 may just be it.

So I think it's totally worth to give Google a last serious push a try. And last but not least ... then really nobody could blame Mozilla for not having tried everything. The blame for WebM having failed would clearly be on Google.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Nicholas Nethercote 3/13/12 6:40 PM
On Tue, Mar 13, 2012 at 6:18 PM,  <mks....@gmail.com> wrote:
>
> So I think it's totally worth to give Google a last serious push a try. And last but not least ... then really nobody could blame Mozilla for not having tried everything. The blame for WebM having failed would clearly be on Google.

I'm pretty sure this "last serious push" has already been tried,
unfortunately.  Various Mozilla people have been trying to get info
from Google for months without success.

The whole situation is quite odd.  Google announced quite loudly they
were dropping h.264 support 14 months ago.  AFAIK they haven't said a
*single* thing about it since.

Nick
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/13/12 6:41 PM
On Mar 13, 6:18 pm, mks.p...@gmail.com wrote:
> On Wednesday, March 14, 2012 12:02:13 AM UTC+1, Asa Dotzler wrote:
> > Even if Google committed today to dropping h.264 from Chrome
> > desktop, the world has changed enough in the last year that I don't
> > think even Chrome desktop's position can have much influence one way or
> > another today.
>
> I believe if Google drops H.264 support from Chrome (even if only the desktop version), no web author could seriously ignore WebM.

No, because Chrome has excellent Flash fallback on desktop.

People keep ignoring this, but it matters. It takes away any "stick"
and there is no "carrot".

It matters that this is true on desktop, too. The contested space is
mobile, where Adobe withdrew Flash last November, and where Flash
wasn't ever as good an option (SJ was right, in part).

Mozilla has been riding on Flash fallback for Firefox via <object>
nested in <video>. Do not turn a blind eye to this, for us or for
Chrome -- especially for Chrome, given their custom Flash integration.

> Chrome + Firefox + Opera

Opera desktop has tiny share. Opera mobile uses h/w h.264 decoders
(right?).

/be

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 7:05 PM
On Wednesday, March 14, 2012 2:40:30 AM UTC+1, Nicholas Nethercote wrote:
> I'm pretty sure this "last serious push" has already been tried,
> unfortunately.  Various Mozilla people have been trying to get info
> from Google for months without success.

This last serious push should certainly be public. If Mozilla is considering supporting H.264, it would be a good opportunity to make a public call to Google (in an official blog for example), whether or not their commitment (and promise) is still valid.

Mozilla considering to support H.264 is certainly going to be picked up by quite some tech media, and if this is combined with a public request to Google that they should finally say whether they are still committed to fully supporting WebM (and at the same time saying that Mozilla will continue their commitment, but only if Chrome drops support, so Mozilla isn't wasting their efforts), it might be quite hard for Google to ignore it again. (And if Google ignores a public request which likely attracts some media coverage, that would also be quite a clear answer, if unfortunately a clear no.)

A private request can much easier be ignored than a (last, but loud) public request. I wouldn't leave this untried.

And again, if Mozilla really needs to support H.264, there will certainly be some blaming. Like who killed WebM. Or is Mozilla really committed to the free web. Or why has Mozilla hesitated for so long (to some people for no apparent reason). With such a public and loud last call, many more people would understand Mozilla's decisions, and those who are going to ignite flamewars against Mozilla (and this certainly will happen; actually it happens all the time) will have very weak arguments at hand.

So even if the low chances to convince Google don't make this seem to be worth the effort, the positive public image by demonstrating that really everything has been tried and by once more explaining why Mozilla acted this way (which also is yet another opportunity to explain the problem with patent encumbered software), may make it worth the effort.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/13/12 7:58 PM
You are still ignoring the Flash fallback issue (on desktop; no
fallback on mobile and zero chance Google pulls h/w h.264 decoding
from Android Stock Browser or Chrome on ICS).

Also what's this double-talk about us "hesitating?" If you are on the
side of unencumbered video, we stayed that course. You should be
praising us. But if you think we should have given up without a fight,
how principled would that have been? Not very.

At first, with Google seemingly along side, we thought we had a chance
to win WebM adoption. It did not work out, in part because Google
didn't pull h.264 support from Chrome <video> -- but even if they had,
it's not clear we all could have won. There was still Flash fallback
saving us from broken video icons. There was still a huge pipe of
camera, etc., h/w burning h.264 into SiO2.

Fault Mozilla for not seeing the inevitable, fault us for relying on
Flash fallback on desktop and not seeing it as insufficient or
canceled on mobile, but don't fault us for not living up to ideals.
That's not our issue.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Nicholas Nethercote 3/13/12 8:19 PM
On Tue, Mar 13, 2012 at 7:05 PM,  <mks....@gmail.com> wrote:
>
> This last serious push should certainly be public.

This thread will serve that purpose.  It's getting plenty of
attention, and I'm sure the relevant Google people are aware of it.
Petitioning them publicly won't make any difference -- I seriously
doubt that anyone within Google is thinking "if Mozilla asks really
nicely *one* more time, we'll actually do what we promised".

Maybe you think a public appeal is still worthwhile in order to make
the blame-Google case stronger, but that case is already strong and
well understood.  E.g., from
http://arstechnica.com/gadgets/news/2012/03/idealism-vs-pragmatism-mozilla-debates-supporting-h264-video-playback.ars:

"Google's major investment in advancing its unencumbered VP8 codec
gave open Web advocates hope that H.264 could still be displaced, but
it hasn't happened. The lack of follow-through from Google on its
promise to remove H.264 from Chrome has eroded faith in the search
giant's ability to popularize VP8."

Having said that, if/when this decision is made, I think Mozilla
should definitely make a clear and detailed statement explaining what
happened, and what changed in the past 14 months.

Nick
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) jabcre...@gmail.com 3/13/12 8:42 PM
Even thinking about supporting a codec which corporations wanted to send bills to individuals and small companies just for using means you should be fired.

Funny how this discussion is on Google whose own Chrome browser has LONG claimed they would drop H.264 support yet haven't.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Hofmann 3/13/12 8:55 PM
> _______________________________________________
>

In the end does it really matter what google will do, or not do?
Chances are they will opt for being just a pragmatic as we have chosen
to be.

What's really needed from here forward is a heavy dose of website
evangelism,
and/or some incentives for websites to convert over, and some easy
and guaranteed path they can deliver the kind of experience with either
codec.

Do we think all that is in place?  That's the best focus for this discussion
at this point.

-chofmann
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) jno...@gmail.com 3/13/12 9:15 PM
On Tuesday, March 13, 2012 4:08:20 AM UTC-7, Gervase Markham wrote:
> On 13/03/12 06:39, Henri Sivonen wrote:
> > Basically, I think Mozilla needs to pay for the patent licenses in
> > order to deal with Windows XP and not leave functionality of Firefox
> > on XP up to a third party. Once the patent licenses are sunk cost, I
> > think it makes sense to hack the MPEG-LA scheme to the maximum extent
> > possible and give a codec-lib-only product away for free to everyone.
>
> All of what Henri said, but particularly this.
>
> cdouble suggests the "fallback to Flash" option requires changes to the
> Flash player, and we aren't going to get anything out of Adobe right
> now. So to support XP, we need to ship codecs. And if we are going to
> ship codecs, we should use and ship our own codecs everywhere where it
> doesn't lead to big performance or battery issues - for consistency, for
> control, and to give others the ability to leverage them and avoid the
> patent tax.
>
> Gerv

I don't know how patent licensing works but I think it would be better if system codecs were checked for by the installer.  Where the codecs are not found, they are downloaded and installed system wide.  Then these codecs can be accessed by the MPAPI like any other codec.  This would only require Mozilla to pay for licenses where they are needed, and only once per individual.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/13/12 9:17 PM
On 3/13/2012 8:55 PM, Chris Hofmann wrote: > What's really needed from
here forward is a heavy dose of website
> evangelism, and/or some incentives for websites to convert over,
> and some easy and guaranteed path they can deliver the kind of
> experience with either codec.
>
> Do we think all that is in place?

No. And I don't think it can be put in place.

There is no easy or guaranteed path they can deliver the same
experience. They have to first re-encode their library of content -
something that no one we've talked to other than YouTube (and Blizzard
has talked to other large catalogs) is willing to do. It's not easy or
cheap and it's got zero reward when everything just works with H.264
(even for Firefox because they'll have Flash fallback using that same
h.264 encoding for quite a while.) Next they have to re-build their
sites to do the right thing with H.264, WebM, and Flash fallback (and
probably several quality versions of each of those encodings) for all
the different desktop and mobile browsers. That's no easy task, not
cheap, and again with zero reward.

This isn't a "do x, y, and z, and it will work in Firefox" like the old
days. It currently "works" in Firefox as far as they're concerned. This
isn't a "re-encode once and never again" like "re-write for standards"
was in the old days. It's "double the encoding load, the site burden,
and user support issues" for the foreseeable future.

There is no carrot here. A free codec doesn't help when they still have
to pay for the non-free one to reach all of their Microsoft and Apple
users. They don't get better performance or quality from WebM. They get
something slightly less performant and slightly lower quality.

We are not going to win this one evangelizing a more expensive, slightly
lower quality solution that requires a whole lot of effort with no
obvious benefits. No matter how well it fits our values, it doesn't fit
their values.

Had adobe included WebM in Flash and Flash Media Server; had Google
dropped H.264 on Chrome and Android; maybe there'd be some opportunities
to try to get people to move to WebM as a global web solution but I
think that opportunity past us about 8 months ago with the massive rise
in Android and iPad usage and with the further cementing of h.264 as the
<video> element solution - both with more site use and with Adobe's
launch of a streaming h.264 to the <video> tag server product.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/13/12 10:11 PM
On 3/12/12 11:39 PM, Henri Sivonen wrote:

> [...]
>
> Therefore, if/when we expose H.264 to the Web in some Mozilla product,
> we should have it ready across the product line--including Firefox on
> Windows XP and Firefox on desktop Linux.

Well, here's a counter argument.

Windows XP is a dead-end OS. It's old and on its way to becoming
unsupported by its vendor, and eventually by us. The only problem is
that it's still got a lot of existing users. But these users already
don't have H.264 support in Firefox, and already have limited options
for a modern browser (i.e., not IE. :). [I guess Chrome still supports
XP (with H.264?), I wonder what their plans are.]

So, it would not be unreasonable to find that the cost (literally!) and
effort for supporting H.264 on XP is not worth it. It's a slight
fracturing of the web/gecko platform, but also one that will solve
itself naturally over time. The only part I'm not sure of is if the XP
life-support system will actually keep it around long enough to make it
a problem we have to deal with (wrt to H.264 support).

Linux is a trickier issue. Haters gonna hate, but none of the above
applies to Linux. However I am pleasantly surprised to find a number of
comments in this thread indicating that Linux already has a number of
options for H.264 support. It actually sounds like H.264 is less of a
problem on Linux than XP. So I'm not sure what to make of that. :)

Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Justin Dolske 3/13/12 10:14 PM
On 3/13/12 12:18 PM, Boris Zbarsky wrote:

> Just to be clear, I don't think a single person involved in this
> discussion is _happy_. The current situations sucks. A lot.

So very true!

Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/13/12 10:36 PM
On Mar 13, 10:11 pm, Justin Dolske <dol...@mozilla.com> wrote:
> On 3/12/12 11:39 PM, Henri Sivonen wrote:
>
> > [...]
>
> > Therefore, if/when we expose H.264 to the Web in some Mozilla product,
> > we should have it ready across the product line--including Firefox on
> > Windows XP and Firefox on desktop Linux.
>
> Well, here's a counter argument.
>
> Windows XP is a dead-end OS. It's old and on its way to becoming
> unsupported by its vendor, and eventually by us. The only problem is
> that it's still got a lot of existing users. But these users already
> don't have H.264 support in Firefox,

Flash fallback...

Again <video>-using authors generally nest an <object> in the <video>
container to handle browsers that don't grok <video> at all. This also
works for <video> with a type not handled by the browser. Flash has
its own H.264 decoder along with decoders for many other encumbered
formats. Flash is widely distributed, including on XP.

So the users you speak of do, generally speaking (based on standard
author practice and templates, e.g., from youtube "Share" widgets)
have "H.264 support in Firefox".

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 10:42 PM
On Wednesday, March 14, 2012 4:55:30 AM UTC+1, Chris Hofmann wrote:
> Chances are they will opt for being just a pragmatic as we have chosen
> to be.

Mozilla should not make a pragmatic decision. If Mozilla decides to implement H.264 it should be only if there is no longer a chance to win the fight. And Google is the last chance, so yes, it matters what Google does.

> What's really needed from here forward is a heavy dose of website
> evangelism,
> and/or some incentives for websites to convert over, and some easy
> and guaranteed path they can deliver the kind of experience with either
> codec.

One thing should be clear to everybody: as soon as all major browsers (and even without Opera, though Opera will have no choice other than implementing H.264 as well) support one common format, WebM will be dead for the web. Nobody will implement 2 formats if one isn't necessary to reach any additional audience. Because having to deal with 2 formats is a pain.

Evangelism is very difficult for various reasons. Too many people don't know or care about patent issues, because they don't understand why it is an issue which affects them, and it's not a simple thing to explain in 3 sentences. For people to really understand the problems, they must invest 30 minutes, 60 minutes, 90 minutes (depending on how compressed the issues are presented), and you don't get people to invest this much amount of time on a topic which they don't believe affects them. Listen around what most people are saying? They are pissed at Mozilla because they blame Mozilla that they have to deal with multiple formats. That giving in may mean to sell the web to the patent mafia is too abstract for them, also because many of the consequences would happen in very hidden ways (e.g. they would pay although they don't know that what they pay for includes patent license fees). Many aspects of patents help the patent mafia ... which is no surprise if you think of it as what it actually is: a (legalized) crime.

Mozilla has done the right thing in trying to resist. But even I as very strong supporter of WebM and a patent free open web must admit that this war probably can't be won (certainly not without strong efforts from Google).
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/13/12 11:48 PM
On Wed, Mar 14, 2012 at 6:15 AM,  <jno...@gmail.com> wrote:
> I don't know how patent licensing works but I think it would be better if system codecs were checked for by the installer.  Where the codecs are not found, they are downloaded and installed system wide.  Then these codecs can be accessed by the MPAPI like any other codec.  This would only require Mozilla to pay for licenses where they are needed, and only once per individual.

There's per-unit pricing with an annual cap per licensee (i.e. legal
entity shipping a product). Mozilla can't tally the units precisely
enough (without Draconian changes to how a user can obtain Firefox)
and ships so many units to Windows XP alone that it's safe to assume
that if Mozilla were to get patent licenses, Mozilla would be paying
the maximum annual price anyway.

--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/13/12 11:52 PM
On Mar 13, 10:42 pm, mks.p...@gmail.com wrote:
> On Wednesday, March 14, 2012 4:55:30 AM UTC+1, Chris Hofmann wrote:
> > Chances are they will opt for being just a pragmatic as we have chosen
> > to be.
>
> Mozilla should not make a pragmatic decision.

Sorry, it's foolish to say Mozilla should only do non-pragmatic,
idealistic moves. That can lead to extinction and then what good is it
for our users or mission?

> If Mozilla decides to implement H.264 it should be only if there is no longer a chance to win the fight.

No one has perfect knowledge of the future, so this is a kind of
comforting nonsense.

> And Google is the last chance, so yes, it matters what Google does.

So Mozilla should wait for Google? Sorry, that's bad on mission ideals
and pragmatic grounds.

> > What's really needed from here forward is a heavy dose of website
> > evangelism,
> > and/or some incentives for websites to convert over, and some easy
> > and guaranteed path they can deliver the kind of experience with either
> > codec.
>
> One thing should be clear to everybody: as soon as all major browsers (and even without Opera, though Opera will have no choice other than implementing H.264 as well) support one common format, WebM will be dead for the web. Nobody will implement 2 formats if one isn't necessary to reach any additional audience.

Why do you keep ignoring my mail about Flash fallback? It is not
necessary *now* even with Firefox <video> not handling H.264 for any
author to encode twice or even consider WebM. Authors use H.264 and
write Flash fallback. Please respond to this, I know you're reading my
posts.

> Because having to deal with 2 formats is a pain.

Yes, that's a fact and there's nothing we can do to reduce that pain,
unless you think we should wait for Apple and Microsoft as well as
Google.

> Mozilla has done the right thing in trying to resist. But even I as very strong supporter of WebM and a patent free open web must admit that this war probably can't be won (certainly not without strong efforts from Google).

Google is not going to remove h/w H.264 decoding from Android, stock
browser or Chrome. You can count on that.

Desktop doesn't matter because of Flash fallback, but Chrome hasn't
dropped H.264 from <video> to run even a little risk of bouncing into
the fallback content.

I think you are being relentlessly unrealistic in how you write about
pragmatism, and in particular in continuing to ignore the fallback
issue.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/14/12 12:39 AM
On Wed, Mar 14, 2012 at 7:11 AM, Justin Dolske <dol...@mozilla.com> wrote:
> On 3/12/12 11:39 PM, Henri Sivonen wrote:
>> Therefore, if/when we expose H.264 to the Web in some Mozilla product,
>> we should have it ready across the product line--including Firefox on
>> Windows XP and Firefox on desktop Linux.
>
> Well, here's a counter argument.
>
> Windows XP is a dead-end OS. It's old and on its way to becoming unsupported
> by its vendor, and eventually by us. The only problem is that it's still got
> a lot of existing users.

Agreed. Though at present and probably next year still, it has such a
high proportion of the total of Firefox users that I think the
"dead-end OS" consideration isn't particularly persuasive yet. I do
realize the "dead-end OS" consideration will become very persuasive
long before the H.264 patents expire.

> But these users already don't have H.264 support in
> Firefox, and already have limited options for a modern browser (i.e., not
> IE. :). [I guess Chrome still supports XP (with H.264?), I wonder what their
> plans are.]

Indeed, they can't switch to IE9 on XP. However, they could switch to
Chrome or Safari--more likely to Chrome than to Safari. Or Opera if
Opera figures out a way to support H.264 on XP.

> So, it would not be unreasonable to find that the cost (literally!) and
> effort for supporting H.264 on XP is not worth it. It's a slight fracturing
> of the web/gecko platform, but also one that will solve itself naturally
> over time. The only part I'm not sure of is if the XP life-support system
> will actually keep it around long enough to make it a problem we have to
> deal with (wrt to H.264 support).

The announced EOL for XP is April 8, 2014. Chances are that XP will be
in use beyond that date, but even two years is plenty of time for
Firefox users to become unhappy with Firefox, switch to Chrome (and
stick with Chrome when they eventually migrate away from XP).

If we expose H.264 support on all platforms except XP, I fear the
following will happen:
1) Web authors will rejoice that they can now use H.264 in <video> in
all major browsers except IE8 and below. (They don't notice Firefox on
XP, since they aren't treating it currently as a separate thing from
Firefox in general.)
2) A substantial part of Web authors will go "Down with IE8 and below!
Down with Flash!"
3) They'll publish H.264 in <video> without Flash fallback.
4) They'll test in Firefox on Windows 7 and see things are OK, since
they assume Firefox is the same everywhere.
5) Firefox on XP users don't get video a substantial proportion of
time, not even Flash fallback.
6) When someone reminds the authors, they'll go "Who cares about XP
anymore?" and leave things unchanged.
7) Firefox users on XP are people who've made a browser choice before.
They'll make a browser choice again and switch to Chrome.
8) When the users eventually migrate away from XP (to a newer Windows
release, to OS X, to Ubuntu), they'll stick with Chrome.
9) We will have wasted double-digit percentages of our user base away.

Again, two years is plenty of time for the above steps to play out
(though I expect the time until true XP insignificance to be more than
the two years until the official EOL).

I also worry that even if we decide to support system codecs via
Gstreamer and QuickTime on desktop Linux and Mac if those don't ship
in the same release train as system codec support for Android and
Windows >= Vista, users may have perceived their platform as having
been left behind and switched to Chrome by the time we ship Gstreamer
and QuickTime support. So I really hope the H.264 enablement--by
whatever means--happens in the same release train across all operating
systems.

Instead of Mozilla getting patent licenses, maybe this can be solved
by system codecs everywhere but XP and a third-party solution for XP,
but I think it's very important that all platforms be solved at the
same time and that there be a solution for XP that lasts until people
who use Firefox on XP have migrated away from XP.

> Linux is a trickier issue. Haters gonna hate, but none of the above applies
> to Linux. However I am pleasantly surprised to find a number of comments in
> this thread indicating that Linux already has a number of options for H.264
> support. It actually sounds like H.264 is less of a problem on Linux than
> XP. So I'm not sure what to make of that. :)

We would probably be in good enough shape on desktop Linux by using
Gstreamer (with a whitelist allowing only H.264/AAC/MP4 to be passed
to Gstreamer to avoid fragmentation and low-reward security exposure)
and leaving it up to the user / distro to arrange one of the
following:
1) Licensed closed-source codec pack from Fluendo
2) "ugly" and "ffmpeg" codec sets with the distro paying the protection money
3) "ugly" and "ffmpeg" codec sets with the user making sure (s)he is
in a country that hasn't been foolish enough to grant patents of this
kind or where personal use doesn't lead to patent liability.
4) Specialized hardware-specific Gstreamer plug-in that delegates to
licensed hardware.

This would make us better off than Chromium but worse off than Chrome.
This would put us on equal footing with Midori, Epiphany and Opera (if
Opera issues a minor update to re-whitelist H.264/AAC/MP4 in their
Gstreamer support; they had H.264/AAC/MP4 support via Gstreamer on
Linux first but then blocked it to make desktop Opera consistent
across systems).

(The Fluendo pack is $34.95 now. Of course, that sort of thing isn't
going to scale if this encourages others to inject more
royalty-bearing stuff to the Web platform and support for each new
thing is $34.95 per user. No disrespect to Fluendo implied. They are
doing good things with hard 3rd-party constraints.)
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Panos Astithas 3/14/12 1:58 AM
On Wed, Mar 14, 2012 at 1:44 AM, Brendan Eich
<therealbr...@gmail.com>wrote:

> You have to realize we have very little mobile market share right now.
> As Asa points out, Apple has moved the needle hard, and Android is not
> going WebM-only or even WebM-over-H.264. We're emtering the game late
> with few chips.
>
> B2G is our big bet, it's a good one. But it won't even get on phones
> if it doesn't do H.264 video. No Flash fallback, remember.
>

I think this point is worth emphasizing. Video on the Web was an important
battle and to be fair, the situation today is far better than it was before
<video>, even if not as good as we'd hoped. However we are now entering a
fight for a far more important goal: keeping the web relevant in the mobile
space. We simply cannot afford to kill our efforts there in the vain hope
that the world might change while we wait.

Panos
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/14/12 3:12 AM
To complete Henri Sivonen's answer :

GNOME (Linux) has an automatic installation process of codecs when
needed with the default video player (Totem, using GStreamer) since
2.20 version http://library.gnome.org/misc/release-notes/2.20/index.html.en#rnusers-sound-and-video

Justin Dolske a écrit :
> On 3/12/12 11:39 PM, Henri Sivonen wrote:
>
> > [...]
> >
> > Therefore, if/when we expose H.264 to the Web in some Mozilla product,
> > we should have it ready across the product line--including Firefox on
> > Windows XP and Firefox on desktop Linux.
>
> Well, here's a counter argument.
>
> Windows XP is a dead-end OS. It's old and on its way to becoming
> unsupported by its vendor, and eventually by us. The only problem is
> that it's still got a lot of existing users. But these users already
> don't have H.264 support in Firefox, and already have limited options
> for a modern browser (i.e., not IE. :). [I guess Chrome still supports
> XP (with H.264?), I wonder what their plans are.]
>
> So, it would not be unreasonable to find that the cost (literally!) and
> effort for supporting H.264 on XP is not worth it. It's a slight
> fracturing of the web/gecko platform, but also one that will solve
> itself naturally over time. The only part I'm not sure of is if the XP
> life-support system will actually keep it around long enough to make it
> a problem we have to deal with (wrt to H.264 support).
>
> Linux is a trickier issue. Haters gonna hate, but none of the above
> applies to Linux. However I am pleasantly surprised to find a number of
> comments in this thread indicating that Linux already has a number of
> options for H.264 support. It actually sounds like H.264 is less of a
> problem on Linux than XP. So I'm not sure what to make of that. :)
>
> Justin
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) bruce.c...@gmail.com 3/14/12 3:29 AM
Opera mobile uses h/w h.264 decoders
> (right?).

We use whatever is installed on the device, which in most cases means hardware decoding.

FWIW, I've asked various Google people about their dropping h.264 and they've told me - I assume not disingenuously - that they don't know and they've heard nothing.

Odd to spend $104m on webM and then not "big it up" (as da kidz say).

Hugz

Bruce (from Opera, but speaking privately)

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) spec...@gmx.de 3/14/12 4:36 AM
Why Mozilla doesn't support FFDShow for Desktop, plays everything and it is free?
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/14/12 4:55 AM
On 14 mar, 12:36, speci...@gmx.de wrote:
> Why Mozilla doesn't support FFDShow for Desktop, plays everything and it is free?

FFDShow is free from the copyright point of view (as an independant
implementatoon of H264 under a free license) but not from the patent
point of view in countries that deals with patents on software since
it's stil an implemntation of H264.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 3/14/12 7:03 AM
On 13/03/12 19:28, Chris Peterson wrote:
> I agree. Supporting H.264 for the <video> element allows content
> providers to "level up" from proprietary Flash video to HTML5 video.
>
> H.264 may be patent-encumbered, but at least HTML5 video is an open
> framework that gives content providers a roadmap for unifying their
> production pipelines and adopting open codecs in the future.

To generalise: if we concede this battle (as Brendan says, not "war"),
we should concede it in a way which makes us most likely to win the next
one.

Doing something which allows everyone to get on to standard HTML 5 video
markup with no need for a Flash fallback would be one such way of conceding.

Gerv

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 3/14/12 7:09 AM
On 13/03/12 16:44, Asa Dotzler wrote:
> On 3/13/2012 4:33 AM, Gervase Markham wrote: > Given that doing this
> requires us changing our current policy on
>> supporting non-RF video formats[0],
>
>> [0] Stated several times publicly by Roc, e.g. here:
>> http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html
>
> I have stupid question. How is ROC's position on non-RF formats "policy"
> and not simply "the position of a module owner"?
>
> I looked all over mozilla (newsgroups, sites, blogs, etc) and couldn't
> find any Mozilla Policy on this issue.
>
> If it is simply the position of some of the leaders in our community,
> does that automatically make it a Mozilla Policy?
>
> I'm not asserting with 100% certaintly that it isn't Mozilla Policy, but
> from what I've been able find in documentation, it's not.

What is "Mozilla's opinion" or "Mozilla's policy" on a technical issue
is or can be, as you note, a nebulous concept. But roc, as module owner
for Layout and the person generally in charge of our media-playing code
and teams, has been asserting that this is Mozilla's view for some time
now and no-one higher up the organization has contradicted him. I think
a "the most important and qualified person to give an opinion on an
issue sets the policy, particularly if no-one contradicts him" policy
fits what our current practice is.

Also, if I said "until now, it's been our policy that we will not ship
with non-RF codecs", and you said "no, it's not - we were perfectly
willing to do that at any point, we were stopped by (e.g.) technical
issues", that would come as a large surprise to many people both inside
and outside Mozilla.

Gerv
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Timothy Terriberry 3/14/12 7:22 AM
On Mar 14, 7:03 am, Gervase Markham <g...@mozilla.org> wrote:
> To generalise: if we concede this battle (as Brendan says, not "war"),
> we should concede it in a way which makes us most likely to win the next
> one.

Every word of this: http://bemasc.net/wordpress/2012/03/14/ethics-in-an-unethical-world-ethics-offsets/
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Pimm Hogeling 3/14/12 7:25 AM
Op woensdag 14 maart 2012 03:05:21 UTC+1 schreef mks....@gmail.com het volgende:
> This last serious push should certainly be public. If Mozilla is considering supporting H.264, it would be a good opportunity to make a public call to Google (in an official blog for example), whether or not their commitment (and promise) is still valid.
>
> Mozilla considering to support H.264 is certainly going to be picked up by quite some tech media, and if this is combined with a public request to Google that they should finally say whether they are still committed to fully supporting WebM (and at the same time saying that Mozilla will continue their commitment, but only if Chrome drops support, so Mozilla isn't wasting their efforts), it might be quite hard for Google to ignore it again. (And if Google ignores a public request which likely attracts some media coverage, that would also be quite a clear answer, if unfortunately a clear no.)
>
> A private request can much easier be ignored than a (last, but loud) public request. I wouldn't leave this untried.
>
> And again, if Mozilla really needs to support H.264, there will certainly be some blaming. Like who killed WebM. Or is Mozilla really committed to the free web. Or why has Mozilla hesitated for so long (to some people for no apparent reason). With such a public and loud last call, many more people would understand Mozilla's decisions, and those who are going to ignite flamewars against Mozilla (and this certainly will happen; actually it happens all the time) will have very weak arguments at hand.
>
> So even if the low chances to convince Google don't make this seem to be worth the effort, the positive public image by demonstrating that really everything has been tried and by once more explaining why Mozilla acted this way (which also is yet another opportunity to explain the problem with patent encumbered software), may make it worth the effort.

Those are some wise words you're posting, my esse.

Getting some media attention sounds like a good idea to me. H.264 is bad for the web, but most of the n00bs that have "webdesigner" in their Twitter bio have no idea…

Whatever Mozilla does, there will be blaming. Supporting H.264? Mozilla kills the free web. Not supporting H.264? Mozilla can't build a decent browser that "plays HTML5 video".

Explaining decisions in public and calling Google to either put their money where their mouth is (by dropping H.264) or dropping their promise both help raise awareness of the situation.

I don't believe this thread raises nearly as much awareness and understanding as a press release would.
unk...@googlegroups.com 3/14/12 7:25 AM <This message has been deleted.>
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Sander 3/14/12 8:06 AM
Nicholas Nethercote wrote:
> The whole situation is quite odd.  Google announced quite loudly they
> were dropping h.264 support 14 months ago.  AFAIK they haven't said a
> *single* thing about it since.

Tab Atkins said on public-html a few days ago:
> we (Chrome) made a promise
> over a year ago to *drop* H.264 support because of how troublesome it
> was for the open web; we have yet to honor that promise, but I'm aware
> that we are still working toward it.

source: http://lists.w3.org/Archives/Public/public-html/2012Mar/0130.html

~ Sander
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Hubert Figuière 3/14/12 8:17 AM
On 14/03/12 11:06 AM, Sander wrote:

> Tab Atkins said on public-html a few days ago:
>> we (Chrome) made a promise
>> over a year ago to *drop* H.264 support because of how troublesome it
>> was for the open web; we have yet to honor that promise, but I'm aware
>> that we are still working toward it.
>
> source: http://lists.w3.org/Archives/Public/public-html/2012Mar/0130.html

I think it is important to quote the whole paragraph:

> Further, it seems frankly *bizarre* for a Google representative to
> hold up H.264 as a positive example when we (Chrome) made a promise
> over a year ago to *drop* H.264 support because of how troublesome it
> was for the open web; we have yet to honor that promise, but I'm aware
> that we are still working toward it.  I know that we as a company
> rarely, if ever, have a coherent voice, but still, mixed messages.

The context is actually important. Analysis is left as an exercise to
the reader.


Hub
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) brad.o...@gmail.com 3/13/12 4:35 PM
Allowing system decoders would be nice for open source UNIX OS's. As long as there is an option for using FFmpeg and *NOT* GStreamer.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) justi...@gmail.com 3/13/12 3:24 PM
This can't happen fast enough. I hate having to store multiple copies of the same video and audio files. Just make is simple. If the user want to watch a video any browser that refuses to show it for any non-technical reason is a bad browser.

I can already tell you *I* don't target or use Firefox on a daily basis any more because the majority of video/audio sites I go to visit dont work. Their developers don't want to deal with transcoding and storing multipule copies any more than the rest of us.

If you don't want to provide a codec because of a believe or philosophical stance fine, your right. But if its already installed why not use it? I suppose if your stance not to, but in the end more and more people are leaving Firefox as a result.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 1:23 PM
Here are my $0.02 on this topic ...

Whether or not WebM has a chance to become the #1 video format for the web obviously depends much on what Google is doing, and whether they are willing to fulfill their commitment to remove H.264 from Chrome, as announced in January 2011. This is the only potential game changer that I currently see, as long as Chrome along with IE and Safari support H.264, I don't think anything big will change towards WebM's favor.

So I think it's hard to make a good decision as long as Google doesn't make their plans clear. Which is why I believe that the best thing to do for now is to put pressure on Google, also publicly, to finally say whether or not they still plan to remove H.264 support from Chrome, and if yes, when.

If they don't (or don't reply/react at all), there is no use to keep resisting, then the fight would be lost.

If however there is a positive reply from Google, Mozilla shouldn't go forward supporting H.264.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 7:05 PM
On Wednesday, March 14, 2012 2:40:30 AM UTC+1, Nicholas Nethercote wrote:
> I'm pretty sure this "last serious push" has already been tried,
> unfortunately.  Various Mozilla people have been trying to get info
> from Google for months without success.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) mks....@gmail.com 3/13/12 10:42 PM
On Wednesday, March 14, 2012 4:55:30 AM UTC+1, Chris Hofmann wrote:
> Chances are they will opt for being just a pragmatic as we have chosen
> to be.

Mozilla should not make a pragmatic decision. If Mozilla decides to implement H.264 it should be only if there is no longer a chance to win the fight. And Google is the last chance, so yes, it matters what Google does.

> What's really needed from here forward is a heavy dose of website
> evangelism,
> and/or some incentives for websites to convert over, and some easy
> and guaranteed path they can deliver the kind of experience with either
> codec.

One thing should be clear to everybody: as soon as all major browsers (and even without Opera, though Opera will have no choice other than implementing H.264 as well) support one common format, WebM will be dead for the web. Nobody will implement 2 formats if one isn't necessary to reach any additional audience. Because having to deal with 2 formats is a pain.

Evangelism is very difficult for various reasons. Too many people don't know or care about patent issues, because they don't understand why it is an issue which affects them, and it's not a simple thing to explain in 3 sentences. For people to really understand the problems, they must invest 30 minutes, 60 minutes, 90 minutes (depending on how compressed the issues are presented), and you don't get people to invest this much amount of time on a topic which they don't believe affects them. Listen around what most people are saying? They are pissed at Mozilla because they blame Mozilla that they have to deal with multiple formats. That giving in may mean to sell the web to the patent mafia is too abstract for them, also because many of the consequences would happen in very hidden ways (e.g. they would pay although they don't know that what they pay for includes patent license fees). Many aspects of patents help the patent mafia ... which is no surprise if you think of it as what it actually is: a (legalized) crime.

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/14/12 8:38 AM
On Wed, Mar 14, 2012 at 12:20 AM, bawjaws <david....@gmail.com> wrote:
> I'd like to propose that, even if platform codecs are used to provide
> H.264 and AAC for the video tag, Firefox should still blacklist AAC
> for the audio tag and instead support MP3 and Vorbis.

If we support H.264 for video, we will have to support LC-AAC, for the
audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
<audio> if we had code & license to play AAC but only did so in
<video> when accompanied by an H.264 video track, people would see
this as Mozilla causing them grief for no good reason. We wouldn't
really win anything and we wouldn't make any friends. We'd just annoy
people.

> MP3 is patent encumbered, but depending on what you read, the decoding
> patents should expire sometime between the end of 2012 and 2020. I'd
> love it if Mozilla could help locate a definitive answer on what that
> end date might be, including actively seeking both legal and technical
> workarounds to the remaining patents (decode and encode) if it could
> bring that date forward.

It's probably dangerous to try to find out when, but it might be
worthwhile to establish a legal opinion on when. That opinion had
better be correct.

> Fluendo seems able to provide an MP3 decoder at no cost to the Linux
> end user, which Ubuntu for example optionally package, for use by any
> Gstreamer enabled apps and have done for several years now. I believe
> they pay for this right, but don't know the details.

Also worth noting that Windows XP comes with MP3 support, so MP3 is
very different from AAC and H.264 in that regard. (Well, at least
Windows Media Player bundled with XP plays MP3 on a fresh install.
That the codec is available as a library available to other apps like
other Windows Media stuff is conjecture on my part, but probably
pretty good conjecture.)

> Henri Sivonen in this discussion claims mp3 licensing is even worse
> than AAC. I thought I'd heard the opposite from an x264 developer.

If Mozilla made MP3 the one audio format all browsers support, Mozilla
would end up baiting various Web publishers, including game
developers, to use MP3. The reason why Vorbis is so popular for game
sounds for native code games is that the money demands for games.
Also, there are other use fees on MP3.

The main thing AAC has going for it in the MPEG land is that it has
modest (compared to other MPEG stuff) patent license fees for
implementations and no (known) usage fees.
http://www.vialicensing.com/licensing/aac-faq.aspx

If we baited e.g. games to use MP3 but we were wrong about patent
expiry or we were right but the patent holders didn't agree and it was
too expensive for small game developers to show that the patents have
expired, it would be really bad if Mozilla induced e.g. game
developers to suffer MP3 usage fees. See
http://www.scirra.com/blog/64/why-you-shouldnt-use-mp3-in-your-html5-games

If we supported AAC, there'd be a codec that works everywhere and has
no (known) usage fees, so at least we wouldn't bait Web publishers
into jeopardy.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Hofmann 3/14/12 8:39 AM
There has to be a carrot, and there has to be adoption on websites for
for WebM to survive and thrive. The picture Asa paints here is not a
good one for getting adoption on websites to take place,  and we have to
think beyond anything that google or adobe, or any other client software
provider may or may not do, since it really won't have influence on use
at this point.

It might be worth thinking about ways to pressure websites to adopt the
format and get users involved in that using much the same strategy that
we did for DNT.   We have just over 7.5% of firefox users expressing
there desire for not being tracked a year after shipping DNT and its
starting to have influence.   In the same way we could have a percent of
users expressing their desire for sites to sent royalty free video (SRFV:1).

It seems a shame to start using request headers to send a bunch of these
kind of "advocacy bits" to try and influence websites to do the right
thing, but this one certainly rises to the same level as Do Not Track.  
It might lead to  something that could help them site with accounting
and negotiation on their licensing.

> We are not going to win this one evangelizing a more expensive,
> slightly lower quality solution that requires a whole lot of effort
> with no obvious benefits. No matter how well it fits our values, it
> doesn't fit their values.

If we extend serving user preference into the conversation there might
be a chance to turn this around.  DNT also cost sites money in terms of
monetization  opportunities and needing to set up duplicate content
serving systems, but sites some sites want to do this because they can
see its the right thing for their users.

We can't drop the efforts to improve the format quality and tools.  That
gap has to be closed as well if WebM is going to have a chance.

-chofmann
"give the people tools to save the open web..."
>
> Had adobe included WebM in Flash and Flash Media Server; had Google
> dropped H.264 on Chrome and Android; maybe there'd be some
> opportunities to try to get people to move to WebM as a global web
> solution but I think that opportunity past us about 8 months ago with
> the massive rise in Android and iPad usage and with the further
> cementing of h.264 as the <video> element solution - both with more
> site use and with Adobe's launch of a streaming h.264 to the <video>
> tag server product.
>
> - A
> _______________________________________________


Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/14/12 9:18 AM

On 3/14/2012 7:09 AM, Gervase Markham wrote:
> On 13/03/12 16:44, Asa Dotzler wrote:
>> On 3/13/2012 4:33 AM, Gervase Markham wrote: > Given that doing this
>> requires us changing our current policy on
>>> supporting non-RF video formats[0],
>>
>>> [0] Stated several times publicly by Roc, e.g. here:
>>> http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html
>>
>> I have stupid question. How is ROC's position on non-RF formats "policy"
>> and not simply "the position of a module owner"?
>>
>> I looked all over mozilla (newsgroups, sites, blogs, etc) and couldn't
>> find any Mozilla Policy on this issue.
>>
>> If it is simply the position of some of the leaders in our community,
>> does that automatically make it a Mozilla Policy?
>>
>> I'm not asserting with 100% certaintly that it isn't Mozilla Policy, but
>> from what I've been able find in documentation, it's not.
>
> What is "Mozilla's opinion" or "Mozilla's policy" on a technical issue
> is or can be, as you note, a nebulous concept. But roc, as module owner
> for Layout and the person generally in charge of our media-playing code
> and teams, has been asserting that this is Mozilla's view for some time
> now and no-one higher up the organization has contradicted him. I think
> a "the most important and qualified person to give an opinion on an
> issue sets the policy, particularly if no-one contradicts him" policy
> fits what our current practice is.

And ROC has changed his view over time. If it was his decision in the
past, is it still his decision today? Here's a comment from him up-thread:

> I don't think it's working. With Google effectively abandoning us by not
> removing H.264 from Chrome*, and Adobe's support for WebM never
> materializing, I believe our stance is doing little more than hurting Web
> developers and Firefox users.

If, as you say, the module owner was reasonably asserting Mozilla's
view, then why is that not acceptable today as Mozilla's policy
(somewhat rhetorical).

The world has changed. The people leading this effort within Mozilla
have changed their views -- most of them towards using h.264. I realize
we all are struggling with this, but presumably it's the moduler
owners/leaders on video like ROC who get to make the final call and not
something that has to be escalated to some higher "policy" debate.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Chris Hofmann 3/14/12 9:18 AM
On 3/13/12 10:42 PM, mks....@gmail.com wrote:
> and it's [royalty free video as a key part of the future of an open web] not a simple thing to explain in 3 sentences.

the issues around privacy and tracking share some of the problems.  we
found a way for people to tune into the issues at many different levels,
including just 4 words, and one preference setting.

Do Not Track Me!

Getting a community of influencers going and engaged in spreading the
word can work if we put some effort into it.  Send Royalty Free Video
[SRFV:1]

-chofmann
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/14/12 9:24 AM
On 3/13/2012 10:42 PM, mks....@gmail.com wrote:
> On Wednesday, March 14, 2012 4:55:30 AM UTC+1, Chris Hofmann wrote:
>> Chances are they will opt for being just a pragmatic as we have
>> chosen to be.
>
> Mozilla should not make a pragmatic decision. If Mozilla decides to
> implement H.264 it should be only if there is no longer a chance to
> win the fight. And Google is the last chance, so yes, it matters what
> Google does.

I don't think that the people who have been tracking this closely
believe that Google can still turn the tides. Perhaps they could have a
year ago, but we're in a much different world and Google's decisions
around Chrome desktop alone certainly will not change the game in any
meaningful way. To much has happened with the rise of Android and the
ginormous growth of iPads. Too little has happened with Adobe on the
WebM front -- they're now happily streaming h.264 to the <video> tag
with their popular streaming servers. Too many sites have converted from
Flash to h.264 powered <video>.  Dropping h.264 from Chrome would be too
little too late.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/14/12 9:26 AM
On Wed, Mar 14, 2012 at 2:22 PM, Timothy Terriberry <tter...@webrtc.org>wrote:
Cool idea, but "Mozilla risks becoming dangerously complacent and complicit
to the cartel-controlled multimedia monopolies" makes me snort.

Rob
--
“You have heard that it was said, ‘Love your neighbor and hate your enemy.’
But I tell you, love your enemies and pray for those who persecute you,
that you may be children of your Father in heaven. ... If you love those
who love you, what reward will you get? Are not even the tax collectors
doing that? And if you greet only your own people, what are you doing more
than others?" [Matthew 5:43-47]
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joshua Cranmer 3/14/12 9:30 AM
On 3/14/2012 10:38 AM, Henri Sivonen wrote:
>> MP3 is patent encumbered, but depending on what you read, the decoding
>> patents should expire sometime between the end of 2012 and 2020. I'd
>> love it if Mozilla could help locate a definitive answer on what that
>> end date might be, including actively seeking both legal and technical
>> workarounds to the remaining patents (decode and encode) if it could
>> bring that date forward.
> It's probably dangerous to try to find out when, but it might be
> worthwhile to establish a legal opinion on when. That opinion had
> better be correct.

An attempt to guess by a non-lawyer: according to the patents listed in
the pool, the last US patent will expire sometime in 2017. However, any
patent filed more than a year after the publishing of the MP3 spec could
probably be ruled invalid (that's where the 2012 end comes from: 21
years after the spec was published). The last patent in the pool which
was filed in this timeslot would expire in late 2013.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 3/14/12 9:39 AM
On 14/03/12 16:18, Asa Dotzler wrote:
> And ROC has changed his view over time.

Sure, he has. I am not invoking past-roc to say "we shouldn't do this,
because roc has said so in the past", or invoking past-roc to say "roc
still thinks we shouldn't do this", I'm invoking past-roc to say "hey, I
do think this is a change of policy, even if the root message suggests
it's not".

> The world has changed. The people leading this effort within Mozilla
> have changed their views -- most of them towards using h.264. I realize
> we all are struggling with this, but presumably it's the moduler
> owners/leaders on video like ROC who get to make the final call and not
> something that has to be escalated to some higher "policy" debate.

Er, I think you are mistaking me for someone who disagrees with this ;-)

Gerv

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/14/12 9:39 AM
On Wed, Mar 14, 2012 at 4:18 PM, Asa Dotzler <a...@mozilla.org> wrote:

> And ROC has changed his view over time. If it was his decision in the
> past, is it still his decision today? Here's a comment from him up-thread:
>

Yes, my views on what our best strategy is have changed in light of new
data (including Google and Adobe not following through on their
announcements).

FWIW, I still stand by everything I wrote in my post at
http://robert.ocallahan.org/2010/01/video-freedom-and-mozilla_23.html.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/14/12 9:41 AM
On Tue, Mar 13, 2012 at 7:23 PM, Chris Pearce <cpe...@mozilla.com> wrote:

> Matthew Gregan will know OSX media frameworks better than I as he worked
> on the original Quicktime HTML5 backend we developed before we decided not
> to use platform codecs (bug 435298), but IIRC it took significant bashing
> to get QuickTime to accept data downloaded by our network stack. IIRC that
> was on OSX 10.5, and on at least one of 10.6 and 10.7 we'd have to use a
> different API to get playback support, possibly both, so we'd need 2 maybe
> 3 backends to cover OSX using platform codecs.
>

Someone pointed me to docs for an interface that would let us get at the
H.264 codec directly (bypassing Quicktime). Not sure about AAC.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 3/14/12 9:41 AM
I heard 2015, perhaps due to extensions:
"If you only look at the MP3 patents filed before December 1992 (one
year after the decoding spec was published), then the last decoding
patent expires in September of 2015."

http://www.osnews.com/story/24954/US_Patent_Expiration_for_MP3_MPEG-2_H_264/

But I'm just using Google; I have no special knowledge.

Gerv

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert Kaiser 3/14/12 9:51 AM
Chris Hofmann schrieb:
> In the end does it really matter what google will do, or not do?

No. In the end what matter is that apparently open video is dead, and
therefore the royalty-free web is dead, at least for those who want to
publish video on the web.
And we seem to be able to change that, so we're giving into it.

What that means in a broader scope is to be seen. One thing is sure, and
that's that everyone at Mozilla is sad that it apparently has to go this
way.

Robert Kaiser
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/14/12 9:54 AM
On Mar 14, 8:38 am, Henri Sivonen <hsivo...@iki.fi> wrote:
> On Wed, Mar 14, 2012 at 12:20 AM, bawjaws <david.scot...@gmail.com> wrote:
> > I'd like to propose that, even if platform codecs are used to provide
> > H.264 and AAC for the video tag, Firefox should still blacklist AAC
> > for the audio tag and instead support MP3 and Vorbis.
>
> If we support H.264 for video, we will have to support LC-AAC, for the
> audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
> <audio> if we had code & license to play AAC but only did so in
> <video> when accompanied by an H.264 video track, people would see
> this as Mozilla causing them grief for no good reason. We wouldn't
> really win anything and we wouldn't make any friends. We'd just annoy
> people.

Too right. Lack of AAC is a real issue for Pandora's HTML5 player on
Firefox, last I heard. MP3 doesn't cut it.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert Kaiser 3/14/12 9:57 AM
mks....@gmail.com schrieb:
> Nobody will implement 2 formats if one isn't necessary to reach any additional audience.

That's exactly why H264 has won this battle in the first place. Using
WebM doesn't reach any additional audience, as basically everyone
without in-browser H.264 has Flash with H.264 anyhow.
The other way round, those without in-browser WebM usually don't have
any WebM support at all.

Robert Kaiser
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/14/12 10:00 AM
On Tue, Mar 13, 2012 at 6:39 AM, Henri Sivonen <hsiv...@iki.fi> wrote:
Mozilla-shipped Firefox releases would come with a bundled Mozilla

> MPEG Components release, so users of Mozilla-shipped Firefox wouldn't
> have to go through any special steps to get H.264/AAC support. Parties
> who self-compile the Firefox codebase (either under the Firefox name
> or some ElementAnimal name) could use a copy of Mozilla MPEG
> Components binaries shipped by Mozilla Corporation for free. If they
> wanted to, they could compile Mozilla MPEG Components themselves and
> get patent licenses on their own. Even non-Gecko products would be
> welcome to use Mozilla MPEG Components as shipped by Mozilla
> Corporation for free, so if Opera wanted to use it on desktop, I think
> Mozilla should let them. If someone wanted to write a Gstreamer
> plug-in that wrapped Mozilla MPEG Components for use in other Linux
> apps, that would be cool too.
>

I don't think we can provide an H.264 codec that anyone can download and
use for free in any product. That would effectively short-circuit MPEG-LA
licensing and therefore I am quite convinced it would be against their
license agreement.

The rest of your post sounds reasonable.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tonyto...@gmail.com 3/14/12 10:17 AM
I have a few ideas for how using h.264 on the desktop could work:

1. ask google/apple/microsoft to pay the $5m or so fee the MPEG-LA requires and bundle it in the installer.
2. ask users on operating systems without h.264 support to pay a fee via paypal of whatever it costs say 10 cents and give them an option to instead watch say 3 minutes of commercials. If you have an "unlock h264 support" option in the firefox menu and have these 2 options that would be great for XP users. I'm sure 2-5 mins of commercials would cover the cost of the h.264 licence. You should allow users to backup the licence file so that if they format their pc they don't have to watch commercials again. The h.264 file would be downloaded from mozilla's servers after the commercials have finished, that way it doesn't need to be bundled.
3. put a link to microsoft's h.264 installer on the start page of mozilla for when users install firefox.
4. ask adobe to make a new version of flash that allows mozilla to connect to the h.264 capability without having to use flash itself, this would mean 98% of windows users (that have flash) could use h.264 for free and hopefully without the security concerns of flash if they can have a file in their installation directory that you could link in to.

p.s is adobe still planning to include support for vp8 in flash as they still haven't done so.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/14/12 10:24 AM
On Wed, Mar 14, 2012 at 7:00 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> I don't think we can provide an H.264 codec that anyone can download and use
> for free in any product. That would effectively short-circuit MPEG-LA
> licensing and therefore I am quite convinced it would be against their
> license agreement.

Fluedo has a codec-only product albeit not gratis. The whole purpose
of Fluendo Complete Codec Pack is to supply codec capabilities for
apps.

Adobe and Apple have codec plus other stuff products for gratis (Flash
Player and QuickTime for Windows) that allow third parties (Web sites
and Windows apps) to leverage them as MP4 players.

Of course, it's possible that codec-only gratis products are banned. Dunno.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tonyto...@gmail.com 3/14/12 10:54 AM
also MS could distribute h.264 via a windows update on XP, it would cost them hardly anything.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/14/12 11:41 AM
OK. Sorry. Happy to know I'm not thinking crazythink.

- A

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/14/12 12:18 PM
As I've said elsewhere in this thread, that would be too little too
late. Had this happened a year ago, along with the other Google and
Adobe promises, we might be in a different place today, but we're no
longer where we were a year ago and the forces of h.264 are considerably
stronger today than they were then.

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/14/12 1:02 PM
This thread is about "we need H264 since the web has chosen it and
since it's hardware accelerated on mobile devices" if i'm not
mistaken.

However YouTube provides WebM versions of its catalogue, and at least
3 SoC provide hardware acceleration for WebM.

Therefore could you precise your first argument :
1°) I know YouTube is not the only videos sharing site, but it has
huge market share especially here in France where even the french
DailyMotion is far behind. Do mobile user want to access other videos
sharing site ? i mean what is the videos sharing site market share
Mozilla aims to reach with H264 ? Do we speak of 5%, 30%, 50 % ?
2°) Is B2G supposed to target hardware that doesn't provide WebM
acceleration at the end of 2012 (considering present and future SoC
that provide such acceleration) or is it theorical ?

To sum up things : can we quantify the need of H264 on mobile
plateform ?

Thank you

For reference :
Here is a list of Tegra 2 & 3 smartphones http://www.nvidia.com/object/tegra-superphones.html
Here is a list of Omap 4 smartphones https://en.wikipedia.org/wiki/OMAP#OMAP_4

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) brian.b...@gmail.com 3/14/12 1:24 PM
On Monday, March 12, 2012 7:29:14 AM UTC-7, Joe Drew wrote:
> On 2012-03-12 9:00 AM, Doug Turner wrote:
> > This is great.  We really need this on Android where many mobile sites only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM),
> most _desktop_ sites only supported h.264 (mostly through Flash).
>
> I am very concerned that, by supporting system codecs, we're simply
> capitulating on Free codecs. If, as we believe, mobile is the future,
> and we support h.264 or other non-free codecs on mobile, then the future
> of codecs is non-free.
>
> joe

Except Webm isn't a free codec and even though it has not been challenged in court yet, its pretty clear from the claims of many who have looked deeply into it when that does occur WebM will be in violation of patents. Putting all the eggs in one basket to back something that is "free" but isn"t proved free could be a mistake. I am happy with H.264, all my stuff is compatible with it. The quality is better and technically is a better codec under the hood. I have zero interest in WebM for any reasons because in order for WebM to be as good as H.264 it is going to require issues with patents.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/14/12 1:29 PM
That kind of opinion has already be debated everywhere else. It's off
topic, thank you.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) brian.b...@gmail.com 3/14/12 1:30 PM
On Tuesday, March 13, 2012 9:36:16 AM UTC-7, antistress wrote:
> Also, on a legal side, to people who ask to bundle Firefox with open
> source H264 codec, please check if it's legally possible. I remember
> an article (OSnews ?) that explained that once you're a MPEG-LA
> licencee you have to use the official proprietay codec.
> But i may be wrong.
> Thanks

Didn't Microsoft even offer to pay for the license so firefox could use h.264 codec to end any debate on HTML5 standard.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) shte...@gmail.com 3/14/12 2:21 PM
On Monday, March 12, 2012 11:46:13 PM UTC-4, Asa Dotzler wrote:
> Our shipping or not shipping h.264 doesn't make the Web unimplementable
> in a free stack. The dominance of h.264 video on the Web -- something
> our 3 years of Open Video support has not changed or even slowed down,
> is what makes the Web unimplementable in a free stack.
>

How do you know that your firm stance didn't make things as good as they are? I.e. if not for that stance, things could be even worse. Now you are saying that giving up is OK (i.e. making things even worse). Slow (or even very slow) progress is not the same as rapid regression which will happen the moment you give up.

Regards,

Hillel.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert O'Callahan 3/14/12 2:41 PM
On Wed, Mar 14, 2012 at 5:00 PM, Robert O'Callahan <rob...@ocallahan.org>wrote:

> The rest of your post sounds reasonable.
>

Actually I need more time to think about system codecs vs licensing.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) jacob.b...@gmail.com 3/14/12 4:20 PM
I'm late in this discussion but wanted to throw this idea around:

How about saying that the real fight worth fighting here, the big picture, is the fight against the very concept of software patentability?

My concern about saying "we won't support patent-encumbered codecs" is that it might be implicitly conceding some amount of meaningfulness to software patents. Isn't that doing them too big a favor?

I'd argue that here, the pragmatic and idealistic approaches agree on supporting system codecs regardless of patent concerns --- I wonder though how many people would support the idealistic step of openly denouncing the absurdity of the concept of software patentability, without any exceptions or apologies (I do, but that's just one random guy's voice).

Benoit
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) jacob.b...@gmail.com 3/14/12 4:20 PM
I'm late in this discussion but wanted to throw this idea around:

How about saying that the real fight worth fighting here, the big picture, is the fight against the very concept of software patentability?

My concern about saying "we won't support patent-encumbered codecs" is that it might be implicitly conceding some amount of meaningfulness to software patents. Isn't that doing them too big a favor?

I'd argue that here, the pragmatic and idealistic approaches agree on supporting system codecs regardless of patent concerns --- I wonder though how many people would support the idealistic step of openly denouncing the absurdity of the concept of software patentability, without any exceptions or apologies (I do, but that's just one random guy's voice).

Benoit

On Wednesday, March 14, 2012 10:41:54 PM UTC+1, Robert O&#39;Callahan wrote:
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joshua Cranmer 3/14/12 5:09 PM
On 3/14/2012 6:20 PM, jacob.b...@gmail.com wrote:
> I'm late in this discussion but wanted to throw this idea around:
>
> How about saying that the real fight worth fighting here, the big picture, is the fight against the very concept of software patentability?
>
The European Union explicitly forbids "programs for computers" as being
unpatentable, yet looking at a list of, say, H.264 patents turns up a
large number of patents that apply within the EU. The Supreme Court of
the United States could not arrive at a patentability test in Bilski v.
Kappos despite a record number of amici curiae briefs which basically
begged them to resolve the issue. Given that the last prior SCOTUS case
involving patentability of software basically said "software per se
isn't patentable", it's not clear if software is actually patentable in
the US either, but that hasn't stopped the USPTO from granting software
patents. The current situation is as clear as mud, and no one wants to
risk losing a lawsuit to find it if software is truly patentable or not.

Mozilla has far less leverage over the question of software
patentability then it does over the question of which codecs to use on
the Open Web. As this thread demonstrates, it has practically none on
the latter case, so why would you expect that campaigning against
software patentability would be any more fruitful.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tch...@gmail.com 3/14/12 6:01 PM
Google hasn't given up on pushing webm, they are acquiring Motorola and allowed moto to litigate with its patent essential to h.264 to increase the price of and perceived risk of the codec to the point its un attractive. I say stick with open codecs.  
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tch...@gmail.com 3/14/12 6:06 PM
Extra codecs should be installable  with a short warning and people should be able to get codec from any source.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Nukeador 3/14/12 6:16 PM
El 14/03/12 03:05, mks....@gmail.com escribió:
> This last serious push should certainly be public. If Mozilla is considering supporting H.264, it would be a good opportunity to make a public call to Google (in an official blog for example), whether or not their commitment (and promise) is still valid.
>
> Mozilla considering to support H.264 is certainly going to be picked up by quite some tech media, and if this is combined with a public request to Google that they should finally say whether they are still committed to fully supporting WebM (and at the same time saying that Mozilla will continue their commitment, but only if Chrome drops support, so Mozilla isn't wasting their efforts), it might be quite hard for Google to ignore it again. (And if Google ignores a public request which likely attracts some media coverage, that would also be quite a clear answer, if unfortunately a clear no.)
>
> A private request can much easier be ignored than a (last, but loud) public request. I wouldn't leave this untried.
>
> And again, if Mozilla really needs to support H.264, there will certainly be some blaming. Like who killed WebM. Or is Mozilla really committed to the free web. Or why has Mozilla hesitated for so long (to some people for no apparent reason). With such a public and loud last call, many more people would understand Mozilla's decisions, and those who are going to ignite flamewars against Mozilla (and this certainly will happen; actually it happens all the time) will have very weak arguments at hand.
>
> So even if the low chances to convince Google don't make this seem to be worth the effort, the positive public image by demonstrating that really everything has been tried and by once more explaining why Mozilla acted this way (which also is yet another opportunity to explain the problem with patent encumbered software), may make it worth the effort.
I'm totally with you, we don't lose anything doing this.

Some of you are talking about the end of the battle, a surrender without
a last fight, I vote to show everyone that open webs is so important to
us that we will fight tooth and nail, even if at the end we lose and we
have to surrender.

Regards.

--
Rubén Martín [Nukeador]
Mozilla Reps Council Member
http://www.mozilla-hispano.org
http://twitter.com/mozilla_hispano
http://facebook.com/mozillahispano


Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) rast...@gmail.com 3/14/12 6:18 PM
On Wednesday, March 14, 2012 8:28:04 AM UTC+11, John Jensen wrote:
> Or, said another way, "the market has spoken". 80% of video content, as
> surveyed by a web video indexing company in December 2011, is served via
> H.264. This proportion has increased every time they've measured it:
>
> http://blog.mefeedia.com/html5-dec-2011


It seems to me that this MeFeedia blog post is widely misunderstood and consequently it's often misused in web video discussions. The MeFeedia blog post absolutely does not say that 80% of web video content is in H.264. The title of the graph is "% of H.264 Video Available for HTML5 Playback." This means that of all the H.264 videos surveyed by MeFeedia, 80% of them have an HTML5 playback option.

Consider also that the MeFeedia blog post says "WebM has little traction (less than 2% of out [sic] index)." MeFeedia's index isn't very big (only 50 million videos as they note in the blog post) and I don't think they index videos in proportion to video site usage or the number of videos on each site. It's more instructive to look at a comparison of which video sites are actually in use by web users. This is what comScore Video Metrix does:

http://www.comscore.com/Press_Events/Press_Releases/2011/12/YouTube_Accounts_for_At_Least_34_Percent_of_All_Videos

World wide, Google owned video sites accounted for 43.8% of all video views in October, 2011 (and in fact it's slightly higher than that as 97% of Vevo video views were via YouTube). That's 88,278,970,000 video views which have the potential to be served as WebM videos. Even if only a third on the videos which were part of those 88 billion views have WebM transcodes available, it's difficult to reconcile MeFeedia's 2% WebM count with Google's dominance in web video viewership. MeFeedia is simply not a strong indicator of practical WebM availability and visibility.

In any case, I think if Mozilla starts supporting H.264 playback in Firefox it will effectively kill off progress towards royalty-free video on the web. The issues around web video are poorly understood by users and developers alike as it is. If Firefox plays H.264 video there'll be even less incentive to understand those issues or to understand why a royalty-free web matters.

Google talks the talk when it comes to royalty-free video. It's a shame that when it comes to walking the walk Google is still taking baby-steps.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Pimm Hogeling 3/14/12 6:56 PM
Op donderdag 15 maart 2012 02:18:54 UTC+1 schreef rast...@gmail.com het volgende:
> It seems to me that this MeFeedia blog post is widely misunderstood and consequently it's often misused in web video discussions. The MeFeedia blog post absolutely does not say that 80% of web video content is in H.264. The title of the graph is "% of H.264 Video Available for HTML5 Playback." This means that of all the H.264 videos surveyed by MeFeedia, 80% of them have an HTML5 playback option.
>
> Consider also that the MeFeedia blog post says "WebM has little traction (less than 2% of out [sic] index)." MeFeedia's index isn't very big (only 50 million videos as they note in the blog post) and I don't think they index videos in proportion to video site usage or the number of videos on each site. It's more instructive to look at a comparison of which video sites are actually in use by web users. This is what comScore Video Metrix does:
>
> http://www.comscore.com/Press_Events/Press_Releases/2011/12/YouTube_Accounts_for_At_Least_34_Percent_of_All_Videos
>
> World wide, Google owned video sites accounted for 43.8% of all video views in October, 2011 (and in fact it's slightly higher than that as 97% of Vevo video views were via YouTube). That's 88,278,970,000 video views which have the potential to be served as WebM videos. Even if only a third on the videos which were part of those 88 billion views have WebM transcodes available, it's difficult to reconcile MeFeedia's 2% WebM count with Google's dominance in web video viewership. MeFeedia is simply not a strong indicator of practical WebM availability and visibility.
>
> In any case, I think if Mozilla starts supporting H.264 playback in Firefox it will effectively kill off progress towards royalty-free video on the web. The issues around web video are poorly understood by users and developers alike as it is. If Firefox plays H.264 video there'll be even less incentive to understand those issues or to understand why a royalty-free web matters.
>
> Google talks the talk when it comes to royalty-free video. It's a shame that when it comes to walking the walk Google is still taking baby-steps.

I would like to echo that sentence: "The issues around web video are poorly understood by users and developers alike as it is."
unk...@googlegroups.com 3/14/12 6:56 PM <This message has been deleted.>
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) rast...@gmail.com 3/14/12 9:33 PM
On Wednesday, March 14, 2012 5:31:28 AM UTC+11, Andreas Gal wrote:
> Very true. We should move the focus to providing an open replacement for the upcoming H.265 standard from the start so history won't repeat itself.

History will repeat itself. H.265 is probably a year away and Qualcomm, for example, is already doing H.265 demonstrations:

http://reviews.cnet.com/8301-13970_7-57387626-78/qualcomm-shows-horsepower-of-next-gen-h.265-video/

I doubt there's enough time to produce anything competitive with H.265, unless Google can magic up a VP9 in the next year. Once H.265 encoder and decoder hardware starts appearing I imagine there will be a push to use H.265 in things like WebRTC.

The only hopes I see are that the H.265 encoder is so slow it's impratical (although encoder hardware would fix that) or that H.265 can somehow be made royalty-free (seems unlikely) or that the licencing is so awful that no one wants to use it. However, I suspect that giving up on avoiding H.264 also means giving up on avoiding H.265 and Mozilla will need to prepare to support H.265 in a couple of years. Given that situation, how would royalty-free video usage on the web become widespread in, say, the next half-decade?

On the audio side, however, I think Opus has a chance of becoming the standard audio codec for the web. Hardware support isn't such a big deal for audio and even large audio sites like SoundCloud wouldn't find transcoding to Opus an impossible mountain to climb. Maybe Mozilla should focus on developing tools and advocating for Opus in order to get at least one royalty-free codec in widespread use on the web.

-Ray
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/14/12 11:11 PM
On Mar 14, 9:33 pm, rasta...@gmail.com wrote:
> Once H.265 encoder and decoder hardware starts appearing I imagine there will be a push to use H.265 in things like WebRTC.

I think H.265 is too asymmetric -- encoding takes too many cycles
compared to decoding.

WebRTC is the next battle and it should be unencumbered WebM.

> The only hopes I see are that the H.265 encoder is so slow it's impratical (although encoder hardware would fix that)

No, hardware can't fix the asymmetry without power and area blow-up. H.
265 looks like a bad fit for mobile two-way realtime. Experts should
correct me if I'm wrong.

/be

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/14/12 11:53 PM
I'm sorry, Hillel, I don't understand what you're saying. You think the
situation is good today?

- A

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/14/12 11:56 PM
On 3/14/2012 6:01 PM, tch...@gmail.com wrote:
> Google hasn't given up on pushing webm, they are acquiring Motorola and allowed moto to litigate with its patent essential to h.264 to increase the price of and perceived risk of the codec to the point its un attractive. I say stick with open codecs.
>
Source?

- A

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/15/12 12:02 AM
On Wed, Mar 14, 2012 at 11:41 PM, Robert O'Callahan
<rob...@ocallahan.org> wrote:
> On Wed, Mar 14, 2012 at 5:00 PM, Robert O'Callahan <rob...@ocallahan.org>
> wrote:
>>
>> The rest of your post sounds reasonable.
>
> Actually I need more time to think about system codecs vs licensing.

Yeah. The price of the patent licenses could buy a lot of labor to do
stuff that advances Mozilla's Mission.

Desktop Linux with Gstreamer is in a better shape than I thought when
I wrote that post. I still think that having a solution for XP for the
next two years is important and that it's important to have Mac,
Linux, XP and Windows Vista/7 support released in the same train. With
the latest patch on the Gstreamer support bug, it seems that Linux
will be ready to go in time.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/15/12 12:20 AM
On 3/15/2012 12:02 AM, Henri Sivonen wrote:
> I still think that having a solution for XP for the
> next two years is important

Why for only the next two years?

- A
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/15/12 12:53 AM
If Microsoft sticks to their plan, XP will stop receiving security
updates after two years.

It's of course possible that Firefox will still have lots of users on
XP when the XP EOL date comes and our H.264 solution for XP needs to
last longer. (I wonder what kind of botnet zombie apocalypse ensues if
XP still has lots of users when the security updates stop...)
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/15/12 1:19 AM
My bet is that we're looking at a 4 to 5 year horizon and not two years.
In two years, XP will still have at least twice as many Firefox users ad
Mac and Linux combined.

- A

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gijs Kruitbosch 3/15/12 2:01 AM
On 15/03/2012 07:11 AM, Brendan Eich wrote:
> <snip>
> WebRTC is the next battle and it should be unencumbered WebM.
> <snip>

Something I don't understand about this debate is how we're going to win that
one if there's no (mobile) hardware support for WebM. Conversely, if there is
hardware support for WebM, then isn't there less of a mobile-battery-life
incentive to get H264 support?

Or is this just a timing thing where we expect WebRTC to take 1-2 years until
it's finalized, and by then we hope (and will fight for?!) the hardware support
to have improved compared to the current situation?

~ Gijs
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Mario Moreira 3/15/12 2:44 AM
As "just a user" of Firefox on the desktop, I would like to say that one of the main reasons I haven't changed to Chrome, even after some issues with Firefox, is that Firefox represents the Free side of the web, and is not ruled by some corporation that someday my become greedy and may have other priorities than "what is best for the users".

If mozilla starts to think in "market share" instead of "what's better for the users", it's starting to behave like any corporation.

Even if Firefox is not as good as browser X from some corporation (unless it lags too much behind competition), you will have supporters.

Google seems to want to do the right thing, but money and PR talks louder, and they don't have the guts to do what should be done (and drop h.264 support).

If mozilla does the same thing, the message will be: ideologies doesn't matter even the "Free" groups will eventually give up, so lets just keep pressing.

So think about what's more important, it's market share, or to contribute to an unencumbered web?
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) rast...@gmail.com 3/15/12 2:50 AM
On Thursday, March 15, 2012 5:11:31 PM UTC+11, Brendan Eich wrote:
> No, hardware can't fix the asymmetry without power and area blow-up. H.
> 265 looks like a bad fit for mobile two-way realtime. Experts should
> correct me if I'm wrong.

Okay. Even if H.265 encoders don't arrive for mobile any time soon, H.265 decoders already work in software on mobile CPUs and the web is bigger than just mobile and real time video. In the future, why isn't H.265 the logical successor to H.264 for web video generally?

It seems likely that in not so very long at all there will be fast Intel CPUs and Nvidia GPUs with fast H.265 encoding built-in. As fast encoders become available and as Microsoft and Apple roll out H.265 decoders as part of their operating systems, why won't the Microsoft Store and iTunes start selling H.265 video? Why won't web video sites follow suit and move to H.265 as their future format?

I'm sure H.264 will be around for a long time for much the same reasons that MP3 has been around for a long time, and maybe that means H.265 can be avoided for a while longer. But I don't see that H.264 will be superseded by anything other than another royalty-bearing format, namely H.265. I see Hopes but I don't see any Strategies. Maybe some kind of royalty-free super video codec will arrive on the scene but that doesn't seem likely.

Perhaps there can't be any strategy because the interests that think royalty-free web video is a Really Good Thing just don't have enough influence in web video. And the interest that does have enough influence in web video (Google) doesn't seem to be interested in influencing web video enough.


-Ray
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Jeff Muizelaar 3/15/12 7:09 AM

On 2012-03-15, at 2:11 AM, Brendan Eich wrote:

> On Mar 14, 9:33 pm, rasta...@gmail.com wrote:
>> Once H.265 encoder and decoder hardware starts appearing I imagine there will be a push to use H.265 in things like WebRTC.
>
> I think H.265 is too asymmetric -- encoding takes too many cycles
> compared to decoding.

This would be surprising to me. H.265 gives encoders more flexibility so they have the choice to spend more cycles making better decisions to improve quality but if they want quality similar to current HW H.264 encoders I'd expect they can make similar tradeoffs to get similar performance. I don't know of anything in H.265 that makes it fundamentally harder to encode.

-Jeff

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Mario Moreira 3/15/12 2:44 AM
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) rast...@gmail.com 3/14/12 6:18 PM
On Wednesday, March 14, 2012 8:28:04 AM UTC+11, John Jensen wrote:
> Or, said another way, "the market has spoken". 80% of video content, as
> surveyed by a web video indexing company in December 2011, is served via
> H.264. This proportion has increased every time they've measured it:
>
> http://blog.mefeedia.com/html5-dec-2011


Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) rast...@gmail.com 3/14/12 9:33 PM
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tch...@gmail.com 3/14/12 6:06 PM
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tonyto...@gmail.com 3/14/12 10:17 AM
I have a few ideas for how using h.264 on the desktop could work:

1. ask google/apple/microsoft to pay the $5m or so fee the MPEG-LA requires and bundle it in the installer.
2. ask users on operating systems without h.264 support to pay a fee via paypal of whatever it costs say 10 cents and give them an option to instead watch say 3 minutes of commercials. If you have an "unlock h264 support" option in the firefox menu and have these 2 options that would be great for XP users. I'm sure 2-5 mins of commercials would cover the cost of the h.264 licence. You should allow users to backup the licence file so that if they format their pc they don't have to watch commercials again. The h.264 file would be downloaded from mozilla's servers after the commercials have finished, that way it doesn't need to be bundled.
3. put a link to microsoft's h.264 installer on the start page of mozilla for when users install firefox.
4. ask adobe to make a new version of flash that allows mozilla to connect to the h.264 capability without having to use flash itself, this would mean 98% of windows users (that have flash) could use h.264 for free and hopefully without the security concerns of flash if they can have a file in their installation directory that you could link in to.

p.s is adobe still planning to include support for vp8 in flash as they still haven't done so.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) david....@gmail.com 3/15/12 9:14 AM
On Wednesday, 14 March 2012 15:38:58 UTC, Henri Sivonen  wrote:
> If we support H.264 for video, we will have to support LC-AAC, for the
> audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
> <audio> if we had code & license to play AAC but only did so in
> <video> when accompanied by an H.264 video track, people would see
> this as Mozilla causing them grief for no good reason. We wouldn't
> really win anything and we wouldn't make any friends. We'd just annoy
> people.

The "good reason" would be that at some time within the next 8 months to 8 years (possibly sooner if someone does the legwork to create a patent free subset of MP3 or overturn some patents) they would have a totally royalty free audio format that is globally supported, not just by browsers but by websites as well as nearly every piece of personal audio technology on the planet and probably 90% of people's existing lossily-encoded music collections. Annoying people to that end seems, to me at least, to be part of Mozilla's mission.

> It's probably dangerous to try to find out when, but it might be
> worthwhile to establish a legal opinion on when. That opinion had
> better be correct.

What's the worst case scenario? Some internet music service or game being sued and forced to cough up the usage fees they've not paid? Or like P2P downloaders would they be hit for billions of dollars (though that's copyright not patents)? (I genuinely don't know, this is the kind of thing the Mozilla legal team could clear up though).

> If Mozilla made MP3 the one audio format all browsers support, Mozilla
> would end up baiting various Web publishers, including game
> developers, to use MP3. The reason why Vorbis is so popular for game
> sounds for native code games is that the money demands for games.
> Also, there are other use fees on MP3.

Note that the first post in this thread suggests B2G and Firefox on Android will soon support mp3 and further discussion suggests desktop Firefox will follow for consistency. Your own argument about dropping AAC annoying people applies here for not supporting MP3 when other browsers do. So this problem already exists, my suggestion only makes it worse by some unknown degree for an unknown period of time and then better afterwards (since e.g. open source browsers could have mp3 encoders built in to help with audio uploads). I don't know how that equation works out in the end, but I certainly think it's worth looking into.

> The main thing AAC has going for it in the MPEG land is that it has
> modest (compared to other MPEG stuff) patent license fees for
> implementations and no (known) usage fees.
> http://www.vialicensing.com/licensing/aac-faq.aspx

Ironic historical aside: it was due to Apple and Real Networks refusing to implement AAC that they dropped the web usage fees they originally intended to charge.

> If we supported AAC, there'd be a codec that works everywhere and has
> no (known) usage fees, so at least we wouldn't bait Web publishers
> into jeopardy.

This would be a stronger argument if H.264 didn't also carry usage fees. You'd simply need to place a notice about mp3 next to the notice warning developers about fees for using H.264 (which as far as I can tell no other browser has done for either codec). And again, since mp3 support seems to be happening anyway you'd need to do this regardless of if you withhold AAC support in audio tags. (I'll note that I'd much rather read Mozilla's take on exactly what circumstances require payment rather than take the patent owners word anyway.)

I guess these usage fees explain the Pandora requirement for AAC mentioned in another response (though citing a US-only service, where Internet Explorer, Mac OS X, iOS usage and software patents are all stronger than elsewhere is unfortunate).

If Mozilla is going to add mp3 support, I would hope that it would do the patent investigation anyway in order to minimize the impact of patents on the web. If this is done early enough, and has promising enough results then I hope the idea of not adding (or dropping) support for AAC and promoting royalty-free MP3 is at least evaluated by those with the legal and technical capabilities before it is dismissed.

regards,

dave

On Wednesday, 14 March 2012 15:38:58 UTC, Henri Sivonen  wrote:
> On Wed, Mar 14, 2012 at 12:20 AM, bawjaws <david....@gmail.com> wrote:
> > I'd like to propose that, even if platform codecs are used to provide
> > H.264 and AAC for the video tag, Firefox should still blacklist AAC
> > for the audio tag and instead support MP3 and Vorbis.
>
> If we support H.264 for video, we will have to support LC-AAC, for the
> audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
> <audio> if we had code & license to play AAC but only did so in
> <video> when accompanied by an H.264 video track, people would see
> this as Mozilla causing them grief for no good reason. We wouldn't
> really win anything and we wouldn't make any friends. We'd just annoy
> people.
>
> > MP3 is patent encumbered, but depending on what you read, the decoding
> > patents should expire sometime between the end of 2012 and 2020. I'd
> > love it if Mozilla could help locate a definitive answer on what that
> > end date might be, including actively seeking both legal and technical
> > workarounds to the remaining patents (decode and encode) if it could
> > bring that date forward.
>
> It's probably dangerous to try to find out when, but it might be
> worthwhile to establish a legal opinion on when. That opinion had
> better be correct.
>
> > Fluendo seems able to provide an MP3 decoder at no cost to the Linux
> > end user, which Ubuntu for example optionally package, for use by any
> > Gstreamer enabled apps and have done for several years now. I believe
> > they pay for this right, but don't know the details.
>
> Also worth noting that Windows XP comes with MP3 support, so MP3 is
> very different from AAC and H.264 in that regard. (Well, at least
> Windows Media Player bundled with XP plays MP3 on a fresh install.
> That the codec is available as a library available to other apps like
> other Windows Media stuff is conjecture on my part, but probably
> pretty good conjecture.)
>
> > Henri Sivonen in this discussion claims mp3 licensing is even worse
> > than AAC. I thought I'd heard the opposite from an x264 developer.
>
> If Mozilla made MP3 the one audio format all browsers support, Mozilla
> would end up baiting various Web publishers, including game
> developers, to use MP3. The reason why Vorbis is so popular for game
> sounds for native code games is that the money demands for games.
> Also, there are other use fees on MP3.
>
> The main thing AAC has going for it in the MPEG land is that it has
> modest (compared to other MPEG stuff) patent license fees for
> implementations and no (known) usage fees.
> http://www.vialicensing.com/licensing/aac-faq.aspx
>
> If we baited e.g. games to use MP3 but we were wrong about patent
> expiry or we were right but the patent holders didn't agree and it was
> too expensive for small game developers to show that the patents have
> expired, it would be really bad if Mozilla induced e.g. game
> developers to suffer MP3 usage fees. See
> http://www.scirra.com/blog/64/why-you-shouldnt-use-mp3-in-your-html5-games
>
> If we supported AAC, there'd be a codec that works everywhere and has
> no (known) usage fees, so at least we wouldn't bait Web publishers
> into jeopardy.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) tch...@gmail.com 3/15/12 9:17 AM
http://arstechnica.com/tech-policy/news/2012/02/microsoft-to-ec-motorola-hamstringing-xbox-pc-with-huge-patent-royalties.ars is one source that comes to mind off the top of my head
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/15/12 10:24 AM
On Mar 15, 2:44 am, mario.more...@gmail.com wrote:
> As "just a user" of Firefox on the desktop, I would like to say that one of the main reasons I haven't changed to Chrome, even after some issues with Firefox, is that Firefox represents the Free side of the web, and is not ruled by some corporation that someday my become greedy and may have other priorities than "what is best for the users".

That's simplistic. Read on, but note that Mozilla still has no agenda
other than putting users first and keeping the Web evolving and open
to the limits of human imperfection (which limits are on parade when
it comes to video).

> If mozilla starts to think in "market share" instead of "what's better for the users", it's starting to behave like any corporation.

You left out "only" after "think".

We've *never* ignored market share. Doing so easily leads down paths
to extinction, and then we have zero users, zero influence on
standards bodies, and zero ability to fulfill our mission.

This should be clear. Firefox took market share from IE in part by
reverse-engineering .innerHTMl, undetected document.all, and other
stuff, which eventually wound up in HTML5 (which we kicked off with
Opera and Apple via the WHAT WG in 2004). We have never ignored market
share or browse interop game theory. Not ever.

> Even if Firefox is not as good as browser X from some corporation (unless it lags too much behind competition), you will have supporters.

Too few, particularly on mobile which is growing 3x faster than
desktop, and we die. The trend is your friend, and Mozilla going to
zero users will be predicted by no growth on desktop and no share
(absolute share) to speak of on mobile. That is the threat right now.

B2G and (probably more growth-limited) Firefox on Android done right
(our native front end and slick off-main-thread-compositing/GPU-
optimized rendering under touch control) are our proposed solutions.
Neither will work without H.264 -- B2G simply won't get on phones
without it. Firefox on Android without Flash fallback will also fail.

If you think we should shrink to desktop-Linux-any-year-now
dimensions, from there to keep shrinking, then we must part company.
We're not going to sign up for purity at the price of irrelevance and
virtual extinction. On this Mitchell and I have been clear for almost
a decade.

> Google seems to want to do the right thing, but money and PR talks louder, and they don't have the guts to do what should be done (and drop h.264 support).

Really, Google seems to want something not in evidence by their
actions? Dream on. The Jan. 11, 2011 post by Mike Jazayeri promising
to drop H.264 support from <video> in Chrome not only lacked any
follow through, it is meaningless on desktop due to Chrome's excellent
Flash fallback. On Android, Google will not and has not promised to
drop H.264 h/w decoding from <video> in stock browser or Chrome on ICS
-- and they will not make such a suicidal promise.

Stop imputing your own wishes to Google. It's embarrassing and it
gives them more credit than you give Mozilla in your lede.

> If mozilla does the same thing, the message will be: ideologies doesn't matter even the "Free" groups will eventually give up, so lets just keep pressing.

Mozilla going down, dying a noble death for purity, while Google
laughs all the way to the bank, is terrible for our mission and users.
That you seem to think otherwise is alarming. Again we may have to
part company on this, but Firefox did not gain market share by purity
at the price of market loss leading to extinction.

> So think about what's more important, it's market share, or to contribute to an unencumbered web?

Obviously too little market share contributes nothing to an
unencumbered web. Your argument is simplistic on its face. Please
improve it or adapt to the fact that we've never put purity above all
else when the likely outcome is death.

There are points of principle, dogma really, over which we would die
rather than compromise. Closed source is one. Plugging in encumbered
decoders? We've been doing that via Flash and other plugins forever.
There's a degree of hypocrisy (not in your post but in general) in
ignoring this and pretending we're pure just because of how our
<video> element works.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/15/12 10:33 AM
On Mar 15, 7:09 am, Jeff Muizelaar <jmuizel...@mozilla.com> wrote:
> On 2012-03-15, at 2:11 AM, Brendan Eich wrote:
>
> > On Mar 14, 9:33 pm, rasta...@gmail.com wrote:
> >> Once H.265 encoder and decoder hardware starts appearing I imagine there will be a push to use H.265 in things like WebRTC.
>
> > I think H.265 is too asymmetric -- encoding takes too many cycles
> > compared to decoding.
>
> This would be surprising to me. H.265 gives encoders more flexibility so they have the choice to spend more cycles making better decisions to improve quality but if they want quality similar to current HW H.264 encoders I'd expect they can make similar tradeoffs to get similar performance. I don't know of anything in H.265 that makes it fundamentally harder to encode.

Sure, in this light H.265 degrades to H.264 but then what's the point?
Anyway, the next battle is not H.anything, it's WebRTC and the
opportunity to mandate an unencumbered default.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/15/12 10:33 AM
On Mar 15, 2:50 am, rasta...@gmail.com wrote:
> On Thursday, March 15, 2012 5:11:31 PM UTC+11, Brendan Eich wrote:
> > No, hardware can't fix the asymmetry without power and area blow-up. H.
> > 265 looks like a bad fit for mobile two-way realtime. Experts should
> > correct me if I'm wrong.
>
> Okay. Even if H.265 encoders don't arrive for mobile any time soon, H.265 decoders already work in software on mobile CPUs and the web is bigger than just mobile and real time video. In the future, why isn't H.265 the logical successor to H.264 for web video generally?

Possibly it is, and that's not the next battle that I see as winnable.
WebRTC has its own requirements and stack, and as a standard it can
stipulate an unencumbered format as mandatory default. The analogous
battle was lost with HTML5 video, but WebRTC has a real shot of
mandating an unencumbered default, from what I can see.

Timing is everything. Having VP8 decoding in hardware in 1-2 years
will help, even if the video content on the web is H.264 and we must
also use h/w H.264 decoding to compete on mobile. Two decoders (both
iDCT based, I will refrain from speculating on patent issues) means
some amount of design if not chip area sharing.

Anyway the h/w VP8 support is coming but it's too late to tilt HTML5
video away from H.264. That's the last battle. WebRTC is the next one.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Joshua Cranmer 3/15/12 10:47 AM
On 3/15/2012 11:14 AM, david....@gmail.com wrote:
> On Wednesday, 14 March 2012 15:38:58 UTC, Henri Sivonen  wrote:
>> If we support H.264 for video, we will have to support LC-AAC, for the
>> audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
>> <audio>  if we had code&  license to play AAC but only did so in
>> <video>  when accompanied by an H.264 video track, people would see
>> this as Mozilla causing them grief for no good reason. We wouldn't
>> really win anything and we wouldn't make any friends. We'd just annoy
>> people.
> The "good reason" would be that at some time within the next 8 months to 8 years (possibly sooner if someone does the legwork to create a patent free subset of MP3 or overturn some patents) they would have a totally royalty free audio format that is globally supported, not just by browsers but by websites as well as nearly every piece of personal audio technology on the planet and probably 90% of people's existing lossily-encoded music collections. Annoying people to that end seems, to me at least, to be part of Mozilla's mission.
MP3 is certainly not patent-free for another 18 months (September 2013
with an interpretation which may not match up with law), probably not
for 3.5 years; anything before 8 years runs a high risk of a lawsuit.
>> It's probably dangerous to try to find out when, but it might be
>> worthwhile to establish a legal opinion on when. That opinion had
>> better be correct.
> What's the worst case scenario? Some internet music service or game being sued and forced to cough up the usage fees they've not paid? Or like P2P downloaders would they be hit for billions of dollars (though that's copyright not patents)? (I genuinely don't know, this is the kind of thing the Mozilla legal team could clear up though).
The usage fees not paid, times 3 for willful infringement of a patent,
interest charges, and all legal fees. Plus forced removal of the
product. Legal risk of patent infringement is much higher than copyright
infringement.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Asa Dotzler 3/15/12 2:52 PM
I don't see anything beyond speculation there on motives and I can't
imagine that it's, as you frame it, to make 264 less appealing. This is
far more likely motorola trying to get more out of licensing to benefit
moto than to benefit open codecs.

- A

Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Ralph Giles 3/15/12 6:03 PM
On 12-03-13 2:54 AM, Henri Sivonen wrote:
> On Tue, Mar 13, 2012 at 9:48 AM, Mike Hommey<m...@glandium.org>  wrote:
>> As for Linux, I'm not sure to what extent h264 decoding is available to
>> them by default. I know Debian has it, now.
>
> By default? Really? That sounds *very* un-Debian.

Indeed. I'd say having libavcodec in main is a direct violation of the
recently posted patent policy[1]. When I've asked developers about this
I've gotten some mixture of willful ignorance and civil disobedience as
a justification.

In any case, I don't think we can provide support to our users this way.

> The  "bad", "ugly" and "ffmpeg" Gstreamer plug-in sets are marked as
> not supported by Canonical. Canonical is listed as an MPEG-LA H.264
> pool licensee. I don't know how the legal side of this works. The
> "check your laws" legal warning is now gone from the install wizards.

Interesting. I tried it through the Gnome popup totem displays, which
prompted me to install gstreamer-ffmpeg, but required I affirm I had
obtained a patent license or did not need one.

Up to Ubuntu 10.10, the documentation tells you what to install to play
back non-free codecs but warns about legal restrictions.[2]

Since the 10.10 release, Ubuntu's documentation only refers to DVD
decryption being legally restricted, and doesn't specifically warn about
the codecs. However, I can't find any affirmation that the ubuntu
packages include a patent license.

Fedora doesn't include mp4 decoders at all. The GNOME search box says
it's unable to find a plugin, and the 'More information' button refers
the user to a wiki page about how non-free codecs aren't included, and
suggests searching third-party documentation for more information.[3]
Unlike Ubuntu, Fedora doesn't give instructions on what to install.

I wouldn't say the situation with Linux system codecs is good at all.

  -r


[1] http://www.debian.org/legal/patent
[2] https://help.ubuntu.com/10.10/musicvideophotos/C/video-playback.html
[3] https://fedoraproject.org/wiki/PackageKit_Items_Not_Found#Missing_Codec

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Henri Sivonen 3/16/12 1:08 AM
On Thu, Mar 15, 2012 at 6:14 PM,  <david....@gmail.com> wrote:
>> It's probably dangerous to try to find out when, but it might be
>> worthwhile to establish a legal opinion on when. That opinion had
>> better be correct.
>
> What's the worst case scenario?

I think I'll stick to "IANAL" here instead of posting my speculation.

>> If Mozilla made MP3 the one audio format all browsers support, Mozilla
>> would end up baiting various Web publishers, including game
>> developers, to use MP3. The reason why Vorbis is so popular for game
>> sounds for native code games is that the money demands for games.
>> Also, there are other use fees on MP3.
>
> Note that the first post in this thread suggests B2G and Firefox on Android will soon support mp3 and further discussion suggests desktop Firefox will follow for consistency. Your own argument about dropping AAC annoying people applies here for not supporting MP3 when other browsers do.

In my reply to the idea of blocking AAC from <audio>, I wasn't
advocating not supporting MP3. In fact, since MP3 is supported as a
platform codec more broadly than H.264 and AAC, it probably makes
sense to support it using platform codecs everywhere.

>> The main thing AAC has going for it in the MPEG land is that it has
>> modest (compared to other MPEG stuff) patent license fees for
>> implementations and no (known) usage fees.
>> http://www.vialicensing.com/licensing/aac-faq.aspx
>
> Ironic historical aside: it was due to Apple and Real Networks refusing to implement AAC that they dropped the web usage fees they originally intended to charge.

Kudos to Apple and Real. Too bad Apple wasn't/hasn't been as
successful on the video side of MPEG.

>> If we supported AAC, there'd be a codec that works everywhere and has
>> no (known) usage fees, so at least we wouldn't bait Web publishers
>> into jeopardy.
>
> This would be a stronger argument if H.264 didn't also carry usage fees.

AAC alone without H.264 is useful for music, talk shows and game
sounds. MPEG-LA waived usage fees for H.264 when the content is gratis
to end users.

Unfortunately, there doesn't appear to be any video codec that has a
real chance of getting supported by default by all major browsers and
has no usage fees for the case where the end user pays directly for
the service.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Henri Sivonen 3/16/12 3:11 AM
On Fri, Mar 16, 2012 at 3:03 AM, Ralph Giles <gi...@mozilla.com> wrote:
> On 12-03-13 2:54 AM, Henri Sivonen wrote:
>>
>> On Tue, Mar 13, 2012 at 9:48 AM, Mike Hommey<m...@glandium.org>  wrote:
>>>
>>> As for Linux, I'm not sure to what extent h264 decoding is available to
>>> them by default. I know Debian has it, now.
>>
>>
>> By default? Really? That sounds *very* un-Debian.
>
> Indeed. I'd say having libavcodec in main is a direct violation of the
> recently posted patent policy[1]. When I've asked developers about this I've
> gotten some mixture of willful ignorance and civil disobedience as a
> justification.
>
> In any case, I don't think we can provide support to our users this way.

Yeah, linking with system libavcodec seems like a bad idea.

>> The  "bad", "ugly" and "ffmpeg" Gstreamer plug-in sets are marked as
>> not supported by Canonical. Canonical is listed as an MPEG-LA H.264
>> pool licensee. I don't know how the legal side of this works. The
>> "check your laws" legal warning is now gone from the install wizards.
>
> Interesting. I tried it through the Gnome popup totem displays, which
> prompted me to install gstreamer-ffmpeg, but required I affirm I had
> obtained a patent license or did not need one.

Sorry. The assertion about the Gnome wizard I made above was bogus; I
should have taken the process further in my testing. The "check your
laws" dialog is still there.

When checking the third-party box in the Ubuntu system installer,
there's no such message, though. No idea how that works legally.

> Fedora doesn't include mp4 decoders at all. The GNOME search box says it's
> unable to find a plugin, and the 'More information' button refers the user
> to a wiki page about how non-free codecs aren't included, and suggests
> searching third-party documentation for more information.[3] Unlike Ubuntu,
> Fedora doesn't give instructions on what to install.

The Fluendo pack is available for Fedora:
http://www.fluendo.com/shop/product/complete-set-of-playback-plugins/

I wonder at what price Fluendo would be able to offer a "Fluendo Web
pack" with MP3, AAC and H.264. MP3 they are able to offer for $0. The
full pack includes extra stuff: Windows Media, MPEG-4 Visual and
MPEG-2. MPEG-2 is particularly expensive. VC-1 is more expensive than
H.264 and MPEG-4 Visual isn't exactly inexpensive.

I also wonder if Fluendo is going to produce codec packs for GStreamer
on Windows to go along with the new announcement to bring GStreamer to
Windows, Mac and Android.
http://www.collabora.com/press/2012/02/collabora-and-fluendo-invest-in-gstreamer-open-source-multimedia-framework.html

Might GStreamer for Windows be a part of a solution for XP? Opera
already uses GStreamer on Windows and Mac albeit with RF codecs only
for the time being. Maybe Opera is in the market to buy H.264 and AAC
codecs for GStreamer on Windows. Considering how the MPEG-LA pricing
is structured, it makes sense for as many entities as possible to pool
their codec blob acquisition through one legal entity that has
productized codecs. At least it theory, it should be cheaper if both
Mozilla and Opera bought codec blobs from Fluendo than if Mozilla and
Opera both took licenses to produce their own codec blobs.

(Aside: Fluendo is now advertising they have hardware acceleration on
Intel, AMD and Nvidia hardware.)

> I wouldn't say the situation with Linux system codecs is good at all.

OK.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) david....@gmail.com 3/15/12 9:14 AM
On Wednesday, 14 March 2012 15:38:58 UTC, Henri Sivonen  wrote:
> If we support H.264 for video, we will have to support LC-AAC, for the
> audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
> <audio> if we had code & license to play AAC but only did so in
> <video> when accompanied by an H.264 video track, people would see
> this as Mozilla causing them grief for no good reason. We wouldn't
> really win anything and we wouldn't make any friends. We'd just annoy
> people.

The "good reason" would be that at some time within the next 8 months to 8 years (possibly sooner if someone does the legwork to create a patent free subset of MP3 or overturn some patents) they would have a totally royalty free audio format that is globally supported, not just by browsers but by websites as well as nearly every piece of personal audio technology on the planet and probably 90% of people's existing lossily-encoded music collections. Annoying people to that end seems, to me at least, to be part of Mozilla's mission.

> It's probably dangerous to try to find out when, but it might be
> worthwhile to establish a legal opinion on when. That opinion had
> better be correct.

What's the worst case scenario? Some internet music service or game being sued and forced to cough up the usage fees they've not paid? Or like P2P downloaders would they be hit for billions of dollars (though that's copyright not patents)? (I genuinely don't know, this is the kind of thing the Mozilla legal team could clear up though).

> If Mozilla made MP3 the one audio format all browsers support, Mozilla
> would end up baiting various Web publishers, including game
> developers, to use MP3. The reason why Vorbis is so popular for game
> sounds for native code games is that the money demands for games.
> Also, there are other use fees on MP3.

Note that the first post in this thread suggests B2G and Firefox on Android will soon support mp3 and further discussion suggests desktop Firefox will follow for consistency. Your own argument about dropping AAC annoying people applies here for not supporting MP3 when other browsers do. So this problem already exists, my suggestion only makes it worse by some unknown degree for an unknown period of time and then better afterwards (since e.g. open source browsers could have mp3 encoders built in to help with audio uploads). I don't know how that equation works out in the end, but I certainly think it's worth looking into.

> The main thing AAC has going for it in the MPEG land is that it has
> modest (compared to other MPEG stuff) patent license fees for
> implementations and no (known) usage fees.
> http://www.vialicensing.com/licensing/aac-faq.aspx

Ironic historical aside: it was due to Apple and Real Networks refusing to implement AAC that they dropped the web usage fees they originally intended to charge.

> If we supported AAC, there'd be a codec that works everywhere and has
> no (known) usage fees, so at least we wouldn't bait Web publishers
> into jeopardy.

This would be a stronger argument if H.264 didn't also carry usage fees. You'd simply need to place a notice about mp3 next to the notice warning developers about fees for using H.264 (which as far as I can tell no other browser has done for either codec). And again, since mp3 support seems to be happening anyway you'd need to do this regardless of if you withhold AAC support in audio tags. (I'll note that I'd much rather read Mozilla's take on exactly what circumstances require payment rather than take the patent owners word anyway.)

I guess these usage fees explain the Pandora requirement for AAC mentioned in another response (though citing a US-only service, where Internet Explorer, Mac OS X, iOS usage and software patents are all stronger than elsewhere is unfortunate).

If Mozilla is going to add mp3 support, I would hope that it would do the patent investigation anyway in order to minimize the impact of patents on the web. If this is done early enough, and has promising enough results then I hope the idea of not adding (or dropping) support for AAC and promoting royalty-free MP3 is at least evaluated by those with the legal and technical capabilities before it is dismissed.

regards,

dave

On Wednesday, 14 March 2012 15:38:58 UTC, Henri Sivonen  wrote:
> On Wed, Mar 14, 2012 at 12:20 AM, bawjaws <david....@gmail.com> wrote:
> > I'd like to propose that, even if platform codecs are used to provide
> > H.264 and AAC for the video tag, Firefox should still blacklist AAC
> > for the audio tag and instead support MP3 and Vorbis.
>
> If we support H.264 for video, we will have to support LC-AAC, for the
> audio tracks. Since Chrome, IE9 and Safari support both AAC and MP3 in
> <audio> if we had code & license to play AAC but only did so in
> <video> when accompanied by an H.264 video track, people would see
> this as Mozilla causing them grief for no good reason. We wouldn't
> really win anything and we wouldn't make any friends. We'd just annoy
> people.
>
> > MP3 is patent encumbered, but depending on what you read, the decoding
> > patents should expire sometime between the end of 2012 and 2020. I'd
> > love it if Mozilla could help locate a definitive answer on what that
> > end date might be, including actively seeking both legal and technical
> > workarounds to the remaining patents (decode and encode) if it could
> > bring that date forward.
>
> It's probably dangerous to try to find out when, but it might be
> worthwhile to establish a legal opinion on when. That opinion had
> better be correct.
>
> > Fluendo seems able to provide an MP3 decoder at no cost to the Linux
> > end user, which Ubuntu for example optionally package, for use by any
> > Gstreamer enabled apps and have done for several years now. I believe
> > they pay for this right, but don't know the details.
>
> Also worth noting that Windows XP comes with MP3 support, so MP3 is
> very different from AAC and H.264 in that regard. (Well, at least
> Windows Media Player bundled with XP plays MP3 on a fresh install.
> That the codec is available as a library available to other apps like
> other Windows Media stuff is conjecture on my part, but probably
> pretty good conjecture.)
>
> > Henri Sivonen in this discussion claims mp3 licensing is even worse
> > than AAC. I thought I'd heard the opposite from an x264 developer.
>
> If Mozilla made MP3 the one audio format all browsers support, Mozilla
> would end up baiting various Web publishers, including game
> developers, to use MP3. The reason why Vorbis is so popular for game
> sounds for native code games is that the money demands for games.
> Also, there are other use fees on MP3.
>
> The main thing AAC has going for it in the MPEG land is that it has
> modest (compared to other MPEG stuff) patent license fees for
> implementations and no (known) usage fees.
> http://www.vialicensing.com/licensing/aac-faq.aspx
>
> If we baited e.g. games to use MP3 but we were wrong about patent
> expiry or we were right but the patent holders didn't agree and it was
> too expensive for small game developers to show that the patents have
> expired, it would be really bad if Mozilla induced e.g. game
> developers to suffer MP3 usage fees. See
> http://www.scirra.com/blog/64/why-you-shouldnt-use-mp3-in-your-html5-games
>
> If we supported AAC, there'd be a codec that works everywhere and has
> no (known) usage fees, so at least we wouldn't bait Web publishers
> into jeopardy.
>
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Ralph Giles 3/16/12 11:15 AM
On 12-03-16 3:11 AM, Henri Sivonen wrote:

> Sorry. The assertion about the Gnome wizard I made above was bogus; I
> should have taken the process further in my testing. The "check your
> laws" dialog is still there.

Ok, we're seeing the same thing then.

> When checking the third-party box in the Ubuntu system installer,
> there's no such message, though. No idea how that works legally.

Clear as mud!

> The Fluendo pack is available for Fedora:
> http://www.fluendo.com/shop/product/complete-set-of-playback-plugins/

Yes. I use those on my work desktop, which is Fedora. They generally
work well.

> I wonder at what price Fluendo would be able to offer a "Fluendo Web
> pack" with MP3, AAC and H.264. MP3 they are able to offer for $0.
 > [...]
> Considering how the MPEG-LA pricing
> is structured, it makes sense for as many entities as possible to pool
> their codec blob acquisition through one legal entity that has
> productized codecs.

Interesting idea. When Fluendo first started distributing their no-cost
mp3 decoder, there was a $50k annual royalty cap. My understanding is
they paid the full cap so they could distribute the decoder as widely as
possible. This was intended to support the gstreamer-using community and
to advertise their services. I note that their decoder is not open
source, despite many of the people involved being open source developers.

That cap was orders of magnitude lower than the cap for h.264 (now set
at $6.5M through 2015). We'd cross it if we bought a copy for every
Firefox user, though. The h.264 cap starts at about 55M units.

Some time in the last few years, the cap disappeared from
http://mp3-licensing.com/royalty/ I'm not sure when; their robots.txt
explicitly blocks archive.org. I don't see a cap for AAC at
vialicensing.com either.

We've discussed using gstreamer several times since we first added media
support, not least because wrapping it would have let us expose
filter-graph APIs much earlier. Writing a media framework is hard. But
we were wary of the maintenance burden for the same reasons, and
gstreamer didn't work well cross-platform. If we went this direction I
expect we'd need to invest in its development, probably by hiring Collabora.

  -r
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Ralph Giles 3/16/12 11:58 AM
On 12-03-16 11:15 AM, Ralph Giles wrote:

> I note that their decoder is not open
> source, despite many of the people involved being open source developers.

It is, in fact, open source, under a BSD copyright license. I've just
been pointed at
https://core.fluendo.com/gstreamer/svn/trunk/gst-fluendo-mp3/

Apologies,
  -r

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) pd 3/17/12 1:21 AM
I run XP. I have done for a very happy long time. In that same period of time I've successfully tried and used any number of media players that have zero direct relationship to Windows Media Player because, frankly, WMP is ... not good.

Most of these media players are just front ends of varying quality to libav/ffmpeg. If several software developers (Media Player Classic HC, The KMPlayer, GOM Player) can deliver H.264 and other codec support through ffmpeg or x264, on Windows XP, there really should be no reason why Mozilla cannot do the same through Firefox.

In previous iterations of this debate, bugs providing DirectShow support and GStreamer support have been retarded mostly by politics. As far back as October 2008 a Firefox version was compiled with DirectShow support built in. You can *still* get it here:

http://pearce.org.nz/firefox-directshow-video-2008_10_24.win32.zip

and test it here:

http://pearce.org.nz/player/

In addition a completely new developer, Alessandro Decina, has implemented a <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=422540">GStreamer backend for HTML5 video element</a> as his very first bug fix!

Others have mentioned that they see H.264 as simply a reality Firefox needs to reluctantly engage with. However there is also the momentum argument. Attracting developers capable of tackling the task Alessandro has with their very first bug is surely a big indicator of momentum for the Firefox project, is it not? Firefox formerly had so much momentum behind it but that has dropped off. Supporting system-installed codecs could prove almost a resource-neutral means of getting some Firefox momentum again.

On Monday, March 12, 2012 6:28:04 PM UTC+11, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Henri Sivonen 3/19/12 2:54 AM
On Fri, Mar 16, 2012 at 8:15 PM, Ralph Giles <gi...@mozilla.com> wrote:
>> Considering how the MPEG-LA pricing
>> is structured, it makes sense for as many entities as possible to pool
>> their codec blob acquisition through one legal entity that has
>> productized codecs.
>
> Interesting idea. When Fluendo first started distributing their no-cost mp3
> decoder, there was a $50k annual royalty cap. My understanding is they paid
> the full cap so they could distribute the decoder as widely as possible.
> This was intended to support the gstreamer-using community and to advertise
> their services.

> I don't see a cap for AAC at vialicensing.com either.

See http://www.vialicensing.com/licensing/aac-fees.aspx

If I'm parsing that right, the "Standard Rates" don't apply to "PC
software". "Consumer PC Software" has annual caps for decoder-only and
for both encode and decode. "PC Enabling Software" has annual fees
(i.e. nothing lower than the cap price is even an option) for
decoder-only and for both encode and decode.

I'd guess if Firefox included a decoder compiled in, it would be
"Consumer PC Software". I would guess a GStreamer plug-in for use on
desktop computers would be "PC Enabling Software", though I don't see
the definitions of the terms anywhere.

I'm guessing that Fluendo is paying $300000 per year to ship AAC
decoders and funding that at least in part from the end user price of
the Complete Codec Pack. (Their product description doesn't talk about
AAC encode.)

I think interesting questions are:
 * If Fluendo were to max the payments under their H.264 license and
offer an H.264/AAC/MP4 decode GStreamer plug-in gratis to end users
(thereby losing some income from the Complete Codec Pack from users
who only care about H.264/AAC/MP4 and would forgo buying the Complete
Codec Pack if a gratis H.264/AAC/MP4 pack was available), how much new
income they'd need?
 * Would it be possible for organizations who'd obviously benefit from
being able to auto-install such a GStreamer plug-in for their end
users (e.g. Mozilla, Opera, Canonical, Red Hat, SuSE) to pay Fluendo
for the privilege to automate the installation of such a plug-in so
that the payments add up to as much as Fluendo would need per the
previous question?

That sort of thing would seem like a better deal for every
organization involved than each organization solving the problem
alone. Since MPEG-LA allows Microsoft and Apple to offer their
licensed codecs for use by 3rd-party apps without extra fees paid by
the developers of the 3rd-party apps, I'd expect the "ND" part of
"RAND" to require the MPEG-LA to permit Fluendo to do the same.

> gstreamer didn't work well cross-platform.

What was the problem? It seems to work well enough for Opera.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Robert O'Callahan 3/19/12 3:03 AM
On Mon, Mar 19, 2012 at 10:54 PM, Henri Sivonen <hsiv...@iki.fi> wrote:

> That sort of thing would seem like a better deal for every
> organization involved than each organization solving the problem
> alone. Since MPEG-LA allows Microsoft and Apple to offer their
> licensed codecs for use by 3rd-party apps without extra fees paid by
> the developers of the 3rd-party apps, I'd expect the "ND" part of
> "RAND" to require the MPEG-LA to permit Fluendo to do the same.
>

I don't think you can assume that Microsoft and Apple's licenses are on the
same terms as the regular application licenses.

Rob
--
“You have heard that it was said, ‘Love your neighbor and hate your enemy.’
But I tell you, love your enemies and pray for those who persecute you,
that you may be children of your Father in heaven. ... If you love those
who love you, what reward will you get? Are not even the tax collectors
doing that? And if you greet only your own people, what are you doing more
than others?" [Matthew 5:43-47]
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Henri Sivonen 3/19/12 3:38 AM
On Mon, Mar 19, 2012 at 12:03 PM, Robert O'Callahan
<rob...@ocallahan.org> wrote:
> On Mon, Mar 19, 2012 at 10:54 PM, Henri Sivonen <hsiv...@iki.fi> wrote:
>>
>> That sort of thing would seem like a better deal for every
>> organization involved than each organization solving the problem
>> alone. Since MPEG-LA allows Microsoft and Apple to offer their
>> licensed codecs for use by 3rd-party apps without extra fees paid by
>> the developers of the 3rd-party apps, I'd expect the "ND" part of
>> "RAND" to require the MPEG-LA to permit Fluendo to do the same.
>
> I don't think you can assume that Microsoft and Apple's licenses are on the
> same terms as the regular application licenses.

Surely under RAND, anyone should be able to get whatever terms they
are getting for the price they are getting (though as licensees, some
money then flows back from the pool to MS and Apple). Or is the
MPEG-LA H.264 pool not really RAND?

Also: Apple offered QuickTime, including QuickTime for Windows, which
can't qualify for some special "operating system bundle" license type,
for use by 3rd parties, without extra fees even before Apple became a
licensor in the H.264 pool.
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Henri Sivonen 3/19/12 3:40 AM
On Mon, Mar 19, 2012 at 12:38 PM, Henri Sivonen <hsiv...@iki.fi> wrote:
> (though as licensees, some
> money then flows back from the pool to MS and Apple).

...as *licensors*...
Re: Supporting patent-encumbered codecs (was hardware-accelerated audio/video decoding in Gecko (bug 714408)) Henri Sivonen 3/19/12 7:10 AM
On Mon, Mar 19, 2012 at 12:03 PM, Robert O'Callahan
<rob...@ocallahan.org> wrote:
> I don't think you can assume that Microsoft and Apple's licenses are on the
> same terms as the regular application licenses.

Slide 8 of http://www.mpeg-la.com/main/programs/avc/Documents/avcweb.pdf
treats bundling with a PC OS is a different license category, but for
the time spans priced so far, the price has been the same as for PC
non-OS bundle licenses.

(Note that for some MPEG-LA pools, the PC OS license is more
expensive. E.g. for VC-1, the PC OS license is more expensive than a
PC non-OS license.)
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) brian...@brandorr.com 3/19/12 11:22 AM
On Monday, March 12, 2012 3:28:04 AM UTC-4, Andreas Gal wrote:
> I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).
>
> Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.
>
> The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).
>
> Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).
>
> I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.
>
> Let me know if you have any questions.
>
> Andreas

Some thoughts.

1) Open web video is an uphill battle, with Mozilla being the standard bearer for the cause.
2) Currently web developers can get around the lack of H.264 in Firefox by using something like http://mediaelementjs.com/ to hook into Flash for those platforms that don't natively support H.264. (Or multi-encode, if they are a larger shop).
3) Politically, if Mozilla were to start supporting H.264, I don't think W3C, and other standards bodies, would have any choice but to ratify H.264 as the "Web Standard" for video.

This battle is about getting the W3C standards body to bless open video codecs for HTML5. Which would then put the H.264-only browsers in "out of compliance" status.

Google has a dilemma, in the browser/codec wars. They need more users to start using Chrome before switching off H.264, as doing so now would just prevent them from gaining market share. IE: If they had a larger share of the market, then switching off H.264, would be powerful indeed, but not so much with them struggling to hold onto less than 20% of the desktop market. (They are currently seeing small browser share declines.) Google is a comercial entity. Mozilla, is a non-profit that can make decisions based on their mission and not strictly on profit numbers.

Long story short, if there is anything Mozilla can do to avoid blessing H.264 in any form, I think it is a worthy goal, and will only help in efforts to get WebM and Ogg Theora blessed by W3C.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Brendan Eich 3/20/12 7:45 AM
On Mar 19, 11:22 am, brian.gu...@brandorr.com wrote:
> 2) Currently web developers can get around the lack of H.264 in Firefox by using something likehttp://mediaelementjs.com/to hook into Flash for those platforms that don't natively support H.264.

No-go on mobile -- no Flash there. Mobile matters most, to quote from
my blog post:

http://brendaneich.com/2012/03/video-mobile-and-the-open-web/

Everything you wrote assumes stable-ish (for Mozilla's sustainability)
desktop browser share and ignores mobile share. Beware noisy data
showing Chrome in even small decline -- but also don't assume Chrome
takes only from IE share (allowing Google to drop H.264 once they
"pass IE"). Mainly, do not ignore mobile.

/be
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Hubert Figuière 3/20/12 1:53 PM
On 19/03/12 11:22 AM, brian...@brandorr.com wrote:
> 1) Open web video is an uphill battle, with Mozilla being the standard bearer for the cause.
> 2) Currently web developers can get around the lack of H.264 in Firefox by using something like http://mediaelementjs.com/ to hook into Flash for those platforms that don't natively support H.264. (Or multi-encode, if they are a larger shop).
> 3) Politically, if Mozilla were to start supporting H.264, I don't think W3C, and other standards bodies, would have any choice but to ratify H.264 as the "Web Standard" for video.

I can't seem to be able to understand people advocating to use Flash as
a substitute for H264 in HTML5 <video> in the name of the open web.[0]
Definitely not. Flash is even more at that opposite side of the spectrum
than H264. After all, H264 is just patent encumbered [1], while Flash is
a completely proprietary and closed standard, not without security
flaws, that still require a lot of patent encumbered technologies to
implement, and is still not on all platform.[2]

One benefit I see about supporting H264 in Firefox is that it would
contribute to push Flash towards extinction as one of the last hardline
usage of it is to deliver H264 video.

That does not mean the W3C should ratify the use of H264 or any other
non-royalty-free-patented technology in the HTML5 standard. But this is
a different battle.

Nor is this a reason to stop the efforts on WebM.

Hub

[0] Those who know me will know that I have been a hipster Flash hater.
I hated Flash before even the iPhone existed. So much that i only
started watching videos on Youtube when they started to show content in
webm.

[1] this is not a minor issue, and it has been discussed at length.

[2] Case in point: Gnash still not a replacement of the Adobe Flash
player. And Adobe just announced end of 2011 that they were dropping
Flash on Linux (not the first time they do it - Flash 8 never existed on
Linux) and on mobile (not that it has ever been good)
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) antistress 3/20/12 4:34 PM
As far as i'm concerned, i see a huge difference : with Flash plugin
everybody can start developping a free web browser because it's Adobe
who pays the bill.
The obvious drawback is that it's Adobe who choose which OS is able to
run Flash.
It's not perfect but it seems better that HTML5/H264 in countries
which deals with patents.

But it's not the case anymore since Adobe is leaving the mobile
market...

On 20 mar, 21:53, Hubert Figuière <h...@mozilla.com> wrote:
> On 19/03/12 11:22 AM, brian.gu...@brandorr.com wrote:
>
> > 1) Open web video is an uphill battle, with Mozilla being the standard bearer for the cause.
> > 2) Currently web developers can get around the lack of H.264 in Firefox by using something likehttp://mediaelementjs.com/to hook into Flash for those platforms that don't natively support H.264. (Or multi-encode, if they are a larger shop).
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) rob...@gmail.com 3/25/12 1:54 AM
I took some time thinking about it and the repercussions to this technical move. I wrote the details here:

http://robux4.blogspot.fr/2012/03/resistance.html
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) rob...@gmail.com 3/25/12 1:54 AM
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) daoli...@gmail.com 7/4/12 9:55 AM
On Monday, 12 March 2012 14:28:04 UTC+7, Andreas Gal  wrote:
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) daoli...@gmail.com 7/4/12 9:57 AM
On Monday, 12 March 2012 21:29:14 UTC+7, Joe Drew  wrote:
> On 2012-03-12 9:00 AM, Doug Turner wrote:
> > This is great.  We really need this on Android where many mobile sites only use h.264 video.
>
> I don't see why this matters. At the time we shipped Theora (and WebM),
> most _desktop_ sites only supported h.264 (mostly through Flash).
>
> I am very concerned that, by supporting system codecs, we're simply
> capitulating on Free codecs. If, as we believe, mobile is the future,
> and we support h.264 or other non-free codecs on mobile, then the future
> of codecs is non-free.
>
> joe

Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Anthony jones 7/4/12 1:37 PM
On 05/07/12 04:57, daoli...@gmail.com wrote:
> On Monday, 12 March 2012 21:29:14 UTC+7, Joe Drew  wrote:
>> I am very concerned that, by supporting system codecs, we're simply
>> capitulating on Free codecs. If, as we believe, mobile is the future,
>> and we support h.264 or other non-free codecs on mobile, then the future
>> of codecs is non-free.
>>
>> joe

Supporting h264 is an admission of defeat.

Firefox has earned peoples trust by looking after their interests
without forcing them to think about the hazards of the Internet.
Supporting h264 is not in the user's best long term interest but it
becomes a trade off between the user's interests and their perceived
interests. If you don't look after their perceived interest then they'll
find someone who will.

If h264 on mobile is fait accompli then we need to come up with a good
plan B. Something along the lines of allowing them to make informed
decisions.

Anthony
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Robert Kaiser 7/5/12 5:26 AM
Anthony jones schrieb:
> Supporting h264 is an admission of defeat.

We already had exhaustive discussions on this. And yes, we are sad we
need to do this right now, but there's no way around it if we want to be
strong on mobile and give people the experience they expect (also, HTML
<video> with H.264 is still better in terms of the Open Web than Flash
video with H.264). Please read this blog post:
http://hacks.mozilla.org/2012/03/video-mobile-and-the-open-web/

Also, this codec fight is not over. We admittedly lost the first round,
but a new round is coming fast with real-time communication on the web
(WebRTC) and we hope we can make a free-codec-only policy prevail there,
AFAIK. Also, there will be a next generation of video codecs after VP8
and H.264, and Mozilla is actively helping efforts to create a free
codec that will be able to compete with H.265 and be available soon
enough that H.265 will not have the huge head-start H.264 had over WebM.

Robert Kaiser


P.S.: Note that this is all from my POV and what I think I know, I
cannot officially speak for Mozilla. The linked blog post from Brendan
is an official Mozilla statement, though.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) David Rajchenbach-Teller 7/5/12 8:19 AM
I am probably missing something, but perhaps there is a way to use VLC?

As far as I understand, VLC is as open as it gets, it has h.264 decoding
on desktop platforms, hardware-accelerated at least under Windows and
Linux, and there is already a VLC plug-in for Firefox. If more
integration is needed, I am sure that we can work something out with the
VLC team.

Cheers,
 David

On 3/13/12 3:34 AM, Asa Dotzler wrote:
> On 3/12/2012 7:28 PM, Chris Pearce wrote:
>> On 13/03/2012 3:18 p.m., Asa Dotzler wrote:
>>> We should license and ship these decoders. If there's a major cost
>>> difference ( don't know the economics very well) then I'd be fine with
>>> just shipping them where they're missing (more code burden) and
>>> relying on system stuff where it already exists.
>>
>> If the cost isn't too great, licensing and shipping decoders would be my
>> preferred solution, since it's by far the quickest path to market, has
>> the smallest maintenance overhead, and easiest way to ensure consistent
>> cross platform behaviour.
>>
>>
>> Chris P.
>
> I think it's only single digit millions per year. Not outrageous.
>
> - A
>
> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform


--
David Rajchenbach-Teller, PhD
 Performance Team, Mozilla



Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 7/5/12 10:50 AM
On 12-07-05 8:19 AM, David Rajchenbach-Teller wrote:

> I am probably missing something, but perhaps there is a way to use VLC?

We could use vlc's wrapper code, or gstreamer's, or write our own. The
issue is deciding on a strategy and how to change our licensing policy
if we want to ship LGPL code or proprietary decoders for platforms which
don't include their own. That and the small matter of programming to
actually make it work.

 -r


Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Jet Villegas 7/5/12 10:55 AM
These guys already have Gecko-to-gstreamer hooks...
http://wiki.songbirdnest.com/Developer

They used to have Gecko-to-VLC hooks but switched to gstreamer for various (non-technical?) reasons.

-- Jet
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Mook 7/5/12 10:37 PM
On 7/5/2012 10:55 AM, Jet Villegas wrote:
> These guys already have Gecko-to-gstreamer hooks...
> http://wiki.songbirdnest.com/Developer
>
> They used to have Gecko-to-VLC hooks but switched to gstreamer for various (non-technical?) reasons.
>
> -- Jet

Yep, Songbird uses gstreamer on all three platforms.  The old VLC media
core was using the VLC NPAPI plugin (!) and was GPL-only; that licensing
was part of the reason for the switch.  It really was more of a
proof-of-concept.  The fact that the new thing was more designed to be a
media framework and didn't run into the horribleness that is Gecko's
NPAPI support didn't hurt either, of course.

The XPCOM gstreamer bits are GPL (actually, dual-licensed, but I am
unaware of a non-GPL source license for third parties).  Given Mozilla's
historical tendencies, the difference in architecture, and how much
people scream when threading is involved, though, I suspect you'd want
to write your own - but you really want whoever's going to actually
integrate it to look at it and decide.

Feel free to ask around; #songbird (on irc.mozilla.org) doesn't bite,
though there might be snark ;)

--
Mook
(No idea why this is suddenly coming up again...)
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Mike Hommey 7/5/12 10:54 PM
On Thu, Jul 05, 2012 at 10:37:39PM -0700, Mook wrote:
> On 7/5/2012 10:55 AM, Jet Villegas wrote:
> >These guys already have Gecko-to-gstreamer hooks...
> >http://wiki.songbirdnest.com/Developer
> >
> >They used to have Gecko-to-VLC hooks but switched to gstreamer for various (non-technical?) reasons.
> >
> >-- Jet
>
> Yep, Songbird uses gstreamer on all three platforms.  The old VLC
> media core was using the VLC NPAPI plugin (!) and was GPL-only; that
> licensing was part of the reason for the switch.  It really was more
> of a proof-of-concept.  The fact that the new thing was more
> designed to be a media framework and didn't run into the
> horribleness that is Gecko's NPAPI support didn't hurt either, of
> course.
>
> The XPCOM gstreamer bits are GPL (actually, dual-licensed, but I am
> unaware of a non-GPL source license for third parties).  Given
> Mozilla's historical tendencies, the difference in architecture, and
> how much people scream when threading is involved, though, I suspect
> you'd want to write your own - but you really want whoever's going
> to actually integrate it to look at it and decide.

There's already gstreamer support in the tree.
(yeah, bug 422540 landed a while ago)

Mike
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Philip Chee 7/6/12 6:24 AM
I believe that the gstreamer backend has been checked in to
mozilla-central recently, although it's not built by default.

Phil

--
Philip Chee <phi...@aleytys.pc.my>, <phili...@gmail.com>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Gervase Markham 7/10/12 3:33 AM
On 05/07/12 18:50, Ralph Giles wrote:
> We could use vlc's wrapper code, or gstreamer's, or write our own. The
> issue is deciding on a strategy and how to change our licensing policy
> if we want to ship LGPL code or proprietary decoders for platforms which
> don't include their own. That and the small matter of programming to
> actually make it work.

Our licensing policy recently changed to allow us to ship
dynamically-linked LGPLed code. So that, at least, is no longer a problem.

Gerv
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Ralph Giles 7/10/12 4:33 PM
On 12-07-10 3:33 AM, Gervase Markham wrote:

> Our licensing policy recently changed to allow us to ship
> dynamically-linked LGPLed code. So that, at least, is no longer a problem.

I wasn't aware of that! Thanks.

 -r
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) alexande...@gmail.com 7/10/12 9:50 PM
Mozilla decides to implement H.264, because there is a MPEG LA insider pushing for changes in Mozilla's internal structure, i would firstly look into the developer or said person that is pushing for this, and kick them out of the organization.

shabby, when they do implement, I'm going to uninstall Firefox  
Re: hardware-accelerated audio/video decoding in Gecko (bug 714408) Nicolas Silva 7/10/12 10:11 PM
There has been a long thread on this subject about 3 months ago in this mailing list.

It looks like you haven't read it, in which case you should. There are some solid arguments there, and even though I am personally against patent encumbered formats being supported in Gecko, I somehow understand that the realities of the market and what most users and webdevs want/do is not always compatible with Mozilla's mission.

If we want to keep pushing open standards we need users. Right now this is a big deal for users.

----- Original Message -----
From: "alexander alisani" <alexande...@gmail.com>
To: dev-pl...@lists.mozilla.org
Cc: "dev-platform" <dev-pl...@lists.mozilla.org>
Sent: Wednesday, July 11, 2012 12:50:16 AM
Subject: Re: hardware-accelerated audio/video decoding in Gecko (bug 714408)

More topics »