DirectX/Quicktime backends for Video

11 views
Skip to first unread message

DigDug

unread,
Jun 21, 2009, 5:02:15 PM6/21/09
to
So there doesn't seem to be any consensus in Bug 435339 about whether
or not these are wanted or would even be accepted. Can someone with
enough clout to make a decision make one and close the bug if its
WONTFX?

My 2 cents, the open web is different than the Open Linux platform.
The OpenWeb is open because its simple. No strict checking of
someone's HTML ability. No real penalty for writing bad javascript.
Most of the image formats that people use are widely supported, so
there's not a lot to worry about there. Limiting users to what is an
obscure codec (compared to what they're used to at least), purely
because its "open" in the sense that its unencumbered by patents or
whatnot seems contrary to that spirit.

Letting the platform decide what's supported and what isn't (and
providing a better Plugin Finder service to help people when there's
no supported codec) would seem like a no brainer to me. But someone
who can put aside any platform predispositions, better educated about
these things, and who can make an honest decision based on what they
think is best for the web should make decision.

Joshua Cranmer

unread,
Jun 21, 2009, 6:10:38 PM6/21/09
to
DigDug wrote:
> Limiting users to what is an
> obscure codec (compared to what they're used to at least), purely
> because its "open" in the sense that its unencumbered by patents or
> whatnot seems contrary to that spirit.

It might be contrary, but patents present an issue that transcends
debate about openness. Mozilla cannot knowingly infringe on patents, and
it will not license patents if a royalty is required (I think the
software licenses with Mozilla might make licensing legally impossible,
but I am not a lawyer and I don't know this portion of law).

So long as the codec in question is patent-protected in such a manner,
it remains out of the question. If you object to the matter, it might be
better to complain to the patent holder or possibly even your local
government representative (in case he or she would support a law which
would retroactively invalidate software patents).

And, FWIW, if Youtube or Hulu provided multimedia in the Theora format
for <video>, I think a lot of the "obscure" objection would vanish. Also
note that 99% of the Internet-using public has no concept of video
codecs, so complaining that it's obscure compared to what the public is
used to is rather moot. In terms of what servers support and use for
<video> and <audio>, I'm willing to bet that the Ogg Vorbis and Theora
are by far the most popular codecs.

Ben Bucksch

unread,
Jun 21, 2009, 8:32:50 PM6/21/09
to
As for the open web: I agree that allowing arbitrary codecs shouldn't be
the goal, because it doesn't help the open web.

However, I think that a <video> tag that plays *no* usable codec, or
only a codec that nobody has encoded video in, does not help the open
web either. I really like the idea of Theora, but unfortunately, it is
neither an official standard nor is it efficient: files are relatively
large (IIRC even in comparison to MPEG2, even more so MPEG4).
Consequently, I have only 1 or 2 Theora videos. Not many Windows users
have DirectShow codecs for Theora, which means they can't save and play
saved Theora videos with their normal video player. Theora is not in
wide use, and has little chance of getting there (no matter what we do)
due to its bad efficiency.

MPEG2/4 are unfortunately patent encumbered, but are a standard and very
widely used, in DVDs, DVB (almost all TV in Europe) etc.. I have a few
TB of MPEG videos myself, mostly from DVD and TV.

Hardly any website wanting to show video to their users will consider
Theora to be a tempting alternative. It is always a cost/effect
discussion within companies, and Theora is likely to be the object of
ridicule of those people who don't like open source very much, which
doesn't make it easier for our open web friends to push for <video>
support within their web company. The video provider would have to keep
all videos in another format. If <video> would allow MPEG2 and MPEG4,
they could stream their existing video files, which would make support
for <video> a *lot* easier to implement, just some HTML+JS changes
versus changing their content management, encoders and adding tons of
storage.

With MPEG support, I could create a XULrunner application which can play
video as background, preview or a video player. I am writing just such
an TV player app in XULRunner right now and need MPEG2/4 support.
Reencoding my n TB of video files is out of question.

As for the patent issue: We're not implementing MPEG, so we're not in
violation of the MPEG patents. We're merely using generic OS or library
functions. It's the same situation as with any other DirectShow app: Any
application can use the generic DirectShow APIs, without violating
patents. The user may get the codec from anybody (legally), and any
DirectShow application can then legally play the videos without being in
violation of the patents.


So:

* Please enable MPEG2 and 4, and not the other proprietary codecs.
This will not make the video patent/codec situation worse: an open
desktop system needs to deal with MPEG anyways due to DVB and DVD.
* MPEG2/4 support will help <video> tremendously, because it reduces
the effort to use it from very high (change the whole video
library) to small (change one HTML page).
* I don't think patents are an issue, if you don't ship MPEG codes,
but merely use generic OS APIs.


Ben Bucksch

unread,
Jun 21, 2009, 8:39:32 PM6/21/09
to
As said in my last post, the biggest problem with Theora is its
inefficiency (relatively large files/bandwidth). As a result, it's not
in very wide use, and probably won't get in wide use even if it's the
only codec we support.

However, I just read about Dirac, a codec invented and implemented by
the BBC, specifically avoiding patents. It's Wavelet-based, uses motion
compression and other algos, and is announced to be about as efficient
as MPEG4, maybe even with better image quality. (I just cite Wikipedia,
I haven't tried myself yet.) It's suitable for resolutions from 176x144
to 1920x1080.

That makes it a much better candidate for campaigning and support by us
than Theora.
Oh, and the BBC implementation is licenced under MPL 1.1 / GPL 2 / LGPL 2.1.

Yay?

Ben

Boris Zbarsky

unread,
Jun 21, 2009, 9:10:27 PM6/21/09
to
Ben Bucksch wrote:
> However, I just read about Dirac, a codec invented and implemented by
> the BBC, specifically avoiding patents. It's Wavelet-based, uses motion
> compression and other algos, and is announced to be about as efficient
> as MPEG4, maybe even with better image quality. (I just cite Wikipedia,
> I haven't tried myself yet.) It's suitable for resolutions from 176x144
> to 1920x1080.
>
> That makes it a much better candidate for campaigning and support by us
> than Theora.

Except that in fact its patent risk is higher than Theora's and last I
checked the spec for it wasn't quite stable yet... Has that changed?

-Boris

Chris Double

unread,
Jun 21, 2009, 9:52:58 PM6/21/09
to
On Jun 22, 12:32 pm, Ben Bucksch <ben.bucksch.n...@beonex.com> wrote:
> I really like the idea of Theora, but unfortunately, it is
> neither an official standard nor is it efficient: files are relatively
> large (IIRC even in comparison to MPEG2, even more so MPEG4).

This is not true:

http://hacks.mozilla.org/2009/06/open-video-codecs-and-quality/
http://people.xiph.org/~maikmerten/youtube/

These compare Theora encodes to H.264 encodes of the quality that
YouTube is using. The file size and quality comparisons do not match
what you say.

Chris.
--
http://www.bluishcoder.co.nz

Chris Double

unread,
Jun 21, 2009, 9:57:21 PM6/21/09
to
On Jun 22, 12:39 pm, Ben Bucksch <ben.bucksch.n...@beonex.com> wrote:
> As said in my last post, the biggest problem with Theora is its
> inefficiency (relatively large files/bandwidth).

Read my reply, this is false. Tests show size and bandwidth use is
similar.

>
> That makes it a much better candidate for campaigning and support by us
> than Theora.

You've only just read about it and now you're ready to proclaim it a
better candidate. Do some more digging and you'll see it's not quite
ready for the types of video used on the web yet. Do some more reading
and you'll find we're actually working towards getting Dirac support
in the base libraries we use to implement video so when it is ready we
can easily support it (I'm not saying we will, but it's certainly on
the radar).

Chris.
--
http://www.bluishcoder.co.nz

Ben Bucksch

unread,
Jun 21, 2009, 10:26:08 PM6/21/09
to
On 22.06.2009 03:57, Chris Double wrote:
> Tests show size and bandwidth use is similar.
>

You can't compare a fast, crappy H.264 encoder and the best Theora
encoder of your choice.

If Theora (or any free codec) was as good as H.264 (with proper
encoding), I'd be happy, though.

> Do some more digging and you'll see [Dirac is] not quite ready for the

> types of video used on the web yet.

Care to say why or at least point to an article?

Also, "used on the web" (today) is not that relevant for the future:
YouTube is not interesting for me, because it's too small and too short,
and YouTube clips are in fact getting higher res and longer.

> we're actually working towards getting Dirac support

Some pointer to that would aid the discussion, likewise.

Ben Bucksch

unread,
Jun 21, 2009, 10:27:25 PM6/21/09
to
On 22.06.2009 03:52, Chris Double wrote:
> On Jun 22, 12:32 pm, Ben Bucksch<ben.bucksch.n...@beonex.com> wrote:
>
>> I really like the idea of Theora, but unfortunately, it is
>> neither an official standard nor is it efficient: files are relatively
>> large (IIRC even in comparison to MPEG2, even more so MPEG4).
>>
> This is not true
>

Even if so, all the other arguments stand: I got 2 TB of MPEG videos,
and I'm not going to reencode them. That's why I need MPEG support - if
nothing else, then for XULrunner apps. This is .platform, after all.

Robert O'Callahan

unread,
Jun 21, 2009, 10:30:06 PM6/21/09
to
Here's what I just wrote in the bug.

Even if a volunteer produces a patch, I would not want to ship it in
Firefox in the near future, for the following reasons:
-- We want to focus our energy on promoting open unencumbered codecs at
this time.
-- Only a very small fraction of Windows users have a DirectShow codec
for the most important encumbered codec, H.264.
-- DirectShow is underspecified and codecs are of highly variable
quality. Many codecs probably will not work with Web sites that use all
the rich APIs of <video>, and those bugs will be filed against us. We
may not even be able to fix them. (Note that the problem is bad enough
that in Windows 7, Microsoft isn't even going to allow unknown third
parties to install DirectShow codecs.) We could avoid some of this
problem by white-listing codecs, but then a lot of the people who want
DirectShow support wouldn't be satisfied anyway.
-- Many DirectShow codecs are actually malware.
-- DirectShow codecs are quite likely to have security holes. As those
holes are uncovered, we will have to track the issues and almost
certainly the only possible response will be to blacklist insecure
codecs (since we can't fix them ourselves). If we blacklist enough
codecs, DirectShow support becomes worthless. And each rushed-out
Firefox update creates a ton of work, of course.
-- Each new video backend creates additional maintenance headaches as we
evolve our internal video code.

So even if we didn't care about promoting unencumbered codecs and
someone gave us a working patch, shipping DirectShow support in Firefox
is of limited value and creates tons of maintenance work for us.

I think the above reasons explain why so far, no browser or browser
plugin vendor is using codec implementations they don't control.

Rob

Ben Bucksch

unread,
Jun 21, 2009, 10:35:54 PM6/21/09
to
On 22.06.2009 03:10, Boris Zbarsky wrote:
> its patent risk is higher than Theora's

Well, I'd trust BBC's judgment on that unless I have reason to believe
they erred.

> last I checked the spec for it wasn't quite stable yet... Has that
> changed?

Apparently yes.
"The specification was finalised on 21 January 2008 ...
Version 1.0.0 of the reference implementation ... was released on 17
September 2008 ...
A second implementation called Schr�dinger was funded by the BBC and
aims to provide high-performance, portable version ... written in ANSI C
... 22 February 2008, Schr�dinger 1.0.0 was released."
<http://en.wikipedia.org/wiki/Dirac_%28codec%29>

Robert O'Callahan

unread,
Jun 21, 2009, 10:47:35 PM6/21/09
to
On 22/6/09 12:32 PM, Ben Bucksch wrote:
> I really like the idea of Theora, but unfortunately, it is
> neither an official standard nor is it efficient: files are relatively
> large (IIRC even in comparison to MPEG2, even more so MPEG4).

Like Chris said, you're blindly repeating myths, or at least outdated
information.

> Consequently, I have only 1 or 2 Theora videos. Not many Windows users
> have DirectShow codecs for Theora, which means they can't save and play
> saved Theora videos with their normal video player.

That is not an important use case. Currently, you can't even save video
from most sites without the help of Firefox extensions. At least users
will be able to save Theora videos and play them back in Firefox.

> Theora is not in
> wide use, and has little chance of getting there (no matter what we do)
> due to its bad efficiency.

"Bad efficiency" is a myth. We'll see how we go.

> MPEG2/4 are unfortunately patent encumbered, but are a standard and very
> widely used, in DVDs, DVB (almost all TV in Europe) etc.. I have a few
> TB of MPEG videos myself, mostly from DVD and TV.

Fine, but that's not Web video. There is basically no interest in
serving MPEG2 or pre-H.264 MPEG4 on the Web, and they're encumbered, so
they're not interesting at all.

> With MPEG support, I could create a XULrunner application which can play
> video as background, preview or a video player. I am writing just such
> an TV player app in XULRunner right now and need MPEG2/4 support.
> Reencoding my n TB of video files is out of question.

Supporting XULrunner applications for users who have lots of video
locally in obsolete formats is not a priority.

> * Please enable MPEG2 and 4, and not the other proprietary codecs.
> This will not make the video patent/codec situation worse: an open
> desktop system needs to deal with MPEG anyways due to DVB and DVD.

We're not interested in an "open desktop system", whatever that is. We
are focused on the Web.

> * MPEG2/4 support will help <video> tremendously, because it reduces
> the effort to use it from very high (change the whole video
> library) to small (change one HTML page).

Not at all. No-one is interested in streaming those formats.

> * I don't think patents are an issue, if you don't ship MPEG codes,
> but merely use generic OS APIs.

You're probably right, but that's irrelevant. Supporting DirectShow
would be a nightmare (see my other post) and doesn't let us do anything
we care about.

Rob

Robert O'Callahan

unread,
Jun 21, 2009, 10:48:52 PM6/21/09
to
On 22/6/09 1:57 PM, Chris Double wrote:
> You've only just read about it and now you're ready to proclaim it a
> better candidate. Do some more digging and you'll see it's not quite
> ready for the types of video used on the web yet.

What Chris is too polite to say is that Dirac is currently much worse
than Theora or H.264 at typical Web bit rates.

Rob

Boris Zbarsky

unread,
Jun 21, 2009, 10:52:48 PM6/21/09
to
Ben Bucksch wrote:
> You can't compare a fast, crappy H.264 encoder and the best Theora
> encoder of your choice.

Why not, if the use case is one where speed of encoding matters? Note
that generally Theora encoders are faster than the "proper encoding"
H.264 encoders...

Seriously, there are cases where H.264 is better; I doubt you'll find
anyone arguing against that. For some reason, various sites on the web
are not using H.264 in those better ways. It'd be interesting to know why.

> If Theora (or any free codec) was as good as H.264 (with proper
> encoding), I'd be happy, though.

Define "good"? There are several parameters that matter here: encoding
time, decoding time, bitrate, quality. Let's pretend that this last is
objective. Depending on use case, some of these might matter more than
others....

-Boris

Ben Bucksch

unread,
Jun 21, 2009, 11:06:45 PM6/21/09
to
On 22.06.2009 04:47, Robert O'Callahan wrote:
> We are focused on the Web.

Well, this is .platform, and as I said, I have XULrunner app that needs
to play MPEG2 and 4 videos (on Linux at least, preferably even in the
browser). I have a specific need for MPEG playback, and there's a patch.
If you don't want this, OK, but I wanted to state that I have a good use
for that, and others may too.

I believe there are even usecases for HD (1920x1080) video in the
browser - maybe not common yet, but they exist, e.g. network harddrives
/ home media servers. That would need a much more full-featured player,
too, though, like full screen, remote control, different seek modes etc..

These cases, and mine, may be better suited with external video players
launched from the browser / XULRunner.

>> * MPEG2/4 support will help <video> tremendously, because it reduces
>> the effort to use it from very high (change the whole video
>> library) to small (change one HTML page).
>
> Not at all. No-one is interested in streaming those formats.

Oh yes. YouTube is streaming H.264.
(When I say "MPEG4", I mean H.264 video codec in an MPEG TS/PS container.)

> DirectShow would be a nightmare (see my other post)

Your post assumed arbitrary codecs, while I suggested to enable only
MPEG2+4 codecs (possibly only selected, common ones), so many of the
arguments don't apply, but I fully understand that you don't want to
depend on third party code, also for security reasons. So: Yeah, I see.

Ben

Robert O'Callahan

unread,
Jun 21, 2009, 11:59:06 PM6/21/09
to
On 22/6/09 3:06 PM, Ben Bucksch wrote:
> On 22.06.2009 04:47, Robert O'Callahan wrote:
>> We are focused on the Web.
>
> Well, this is .platform, and as I said, I have XULrunner app that needs
> to play MPEG2 and 4 videos (on Linux at least, preferably even in the
> browser). I have a specific need for MPEG playback, and there's a patch.
> If you don't want this, OK, but I wanted to state that I have a good use
> for that, and others may too.

I suggest you use the Totem NPAPI plugin.

>>> * MPEG2/4 support will help <video> tremendously, because it reduces
>>> the effort to use it from very high (change the whole video
>>> library) to small (change one HTML page).
>>
>> Not at all. No-one is interested in streaming those formats.
>
> Oh yes. YouTube is streaming H.264.
> (When I say "MPEG4", I mean H.264 video codec in an MPEG TS/PS container.)

Then I don't understand why you're advocating DirectShow, since almost
no Windows users have an H.264 DirectShow codec.

Rob

Robert O'Callahan

unread,
Jun 22, 2009, 12:00:50 AM6/22/09
to
On 22/6/09 2:26 PM, Ben Bucksch wrote:
> You can't compare a fast, crappy H.264 encoder and the best Theora
> encoder of your choice.
>
> If Theora (or any free codec) was as good as H.264 (with proper
> encoding), I'd be happy, though.

So you believe that Youtube's staff don't know how to encode H.264 properly?

Rob

Neil

unread,
Jun 22, 2009, 6:14:08 AM6/22/09
to
Robert O'Callahan wrote:

> DirectShow codecs are quite likely to have security holes. As those
> holes are uncovered, we will have to track the issues and almost
> certainly the only possible response will be to blacklist insecure
> codecs (since we can't fix them ourselves). If we blacklist enough
> codecs, DirectShow support becomes worthless. And each rushed-out
> Firefox update creates a ton of work, of course.

I'd like to think that blacklisting updates are independent of Firefox
updates.

--
Warning: May contain traces of nuts.

Gervase Markham

unread,
Jun 22, 2009, 7:06:32 AM6/22/09
to Joshua Cranmer
On 21/06/09 23:10, Joshua Cranmer wrote:
> And, FWIW, if Youtube or Hulu provided multimedia in the Theora format
> for <video>, I think a lot of the "obscure" objection would vanish.

And given that Firefox 3.5 hasn't even been released yet, you need to
give it some time to see what impact it'll make and what they do. Not
everyone's as nimble as DailyMotion.

Gerv

blizzard

unread,
Jun 22, 2009, 9:03:15 AM6/22/09
to

I would also say that understanding the role of dirac is important as
well. It apparently does quite well at higher sizes (720p and above)
and if you have a lot of computing power at your fingertips (kind of
like H.264 in this regard.) Theora does much better at lower
resolutions and has much lower computational cost vs. either Dirac or
H.264. So it's all about trade-offs.

--Chris

blizzard

unread,
Jun 22, 2009, 9:06:42 AM6/22/09
to

Oops - careful with this language. It's good for high-bitrate, large
videos where you have lots of computing power. It apparently does
very well in this case. Let's be specific so that people don't repeat
the wrong phrases (again!)

--Chris

blizzard

unread,
Jun 22, 2009, 9:15:10 AM6/22/09
to

I don't think that it's as simple as that. They are probably bound by
computational limits - like a lot of people - in transcoding.* So
they make tradeoffs, just like everyone else does. H.264 does well if
you play with its knobs and are willing to spend many hours
transcoding videos to get things just right. But most people don't,
and google doesn't.

And it turns out that it doesn't matter because most people don't care
about that last 2% better.

This is a story about trade-offs. The same kinds of things that make
people choose jpeg vs png, html vs. xhtml, or a small fuel efficient
car vs. a car that can carry 7 kids and the family dog.

In our case the trade-off is about the ability to innovate both in the
codec space (where I might add the Xiph folks have done a fantastic
job at closing the gap with H.264) and on the web in general. Sites
like wikipedia are building online video editors based on this stuff
because they can build on the web platform and the backend format (ogg
theora) to build really interesting things.

Although you have a lot of content in H.264 - and I'm sorry about that
- we're interested in a pretty good baseline format for video on the
web.

--Chris

Michael Burns

unread,
Jun 22, 2009, 2:39:20 PM6/22/09
to
> However, I think that a <video> tag that plays *no* usable codec, or
> only a codec that nobody has encoded video in, does not help the open
> web either. I really like the idea of Theora, but unfortunately, it is
> neither an official standard nor is it efficient: files are relatively
> large (IIRC even in comparison to MPEG2, even more so MPEG4).
> Consequently, I have only 1 or 2 Theora videos. Not many Windows users
> have DirectShow codecs for Theora, which means they can't save and play
> saved Theora videos with their normal video player. Theora is not in
> wide use, and has little chance of getting there (no matter what we do)
> due to its bad efficiency.

Theora's rapidly improving. When theora 1.1 comes out, it will be a much
better candidate. Also, It will take some time for web sites to begin
implementing html5 video tag. Maybe updates to ogg, vorbis, and theora
could be added to Firefox 3.5 in the security and stability updates.
That way, the viability will be greatly increased as html 5 becomes
popular. This would nullify any shred of argument against theora's
relative quality.

Boris Zbarsky

unread,
Jun 22, 2009, 2:53:55 PM6/22/09
to
Michael Burns wrote:
> Theora's rapidly improving. When theora 1.1 comes out, it will be a much
> better candidate. Also, It will take some time for web sites to begin
> implementing html5 video tag. Maybe updates to ogg, vorbis, and theora
> could be added to Firefox 3.5 in the security and stability updates.

As I understood it, "theora 1.1" was a new encoder, but not a new format
or new decoder. So it shipping will just mean that videos can be
encoded better and not require any changes on our end.

> That way, the viability will be greatly increased as html 5 becomes
> popular. This would nullify any shred of argument against theora's
> relative quality.

Those are pretty strong words, since the point here is that H.264 can
produce lower bitrates for the same quality than Thusnelda does,
assuming you try hard enough. No one's debating that; what's not clear
is whether trying hard enough is worth it to people. It's not in a lot
of cases.

-Boris

Michael Burns

unread,
Jun 22, 2009, 3:20:37 PM6/22/09
to
> As I understood it, "theora 1.1" was a new encoder, but not a new format
> or new decoder. So it shipping will just mean that videos can be encoded
> better and not require any changes on our end.

Ah. I did not know that. It's good to know though. Thanks for clarifying.

>> That way, the viability will be greatly increased as html 5 becomes
>> popular. This would nullify any shred of argument against theora's
>> relative quality.
>
> Those are pretty strong words, since the point here is that H.264 can
> produce lower bitrates for the same quality than Thusnelda does,
> assuming you try hard enough. No one's debating that; what's not clear
> is whether trying hard enough is worth it to people. It's not in a lot
> of cases.

I know that it can produce a better bit/rate, but no one on the web
actually does it. When Hollywood makes their movies and sells them in
high definition on blue ray, then yeah, it's better than Theora. As for
it would be better, I meant from a business perspective, not from a
technical one. It would be lower cost, easier to use (which also saves
money), better performance (if I remember correctly), and superior
quality (when compared to how the vast majority of the web uses it.)

Michael Burns

unread,
Jun 22, 2009, 3:24:07 PM6/22/09
to
> I know that it can produce a better bit/rate, but no one on the web
> actually does it. When Hollywood makes their movies and sells them in
> high definition on blue ray, then yeah, it's better than Theora. As for
> it would be better, I meant from a business perspective, not from a
> technical one. It would be lower cost, easier to use (which also saves
> money), better performance (if I remember correctly), and superior
> quality (when compared to how the vast majority of the web uses it.)

*change "superior quality" to "very competitive quality".

Robert O'Callahan

unread,
Jun 22, 2009, 4:28:42 PM6/22/09
to

Isn't that what I said?

Rob

Reply all
Reply to author
Forward
0 new messages