DirectX/Quicktime backends for Video
DigDug
or not these are wanted or would even be accepted. Can someone with
enough clout to make a decision make one and close the bug if its
WONTFX?
My 2 cents, the open web is different than the Open Linux platform.
The OpenWeb is open because its simple. No strict checking of
someone's HTML ability. No real penalty for writing bad javascript.
Most of the image formats that people use are widely supported, so
there's not a lot to worry about there. Limiting users to what is an
obscure codec (compared to what they're used to at least), purely
because its "open" in the sense that its unencumbered by patents or
whatnot seems contrary to that spirit.
Letting the platform decide what's supported and what isn't (and
providing a better Plugin Finder service to help people when there's
no supported codec) would seem like a no brainer to me. But someone
who can put aside any platform predispositions, better educated about
these things, and who can make an honest decision based on what they
think is best for the web should make decision.
Joshua Cranmer
> Limiting users to what is an
> obscure codec (compared to what they're used to at least), purely
> because its "open" in the sense that its unencumbered by patents or
> whatnot seems contrary to that spirit.
It might be contrary, but patents present an issue that transcends
debate about openness. Mozilla cannot knowingly infringe on patents, and
it will not license patents if a royalty is required (I think the
software licenses with Mozilla might make licensing legally impossible,
but I am not a lawyer and I don't know this portion of law).
So long as the codec in question is patent-protected in such a manner,
it remains out of the question. If you object to the matter, it might be
better to complain to the patent holder or possibly even your local
government representative (in case he or she would support a law which
would retroactively invalidate software patents).
And, FWIW, if Youtube or Hulu provided multimedia in the Theora format
for <video>, I think a lot of the "obscure" objection would vanish. Also
note that 99% of the Internet-using public has no concept of video
codecs, so complaining that it's obscure compared to what the public is
used to is rather moot. In terms of what servers support and use for
<video> and <audio>, I'm willing to bet that the Ogg Vorbis and Theora
are by far the most popular codecs.
Ben Bucksch
the goal, because it doesn't help the open web.
However, I think that a <video> tag that plays *no* usable codec, or
only a codec that nobody has encoded video in, does not help the open
web either. I really like the idea of Theora, but unfortunately, it is
neither an official standard nor is it efficient: files are relatively
large (IIRC even in comparison to MPEG2, even more so MPEG4).
Consequently, I have only 1 or 2 Theora videos. Not many Windows users
have DirectShow codecs for Theora, which means they can't save and play
saved Theora videos with their normal video player. Theora is not in
wide use, and has little chance of getting there (no matter what we do)
due to its bad efficiency.
MPEG2/4 are unfortunately patent encumbered, but are a standard and very
widely used, in DVDs, DVB (almost all TV in Europe) etc.. I have a few
TB of MPEG videos myself, mostly from DVD and TV.
Hardly any website wanting to show video to their users will consider
Theora to be a tempting alternative. It is always a cost/effect
discussion within companies, and Theora is likely to be the object of
ridicule of those people who don't like open source very much, which
doesn't make it easier for our open web friends to push for <video>
support within their web company. The video provider would have to keep
all videos in another format. If <video> would allow MPEG2 and MPEG4,
they could stream their existing video files, which would make support
for <video> a *lot* easier to implement, just some HTML+JS changes
versus changing their content management, encoders and adding tons of
storage.
With MPEG support, I could create a XULrunner application which can play
video as background, preview or a video player. I am writing just such
an TV player app in XULRunner right now and need MPEG2/4 support.
Reencoding my n TB of video files is out of question.
As for the patent issue: We're not implementing MPEG, so we're not in
violation of the MPEG patents. We're merely using generic OS or library
functions. It's the same situation as with any other DirectShow app: Any
application can use the generic DirectShow APIs, without violating
patents. The user may get the codec from anybody (legally), and any
DirectShow application can then legally play the videos without being in
violation of the patents.
So:
* Please enable MPEG2 and 4, and not the other proprietary codecs.
This will not make the video patent/codec situation worse: an open
desktop system needs to deal with MPEG anyways due to DVB and DVD.
* MPEG2/4 support will help <video> tremendously, because it reduces
the effort to use it from very high (change the whole video
library) to small (change one HTML page).
* I don't think patents are an issue, if you don't ship MPEG codes,
but merely use generic OS APIs.
Ben Bucksch
inefficiency (relatively large files/bandwidth). As a result, it's not
in very wide use, and probably won't get in wide use even if it's the
only codec we support.
However, I just read about Dirac, a codec invented and implemented by
the BBC, specifically avoiding patents. It's Wavelet-based, uses motion
compression and other algos, and is announced to be about as efficient
as MPEG4, maybe even with better image quality. (I just cite Wikipedia,
I haven't tried myself yet.) It's suitable for resolutions from 176x144
to 1920x1080.
That makes it a much better candidate for campaigning and support by us
than Theora.
Oh, and the BBC implementation is licenced under MPL 1.1 / GPL 2 / LGPL 2.1.
Yay?
Ben
Boris Zbarsky
> However, I just read about Dirac, a codec invented and implemented by
> the BBC, specifically avoiding patents. It's Wavelet-based, uses motion
> compression and other algos, and is announced to be about as efficient
> as MPEG4, maybe even with better image quality. (I just cite Wikipedia,
> I haven't tried myself yet.) It's suitable for resolutions from 176x144
> to 1920x1080.
>
> That makes it a much better candidate for campaigning and support by us
> than Theora.
Except that in fact its patent risk is higher than Theora's and last I
checked the spec for it wasn't quite stable yet... Has that changed?
-Boris
Chris Double
> I really like the idea of Theora, but unfortunately, it is
> neither an official standard nor is it efficient: files are relatively
> large (IIRC even in comparison to MPEG2, even more so MPEG4).
This is not true:
http://hacks.mozilla.org/2009/06/open-video-codecs-and-quality/
http://people.xiph.org/~maikmerten/youtube/
These compare Theora encodes to H.264 encodes of the quality that
YouTube is using. The file size and quality comparisons do not match
what you say.
Chris.
--
http://www.bluishcoder.co.nz
Chris Double
> As said in my last post, the biggest problem with Theora is its
> inefficiency (relatively large files/bandwidth).
Read my reply, this is false. Tests show size and bandwidth use is
similar.
>
> That makes it a much better candidate for campaigning and support by us
> than Theora.
You've only just read about it and now you're ready to proclaim it a
better candidate. Do some more digging and you'll see it's not quite
ready for the types of video used on the web yet. Do some more reading
and you'll find we're actually working towards getting Dirac support
in the base libraries we use to implement video so when it is ready we
can easily support it (I'm not saying we will, but it's certainly on
the radar).
Chris.
--
http://www.bluishcoder.co.nz
Ben Bucksch
> Tests show size and bandwidth use is similar.
>
You can't compare a fast, crappy H.264 encoder and the best Theora
encoder of your choice.
If Theora (or any free codec) was as good as H.264 (with proper
encoding), I'd be happy, though.
> Do some more digging and you'll see [Dirac is] not quite ready for the
> types of video used on the web yet.
Care to say why or at least point to an article?
Also, "used on the web" (today) is not that relevant for the future:
YouTube is not interesting for me, because it's too small and too short,
and YouTube clips are in fact getting higher res and longer.
> we're actually working towards getting Dirac support
Some pointer to that would aid the discussion, likewise.
Ben Bucksch
> On Jun 22, 12:32 pm, Ben Bucksch<ben.bucksch.n...@beonex.com> wrote:
>
>> I really like the idea of Theora, but unfortunately, it is
>> neither an official standard nor is it efficient: files are relatively
>> large (IIRC even in comparison to MPEG2, even more so MPEG4).
>>
> This is not true
>
Even if so, all the other arguments stand: I got 2 TB of MPEG videos,
and I'm not going to reencode them. That's why I need MPEG support - if
nothing else, then for XULrunner apps. This is .platform, after all.
Robert O'Callahan
Even if a volunteer produces a patch, I would not want to ship it in
Firefox in the near future, for the following reasons:
-- We want to focus our energy on promoting open unencumbered codecs at
this time.
-- Only a very small fraction of Windows users have a DirectShow codec
for the most important encumbered codec, H.264.
-- DirectShow is underspecified and codecs are of highly variable
quality. Many codecs probably will not work with Web sites that use all
the rich APIs of <video>, and those bugs will be filed against us. We
may not even be able to fix them. (Note that the problem is bad enough
that in Windows 7, Microsoft isn't even going to allow unknown third
parties to install DirectShow codecs.) We could avoid some of this
problem by white-listing codecs, but then a lot of the people who want
DirectShow support wouldn't be satisfied anyway.
-- Many DirectShow codecs are actually malware.
-- DirectShow codecs are quite likely to have security holes. As those
holes are uncovered, we will have to track the issues and almost
certainly the only possible response will be to blacklist insecure
codecs (since we can't fix them ourselves). If we blacklist enough
codecs, DirectShow support becomes worthless. And each rushed-out
Firefox update creates a ton of work, of course.
-- Each new video backend creates additional maintenance headaches as we
evolve our internal video code.
So even if we didn't care about promoting unencumbered codecs and
someone gave us a working patch, shipping DirectShow support in Firefox
is of limited value and creates tons of maintenance work for us.
I think the above reasons explain why so far, no browser or browser
plugin vendor is using codec implementations they don't control.
Rob
Ben Bucksch
> its patent risk is higher than Theora's
Well, I'd trust BBC's judgment on that unless I have reason to believe
they erred.
> last I checked the spec for it wasn't quite stable yet... Has that
> changed?
Apparently yes.
"The specification was finalised on 21 January 2008 ...
Version 1.0.0 of the reference implementation ... was released on 17
September 2008 ...
A second implementation called Schr�dinger was funded by the BBC and
aims to provide high-performance, portable version ... written in ANSI C
... 22 February 2008, Schr�dinger 1.0.0 was released."
<http://en.wikipedia.org/wiki/Dirac_%28codec%29>
Robert O'Callahan
> I really like the idea of Theora, but unfortunately, it is
> neither an official standard nor is it efficient: files are relatively
> large (IIRC even in comparison to MPEG2, even more so MPEG4).
Like Chris said, you're blindly repeating myths, or at least outdated
information.
> Consequently, I have only 1 or 2 Theora videos. Not many Windows users
> have DirectShow codecs for Theora, which means they can't save and play
> saved Theora videos with their normal video player.
That is not an important use case. Currently, you can't even save video
from most sites without the help of Firefox extensions. At least users
will be able to save Theora videos and play them back in Firefox.
> Theora is not in
> wide use, and has little chance of getting there (no matter what we do)
> due to its bad efficiency.
"Bad efficiency" is a myth. We'll see how we go.
> MPEG2/4 are unfortunately patent encumbered, but are a standard and very
> widely used, in DVDs, DVB (almost all TV in Europe) etc.. I have a few
> TB of MPEG videos myself, mostly from DVD and TV.
Fine, but that's not Web video. There is basically no interest in
serving MPEG2 or pre-H.264 MPEG4 on the Web, and they're encumbered, so
they're not interesting at all.
> With MPEG support, I could create a XULrunner application which can play
> video as background, preview or a video player. I am writing just such
> an TV player app in XULRunner right now and need MPEG2/4 support.
> Reencoding my n TB of video files is out of question.
Supporting XULrunner applications for users who have lots of video
locally in obsolete formats is not a priority.
> * Please enable MPEG2 and 4, and not the other proprietary codecs.
> This will not make the video patent/codec situation worse: an open
> desktop system needs to deal with MPEG anyways due to DVB and DVD.
We're not interested in an "open desktop system", whatever that is. We
are focused on the Web.
> * MPEG2/4 support will help <video> tremendously, because it reduces
> the effort to use it from very high (change the whole video
> library) to small (change one HTML page).
Not at all. No-one is interested in streaming those formats.
> * I don't think patents are an issue, if you don't ship MPEG codes,
> but merely use generic OS APIs.
You're probably right, but that's irrelevant. Supporting DirectShow
would be a nightmare (see my other post) and doesn't let us do anything
we care about.
Rob
Robert O'Callahan
> You've only just read about it and now you're ready to proclaim it a
> better candidate. Do some more digging and you'll see it's not quite
> ready for the types of video used on the web yet.
What Chris is too polite to say is that Dirac is currently much worse
than Theora or H.264 at typical Web bit rates.
Rob
Boris Zbarsky
> You can't compare a fast, crappy H.264 encoder and the best Theora
> encoder of your choice.
Why not, if the use case is one where speed of encoding matters? Note
that generally Theora encoders are faster than the "proper encoding"
H.264 encoders...
Seriously, there are cases where H.264 is better; I doubt you'll find
anyone arguing against that. For some reason, various sites on the web
are not using H.264 in those better ways. It'd be interesting to know why.
> If Theora (or any free codec) was as good as H.264 (with proper
> encoding), I'd be happy, though.
Define "good"? There are several parameters that matter here: encoding
time, decoding time, bitrate, quality. Let's pretend that this last is
objective. Depending on use case, some of these might matter more than
others....
-Boris
Ben Bucksch
> We are focused on the Web.
Well, this is .platform, and as I said, I have XULrunner app that needs
to play MPEG2 and 4 videos (on Linux at least, preferably even in the
browser). I have a specific need for MPEG playback, and there's a patch.
If you don't want this, OK, but I wanted to state that I have a good use
for that, and others may too.
I believe there are even usecases for HD (1920x1080) video in the
browser - maybe not common yet, but they exist, e.g. network harddrives
/ home media servers. That would need a much more full-featured player,
too, though, like full screen, remote control, different seek modes etc..
These cases, and mine, may be better suited with external video players
launched from the browser / XULRunner.
>> * MPEG2/4 support will help <video> tremendously, because it reduces
>> the effort to use it from very high (change the whole video
>> library) to small (change one HTML page).
>
> Not at all. No-one is interested in streaming those formats.
Oh yes. YouTube is streaming H.264.
(When I say "MPEG4", I mean H.264 video codec in an MPEG TS/PS container.)
> DirectShow would be a nightmare (see my other post)
Your post assumed arbitrary codecs, while I suggested to enable only
MPEG2+4 codecs (possibly only selected, common ones), so many of the
arguments don't apply, but I fully understand that you don't want to
depend on third party code, also for security reasons. So: Yeah, I see.
Ben
Robert O'Callahan
> On 22.06.2009 04:47, Robert O'Callahan wrote:
>> We are focused on the Web.
>
> Well, this is .platform, and as I said, I have XULrunner app that needs
> to play MPEG2 and 4 videos (on Linux at least, preferably even in the
> browser). I have a specific need for MPEG playback, and there's a patch.
> If you don't want this, OK, but I wanted to state that I have a good use
> for that, and others may too.
I suggest you use the Totem NPAPI plugin.
>>> * MPEG2/4 support will help <video> tremendously, because it reduces
>>> the effort to use it from very high (change the whole video
>>> library) to small (change one HTML page).
>>
>> Not at all. No-one is interested in streaming those formats.
>
> Oh yes. YouTube is streaming H.264.
> (When I say "MPEG4", I mean H.264 video codec in an MPEG TS/PS container.)
Then I don't understand why you're advocating DirectShow, since almost
no Windows users have an H.264 DirectShow codec.
Rob
Robert O'Callahan
> You can't compare a fast, crappy H.264 encoder and the best Theora
> encoder of your choice.
>
> If Theora (or any free codec) was as good as H.264 (with proper
> encoding), I'd be happy, though.
So you believe that Youtube's staff don't know how to encode H.264 properly?
Rob
Neil
> DirectShow codecs are quite likely to have security holes. As those
> holes are uncovered, we will have to track the issues and almost
> certainly the only possible response will be to blacklist insecure
> codecs (since we can't fix them ourselves). If we blacklist enough
> codecs, DirectShow support becomes worthless. And each rushed-out
> Firefox update creates a ton of work, of course.
I'd like to think that blacklisting updates are independent of Firefox
updates.
--
Warning: May contain traces of nuts.
Gervase Markham
> And, FWIW, if Youtube or Hulu provided multimedia in the Theora format
> for <video>, I think a lot of the "obscure" objection would vanish.
And given that Firefox 3.5 hasn't even been released yet, you need to
give it some time to see what impact it'll make and what they do. Not
everyone's as nimble as DailyMotion.
Gerv
blizzard
I would also say that understanding the role of dirac is important as
well. It apparently does quite well at higher sizes (720p and above)
and if you have a lot of computing power at your fingertips (kind of
like H.264 in this regard.) Theora does much better at lower
resolutions and has much lower computational cost vs. either Dirac or
H.264. So it's all about trade-offs.
--Chris
blizzard
Oops - careful with this language. It's good for high-bitrate, large
videos where you have lots of computing power. It apparently does
very well in this case. Let's be specific so that people don't repeat
the wrong phrases (again!)
--Chris
blizzard
I don't think that it's as simple as that. They are probably bound by
computational limits - like a lot of people - in transcoding.* So
they make tradeoffs, just like everyone else does. H.264 does well if
you play with its knobs and are willing to spend many hours
transcoding videos to get things just right. But most people don't,
and google doesn't.
And it turns out that it doesn't matter because most people don't care
about that last 2% better.
This is a story about trade-offs. The same kinds of things that make
people choose jpeg vs png, html vs. xhtml, or a small fuel efficient
car vs. a car that can carry 7 kids and the family dog.
In our case the trade-off is about the ability to innovate both in the
codec space (where I might add the Xiph folks have done a fantastic
job at closing the gap with H.264) and on the web in general. Sites
like wikipedia are building online video editors based on this stuff
because they can build on the web platform and the backend format (ogg
theora) to build really interesting things.
Although you have a lot of content in H.264 - and I'm sorry about that
- we're interested in a pretty good baseline format for video on the
web.
--Chris
Michael Burns
> only a codec that nobody has encoded video in, does not help the open
> web either. I really like the idea of Theora, but unfortunately, it is
> neither an official standard nor is it efficient: files are relatively
> large (IIRC even in comparison to MPEG2, even more so MPEG4).
> Consequently, I have only 1 or 2 Theora videos. Not many Windows users
> have DirectShow codecs for Theora, which means they can't save and play
> saved Theora videos with their normal video player. Theora is not in
> wide use, and has little chance of getting there (no matter what we do)
> due to its bad efficiency.
Theora's rapidly improving. When theora 1.1 comes out, it will be a much
better candidate. Also, It will take some time for web sites to begin
implementing html5 video tag. Maybe updates to ogg, vorbis, and theora
could be added to Firefox 3.5 in the security and stability updates.
That way, the viability will be greatly increased as html 5 becomes
popular. This would nullify any shred of argument against theora's
relative quality.
Boris Zbarsky
> Theora's rapidly improving. When theora 1.1 comes out, it will be a much
> better candidate. Also, It will take some time for web sites to begin
> implementing html5 video tag. Maybe updates to ogg, vorbis, and theora
> could be added to Firefox 3.5 in the security and stability updates.
As I understood it, "theora 1.1" was a new encoder, but not a new format
or new decoder. So it shipping will just mean that videos can be
encoded better and not require any changes on our end.
> That way, the viability will be greatly increased as html 5 becomes
> popular. This would nullify any shred of argument against theora's
> relative quality.
Those are pretty strong words, since the point here is that H.264 can
produce lower bitrates for the same quality than Thusnelda does,
assuming you try hard enough. No one's debating that; what's not clear
is whether trying hard enough is worth it to people. It's not in a lot
of cases.
-Boris
Michael Burns
> or new decoder. So it shipping will just mean that videos can be encoded
> better and not require any changes on our end.
Ah. I did not know that. It's good to know though. Thanks for clarifying.
>> That way, the viability will be greatly increased as html 5 becomes
>> popular. This would nullify any shred of argument against theora's
>> relative quality.
>
> Those are pretty strong words, since the point here is that H.264 can
> produce lower bitrates for the same quality than Thusnelda does,
> assuming you try hard enough. No one's debating that; what's not clear
> is whether trying hard enough is worth it to people. It's not in a lot
> of cases.
I know that it can produce a better bit/rate, but no one on the web
actually does it. When Hollywood makes their movies and sells them in
high definition on blue ray, then yeah, it's better than Theora. As for
it would be better, I meant from a business perspective, not from a
technical one. It would be lower cost, easier to use (which also saves
money), better performance (if I remember correctly), and superior
quality (when compared to how the vast majority of the web uses it.)
Michael Burns
> actually does it. When Hollywood makes their movies and sells them in
> high definition on blue ray, then yeah, it's better than Theora. As for
> it would be better, I meant from a business perspective, not from a
> technical one. It would be lower cost, easier to use (which also saves
> money), better performance (if I remember correctly), and superior
> quality (when compared to how the vast majority of the web uses it.)
*change "superior quality" to "very competitive quality".
Robert O'Callahan
Isn't that what I said?
Rob