External (ie, gstreamer) handling of HTML5 video content
Jeremy Nickurak
which I'm told is specific to the maemo mobile platform and fennec. I'd
suggest that if there's a good argument to be made for supporting
external handling of the video on that platform, the same argument likely
applies elsewhere. As requested, I'd like to open some discussion here.
What's the outlook for focusing on external handling of video/audio
decoding and presentation? Bug #422540's been drawing a lot of attention
if its CC list is any indication. Proper integration with whatever video
and audio codecs installed on the system, and proper integration with
whatever presentation methods (ie, good pulseaudio support, accelerated
video, accelerated decoding where available in hardware) seems like a
good idea well beyond just the maemo mobile platform. There are a lot of
talented people building a framework that's well tested across many
platforms and application areas.
L. David Baron
For information on why we don't want to do this, see:
http://weblogs.mozillazine.org/roc/archives/2010/01/video_freedom_a.html
http://weblogs.mozillazine.org/roc/archives/2010/01/activex_all_ove.html
http://weblogs.mozillazine.org/roc/archives/2009/06/directshow_and.html
http://shaver.off.net/diary/2010/01/23/html5-video-and-codecs/
http://www.0xdeadbeef.com/weblog/2010/01/html5-video-and-h-264-what-history-tells-us-and-why-were-standing-with-the-web/
-David
--
L. David Baron http://dbaron.org/
Mozilla Corporation http://www.mozilla.com/
Nicholas Nethercote
>> Proper integration with whatever video
>> and audio codecs installed on the system, and proper integration with
>> whatever presentation methods (ie, good pulseaudio support, accelerated
>> video, accelerated decoding where available in hardware) seems like a
>> good idea well beyond just the maemo mobile platform. There are a lot of
>> talented people building a framework that's well tested across many
>> platforms and application areas.
>
> http://weblogs.mozillazine.org/roc/archives/2010/01/video_freedom_a.html
> http://weblogs.mozillazine.org/roc/archives/2010/01/activex_all_ove.html
> http://weblogs.mozillazine.org/roc/archives/2009/06/directshow_and.html
> http://shaver.off.net/diary/2010/01/23/html5-video-and-codecs/
> http://www.0xdeadbeef.com/weblog/2010/01/html5-video-and-h-264-what-history-tells-us-and-why-were-standing-with-the-web/
I understand the arguments against external video handling in Firefox.
I don't understand why Fennec should be any different. The only
reason I've heard (and I may have misheard/misinterpreted) is that
"mobile is smaller and less important" which seems pretty weak.
Can anyone enlighten me? Thanks.
N
Robert O'Callahan
> Follow up from: http://bugzilla.mozilla.org/show_bug.cgi?id=422540 ,
> which I'm told is specific to the maemo mobile platform and fennec. I'd
> suggest that if there's a good argument to be made for supporting
> external handling of the video on that platform, the same argument likely
> applies elsewhere. As requested, I'd like to open some discussion here.
Thanks for doing that.
> What's the outlook for focusing on external handling of video/audio
> decoding and presentation? Bug #422540's been drawing a lot of attention
> if its CC list is any indication. Proper integration with whatever video
> and audio codecs installed on the system, and proper integration with
> whatever presentation methods (ie, good pulseaudio support, accelerated
> video, accelerated decoding where available in hardware) seems like a
> good idea well beyond just the maemo mobile platform. There are a lot of
> talented people building a framework that's well tested across many
> platforms and application areas.
Integration with system codecs is something that I think we shouldn't
distribute in Firefox, for reasons which are outlined in the blog posts
dbaron mentioned (some of which I wrote). I don't think we should stop
other people from doing that, though. If someone wanted to develop code
for system codec integration and distribute their own browser builds
with that code, they can just go ahead and do it. It might even be
reasonable to take such code in mozilla-central, although that would
depend on factors such as how well the implementation conforms to the
HTML5 spec.
Integration with platform-specific presentation methods makes complete
sense.
Whether we should use an existing framework (presumably GStreamer) for
all our playback needs is a different question, and I don't think we
know the long-term answer. There's a lot in GStreamer that we don't
currently need, like the entire filter-graph architecture. But every
time we think about adding functionality that GStreamer already has, we
definitely should be asking ourselves "should we just integrate
GStreamer instead?" Having code to use system GStreamer (which Mike K is
working on) would help us answer that question, although there would
still be uncertainty around issues such as footprint, licensing, and how
easy it would be to add arbitrary features that require changes to the
framework.
Rob
Chris Lord
> On Monday 2010-01-25 12:27 -0600, Jeremy Nickurak wrote:
> http://weblogs.mozillazine.org/roc/archives/2010/01/video_freedom_a.html
> http://weblogs.mozillazine.org/roc/archives/2010/01/activex_all_ove.html
> http://weblogs.mozillazine.org/roc/archives/2009/06/directshow_and.html
> http://shaver.off.net/diary/2010/01/23/html5-video-and-codecs/
> http://www.0xdeadbeef.com/weblog/2010/01/html5-video-and-h-264-what-history-tells-us-and-why-were-standing-with-the-web/
>
Just to play devil's advocate, GStreamer could still be used and
non-HTML5 video codecs could purposefully be disabled. Wouldn't it be
good to use GStreamer and massively reduce the maintenance?
If it's not being used solely because it would allow more than theora to
be supported, that's a pretty flimsy reason not to take advantage of the
hard work of others.
It would be easier to enable all codecs if non-theora was purposefully
disabled, but then anyone that recompiles Firefox themselves could
already circumvent this limitation.
Thoughts?
--Chris
Mike Kristoffersen
<SNIP>
>
> Just to play devil's advocate, GStreamer could still be used and
> non-HTML5 video codecs could purposefully be disabled. Wouldn't it be
> good to use GStreamer and massively reduce the maintenance?
>
> If it's not being used solely because it would allow more than theora to
> be supported, that's a pretty flimsy reason not to take advantage of the
> hard work of others.
>
> It would be easier to enable all codecs if non-theora was purposefully
> disabled, but then anyone that recompiles Firefox themselves could
> already circumvent this limitation.
>
> Thoughts?
>
> --Chris
>
>
I don't see any reason why we should actively try to prevent anyone from
using whatever codec they want (like we don't try to prevent people
from watching flash content by using plug-ins, but rather we make sure
it's working before we release new software).
I can understand the reasoning behind not wanting to directly
include/promote/support non-free codecs, but thats not the same as
saying that I agree.
With GStreamer we get support for current and future free codecs that
the GStreamer community will work on - that has value for us, there is
no reason for us to create and maintain our own framework for handling
different codecs or worse hardcode the codecs we think are "worthy" into
our code base. If we want to support a codec that the GStreamer
comunity doesn't supply, then it will hardly be harder for us to
integrate it into the GStreamer framework, than it would be to integrate
it into our own framework.
It is true that there is a side effect by using GStreamer which allows
the end user to install codecs into the system that can have patent
issues, copyright issues, and/or hurts his web-experience, but that will
be his choice - in the same way we don't disable Javascript, because
some sites might use patented algorithms in their scripts. In general
we don't prevent users from accessing or downloading content that are
non-free in any sense but rather we let the user make the choice.
We might need to put GStreamer into it's own process for security
reasons and the stability of the browser, in case faulty codecs are
used, but that is kind of a separate discussion.
^__^
MikeK
Mike Kristoffersen
>
> I can understand the reasoning behind not wanting to directly
> include/promote/support non-free codecs, but thats not the same as
> saying that I agree.
>
Just to avoid confusion - with the above I DON'T mean that Mozilla
should promote, develop or include non-free codecs, I'm definitely in
favor of any kind of support to the free codecs, I just don't think we
should actively prevent the non-free versions.
^__^
MikeK
Mark Finkle
> I understand the arguments against external video handling in Firefox.
> I don't understand why Fennec should be any different. The only
> reason I've heard (and I may have misheard/misinterpreted) is that
> "mobile is smaller and less important" which seems pretty weak.
>
> Can anyone enlighten me? Thanks.
IIRC, the main reasons were performance related. See
http://blog.vlad1.com/2009/08/18/theora-video-soon-in-firefox-on-arm-cpus/
That said, projects like theorarm (http://wss.co.uk/pinknoise/theorarm/)
show better performance is possible on ARM devices.
Robert Kaiser
> Wouldn't it be
> good to use GStreamer and massively reduce the maintenance?
How does making inter-platform differences larger reduce maintenance?
From most things I have seen where we hook directly into or reuse
platform libraries, things get much harder to maintain esp. for people
who don't have other platforms available to test on, and our feature set
starts to vastly diverge between platforms - see. e.g. the complete
substandard and sucking print settings support on Linux, while things
seem to be in a much better state on other platforms.
Robert Kaiser
Chris Double
> Chris Lord wrote:
> > Wouldn't it be
> > good to use GStreamer and massively reduce the maintenance?
>
> How does making inter-platform differences larger reduce maintenance?
I think his suggestion would be to use GStreamer on all platforms -
like Opera is doing (and Songbird does I believe).
Chris.
--
http://bluishcoder.co.nz
Robert O'Callahan
> Just to play devil's advocate, GStreamer could still be used and
> non-HTML5 video codecs could purposefully be disabled. Wouldn't it be
> good to use GStreamer and massively reduce the maintenance?
If it was obvious that using GStreamer would massively reduce
maintenance, and didn't create any new problems with HTML5
compatibility, licensing, footprint, performance or ease of implementing
new features or performance improvements, we should definitely use it!
> If it's not being used solely because it would allow more than theora to
> be supported, that's a pretty flimsy reason not to take advantage of the
> hard work of others.
Indeed. That reason has never factored into the decision of whether to
use GStreamer. It might factor into the decision of whether, on Linux,
we would use system GStreamer vs a built-in GStreamer, in builds
distributed by Mozilla.
Rob
King InuYasha
> On 28/01/10 12:35 AM, Chris Lord wrote:
>
> > Just to play devil's advocate, GStreamer could still be used and
> > non-HTML5 video codecs could purposefully be disabled. Wouldn't it be
> > good to use GStreamer and massively reduce the maintenance?
>
> If it was obvious that using GStreamer would massively reduce
> maintenance, and didn't create any new problems with HTML5
> compatibility, licensing, footprint, performance or ease of implementing
> new features or performance improvements, we should definitely use it!
>
Then why not use it? Opera is currently shipping their Windows version
of 10.50
with GStreamer, and their soon-to-be-released Mac and Linux builds
will also
use GStreamer. In the case of Linux, it will use system GStreamer. But
for
Mac and Windows, a bundled version of GStreamer is going to be used.
> > If it's not being used solely because it would allow more than theora to
> > be supported, that's a pretty flimsy reason not to take advantage of the
> > hard work of others.
>
> Indeed. That reason has never factored into the decision of whether to
> use GStreamer. It might factor into the decision of whether, on Linux,
> we would use system GStreamer vs a built-in GStreamer, in builds
> distributed by Mozilla.
In builds distributed by Mozilla, I'd put a huge bet on Mozilla
bundling for
all platforms. It ensures that Mozilla can do effective quality
control and
only bundle codecs that fit within Mozilla's ideology. They probably
would
allow Linux distributions to use system GStreamer with their own
Firefox
builds, since it would be pointless to require duplication of code
like that.
Would it really be so bad to use GStreamer instead of your own weird
backend that only supports Ogg Vorbis and Theora with the Ogg
container, which by the way, is really only designed for streaming
without seeking, as opposed to containers like Matroska, which support
both methods of operation.
Robert O'Callahan
> On Jan 27, 8:23 pm, Robert O'Callahan<rob...@ocallahan.org> wrote:
>> If it was obvious that using GStreamer would massively reduce
>> maintenance, and didn't create any new problems with HTML5
>> compatibility, licensing, footprint, performance or ease of implementing
>> new features or performance improvements, we should definitely use it!
>
> Then why not use it?
Because regardless of what Opera's doing, the answers to those questions
are not clear.
Just to pick one, licensing, GStreamer is LGPL but we have a policy of
our builds being distributable under any of our three licenses --- LGPL,
GPL or MPL. In practice that means libraries we import are usually BSD
licensed or LGPL/MPL dual licensed. So we'd either want GStreamer to be
relicensed (unlikely I assume), or we'd have to change our policy.
Rob