Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Re: Codecs plan

42 views
Skip to first unread message

Andreas Gal

unread,
Mar 20, 2012, 4:48:01 AM3/20/12
to rob...@ocallahan.org, dev-...@lists.mozilla.org

I would do the following order: 1) 4) 2) 3)

Andreas

On Mar 20, 2012, at 1:18 AM, Robert O'Callahan wrote:

> Here's what I think we're going to do next:
>
> 1) Review, polish and land existing patches for using system codecs on
> Android/B2G (bug 714408). Those patches create an "MPAPI" interface into
> which various codec libraries can be plugged. These libraries will be
> wrappers around libstagefright or other Android system codec interfaces ---
> different libraries customized for different Android versions and/or
> devices as required. Currently these patches only work on devices where we
> can get the decoded YUV data in main memory (MPAPI currently depends on
> this).
> Chris Double is currently working on this. Some contributors are also
> involved.
>
> 2) Following on from #1, extend MPAPI to handle cases where we don't get
> YUV data. The frames might be made available only as some kind of overlay.
> To integrate this into Gecko as best we can, we'll need to modify the
> layers system to work with those overlays. The exact details of this depend
> on exactly what the hardware and APIs can do; needs investigation.
> Not sure who'll work on this yet, but Chris D will be involved and I'll
> probably help.
>
> 3) Review and land latest contributed GStreamer patch (bug 422540), which
> is quite good. We currently have no plans to ship this in a Mozilla
> product, but contributors want it so it might as well be in the tree as
> they're using it.
> This is Chris Double.
>
> 4) Investigate MP3 support using system codecs on desktop platforms.
> 4a) On Windows (XP and up) we can use DirectShow. Chris Pearce is going to
> look into this.
> 4b) on Mac we can use some Mac APIs. Edwin Flores is going to look into
> this.
> 4c) on Linux we could use libav (when MP3 codec is available) directly, or
> possibly via GStreamer (see #3). Not sure who's going to work on that.
> The currently plan is to use these system APIs directly in dedicated codec
> backends rather than wrap them in some other abstraction such as MPAPI, but
> this may be revisited.
>
> Rob
> --
> “You have heard that it was said, ‘Love your neighbor and hate your enemy.’
> But I tell you, love your enemies and pray for those who persecute you,
> that you may be children of your Father in heaven. ... If you love those
> who love you, what reward will you get? Are not even the tax collectors
> doing that? And if you greet only your own people, what are you doing more
> than others?" [Matthew 5:43-47]
> _______________________________________________
> dev-media mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-media

Robert O'Callahan

unread,
Mar 20, 2012, 5:10:59 AM3/20/12
to Andreas Gal, dev-...@lists.mozilla.org
On Tue, Mar 20, 2012 at 9:48 PM, Andreas Gal <g...@mozilla.com> wrote:

> I would do the following order: 1) 4) 2) 3)
>

Agreed, except we'll probably do 3) earlier since it's not much work on our
side and keeping contributors unblocked is important.

Chris Double

unread,
Mar 20, 2012, 5:31:56 AM3/20/12
to rob...@ocallahan.org, dev-...@lists.mozilla.org, Andreas Gal
On Tue, Mar 20, 2012 at 10:10 PM, Robert O'Callahan
<rob...@ocallahan.org> wrote:
>
> Agreed, except we'll probably do 3) earlier since it's not much work on our
> side and keeping contributors unblocked is important.

I agree, I'll keep 3 active to keep the contributor unblocked and
since the bug has a history of being blocked on things.

Andreas, is it worthwhile working on the getting access to decoded
frames on Samsung device issue? Is someone working on that? For those
who aren't aware, the current omx/stagefright approach doesn't work on
devices that prevent access to frame data via the hardware decoders.
Andreas I believe has a workaround in mind for this but I don't know
if it's being worked on.

Chris.
--
http://www.bluishcoder.co.nz

Andreas Gal

unread,
Mar 20, 2012, 5:36:38 AM3/20/12
to Chris Double, dev-...@lists.mozilla.org, rob...@ocallahan.org
I actually solved this for Samsung devices specifically. Look at the crespo (=Nexus S) omx stack (its open source and part of any android tree). For some reason only Samsung knows, instead of using the proper API to pass the address of the unmapped buffer, Samsung allocates frames but then fills them with a little header only. The header points to the actual physical address of the video frame. You can read that header and find the frame in /dev/mem. This is a pretty ugly hack and requires root access. I haven't investigated whether there is a way to make this work for stock Android. It definitely works for Gonk (B2G). This will likely work for any Samsung device with Samsung chipset. They all tend to have the same decoder hardware.

The nicer fix for this is to add overlay support as outlined by roc. I have a partial patch for that somewhere. Its really not a huge deal to implement.

Andreas

>
> Chris.
> --
> http://www.bluishcoder.co.nz

Chris Double

unread,
Mar 20, 2012, 5:45:50 AM3/20/12
to dev-...@lists.mozilla.org
On Tue, Mar 20, 2012 at 10:36 PM, Andreas Gal <g...@mozilla.com> wrote:
> It definitely works for Gonk (B2G). This will likely work for any Samsung device with Samsung chipset.
> They all tend to have the same decoder hardware.

I tested video playback via the plugin on a Nexus S running B2G today
and I just got a green frame with logcat showing "cannot access frame
data" or something like that. Is it a fairly recent fix (ie. last 12
hours or so?).

Chris.
--
http://www.bluishcoder.co.nz

Henri Sivonen

unread,
Mar 20, 2012, 5:51:09 AM3/20/12
to dev-...@lists.mozilla.org
> 4c) on Linux we could use libav (when MP3 codec is available) directly, or
> possibly via GStreamer (see #3). Not sure who's going to work on that.

It seems to me it would be good to use GStreamer for this in order to
benefit from Fluendo's patent-licensed decoder (when available).

--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/

Andreas Gal

unread,
Mar 20, 2012, 5:51:13 AM3/20/12
to Chris Double, dev-...@lists.mozilla.org

I never landed the patch. The green means that the image is all 0,0,0 which is green in the YUV color space. If you look carefully in the left upper corner you will see some random pixels, thats the 16 byte header that contains the physical address of the actual video data. Finding the video frame works only with B2G, not with Android, thats why I never pursued this beyond a proof of concept.

Andreas

Chris Pearce

unread,
Mar 20, 2012, 4:58:44 AM3/20/12
to mozilla-...@lists.mozilla.org
On 20/03/2012 9:48 p.m., Andreas Gal wrote:
>
> I would do the following order: 1) 4) 2) 3)

With multiple people involved, there's no need to do these in serial. ;)

Chris P.

Chris Double

unread,
Mar 20, 2012, 4:50:25 PM3/20/12
to dev-...@lists.mozilla.org
On Tue, Mar 20, 2012 at 10:51 PM, Andreas Gal <g...@mozilla.com> wrote:
>
> I never landed the patch. The green means that the image is all 0,0,0 which is green in the YUV color space. If you look carefully in the left upper corner you will see some random pixels, thats the 16 byte header that contains the physical address of the actual video data. Finding the video frame works only with B2G, not with Android, thats why I never pursued this beyond a proof of concept.

Can I get access to the code for the proof of concept to do this? Or
is it available publicly somewhere?

--
http://www.bluishcoder.co.nz
0 new messages