Google Groups

hardware-accelerated audio/video decoding in Gecko (bug 714408)


Andreas Gal Mar 12, 2012 12:28 AM
Posted in group: mozilla.dev.platform

I want to land bug 714408 on mozilla-central as soon as I get review for it. It adds hardware-accelerated audio/video decoding support to Gecko using system decoders already present on the system. Android, for example, ships by default with a number of decoders, and in particular for such mobile devices we really have to use these hardware-accelerated decoders for good battery life (and performance).

Initially this will be enabled on Gonk (B2G). In a few weeks we will add support for Android as well. We will support decoding any video/audio format that is supported by existing decoders present on the system, including H.264 and MP3. There is really no justification to stop our users from using system decoders already on the device, so we will not filter any formats.

The system decoders are loaded using a plugin framework called MPAPI, which is currently internal to Gecko, but we might expose it via NPAPI to external decoder providers as well. MPAPI helps us deal with incompatibilities in systems decoders (without risking Gecko not starting up which would happen if we try to link against the system decoders directly).

Currently MPAPI delivers audio and video frames back to the browser and fully integrates with our rendering pipeline (CSS and all). On Android we might have to add a second video path using overlays which would only work with a small subset of CSS since extracting video frames isn't supported on all versions of Android (and all devices).

I don't think this bug significantly changes our position on open video. We will continue to promote and support open codecs, but when and where existing codecs are already installed and licensed on devices we will make use of them in order to provide people with the best possible experience.

Let me know if you have any questions.

Andreas