Hardware jpeg decoding in chromium

[rbha...@nvidia.com]

Here is an initial proposal for hardware jpeg decoding in chromium on tegra platform.

In renderer process, skia side changes:

1) Add gl chromium extension or existing API to transfer jpeg encoded buffer to gpu process.  Buffer can be copied to shared memory and it can be transferred to service side (gpu thread).

2) Above extension or API can upload or share gl texture to gpu and this texture is used in skia to render image

In gpu process, chromium side changes:

1) Retrieve shared buffer which contains jpeg encoded data.

2) Add tegra jpeg decoder device in content/common/gpu/media/ . Change https://codereview.chromium.org/825863002/  is uploaded

a. Platform library can be loaded using dlopen and APIs are resolved using dlsym.

b. Mainly read header and decode functions are implemented in this file.

3) Add generic interface class to decode jpeg image in content/common/gpu/media/

a. This interface can be used to create any platform specific jpeg decoder.

4) This class loads and initializes platform specific device class by matching device node or platform library present on device file system.

Open points:

1) Skia side changes, How to integrate gl extension to transfer encoded buffer in skia?

2) Sharing decoder output to gpu. glTexImage2D may not be appropriate as it will again upload buffer which is in hardware memory.

  Dma buf extension can be used to share decoded buffer to gpu w/o copy but it uses External_OES texture and may not be supported in skia.

3) How to invoke jpeg decoder in content/comman/gpu/media from gles2_cmd_decoder.cc. Currently there is no method or framework available.

For chromeOS there is already discussion in progress to implement this feature on tegra platform. Need to sync with this.

Check bug https://code.google.com/p/chrome-os-partner/issues/detail?id=34565​

[Justin Novosad]

+ su...@chromium.org

[Pawel Osciak]

Hi,

We already have a generic interface for JPEG decoders: please see JpegDecodeAccelerator.

We also have a JPEG parser implemented and about to go in: https://chromiumcodereview.appspot.com/748023002/ (please ignore the "vaapi" part in description, it's not longer vaapi-specific). So we should not need any additional header parsing code.

Instead, your driver should implement a standard V4L2 mem-to-mem API for JPEG decoders. On Chrome side, it should be talking to a V4L2JpegDecodeAccelerator, a platform-independent V4L2 implementation of JpegDecodeAccelerator.

I'm happy to ask any detailed questions you may have.

Thanks,

Pawel

[Stéphane Marchesin]

On Tue, Jan 13, 2015 at 5:34 PM, Pawel Osciak <pos...@chromium.org> wrote:
> Hi,
> We already have a generic interface for JPEG decoders: please see
> JpegDecodeAccelerator.
>
> We also have a JPEG parser implemented and about to go in:
> https://chromiumcodereview.appspot.com/748023002/ (please ignore the "vaapi"
> part in description, it's not longer vaapi-specific). So we should not need
> any additional header parsing code.
>
> Instead, your driver should implement a standard V4L2

This isn't for Chrome OS though.

> mem-to-mem API for
> JPEG decoders.

Actually, mem-to-mem isn't quite what we need here, most importantly
it should be possible to bind the output to an eglimage to use it from
GL without an extra copy...

Stéphane

[Pawel Osciak]

On Wed, Jan 14, 2015 at 10:43 AM, Stéphane Marchesin <mar...@chromium.org> wrote:

On Tue, Jan 13, 2015 at 5:34 PM, Pawel Osciak <pos...@chromium.org> wrote:
> Hi,
> We already have a generic interface for JPEG decoders: please see
> JpegDecodeAccelerator.
>
> We also have a JPEG parser implemented and about to go in:
> https://chromiumcodereview.appspot.com/748023002/ (please ignore the "vaapi"
> part in description, it's not longer vaapi-specific). So we should not need
> any additional header parsing code.
>
> Instead, your driver should implement a standard V4L2

This isn't for Chrome OS though.

The code in the proposed CL is not ifdefed to be Android-specific and nothing in its description nor the above proposal says that it would be...

 

> mem-to-mem API for
> JPEG decoders.

Actually, mem-to-mem isn't quite what we need here, most importantly
it should be possible to bind the output to an eglimage to use it from
GL without an extra copy...

I think you might be misunderstanding, the "mem-to-mem" term refers to the device taking jpeg and decoding to raw, unless we have a way to decode a jpeg in place, you need two buffers: source JPEG and destination raw. That's what mem-to-mem is referring to... Binding is a separate topic and of course the HW should produce output to a zero-copyable buffer.

[Stéphane Marchesin]

On Mon, Jan 12, 2015 at 11:03 PM, <rbha...@nvidia.com> wrote:
> Here is an initial proposal for hardware jpeg decoding in chromium on tegra
> platform.
> In renderer process, skia side changes:
> 1) Add gl chromium extension or existing API to transfer jpeg encoded buffer
> to gpu process. Buffer can be copied to shared memory and it can be
> transferred to service side (gpu thread).

Here I think eglImage could be used; I don't think we need extra APIs.
The same way we use eglImage to import video frames from the hardware
video decoder without extra copies, we could use eglImage to import
from the jpeg decoder. After all a jpeg is just a single-frame video
:)

Stéphane

[Rahul Bhagwat]

My proposed solution is OS independent. It should benefit both android and chromeOS. I'm not getting why there should be a separate solutions for each OS?

Thanks and Regards,
Rahul Bhagwat

To unsubscribe from this group and stop receiving emails from it, send an email to graphics-dev...@chromium.org.

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------

[Pawel Osciak]

On Wed, Jan 14, 2015 at 5:40 PM, Rahul Bhagwat <rbha...@nvidia.com> wrote:

My proposed solution is OS independent. It should benefit both android and chromeOS. I'm not getting why there should be a separate solutions for each OS?

The point is that the solution should be platform/vendor-independent as well. Our goal is to minimize platform-specific code in Chrome, so we prefer to base on standard, common APIs, like V4L2, which is the standard for this on Linux.