HW video acceleration on Linux

[binoy]

Hi 

Some clarification regarding "Hardware Accelerated Video Decoding in Linux". Your help is greatly appreciated.

I have seen couple of old post's regarding the same, re-posting this as there is no right direction for this based on the latest source tree. 

First of all it is well understood this feature is not supported out of the box in Linux build (https://bugs.chromium.org/p/chromium/issues/detail?id=137247)

Looking for a workaround way to to achieve this.

Working environment is chromium version 53, based on Arm8 architecture and running on a Renesas / powerVR based GPU and wayland back end. In this setup, high res videos shows very poor performance. So basically the video decoding is done by sw here.

Saw some reference related to chromium on chromeOS supports the hw video decoding through VAVDA / VAAPI (https://chromiumcodereview.appspot.com/9814001) and there also some reference on OpenMax based HW decoder (removed at ToT?) and there is a V4L2 based solution for chromeOS for Arm (https://codereview.chromium.org/137023008) etc.

I would like to get some reference on what would be the best way to approach this problem for an arm8 based Linux - chromium build, any reference to already proved out solution or direction would be helpful. Here the target is equipped with OMX IL (+gst-omx) and V4L2 libraries.

Thanks,

Binoy

[Dan Sanders]

Working environment is chromium version 53

This version of Chrome is nearly a year old. I can't offer detailed analysis of the state of the media stack back then, nor would I feel safe using a browser that old.

I would like to get some reference on what would be the best way to approach this problem for an arm8 based Linux - chromium build, any reference to already proved out solution or direction would be helpful. Here the target is equipped with OMX IL (+gst-omx) and V4L2 libraries.

The existing V4L and VAAPI decoders are workable on Linux. You may need to tweak them a bit if your driver isn't similar enough to what CrOS has.

There are several unfinished code reviews around that enable the use of VDAs on OS_LINUX builds, eg. https://codereview.chromium.org/1743543004/.

- Dan

[binoy]

On Saturday, 3 June 2017 05:51:40 UTC+5:30, Dan Sanders wrote:

Thanks Dan for replying me,

Working environment is chromium version 53

This version of Chrome is nearly a year old. I can't offer detailed analysis of the state of the media stack back then, nor would I feel safe using a browser that old.

 The reason behind choosing v53 is wayland, and a stable version currently we can get on ozone wayland is 53, plans are to move to latest TOT in future as the native wayland support is kind of forming up there

I would like to get some reference on what would be the best way to approach this problem for an arm8 based Linux - chromium build, any reference to already proved out solution or direction would be helpful. Here the target is equipped with OMX IL (+gst-omx) and V4L2 libraries.

The existing V4L and VAAPI decoders are workable on Linux. You may need to tweak them a bit if your driver isn't similar enough to what CrOS has.

There are several unfinished code reviews around that enable the use of VDAs on OS_LINUX builds, eg. https://codereview.chromium.org/1743543004/.

Thanks for the links Dan, V4L is one option currently i am focusing as the VAAPI support on my hardware is doubtful. 

Also the OMX libraries is available on the hardware, but i see the OMX based decoding (OmxVideoDecodeAccelerator) is removed in the latest TOT, is it known to be of any reason

- Dan

[Dan Sanders]

Also the OMX libraries is available on the hardware, but i see the OMX based decoding (OmxVideoDecodeAccelerator) is removed in the latest TOT, is it known to be of any reason

OmxVideoDecodeAccelerator was last used in M26. I can't offer any knowledge of things that happened years before I started working on hardware decode.

Presumably these devices are all using V4L or VAAPI now.

- Dan

[binoy]

Thanks Dan,

Yes, i am not much worried about the OMX path, and actually working on the V4L2 based approach similar to CrOS-arm, assuming i made necessary changes to enable the V4l2 for Linux build (libraries are loaded properly from generic_v4l2_device.cc and stubs are initialized i could see), also ensure disabling the ffmpeg based decoding using the option DISABLE_FFMPEG_VIDEO_DECODERS. 

But still did not see the v4l2 HVA is correctly been considered in the pipeline, ended up in "PIPELINE_ERROR decoder: not supported error". debugging on this now. 

Could not see a technical document on the gpu decoder pipeline, do you have any to share

[Dan Sanders]

But still did not see the v4l2 HVA is correctly been considered in the pipeline, ended up in "PIPELINE_ERROR decoder: not supported error". debugging on this now.

Once the VDA is compiling, the main hurdle is the GpuVideoDecodeAcceleratorFactory:

• GetDecoderCapabilities() must list support for the codecs, profiles, and resolutions of interest, otherwise the VDA will never be constructed.
  
• CreateVDA() must return an instance of the VDA.

Could not see a technical document on the gpu decoder pipeline, do you have any to share

There is no up-to-date documentation of the VDA pipeline that I am aware of.

- Dan

[binoy]

On Friday, 16 June 2017 01:09:57 UTC+5:30, Dan Sanders wrote:

But still did not see the v4l2 HVA is correctly been considered in the pipeline, ended up in "PIPELINE_ERROR decoder: not supported error". debugging on this now.

Once the VDA is compiling, the main hurdle is the GpuVideoDecodeAcceleratorFactory:

• GetDecoderCapabilities() must list support for the codecs, profiles, and resolutions of interest, otherwise the VDA will never be constructed.
  
• CreateVDA() must return an instance of the VDA.

Thanks a lot Dan, found actually the VDA creation is failed in the gpu decoder, and there is some problem with the driver,  following up for the vendor support on this

[binoy]

Hi Dan, got it working after modifying the V4L2 driver, currently i could see only with --in-process-gpu makes this work in Intel wayland, yet to see with M60 native wayland support which Igalia is doing