H264 Hardware Accelerated Decoding of WebRTC inbound streams

[Gagandeep Singh]

Hi,

I  managed to get MojoVideoDecoder working for H.264 streams (both HLS and MP4 types) but to my horror yesterday I found in chrome://webrtc-internals that inbound video streams still get decoded via FFmpegVideoDecoder (which from my lurking around on the internet, and htop, and Chrome task manager seem to suggest is CPU based decoder). 

I am trying to run Chrome (or Chromium) on a vGPU provisioned server and will be capturing webRTC streams to push to further video processing pipelines. And as such instead of throwing more vCPUs at the problem, I'd like to be able to use the 90% remaining gpu on the server instance :) The instance is Linux (debian) with T4 Tesla GPUs.

I am at the edge of my knowledge for this issue. Any suggestions as to how I can enable hardware acceleration for webRTC H264 streams would be really appreciated. Compile from source? Experimental branch? any and all suggestions welcomed!

Best,

Gagandeep Singh

[Andres Calderon Jaramillo]

sprang@ Any reason why normal playback can use hardware acceleration but webRTC doesn't?

Gagandeep: just out of curiosity, what's your video acceleration setup? Are you using the T4 Tesla or are you using something else for video decoding? What interface are you using (V4L2, VA-API, or other)? Does the webRTC stream use the same H.264 profile as when playing an MP4?

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/075642db-1900-40c3-bceb-b33353ae3576n%40chromium.org.

[Gagandeep Singh]

Hi,

Thanks for the reply! Yes, using Tesla T4 for decoding. I was not able to gather H.264 profile, but given how mainstream Main and high are, I doubt that'd be a problem, though will check again and let you know. For interface, from what I understand it is Nvidia proprietary drivers along with a VA-API patch. 

https://www.linuxuprising.com/2021/01/how-to-enable-hardware-accelerated.html?m=1

I followed this webpage for getting the hardware acceleration up and running and you will see that it provides a link to a driver code patch for debian to enable decoding on GPU. Also decoding using MojoVideoDecoder works on both my local machine's GTX 1050 and on server's Tesla T4 GPU for normal video files.

Best,

Gagandeep

[Gagandeep Singh]

One more thing. I couldn't get webrtc to be decoded using GPU even in windows. That is more wierd given how much more compatible Nvidia is with windows.

[Gagandeep Singh]

Sorry for another email, i should gather my thoughts before pressing send. The h264 used for encoding webrtc is OpenH264. Can this be the issue?

[Andres Calderon Jaramillo]

I don't think the encoder would be a problem. Does this happen with multiple webRTC applications? Can you also try https://appr.tc/?debug=loopback&vrc=h264 ?

tmathmeyer@ Did you have to do anything fancy to get hw video decoding working with Meet?

[Gagandeep Singh]

Hi,

So did some digging after waking up. Here's all that I could find.

H264 (102, level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f;x-google-min-bitrate=120)

This is shown for the Codec in chrome://webrtc-internals for outbound stream. profile-level-id is a combination of 3 hexes, 42, 00, 1f.

https://en.wikipedia.org/wiki/Advanced_Video_Coding#Profiles

From here, the hex of 42 represents baseline profile. This is the same profile-level-id used in the appr.tc link @Andres Calderon Jaramillo posted above and our webrtc service provider. I wonder if I can force it to main profile. Nvidia gpu only support main and high profiles. Though not able to easily find what the other two hexes stand for.

Suggestions? This seems to be the issue, in all likelihood.

On Fri, Mar 26, 2021 at 1:48 PM Erik Språng <spr...@google.com> wrote:

Don't know anything about this system in particular, but quite frequently profiles seem to be the issue for H264.

Note that baseline and constrained baseline are the most common profiles used for rtc.

I have rarely used h26x stuff personally, https://groups.google.com/g/discuss-webrtc might be a better place to ask.

E

[Gagandeep Singh]

Found the meaning of other two hexes (just so anyone says I'm sloppy with research :P ) on this page:

https://www.programmersought.com/article/15615289761/

So, our webrtc service provider also supports vp9 codec. Nvidia can decode vp9 Profile 0. If I request the codec to change to vp9 will that be a surefire way to getting vp9 decoded on gpu? Though there is another issue, for the life of me I cant get nvidia to decode vp9 codec even in media-internals. Any vp9 stream in chrome --use-gl=desktop just crashes the MojoVideoDecoder. H264 started working with at least non-webrtc streams but vp9 is giving me serious troubles. VP9 shows up in chrome://gpu in Video Acceleration Info, but it doesn't show up in my vdpauinfo command. Maybe that is a clue as to what is wrong? Any suggestions for VP9 with Nvidia and Chrome would also be useful right now as that might be my only shot at this.

Thanks for the help so far!

Gagandeep Singh

[Gagandeep Singh]

Hi, 

Polite Ping. I would really appreciate if someone can help me with the vp9 codec failure in chrome on my Linux machine running nvidia GPU. Any pointers or war stories ( :P ) as to how one got this working would also be really awesome as well.

Gagandeep Singh