Why does WebRTC not support Hardware acceleration

[Sebastian Kunz]

Hello,

As far as I know webrtc does not support hardware encoding/decoding. You have to implement your own encoder and go from there, which works fine for me. However I am wondering why that is. Why does webrtc not support hardware acceleration. When decoding the stream in the browser the decoding runs on my gpu, meaning (in my case) chrome implemented their own gpu decoder. So why does google not support that in the first place with webrtc, if they have to implement it themselfs later on? Can anyone clarify?

Thank you,

Sebastian

[Alexandre GOUAILLARD]

On Thu, Aug 29, 2019 at 6:56 PM Sebastian Kunz <sebasti...@precipoint.de> wrote:

Hello,

As far as I know webrtc does not support hardware encoding/decoding.

You might want to take another, closer, look. 

[...]

 

When decoding the stream in the browser the decoding runs on my gpu, meaning (in my case) chrome implemented their own gpu decoder. So why does google not support that in the first place with webrtc, if they have to implement it themselfs later on? Can anyone clarify?

libwertc standalone, and webrtc implementation in browsers are different by design.

If you refer to the architecture page of the official website (https://webrtc.org/architecture/), you will see that some parts MUST be implemented by the browser, namely network, capture and rendering, among others.

If you add to that that:

- the secure sandboxing of each tab forces the capturing and the encoding to work in different process, forcing the usage of a different memory model than a standalone library which would have shared-memory,

- the echo cancellation must be at browser level to accommodate audio sources coming from different tabs (and keyboard noise), 

- a few others things, some of them described here: https://www.slideshare.net/alexpiwi5/webrtc-browsers-n-stacks-implementation-differences. (It's 3 years old, but the concepts are still the same)

then it makes a lot of sense. 

• 
  

[Sebastian Kunz]

Thank you for your answer!

You mentioned shared memory. And I am wondering how to make use of shared memory with webrtc. I am capturing frames using the desktop_capturer module, but it copies the frames into system memory. I noticed that there is a sharedMemoryFactory, that can be used by the capturer. I want the frame to be allocated on gpu memory so that I don't need any copying when passing it to the hardware encoder. I am using an Nvidia GPU, how would I go along with it? Can you give me any pointers on getting started? I reckon I need to implement the SharedMemoryFactory, but how do I allocate memory on the gpu in the first place?

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAHgZEq402TP--mhC%2B0q061Am4N7XDhPCpx%3D4jrsvm-hTkoyDfw%40mail.gmail.com.

[Alexandre GOUAILLARD]

most of the code is in chrome, and do not used shared memory.

you will have to implement it yourself, and possibly port code back from chrome to standalone libwebrtc.

you might want to look at the "remoting" part of chrome, which uses libwebrtc for desktop capture (remote debugging, chromecast, ..) and has a different pipeline.

look at the different frame types, and you will see one that is in GPU memory (kNative something from the top of my head). That should be your entry point to understand what needs to be done.

Be carefull about the RTP / encoder coupling ..... the capturer-to-encoder part is far from being the most sensitive part of the pipeline.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAMvyZZgMpNBmDaE_6%3DetkmOjmuMG19evuhKT-HpcaxSfa_7gig%40mail.gmail.com.

--

Alex. Gouaillard, PhD, PhD, MBA

------------------------------------------------------------------------------------

President - CoSMo Software Consulting, Singapore

------------------------------------------------------------------------------------

sg.linkedin.com/agouaillard

•