VP8 hardware support now in Android Chrome Beta?

[Leighton Carr]

Hi WebRTC,

I just got my Nexus 5, which I know has hardware VP8 support (Snapdragon 800) and my local WebRTC screen-sharing app is now giving me around 30fps for 1920x1080.

I assume ChromeM-31 has support for the VP8 hardware decoder?

Looks amazing btw.. :)

Cheers,

Leighton.

[Serge Lachapelle]

31 does not support this. But we are working on it. 

New chip, new Android KitKat APIs to access the chip. Lots of new stuff for us to learn :-)

/Serge

--
 
---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Leighton Carr]

So it's decoding 1080p in software?  

That's kinda ridiculous for a smartphone... O.o

[Serge Lachapelle]

I can't speak for the non-webrtc stuff. WebRTC decode and encode incoming. This is brand new stuff. New chip, new driver and new APIs. Give us a minute or two please :-)

[Leighton Carr]

Of course, sorry.  I'm just equal parts excited for hardware support and amazed the Snapdragon 800 is so powerful.

[Justin Uberti]

We're also "uncomfortably excited" about support for hardware acceleration. We'll post more info as it becomes available.

[kaiduan]

Justin,

What is the issue number to track this?

Thanks,

/Kaiduan

[Ami Fischman]

Decode (landed): 305421, encode (in review): 313115.

[Kaiduan Xie]

Ami,

I took a quick look at the patches from https://code.google.com/p/chromium/issues/detail?id=305421 and 

https://code.google.com/p/chromium/issues/detail?id=313115 and have some questions,

1) For encoding, what is the patch for webrtc media stack and chrome? If I understand correctly, the frame comes from hardware encoder is compressed/encoded VP8 frame instead of raw YUV frame. The frame needs to be fed into webrtc media stack for encapsulation and sending out to network.

2) For decoding, what is the frame format in WebMediaPlayerMS::OnFrameAvailable()? For local camera preview/getUserMedia, the VideoFrame is of YUY format.

Thank you very much,

/Kaiduan

[Ami Fischman]

Kaiduan,

 

1) For encoding, what is the patch for webrtc media stack and chrome? If I understand correctly, the frame comes from hardware encoder is compressed/encoded VP8 frame instead of raw YUV frame. The frame needs to be fed into webrtc media stack for encapsulation and sending out to network.

I think you're confusing the capturer (which emits YUV) and the encoder, which (both SW & HW) emits VP8-encoded bitstream buffers.

2) For decoding, what is the frame format in WebMediaPlayerMS::OnFrameAvailable()? For local camera preview/getUserMedia, the VideoFrame is of YUY format.

HW decoder emits to (opaque to the renderer) textures and sends back to the renderer via PictureReady.  The chromium API for HW decode is VideoDecodeAccelerator.

-a

[Kaiduan Xie]

Ami,

On some platform, one camera API call can emit uncompressed YUV frame and compressed frame at the same time. The way you described is more generic.

Can you explain a bit more on the processing for incoming video frame?

For incoming video frame, the UDP packet is received in browser process, the packet is then sent to render process via IPC for de-packetization, which process decodes the video frame via hardware decoder, after decoding how the video frame is painted to the browser window?

Many thanks,

/Kaiduan

--

[Ami Fischman]

For incoming video frame, the UDP packet is received in browser process, the packet is then sent to render process via IPC for de-packetization, which process decodes the video frame via hardware decoder,

The GPU process is where MediaCodec is triggered (see AndroidVideoDecodeAccelerator).

 

after decoding how the video frame is painted to the browser window?

AVDA paints the decoded frame to a texture which is returned (by texture id) to the renderer and from there it eventually participates in compositing to get to the browser window.

-a

[Kaiduan Xie]

Ami,

Thanks for the detailed elaboration. One more question, the hardware encoding is performed in the render process right?

/Kaiduan

--

[Ami Fischman]

No, only the GPU process can talk to MediaCodec (and the hardware), because sandbox.

[Kaiduan Xie]

Ami,

Then how the YUV frame captured in browser process is encoded and sent to render process for packetization?

/Kaiduan

[Leighton Carr]

Is this why we can't get VP8 frames out of the VideoRenderer in libjingle?

[Ami Fischman]

Again I think you're confounding capture with encode.  The sequence is:

Both: Capture in browser process -> YUV frame into shared memory -> IPC to renderer process ->

SW path: encode in renderer to bitstream buffers ->

HW path: IPC to GPU process ->  encode -> bitstreambuffers back to renderer via IPC ->

Both: bitstreambuffers are packetized in renderer -> IPC to browser to be put on the network

[Kaiduan Xie]

Ami,

From your description, it seems that some changes are needed in the libjingle/webrtc media stack to handle the hardware encoded VP8 frame.

Do you have any idea when VP8 hardware encoding/decoding based webrtc is ready on Nexus 7?

Thanks,

/Kaiduan

[Ami Fischman]

From your description, it seems that some changes are needed in the libjingle/webrtc media stack to handle the hardware encoded VP8 frame.

I think you mean pre-encoded camera output.

Some work was done to support encoding cameras ~1y ago but I don't remember whether it landed or not; sorry.

Do you have any idea when VP8 hardware encoding/decoding based webrtc is ready on Nexus 7?

Nexus7 has no VP8 codec HW in either model AFAIK.

-a

[Arik Yaacob]

Is the hardware acceleration only relevant for Chrome? Is there a plan to add it to the Android AppRTCDemo?
Arik

[Alexandre GOUAILLARD]

@arik, it is not chromium specific , it is android specific. Those changes are for chromium on android to support internal android  "MediaCodec" which in turns just added HW encode/decode support for VP8 in 4.4

http://developer.android.com/reference/android/media/MediaCodec.html

The corresponding work on the appRTCDemo is ongoing. You can follow it on a separate ticket:

http://code.google.com/p/webrtc/issues/detail?id=2575

You might want to wait for the new demo code though

http://review.webrtc.org/3669004/

--

[Kaiduan Xie]

Ami,

I saw the commit 238658 today http://src.chromium.org/viewvc/chrome?view=revision&revision=238658, 

So if I build chromium from trunk, VP8 encoding/decoding is done by hardware accelerator on Nexus 5, right?

Thanks,

/Kaiduan

[Ami Fischman]

No, because of http://src.chromium.org/viewvc/chrome/trunk/src/gpu/config/software_rendering_list_json.cc?r1=238658&r2=238657&pathrev=238658 :(

(however, if you just want to test things out, you can remove that blacklist entry, or ignore the gpu blacklist via flag)

[Paul Kapustin]

Hi, could you provide any details on the "local WebRTC screen-sharing app"? Where can one get it? 

I would also like to test hardware-accelerated WebRTC on Nexus 5.

Thanks,
Paul

[William Cooley]

What's the latest status on VP8 hardware encode support in chrome for android?

Is this already available in the Chrome Beta or stable builds?

Is there an issue number that I can track? 

[Alexandre GOUAILLARD]

http://code.google.com/p/chromium/issues/detail?id=313115

https://code.google.com/p/webrtc/issues/detail?id=2575

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Alex. Gouaillard, PhD, PhD, MBA

------------------------------------------------------------------------------------

CTO - Temasys Communications, S'pore / Mountain View

President - CoSMo Software, Cambridge, MA

------------------------------------------------------------------------------------

sg.linkedin.com/agouaillard

• 
  

[William]

Both of these issues are marked as fixed/closed.
It's still not clear to me when this will land in a Chrome build that is available from the Google play store.

You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/dERL1z5dxtc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.

[Alexandre GOUAILLARD]

try with a nexus 5 ….

[PhistucK]

The first issue says that the support is disabled by default due to poor quality, so I do not think it will happen anytime soon.

☆PhistucK

[Mao, Yujie]

Hi William,

 

The VP8 hardware-accelerated support is already exist in latest Android Chrome Beta, but the encoder is not working very well till now so it’s been disabled by default, and the decoder is working on some devices like Nexus 5.

 

If you want to try VP8 hardware encoding/decoding, you can play with latest Android Chrome Beta on Nexus 5 since it has both VP8 hardware encoding/decoding support. After you installed Android Chrome Beta, please enter chrome://flags and Enable an experiment feature named “Override software rendering list”(The first item), then you can try the VP8 hardware encoding/decoding.

 

More information please see these features are in the gpu blacklist:

https://code.google.com/p/chromium/codesearch#chromium/src/gpu/config/software_rendering_list_json.cc&q=software_&sq=package:chromium&type=cs

 

Note that “MediaCodec is still too buggy to use for encoding”, “Samsung Gaxlaxy NOTE II is too buggy to use for video decoding” and “Samsung Gaxlaxy S4 is too buggy to use for video decoding”.

 

Thanks & Best regards

Jeremy

[William]

Thanks Jeremy!!
This was extremely helpful.

If I understand this correctly MediaCodec is a part of android and not the chromium code. So MediaCodec is only going to see an improvement with an Android version update and is not simply going to improve with a new build of Chrome. Is this something that is not likely to happen till Android 4.5 is out?

Regards,
William

[Mao, Yujie]

Yes, your understanding is right. But it does not mean there can’t be any improvement to Android Chrome, they both need some improvements in the future based on my knowledge.

[Serge Lachapelle]

Big improvements coming in the next few months. we'll keep this list updated. Lots of moving parts here, and it is the first time we wire up an all these various encoders for multiple platforms. 

We will also ping the list if we get stuck :-)

/Serge

[William Cooley]

I'm wondering if there's any more news on this?

Or are there new bugs related to this that I could track? 

[William Cooley]

Serge,

Are we going to see anything new related to this topic in Android Lolipop?

[Justin Uberti]

VP8 HW encode support is substantially improved in Lollipop. Nexus 5, 6, and 9 support HW encode/decode in L via MediaCodec, with other devices coming soon as they update to L.

[Leighton Carr]

Congrats and thanks to everyone involved in this.  Haven't had a chance to do any specific testing, but anecdotally my WebRTC app runs *really* well in Chrome on the Nexus 9.  I'm going to be reworking it to deliver to our internal customers early next year so will post some more about it soon.

Being able to develop 1 web app that does everything we need and runs on any platform is really a huge achievement, and a major enabler for us.  Great work and thanks again.

[IHA]

Is there support for HW codec with WebRTCDemo for Lollipop release?  For SW codec, can I use my old webRTCDemo.apk from KK on Lollipop devices?

[IHA]

Hi,

Any support for WebRTCDemo for VP8 in HW on L release?

[gla...@webrtc.org]

WebRTCDemo has VP8 HW decoding support. There is currently no plans to add VP8 HW encoding support to this sample app.

Full VP8 HW support (both encoding and decoding) is implemented in AppRTCDemo. Later also has setting option, which allows to switch VP8 HW acceleration on and off (it is on by default).