Creating render surfaces for both image and video

[ava...@gmail.com]

Hi,

I am learning the architecture of Chrome's media pipeline by browsing the source and I have reached a point where a VideoLayer is created (inside webmediaplayer_impl.cc). 

The layer is wrapped inside a webLayer again, why and what is really happening here?

Where is the actual surface created? What is the graphics context if skia is used? Can someone point me to the code/file? 

If I want to create an OpenGL surface, where do I put the code?

Thanks in advance.

[Dana Jansens]

On Tue, May 6, 2014 at 6:47 AM, <ava...@gmail.com> wrote:

Hi,

I am learning the architecture of Chrome's media pipeline by browsing the source and I have reached a point where a VideoLayer is created (inside webmediaplayer_impl.cc). 

The layer is wrapped inside a webLayer again, why and what is really happening here?

WebLayer is a wrapper around cc::Layer that is our API to blink. The declaration of the the WebLayer abstract class lives in the blink repo so that blink can know about it, while the implementation lives in chromium where it can know about cc::Layer.

Where is the actual surface created? What is the graphics context if skia is used? Can someone point me to the code/file? 

If I want to create an OpenGL surface, where do I put the code?

Perhaps you want to use a (WebExternal)TextureLayer instead. You'd create a GL texture with a context provided by the Platform API: https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/public/platform/Platform.h&l=566

These are probably intersting:

Canvas2d: https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/Source/platform/graphics/Canvas2DLayerBridge.h&q=canvas%202d%20bridge&sq=package:chromium&type=cs&l=26

WebGL: https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/Source/platform/graphics/gpu/DrawingBuffer.h&q=drawingbuffer&sq=package:chromium&type=cs&l=31

 

Thanks in advance.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[ava...@gmail.com]

Thanks Dana, seems a bit complicated and confusing.

I dont see the below examples Canvas2d and webGL being used in the current video rendering path (HTMLMediaElement->WebMediaPlayerImpl->VideoLayer). How do I enable them?

If I need to create only a openGL surface and redirect to this surface instead of the existing and still be able to use the path of WebMediaPlayerImpl::GetCurrentFrame (I am assuming this is the accelerated path). Where does the code go?

What is the need for /skia/gpu/, I understand skia is only software rendered. What is the role of GPU in SKIA?

Thanks a lot in advance.

[Dana Jansens]

On Thu, May 8, 2014 at 9:43 AM, <ava...@gmail.com> wrote:

Thanks Dana, seems a bit complicated and confusing.

I dont see the below examples Canvas2d and webGL being used in the current video rendering path (HTMLMediaElement->WebMediaPlayerImpl->VideoLayer). How do I enable them?

They are not the "video" path, but the simpler OpenGL display path. Video does not work the same at all, has VideoFrames involved and such. You'd want to look at WebMediaPlayerImpl, VideoResourceUpdater and friends.

If I need to create only a openGL surface and redirect to this surface instead of the existing and still be able to use the path of WebMediaPlayerImpl::GetCurrentFrame (I am assuming this is the accelerated path). Where does the code go?

I'm not sure what you mean, you want to draw the video frame into a texture and give that to the compositor? Then you could use a TextureLayer like canvas and the VideoFrame would just be one of your inputs.

WebMediaPlayerImpl::GetCurrentFrame is used for software or hardware video frames, the VideoFrame structure can hold either.

 

What is the need for /skia/gpu/, I understand skia is only software rendered. What is the role of GPU in SKIA?

Take a look for "ganesh" in this document: http://www.chromium.org/developers/design-documents/gpu-accelerated-compositing-in-chrome

[ava...@gmail.com]

Thanks Dana, It took me sometime to understand the flow however I am still confused.

I am following your lead "Draw the video frame into a texture and give that to the compositor? Then you could use a TextureLayer like canvas and the VideoFrame would just be one of your inputs"

I see that the webexternalTextureLayer is called from the compositor (web_compositor_support) and there is no interface to pass VideoFrame as the input directly. Is it the preparemailbox that can do this?

What is the idea behind mailbox concept?

i am sure that I am missing something basic here. can you help?

off topic: Is canvas2D bridge enabled default? or should I enable some compile/runtime switch?

Thanks in advance.

[Dana Jansens]

On Wed, May 21, 2014 at 5:14 AM, <ava...@gmail.com> wrote:

Thanks Dana, It took me sometime to understand the flow however I am still confused.

I am following your lead "Draw the video frame into a texture and give that to the compositor? Then you could use a TextureLayer like canvas and the VideoFrame would just be one of your inputs"

I see that the webexternalTextureLayer is called from the compositor (web_compositor_support) and there is no interface to pass VideoFrame as the input directly. Is it the preparemailbox that can do this?

If you pass a texture (mailbox) you'd use a cc TextureLayer via WebExternalTextureLayer. If you want to pass a VideoFrame, then you'd use a cc::VideoLayer. See the WebMediaPlayerImpl and the VideoFrameProviderClientImpl and VideoLayerImpl for how this works. Also http://www.chromium.org/developers/design-documents/video-playback-and-compositor

 

What is the idea behind mailbox concept?

Here's the documentation: https://code.google.com/p/chromium/codesearch#chromium/src/gpu/GLES2/extensions/CHROMIUM/CHROMIUM_texture_mailbox.txt&q=mailbox.*txt&sq=package:chromium&l=1

 

i am sure that I am missing something basic here. can you help?

off topic: Is canvas2D bridge enabled default? or should I enable some compile/runtime switch?

For hardware accelerated canvases. I believe they have to be large enough to trigger it.

[kenz kiran]

if it is useful, I had traced Android's HW accelerated Video Playback.

I have put a fairly detailed arch diagram and noted import pieces of code:

https://docs.google.com/document/d/1vzPIW0gAhI3m51ozwSY6g2Hz1wzWLOs3GaPGy9Y4y8w/edit?usp=sharing

-Ravi

To unsubscribe from this group and stop receiving emails from it, send an email to graphics-dev...@chromium.org.

[ava...@gmail.com]

Thanks Ravi, I am reading your doc, it looks impressive. Just curious, Are you trying to use accelerated surface for the Android media player to draw on?

[kenz kiran]

Not sure when you say "accelerated surface". If you mean that we don't paint Video Frames using Skia or CPU based libs, then "YES".

We have ported blink on our own platform, and our Arch is similar to Android i.e the real video playback stack is on Browser Process and we use StreamTexture like classes to provide "texture mailboxes" that will be used to composite Video along with other Layers.

-Ravi