Trusted video path with chromium on embedded ARM

[rebilla...@gmail.com]

Hi All,

We are currently building an STB using an ARM SOC with trustzone.
The STB is just running a browser and the interface is therefore HTML5 based.
For the video we are using DASH and Media Source.

Thanks to the linaro guys and their recent demo, we have been able to run CEF, with aura backend and a specific EGL ozone platform.
So, we have now an up and running browser which plays video correctly in our custom HTML5 player.
The problem is that the video is decoded in software using FFMPEG, which is not really good regarding to performances but also cannot allow us to be L1 certified.
For the performances issues, we could implement in the ozone platform a video decoder, but it won't change anything about security concerns.

For their demo, Linaro succeeded in implementing a secure and hardware accelerated video path using gstreamer/v4l2 and PPAPI.
The problem is that we cannot use a video tag and media source with this solution.

So the challenge we are facing is the integration of a secure video path in the browser which can be used by classic HTML5 video tag and media source.
We have a secure player provided by the manufacturer and a simple user-space library to interface with it, but we don't really know how to integrate it inside the chromium code.
We took a look in the media implementation and the video pipeline but it is quite complicated and we don't have any clue on the required modifications.

Therefore, can anyone provide any idea or pointer on how to implement it ?

Best regards,

Aubin

[Andrew Scherkus]

+alexst

I believe one of the goals of having an ozone-provided video decoder is to permit tighter integration with hardware video decoding and compositing and providing a secure path for video.

Does that not work well for the user-space library you're talking about?

Andrew

[Aubin REBILLAT]

Hi Andrew,

Thank you for helping out again.

The ozone provided video decoder in MediaOzonePlatform is of type VideoDecodeAccelerator. 

As i understood from the code, the job of a VideoDecodeAccelerator is to decode frames and map them to GL textures.

Then those textures are transported through several classes before being rendered.

Therefore, using this method, the video path is not secure, the decoded frames are accessible in chrome and therefore to any malicious software.

The recommended method using arm, is to decode in the trustzone (hardware protected memory area).

Using this method, the decrypted frames never get out of this hardware sandbox from decode to rendering.

This is where the library is useful, it is an interface to a secure player which provides :

- functions to initialize and feed the player with elementary streams

- functions to change the player state : play, pause, seek, ...

- events such as timing, aspect ratio, buffer size, ...

This does not correspond to the VideoDecodeAccelerator interface, as no decoded frame is returned and the player is the one tasked to render on the framebuffer.

Maybe using a punching hole method would do the trick ? 

The only problem i see with it, is that there would be 2 separate player running concurrently without synchronization :

- The secure one from the manufacturer, effectively rendering frames

- The chromium one which would manipulate "hole" frames and render audio

Please, feel free to correct me if i'm wrong.

Best regards,

Aubin

[Alex Sakhartchouk]

Hi Aubin,

This is a very popular topic lately. :)

So we do more than just provide ability to render video frames on the GPU, we have an interface that allows you to decode a frame and have it returned to you via the ozone ScheduleOverlayPlane interface. 

The way we are using it on ChromeOS is to have overlays as an optimization, but still be able to render the frame on the GPU for fancy CSS transitions and such.

VAAPI is using this now on intel, a starting point to look at is here. https://code.google.com/p/chromium/codesearch#chromium/src/content/common/gpu/media/vaapi_picture.cc&q=vaapi_pi&sq=package:chromium&l=1

For your arm SOC, you could decode the frames using your custom library give handles to them to Chrome and have those returned to you just before swap buffers to be displayed in your trust zone code. It's unfortunate that the GL fallback isn't available to you as it limits which parts of the HTML spec you can use, but for screen aligned video you should be able to keep chrome mostly unmodified and stay in the trust zone.

I'd be happy to guide you through ozone bits if you have further questions.

Thanks,

Alex

[Aubin REBILLAT]

Hi Alex,

Thank you very much for your inputs. This is indeed a good solution to implement a trusted video path with a minimum of modifications.

In this case, the library won't be useful at all, as it is just an interface to a low level video player.

So, as i understand, it could work like that :

- The demuxer from the media pipeline would provide encoded buffers containing frames to the ozone platform using the VideoDecodeAccelerator

- The ozone platform, interfaced with the trustzone code, would decode these buffers and extract frames. An arbitrary handle would be assigned to each frame.

- Those handles would then be given back to chromium to live in the media pipeline

- When it would be time to display a frame, chromium would use this "ScheduleOverlayPlane" interface to notify the ozone platform which frame should be displayed.

- Finally on the ozone platform, the notification would be forwarded to the trust zone code which will be in charge of rendering the frame

Feel free to correct me or add details !

Additionally, I took a look at the webmedplayer_impl.* files. How difficult do you think it would be to replace this implementation with our own (backed by the library and the low level video player) ?

There is also code from chromecast in chromium, especially the CMA part that seems to expose interface corresponding to our use. What do you think of it ? Is it usable ?

Thank you very much,

Aubin

[Alex Sakhartchouk]

​Hi Aubin,

​

 

So, as i understand, it could work like that :

- The demuxer from the media pipeline would provide encoded buffers containing frames to the ozone platform using the VideoDecodeAccelerator

- The ozone platform, interfaced with the trustzone code, would decode these buffers and extract frames. An arbitrary handle would be assigned to each frame.

- Those handles would then be given back to chromium to live in the media pipeline

- When it would be time to display a frame, chromium would use this "

​​

ScheduleOverlayPlane" interface to notify the ozone platform which frame should be displayed.

- Finally on the ozone platform, the notification would be forwarded to the trust zone code which will be in charge of rendering the frame

​Your understanding is correct. The interface we expose for the handle is NativePixmap​. The expectation is that you provide either a dmabuf fd or an EGLClientBuffer containing video data to Chromium. This way we could render the frame on the GPU using an EGLImage if you are doing a CSS transformation or if your overlay engine can't handle some scenario.

In your particular case, it sounds like you never want video data accessible to the gpu, so you could provide a small 16x16 (or similar) dummy buffer or maybe even a low-res frame for chrome to use as an EGLClientBuffer. You could stash the hi-res internal handle in your NativePixmap implementation, which will be given back to you via ​ScheduleOverlayPlane. This way, Chrome has something to display as a fallback and your video data remains private in your trust zone.

 

Feel free to correct me or add details !

Additionally, I took a look at the webmedplayer_impl.* files. How difficult do you think it would be to replace this implementation with our own (backed by the library and the low level video player) ?

There is also code from chromecast in chromium, especially the CMA part that seems to expose interface corresponding to our use. What do you think of it ? Is it usable ?

Thank you very much,

Aubin

I'm afraid ​I'm a bit out of my element on this particular issue, Andrew should know more​. This topic had come up in the past, but I'm not sure about the exact status of things.

Hope this helps,

Alex

[Andrew Scherkus]

On Mon, Mar 9, 2015 at 11:10 AM, Alex Sakhartchouk <ale...@google.com> wrote:

​Hi Aubin,

​

 

So, as i understand, it could work like that :

- The demuxer from the media pipeline would provide encoded buffers containing frames to the ozone platform using the VideoDecodeAccelerator

- The ozone platform, interfaced with the trustzone code, would decode these buffers and extract frames. An arbitrary handle would be assigned to each frame.

- Those handles would then be given back to chromium to live in the media pipeline

- When it would be time to display a frame, chromium would use this "

​​

ScheduleOverlayPlane" interface to notify the ozone platform which frame should be displayed.

- Finally on the ozone platform, the notification would be forwarded to the trust zone code which will be in charge of rendering the frame

​Your understanding is correct. The interface we expose for the handle is NativePixmap​. The expectation is that you provide either a dmabuf fd or an EGLClientBuffer containing video data to Chromium. This way we could render the frame on the GPU using an EGLImage if you are doing a CSS transformation or if your overlay engine can't handle some scenario.

In your particular case, it sounds like you never want video data accessible to the gpu, so you could provide a small 16x16 (or similar) dummy buffer or maybe even a low-res frame for chrome to use as an EGLClientBuffer. You could stash the hi-res internal handle in your NativePixmap implementation, which will be given back to you via ​ScheduleOverlayPlane. This way, Chrome has something to display as a fallback and your video data remains private in your trust zone.

 

Feel free to correct me or add details !

Additionally, I took a look at the webmedplayer_impl.* files. How difficult do you think it would be to replace this implementation with our own (backed by the library and the low level video player) ?

There is also code from chromecast in chromium, especially the CMA part that seems to expose interface corresponding to our use. What do you think of it ? Is it usable ?

In general I advocate against re-implementing WebMediaPlayer as it's a hard-to-get-right API and likely results in web compatibility issues. We've been working with the chromecast team on the CMA path that should help provide a lower-level and easier-to-implement set of interfaces than WebMediaPlayer that should also help preserve web compatibility.

 

Andrew 

[Aubin REBILLAT]

Thank you Alex, for these explanations, this is quite clearer.

However, We want to decode UHD streams but the framebuffer is not large enough for it so the low level player library is our best solution for now.

Nevertheless, the ScheduleOverlayPlane method as a fall back implementation could be useful if other methods fail.

Andrew, do you know if the CMA path is usable yet ? And starting in which version of chromium ?

This solution, if available, would be of great help to us.

Thank you both for your fast and helpful answers.

Aubin

[Andrew Scherkus]

On Tue, Mar 10, 2015 at 2:26 AM, Aubin REBILLAT <rebilla...@gmail.com> wrote:

Thank you Alex, for these explanations, this is quite clearer.

However, We want to decode UHD streams but the framebuffer is not large enough for it so the low level player library is our best solution for now.

Nevertheless, the ScheduleOverlayPlane method as a fall back implementation could be useful if other methods fail.

Andrew, do you know if the CMA path is usable yet ? And starting in which version of chromium ?

This solution, if available, would be of great help to us.

+servolk@ may be able to provide more detail about the current state of CMA in trunk.

Andrew

[Sergey Volk]

CMA is usable and stable, it's currently the main media pipeline path used for most streams on Chromecast. But you have to be aware that:

1. There are only a few private implementation of CMA backends (i.e. interfaces like chromecast::media::AudioPipelineDevice in chromecast/media/cma/backend/audio_pipeline_device.h and chromecast::media::VideoPipelineDevice in chromecast/media/cma/backend/video_pipeline_device.h, plus a few related interfaces all defined in chromecast/media/cma/backend/). Those private implementations live in private Chromecast source tree. Since they rely on private hardware vendor-provided APIs, we can't make them public easily. But in theory you could implement your own CMA backend.

2. CMA path relies on the video hole feature (search for VIDEO_HOLE in Chromium source code). At the moment CMA backends act as media sinks, they receive audio/video data and send that data straight to the hardware, never receiving anything back. CMA assumes media is going to be rendered into a hardware video plane and that plane is positioned on the screen via chromecast::media::VideoPlane APIs and is made visible through the graphics layer by using a video hole rendered over the HTMLMediaElement.

[Aubin REBILLAT]

Hi Sergey, 

This is great news for us. It is actually what we want to achieve : Display directly from our player without sending back frames and the VIDEO_HOLE feature corresponds to our need.

On the practical side, we are using CEF extended with aura windowing support and custom ozone platform.

We don't know about the dependencies coming with using "chromecast=1" and their (possibly bad) interactions with CEF.

Do you think that it is feasible to use the CMA path with CEF without building chromecast specific code in chromium ?

Thank you for your inputs,

Aubin

[Sergey Volk]

Hi Aubin,

CEF is Chromium Embedded Framework, right? I'm not very familiar with it, we are using Chromium directly. As far as I understand it should still be feasible to use CMA with CEF, at least I can't think of any issues that would prevent that, it's just that we haven't tried. Earlier versions of CMA lived in Chromecast's private source tree and had some internal dependencies, but when we were upstreaming CMA code we have made some changes to make it easier to use CMA outside of Chromecast. So at least in theory it should be possible to do.

[Aubin REBILLAT]

Hi Sergey,

Indeed, CEF is Chromium Embedded Framework.

I tried to compile with chromecast=1 and it actually passes but without any CMA code compiled.

So i played around with GYP files and i managed to compile with the crhomecast code.

The problem is that there is multiple definition : prefs variables, content::ContentMainDelegate, content::SetupSignalHandlers, ...

This is to be expected, as there is redundant code for launching chrome inside CEF and chromecast;

So, by tweaking again the gyp files, i manage to compile CMA without any multiple definitions, but the CMA path is never used when playing a video.

How is the CMA pipeline integrated inside chromium classic pipeline ? 

Is there any GYP_DEFINES to use ?

 Is there some code to modify inside chromium ?

Thank you for your answer,

Aubin

[Jesse Gunsch]

Hi Aubin,

 The hook point is in CastContentRendererClient::CreateMediaRendererFactory. Assuming you have your own ContentRendererClient implementation, you should be able to inject a media::RendererFactory instance that creates chromecast::media::CmaRenderer instances on request.

The usage in RenderFrameImpl should make the hook clear to indicate how WebMediaPlayerImpl ends up getting the CmaRenderer instance to use.

[Aubin REBILLAT]

Hi Jesse,

Thank you for your inputs.

The "ContentRendererClient" implementation was the CEF one so I modified it to return chromecast::media::CmaRenderer instances with media::RendererFactory.

It's compiling and the CMA code is called : Perfect.

I'll work on my CMA backend implementation for now.

Aubin

[Aubin REBILLAT]

Hi All,

I managed to use my library with chromium so i think it could be interesting to share my experience.

First of all, CEF and CMA don't mix well together, they both define their own delegates (ContenBrowserClient, ContentRendererClient, ...).

I tried to mix the code between the two but i ended up having weird behavior with chrome and no video.

So I tried another approach : Implement my own media::Renderer

In the CEF's ContentRendererClient, the method "CreateMediaRendererFactory" was not overridden, so i implemented it with the corresponding factory class.

Then, I implemented the Renderer which serves as an interface between chromium's media implementation and my custom library.

It was a bit hard due to my lack of knowledge in the chromium's code, but i finally managed to make it work.

Thank you very much for all your help !

Aubin