Fake full screen mode of media playback

[William Xie]

Dear all,

 I have one question about the HTML5 media player full screen playback, 

For phone device, the media UI is using standard media UI, see 1.png,

while for some tablet device, the media UI is switched to customized media UI, see 2.png, 

in this customized media UI mode, the fullscreen mode of media playback is actually fake full screen mode of media player.

After some discussion and investigation with media guys, we found that after user click the full screen button, it is the DIV element becomes full screen, not the VIDEO element. That's the root cause.  So anyone knows if this is a design feature or mistake? 

This issue is very common in some other video provider websites besides Youtube.com, for example,

our product team has the power KPI for browser by streaming video playback. Their test website is www.youku.com,

test browsers are chrome and UC browser (https://play.google.com/store/apps/details?id=com.UCMobile.intl&hl=en). 

The power data shows that chrome consumes much more power in full screen.

So can we just force video element to be full-screed instead of the DIV and use the standard media control bar in full screen mode after click the full screen button?

From my local testing with hacking, it works fine with acceptable user experience.

Is there any other good solution for this issue?

Thanks

William

[Chris Watkins]

I think I can provide some context on the question: For fullscreen video on Android we decode to a power efficient SurfaceView underlay rather than compositing the frames with opengl. But we can only use the underlay if we know we don't need to read the frames back, or apply transforms to them, so we only do it today when HTMLVideoElement is fullscreened. 

In many cases, sites fullscreen a <div> wrapping a <video> because it's a reliable way to implement custom media controls. So the question is: how feasible is it to detect this situation in blink, and fullscreen the video instead of the div? The trade off would be losing custom controls, and any other styling, in exchange for power efficiency.

[David Dorwin]

On Wed, Apr 27, 2016 at 6:08 PM, Chris Watkins <wa...@chromium.org> wrote:

I think I can provide some context on the question: For fullscreen video on Android we decode to a power efficient SurfaceView underlay rather than compositing the frames with opengl. But we can only use the underlay if we know we don't need to read the frames back, or apply transforms to them, so we only do it today when HTMLVideoElement is fullscreened. 

My understanding is that Ozone handles this correctly on a per-frame basis but that we don't use Ozone on Android because of <platform limitations>. The platform is supposed to be improved on M and maybe further on N.

This seems related to the replace hole-punching/inline video surfaces/L1 on Android issues. Investigation and improvements on any one should benefit (move forward, if not entirely solve) all of the others.

[Elliott Sprehn]

I don't think we should mess with pages here, we need to just fix the implementation to correctly detect no overlap and layer promote on Android.

Pages use this for custom controls, overlay bubbles, ads, captions, video annotations, links over the video, and more. Ex. You'd break the "Skip this ad" experience in YouTube.

[William Xie]

Thanks Elliott,

The problem is the customized player UI sends full screen event to a DIV not the VIDEO element after user click the full screen button, hence media player isn't aware of the full screen event at all.

This issue is not Android specific, but other OS like Windows.

We need to figure out a appropriate way to notify media player to be full-screened after user click the full screen button on the customized player UI.

Should we broadcast full screen event to each child node if a container node like DIV gets a full screen event?

Thanks

William

[Elliott Sprehn]

I don't think the video should go full screen, that would hide the custom player UI and all the overlays I listed in my previous email.

We should be doing layer hoisting on the video when nothing is occluding it. There's no reason to hook that to full screen, and it gets us the power savings even when the video inside a YouTube page is playing non full screen.

[Frank Liberato]

"layer hoisting": i'm not very familiar with blink's compositing.  The closest match I have for "layer" is that video elements return CompositingReasonVideo for hw accelerated rendering (full screen or not) from LayoutVideo::additionalCompositingReasons, which results in a separate (cc) VideoLayer eventually.  Is this what you're referring to, Elliot?

thanks

-fl