Issue 876226 in chromium: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element

[bill… via monorail]

Updates:
Cc: rak...@chromium.org bil...@chromium.org
Summary: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element

Comment #15 on issue 876226 by bil...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c15

This is not a WebVR issue - AFrame is a javascript framework that can sit on top of WebVR or the polyfill. This bug as described in #1 is talking about the behavior with the polyfill. I see the same behavior as described on linux, which doesn't support webvr.

For jsfiddle pages, it appears that the a-frame content is hosted in another iframe, and feature policy prevents that iframe from accessing webvr - so we go through polyfill on android even if webvr flags are enabled.

When the button to enter VR mode is clicked, a canvas element in the page is made fullscreen. This means that the iframe we are trying to call elementFromPoint on isn't in the tree we are making fullscreen. I don't know what is the spec-compliant behavior here, since the iframe isn't visible at this point.

Adding rakina@, who fixed the bug mentioned in #12.

Updating title to avoid confusion about vr mode.


--
You received this message because:
1. You were specifically CC'd on the issue

You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings

Reply to this email to add a comment.

[bill… via monorail]

Updates:
Components: Blink>DOM

Comment #16 on issue 876226 by bil...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c16

(No comment was entered for this change.)

[bill… via monorail]

Updates:
Status: Available

Comment #17 on issue 876226 by bil...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c17

[rak… via monorail]

Comment #18 on issue 876226 by rak...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c18

Hi, can you add some example calls to elementFromPoint/elementsFromPoint and the expected output? Not too familiar on how it should work in WebVR...

[bill… via monorail]

Comment #19 on issue 876226 by bil...@google.com: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c19

webvr isn't really involved other than a-frame (a javascript library in use on the sample page) typically uses the webvr api. However, in this case a-frame isn't able to use webvr so its polyfilling it and using fullscreen instead.

go to http://jsfiddle.net/afuybnrt/
in debug tools, switch to debug the innermost iframe (it has a string about 404), and run elementFromPoint(100,100) from console to see 4 elements.
click on the google cardboard viewer icon in the lower right iframe. this makes a canvas element fullscreen.
run elementFromPoint(100,100) from console while the canvas is fullscreen.

now see 1 element in results (html element)

The expectation in #1 is that we should see multiple elements still.

[rak… via monorail]

Updates:
Components: Blink>Layout

Comment #20 on issue 876226 by rak...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c20

Thanks for explaining! I don't think my change that was linked in #12 is the cause of this, it's mostly just affecting Shadow DOM retargeting. It seems there was some changes in Hit-testing code that takes place in a similar time, so let me add Layout to the components..

I'm OOO currently but I'll also try debugging to confirm in around a week.

[dominik.… via monorail]

Comment #21 on issue 876226 by dominik....@gmail.com: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c21

Thank you for the investigation. Did you find out any news?

[chris… via monorail]

Comment #22 on issue 876226 by chri...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c22

Is this a fullscreen bug, or just VR?

[bill… via monorail]

Comment #23 on issue 876226 by bil...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c23

Fullscreen. No Chrome VR code is running in the repro - there is a javascript library running that pretends to implement the VR API by using fullscreen and reading sensor data.

[chris… via monorail]

Updates:
Components: -Blink>DOM -Blink>WebXR>VR Blink>Fullscreen
Status: Untriaged

Comment #24 on issue 876226 by chri...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c24

(No comment was entered for this change.)

[msten… via monorail]

Updates:
Owner: dtapu...@chromium.org
Status: Assigned

Comment #25 on issue 876226 by mste...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c25

[dtapu… via monorail]

Updates:
Cc: dom...@chromium.org foo...@chromium.org
Labels: OS-Chrome OS-Linux

Comment #26 on issue 876226 by dtapu...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c26

I bisected this to: https://chromium.googlesource.com/chromium/src/+/818e182ca

It is about propagating inert attribute into subframes.

So this appears to be working according to the spec. In that since a element is going into the top layer the rest of the document and subframes become inert and therefor nothing participates in hit testing.. And elementsFromPoints returnings the document element.

+domenic/foolip to correct me if I am wrong. But this seems to be correct so I'm guessing this is a WontFix.

reporter@ do you have a use case where hit testing an iframe should work?

[dtapu… via monorail]

Updates:
Labels: -Needs-Bisect -TE-Hardware-Dependency

Comment #27 on issue 876226 by dtapu...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c27

(No comment was entered for this change.)

[dominik.… via monorail]

Comment #28 on issue 876226 by dominik....@gmail.com: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c28

How's possible that an iframe is taken different way in full screen mode than in regular? If I'm able to target the elementsFromPoint on regular mode, why am I unable when in fullscreen mode -- shouldn't it act the same?

This is a critical issue that affects our product, please add back the behavior allowing to get proper element(s) from elementsFromPoint called on iframe when in fullscreen mode.

[dtapu… via monorail]

Comment #29 on issue 876226 by dtapu...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c29

Is there a reason you cannot put the canvas in a div and that iframe inside the div and fullscreen the div instead of the canvas?

[dtapu… via monorail]

Comment #30 on issue 876226 by dtapu...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c30

Or fullscreening the document (as well will work instead of the canvas element)

[dome… via monorail]

Comment #31 on issue 876226 by dom...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c31

Right, this seems to be working as intended. When something else is fullscreened on top of the iframe, elementsFromPoint() will not return the iframe. This follows from the spec for elementsFromPoint():

> For each layout box in the viewport, in paint order, starting with the topmost box, *****that would be a target for hit testing***** at coordinates x,y even if nothing would be overlapping it, when applying the transforms that apply to the descendants of the viewport, append the associated element to sequence.

from https://drafts.csswg.org/cssom-view/#dom-document-elementsfrompoint, emphasis mine

When fullscreen is on top of something, nothing besides the fullscreen element (and its subtree) are a target for hit-testing.

That said, this may not be an intended consequence by the spec writers, so folks might want to discuss that further. In either case, it would be good to add a test and ensure that we are interoperable across browsers, one way or another.

[dtapu… via monorail]

Updates:
Status: WontFix

Comment #32 on issue 876226 by dtapu...@chromium.org: elementFromPoint / elementsFromPoint called on i-frame when on fullscreen Mode returns only root element
https://bugs.chromium.org/p/chromium/issues/detail?id=876226#c32

Here is an example of it working. I added a click listener to the a-scene so you need to just click it and not the glasses to go fullscreen.

http://jsfiddle.net/dtapuska/L938kzqt/