Intent to Implement: IntersectionObserver V2 -- Occlusion Reporting

[Stefan Zager‏]

Intent to Implement: IntersectionObserver V2 -- Occlusion Reporting

Contact emails

sza...@chromium.org, chri...@chromium.org

Explainer

This is a copy of the current explainer doc for IntersectionObserver, with edits describing the proposed additional functionality.

Design doc/Spec

This is a (slightly old) copy of the IntersectionObserver spec, with edits describing the proposed spec changes.

Summary

Currently, IntersectionObserver will report the bounding box of a target element relative to the containing viewport, after applying all intervening overflow and CSS clips. However, it does not consider whether a target element has been painted over (“occluded”) by other content, or whether any visual effects have been applied which may alter or obscure the element’s display. The proposal is to augment IntersectionObserver to report information about occlusion and visual effects, in order to provide a strong guarantee about the visibility of a target element in the rendered output.

Motivation

The primary motivation is to eliminate common patterns of fraud and abuse on the web, and to enable trust relationships between embedded third-party iframes and their host documents. Specifically, it gives the iframe a strong guarantee that its content is visible on screen, and has not been painted over or altered in any way by the embedding document.

These slides, from a lightning talk at BlinkOn in 2017, illustrate these use cases in a bit more detail.

Risks

Interoperability and Compatibility

Chrome, Edge, and Firefox have shipped V1 of IntersectionObserver; it is under development in WebKit. The V2 proposal is a relatively small change to the spec, but it is unclear whether other browsers will support or implement it. We have reached out to the other browser vendors to get feedback on the V2 proposal, but haven’t yet received any replies.

Edge: No signals

Firefox: No signals

Safari: No signals

Web developers: Feedback from Google-internal web development teams has been very positive.

Ergonomics

This proposal will not change the behavior or ergonomics of IntersectionObserver in any meaningful way. Occlusion detection is likely to be more expensive to compute than the existing V1 functionality, hence the default behavior will be not to do occlusion detection; it must be explicitly requested when using the API.

Activation

IntersectionObserver is currently used in approximately 15% of page views. Activating the new functionality simply requires an additional flag to the IntersectionObserver constructor, and examining a new boolean field in each notification.  Should be super easy.

Debuggability

It may be useful for DevTools to report the reason(s) why a target is being reported as “possibly occluded.” Currently there is nothing specific planned in this regard, but that may change as we receive feedback from testers.

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Requesting approval to ship?

No.

https://www.chromestatus.com/feature/5878481493688320

[Emil A Eklund‏]

Super exciting, can't wait to see this.

[Ilya Grigorik‏]

Ditto! Thank you for working on this, can't wait to test it out!

On Fri, May 4, 2018 at 1:42 PM Emil A Eklund <e...@chromium.org> wrote:

Super exciting, can't wait to see this.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADu_oUCdkDQNyM6cnVpszNH0aez5iL8DbZx9h9idqmKxS7qiNA%40mail.gmail.com.

[David Benjamin‏]

From the slides, it seems one use case is to prevent click-jacking, which means you care about both element being visible, and the duration it was visible for (otherwise I could hide your iframe and then flash it visible just before the user clicks). Is the intent that the iframe listen for the event and check timestamps to compute this? (Having that be the iframe's job to compute the duration seems reasonable enough to me, though I'm no expert on click-jacking. Just wanted to confirm I understood correctly.)

I am a little dubious that this alone will make it okay to trust events delivered to iframes, though it certainly helps. For instance, suppose the iframe showed the user a dialog with two buttons. An attacker could move the iframe to position the Accept button where the Decline button used to be just before the user clicks.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CADXXVKoDVBLeS7wv94Tv0zL4undiJD-M78Vc3VE6D8wU5T%3DjNA%40mail.gmail.com.

[Stefan Zager‏]

On Fri, May 4, 2018 at 2:10 PM, David Benjamin <davi...@chromium.org> wrote:

From the slides, it seems one use case is to prevent click-jacking, which means you care about both element being visible, and the duration it was visible for (otherwise I could hide your iframe and then flash it visible just before the user clicks). Is the intent that the iframe listen for the event and check timestamps to compute this? (Having that be the iframe's job to compute the duration seems reasonable enough to me, though I'm no expert on click-jacking. Just wanted to confirm I understood correctly.)

It will be the javascript developer's responsibility to check that the element has been visible for enough time for the click to be legitimate.

 

I am a little dubious that this alone will make it okay to trust events delivered to iframes, though it certainly helps. For instance, suppose the iframe showed the user a dialog with two buttons. An attacker could move the iframe to position the Accept button where the Decline button used to be just before the user clicks.

I recall some discussion a while back about a proposed intervention in chromium to suppress input events to iframes that have moved within the embedding page recently (for some definition of "recently"). I don't know whether that actually happened, but it would address the problem you mention.

Probably the biggest obstacle to trusted interaction with iframes is the lack of a URL bar on iframes, and this proposal doesn't help that at all. But, as you say, it's a step in the right direction.

[Daniel Bratell‏]

The main goal (from my point of view) with IntersectionObserver has always been to get ad networks in particular to stop using all kinds of hacks that slow down the browser and dramatically hurt the user experience. Do you know if they will actually do so? Maybe they have already replaced heavy scripts with IntersectionObserver v1?

/Daniel

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHOQ7J_nO9SPa_vr5LApMFdW46k50fnT6ZY7tbdg30rYpknYmw%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[Stefan Zager‏]

On Mon, May 7, 2018 at 8:40 AM, Daniel Bratell <bra...@opera.com> wrote:

The main goal (from my point of view) with IntersectionObserver has always been to get ad networks in particular to stop using all kinds of hacks that slow down the browser and dramatically hurt the user experience. Do you know if they will actually do so? Maybe they have already replaced heavy scripts with IntersectionObserver v1?

The usage numbers for v1 would suggest that the ad networks (and lots of other people) have been using v1 pretty eagerly, despite the fact that v1 doesn't offer any information that can't also be gotten via other means (it's just that the "other means" tend to degrade performance). By contrast, v2 offers functionality that is currently impossible to get by other means; and since it is a small incremental change to the existing v1 spec, my hope is that ad networks will be just as eager to use it.

[Ken Buchanan‏]

On Sat, May 5, 2018 at 6:59 PM, Stefan Zager <sza...@chromium.org> wrote:

On Fri, May 4, 2018 at 2:10 PM, David Benjamin <davi...@chromium.org> wrote:

From the slides, it seems one use case is to prevent click-jacking, which means you care about both element being visible, and the duration it was visible for (otherwise I could hide your iframe and then flash it visible just before the user clicks). Is the intent that the iframe listen for the event and check timestamps to compute this? (Having that be the iframe's job to compute the duration seems reasonable enough to me, though I'm no expert on click-jacking. Just wanted to confirm I understood correctly.)

It will be the javascript developer's responsibility to check that the element has been visible for enough time for the click to be legitimate.

 

I am a little dubious that this alone will make it okay to trust events delivered to iframes, though it certainly helps. For instance, suppose the iframe showed the user a dialog with two buttons. An attacker could move the iframe to position the Accept button where the Decline button used to be just before the user clicks.

I recall some discussion a while back about a proposed intervention in chromium to suppress input events to iframes that have moved within the embedding page recently (for some definition of "recently"). I don't know whether that actually happened, but it would address the problem you mention.

Yes, that intervention has been proposed and is currently assigned to me but I haven't had time to work on it for the last several months.

I believe that if that intervention is shipped, and IOv2 can achieve its stated goals, it would cover all of the known 'easy' modes of clickjacking. Other vectors are either possible to defend against already (e.g. custom mouse cursors), or are not reliably available to attackers (e.g. popunders).

 

 

Probably the biggest obstacle to trusted interaction with iframes is the lack of a URL bar on iframes, and this proposal doesn't help that at all. But, as you say, it's a step in the right direction.

That is an orthogonal problem that basically reduces to phishing, but yes. Right now a website can display a fake embedded widget and ask for login or payment credentials, and the concern is that if similar real widgets actually existed on reputable websites, users might become more likely to fall for fake ones. However, if you look at an application like OpenYOLO they have already designed around that problem, while they are still held back by substantial clickjacking risk. 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHOQ7J_nO9SPa_vr5LApMFdW46k50fnT6ZY7tbdg30rYpknYmw%40mail.gmail.com.

[David Benjamin‏]

There are also uncredentialed scenarios where the user doesn't necessarily need to know the widget is real or not. If I have some social media button iframe that is purely a button, the real social media button needs to know that it wasn't a click was genuine before acting on it, but if a malicious embedder shows a fake one, it doesn't learn the user's credentials or anything. It only learns that the user wanted to click on some button.

[Ken Buchanan‏]

On Sat, May 5, 2018 at 6:59 PM, Stefan Zager <sza...@chromium.org> wrote:

On Fri, May 4, 2018 at 2:10 PM, David Benjamin <davi...@chromium.org> wrote:

From the slides, it seems one use case is to prevent click-jacking, which means you care about both element being visible, and the duration it was visible for (otherwise I could hide your iframe and then flash it visible just before the user clicks). Is the intent that the iframe listen for the event and check timestamps to compute this? (Having that be the iframe's job to compute the duration seems reasonable enough to me, though I'm no expert on click-jacking. Just wanted to confirm I understood correctly.)

It will be the javascript developer's responsibility to check that the element has been visible for enough time for the click to be legitimate.

 

I am a little dubious that this alone will make it okay to trust events delivered to iframes, though it certainly helps. For instance, suppose the iframe showed the user a dialog with two buttons. An attacker could move the iframe to position the Accept button where the Decline button used to be just before the user clicks.

I recall some discussion a while back about a proposed intervention in chromium to suppress input events to iframes that have moved within the embedding page recently (for some definition of "recently"). I don't know whether that actually happened, but it would address the problem you mention.

Yes, that intervention has been proposed and is currently assigned to me but I haven't had time to work on it for the last several months.

I believe that if that intervention is shipped, and IOv2 can achieve its stated goals, it would cover all of the known 'easy' modes of clickjacking. Other vectors are either possible to defend against already (e.g. custom mouse cursors), or are not reliably available to attackers (e.g. popunders).

 

Probably the biggest obstacle to trusted interaction with iframes is the lack of a URL bar on iframes, and this proposal doesn't help that at all. But, as you say, it's a step in the right direction.

That is an orthogonal problem that basically reduces to phishing, but yes. Right now a website can display a fake embedded widget and ask for login or payment credentials, and the concern is that if similar real widgets actually existed on reputable websites, users might become more likely to fall for fake ones. However, if you look at an application like OpenYOLO they have already designed around that problem, while they are still held back by substantial clickjacking risk.

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHOQ7J_nO9SPa_vr5LApMFdW46k50fnT6ZY7tbdg30rYpknYmw%40mail.gmail.com.

[Daniel Bratell‏]

That they use it is good, but more importantly is that they also stop using the old "hacks" so that they don't in some over ambitious effort use all the methods at the same time. It would be comforting with some kind of confirmation that it is getting better. I don't have any numbers here, but I thought maybe you had.

/Daniel

[Stefan Zager‏]

You make a good point; certainly I have come across ad network code that doesn't demonstrate a lot of concern for performance overhead. I have a sense of who the worst offenders are, and it's probably worthwhile to do another round of investigation and outreach. That's likely to be a Google-internal effort, I would think.