Intent to Ship: IntersectionObserver V2

[Stefan Zager]

Contact emails

sza...@chromium.org, chri...@chromium.org

Explainer

This is a copy of the current explainer doc for IntersectionObserver, with edits describing the proposed additional functionality.

Spec

This is a copy of the IntersectionObserver spec, with edits describing the proposed spec changes.

Tag review

Summary

Currently, IntersectionObserver will report the bounding box of a target element relative to the containing viewport, after applying all intervening overflow and CSS clips. However, it does not consider whether a target element has been painted over (“occluded”) by other content, or whether any visual effects have been applied which may alter or obscure the element’s display. The proposal is to augment IntersectionObserver to report information about occlusion and visual effects, in order to provide a strong guarantee about the visibility of a target element in the rendered output.

Intent to Implement

Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes.

Demo link

https://szager-chromium.github.io/IntersectionObserver/demo/cashbomb/hidden/

http://szager-chromium.github.io/IntersectionObserver/demo/svg/

Debuggability

Presently there is no DevTools functionality specifically for IntersectionObserver V2. It has been suggested that it might make sense to add DevTools support to help identify why a particular target element is considered "possibly not visible". This can be investigated after shipping.

Risks

Interoperability and Compatibility

Chrome, Edge, and Firefox have shipped V1 of IntersectionObserver. It is implemented in WebKit and enabled by default in the current Safari developer preview. The V2 proposal is a relatively small change to the V1 spec, and it was discussed extensively at TPAC 2018. Security people from all browsers were uniformly positive. Rendering and web platform people from the other browsers were neutral-to-positive.

Edge: Supportive, no detailed position.

Firefox:Security people are supportive; some concern about interoperability.

Safari: Supportive, willing to accept patches.

Web developers: Positive.

Ergonomics

It is expected that V2 will be signficantly slower than V1, due to the computational complexity of determing occlusion. For that reason, V2, is disabled by default; and it mandates a minimum "delay" parameter to throttle the frequency of computation. Limited experimentation on real-world sites confirms the performance difference, but does not appear to be a significant factor in overall page performance.

Activation

IntersectionObserver V1 is already being used on ~20% of all page views, and V2 is a small additive change to the API, so it should be relatively easy for developers to use. There is no viable strategy for polyfill.

Is this feature fully tested by web-platform-tests? Link to test suite results from wpt.fyi.

There are tests in the chromium repository than can be upstreamed to web-platform-tests when appropriate.

Entry on the feature dashboard

[Mike West]

LGTM1.

As a security person, I am universally positive. :) This seems like our best available shot at making it possible to compose frames in a way that doesn't expose the frames to direct risk of clickjacking/PixelPerfect/etc. attacks, and I'm quite excited to see it make its way to developers out there in the wild.

I see that you've been having good discussions with folks at Mozilla on the standards position thread. I share +L. David Baron's interop concerns, but it appears to me that you're not only engaging in good faith, but that you've made concrete improvements to interop along the way. I'm confident that you'll be equally responsive to interop concerns discovered after shipping.

-mike

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHOQ7J9PU6yOR86orQ3EJvgfJeOaXo6a5CNzH%2B7KhLhFbJ9toQ%40mail.gmail.com.

[Philip Jägenstedt]

LGTM2, very exciting.

https://github.com/szager-chromium/IntersectionObserver/blob/v2/explainer.md is a great read, highly recommended!

Since we are about to ship this, this seems an appropriate time to upstream our tests to WPT. Is there anything that might prevent that from happening before the feature reaches stable and/or other vendors begin to implement?

Somewhat related, is there a PR for folding https://szager-chromium.github.io/IntersectionObserver/ into the main spec? I ask in part for the usual reasons, but also because any spec that isn't on one of the usual places isn't included in https://github.com/tidoust/reffy-reports, which means their IDL won't be automatically extracted and tested in web-platform-tests. The IDL will probably also be useful to determining browser support for MDN.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3DdoGH-Nn3JkOPb1rtQ0A5gh2m2JTEHxgHvVQOvKtAM_1g%40mail.gmail.com.

[Daniel Bratell]

Reading the explainer, it talks about protecting the users from fraud, but it's the ones purchasing ad space (and possibly the ad networks they use) this will protect, right? End users are just used as tools when trying to get illegitimate money from ad purchasers.

The explanation talks about a performance hit. Is there any chance that will be a performance gain because ad networks will use less javascript, or is this solving a problem javascript could never address anyway?

>  There are tests in the chromium repository than can be upstreamed to web-platform-tests when appropriate.

It would be appropriate I believe, unless it's not appropriate for some reason? I believe it's all magic now so it should be easy (I say, without having actually created any recent web platform tests myself). (foolip?)

/Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAKXHy%3DdoGH-Nn3JkOPb1rtQ0A5gh2m2JTEHxgHvVQOvKtAM_1g%40mail.gmail.com.

--

/* Opera Software, Linköping, Sweden: CET (UTC+1) */

[Philip Jägenstedt]

On Thu, Jan 24, 2019 at 1:13 PM Daniel Bratell <bra...@opera.com> wrote:

Reading the explainer, it talks about protecting the users from fraud, but it's the ones purchasing ad space (and possibly the ad networks they use) this will protect, right? End users are just used as tools when trying to get illegitimate money from ad purchasers.

Being misled to click a Like or Retweet button seems like a potential harm to the clicker or their followers.

 

The explanation talks about a performance hit. Is there any chance that will be a performance gain because ad networks will use less javascript, or is this solving a problem javascript could never address anyway?

>  There are tests in the chromium repository than can be upstreamed to web-platform-tests when appropriate.

It would be appropriate I believe, unless it's not appropriate for some reason? I believe it's all magic now so it should be easy (I say, without having actually created any recent web platform tests myself). (foolip?)

Yes, it's just a matter of moving files within the Chromium repo. If the tests don't use any internal APIs it's probably straightforward. 

[Stefan Zager]

On Thu, Jan 24, 2019 at 4:13 AM Daniel Bratell <bra...@opera.com> wrote:

Reading the explainer, it talks about protecting the users from fraud, but it's the ones purchasing ad space (and possibly the ad networks they use) this will protect, right? End users are just used as tools when trying to get illegitimate money from ad purchasers.

Ads are not the only (or even the primary) use case for V2. It is broadly applicable to many kinds of UI redress attacks that victimize end users. The first demo link illustrates just such a case involving an embedded third-party digital payment widget.

 

The explanation talks about a performance hit. Is there any chance that will be a performance gain because ad networks will use less javascript, or is this solving a problem javascript could never address anyway?

My experience is that ad networks are not always conscientious actors in this way. Some of them are still using IntersectionObserver (V1) *in addition* to their pre-existing javascript involving scroll handlers and other anti-patterns. I think it's important to be practical and realistic in our expectations of how this will be used.

[Chris Harrelson]

That being said, the occlusion testing API in this intent is definitely more performant, and much more comprehensive, than attempts to polyfill it by script. For example, ad scripts sometimes use elementFromPoint, or even multiple calls to elementFromPoint, as a partial solution to occlusion testing.

Sites will be able to achieve better performance by removing such a polyfill and using the v2 API.

(BTW I am involved in this intent, and so am not offering any API owner approval or disapproval.)

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHOQ7J9kj84Ns2Lwuz7FwJt2mfrkQTz6QOx609HBinuER5DEkA%40mail.gmail.com.

[Brian Birtles]

2019年1月24日木曜日 1時08分46秒 UTC+9 Stefan Zager:

Spec

This is a copy of the IntersectionObserver spec, with edits describing the proposed spec changes.

Is there any chance we could get this into a proper spec before shipping such that it might encourage wider review?

(Concern is that we've had things ship with obvious problems before because they never graduated beyond WICG before shipping and hence missed wider review.)

 

Firefox:Security people are supportive; some concern about interoperability.

Is there any reference for which security people are supportive?

(In any case, I think the standards-positions issue should be preferred over any other signals.)

[Stefan Zager]

On Thu, Jan 24, 2019 at 4:22 PM Brian Birtles <bbir...@mozilla.com> wrote:

2019年1月24日木曜日 1時08分46秒 UTC+9 Stefan Zager:

Spec

This is a copy of the IntersectionObserver spec, with edits describing the proposed spec changes.

Is there any chance we could get this into a proper spec before shipping such that it might encourage wider review?

(Concern is that we've had things ship with obvious problems before because they never graduated beyond WICG before shipping and hence missed wider review.)

There is a proper spec repository for IntersectionObserver (I am the editor), but I opted not to commit my changes to that repository, because it seems useful to me to be able to compare the existing V1 spec and the proposed V2 spec side-by-side. I also wanted to have a separate issue tracker, to keep V2-specific issues separate. Ultimately -- if everything proceeds as planned -- the V2 spec would replace the V1 spec in the canonical w3c repository.

Interestingly, IntersectionObserver V1 shipped in multiple browsers well before the spec graduated out of WICG. So, graduating to a proper w3c repository has not been an obstacle in the past.

  

Firefox:Security people are supportive; some concern about interoperability.

Is there any reference for which security people are supportive?

(In any case, I think the standards-positions issue should be preferred over any other signals.)

I presented the proposal to the WebAppSec working group at TPAC 2018. There were representatives from Mozilla there, as well as from other browsers, and the feedback was quite positive. Mike West, who chaired the meeting, commented on the Mozilla standards position bug and mentioned two Mozilla security people whom he characterized as being positive on the proposal. I believe those two people were present at the TPAC presentation.

[Stefan Zager]

On Thu, Jan 24, 2019 at 4:02 AM Philip Jägenstedt <foo...@chromium.org> wrote:

Somewhat related, is there a PR for folding https://szager-chromium.github.io/IntersectionObserver/ into the main spec? I ask in part for the usual reasons, but also because any spec that isn't on one of the usual places isn't included in https://github.com/tidoust/reffy-reports, which means their IDL won't be automatically extracted and tested in web-platform-tests. The IDL will probably also be useful to determining browser support for MDN.

I don't have a PR yet, I will create one.

[Brian Birtles]

On Fri, Jan 25, 2019 at 9:49 AM Stefan Zager <sza...@chromium.org> wrote:

On Thu, Jan 24, 2019 at 4:22 PM Brian Birtles <bbir...@mozilla.com> wrote:

2019年1月24日木曜日 1時08分46秒 UTC+9 Stefan Zager:

Spec

This is a copy of the IntersectionObserver spec, with edits describing the proposed spec changes.

Is there any chance we could get this into a proper spec before shipping such that it might encourage wider review?

(Concern is that we've had things ship with obvious problems before because they never graduated beyond WICG before shipping and hence missed wider review.)

There is a proper spec repository for IntersectionObserver (I am the editor), but I opted not to commit my changes to that repository, because it seems useful to me to be able to compare the existing V1 spec and the proposed V2 spec side-by-side.

That's fine, but it would be appreciated it the changes were made put somewhere more visible before shipping.

(Also, for what it's worth, it's not uncommon to commit a delta spec to the main repo for situations like this, e.g. see https://drafts.csswg.org/css-animations-2/#delta)

 

I also wanted to have a separate issue tracker, to keep V2-specific issues separate.

(I can understand that, but I think labels are sufficient and preferable since it alerts people watching the issues in the main repo about upcoming issues. My experience with Web Animations at least is that such early feedback is very helpful.)

 

Interestingly, IntersectionObserver V1 shipped in multiple browsers well before the spec graduated out of WICG. So, graduating to a proper w3c repository has not been an obstacle in the past.

The concern is it probably should have blocked shipping.

For example, overscroll-behavior went to ship whilst still in WICG and didn't get wider review until that point. I suggested we update the property name at that point and we did. Soon after Tab came up with an even better property name but I didn't have the heart to suggest we change it again because I didn't want to delay Blink's shipping schedule. All of that could have been avoided had the spec moved somewhere more visible sooner.

(And unfortunately all that history is lost because when the WICG repository was later archived the issues disappeared.)

Can we please get the spec changes merged somewhere more visible before shipping?

Best regards,

Brian

[Yoav Weiss]

LGTM3

This seems like an important feature that would enable plugging this security hole.

At the same time:

* I'd love to see the V2 specs get merged to be part of the official repo. You could have them be there on a separate branch, and set up pr-preview to have a PR with diffs between V2 and V1.

* +1 to Philip's comment regarding WPTs

* The interop concerns Mozilla raised seem legit. Is it possible to modify the spec to require stricter and more predictable behavior? It doesn't seem like we should block shipping on that, but definitely seems like something worthwhile iterating over, in collaboration with the Mozilla folks.

Brian - not sure what happened there, but archiving is not supposed to make the issues disappear. Maybe the original repo was moved?

Otherwise, as IntersectionObserver is no longer in incubation, I'm not sure how this is relevant.

 

Can we please get the spec changes merged somewhere more visible before shipping?

Best regards,

Brian

--

You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAF-W_FQa0ww%2B3NRdyahZqcPQk31kfAV8qvkSvLLv-gr_dhTt-w%40mail.gmail.com.

[Daniel Bratell]

On Thu, 24 Jan 2019 19:40:08 +0100, Stefan Zager <sza...@chromium.org> wrote:

On Thu, Jan 24, 2019 at 4:13 AM Daniel Bratell <bra...@opera.com> wrote:

Reading the explainer, it talks about protecting the users from fraud, but it's the ones purchasing ad space (and possibly the ad networks they use) this will protect, right? End users are just used as tools when trying to get illegitimate money from ad purchasers.

Ads are not the only (or even the primary) use case for V2. It is broadly applicable to many kinds of UI redress attacks that victimize end users. The first demo link illustrates just such a case involving an embedded third-party digital payment widget.

Ah, that does indeed sound like attacking the user directly.

/Daniel

[Rick Byers]

On Thu, Jan 24, 2019 at 11:06 PM Yoav Weiss <yo...@yoav.ws> wrote:

LGTM3

This seems like an important feature that would enable plugging this security hole.

At the same time:

* I'd love to see the V2 specs get merged to be part of the official repo. You could have them be there on a separate branch, and set up pr-preview to have a PR with diffs between V2 and V1.

FWIW we've successfully used the approach of incubating additions and major changes to a spec within it's existing repo in pointer events - both via a monkey patch and (for a contentious and hard to separate change) a branch. As long as the group is happy with that and it doesn't add process overhead, it seems less of a hassle to me than moving specs between different repos.

But we should point out that the location of the spec is not relevant from the perspective of the blink launch process. Soliciting and reasonably responding to feedback on a precise specification and (where possible) test suite is the bar for shipping, so whatever makes that more effective and efficient is great.

Also note that there is an important IPR difference when asking for feedback in a W3C GitHub repo than a personal one - so it's reasonable for reviewers to ask for a spec to be moved into a repo that provides IPR protections (including WICG, but not random personal repos) before they are able to provide design feedback.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEif55Ouc0jPgJ-y6tP%3Dtyn3hHxny7OTt1zB5u_UKebT4w%40mail.gmail.com.

[Stefan Zager]

On Fri, Jan 25, 2019 at 2:18 PM Rick Byers <rby...@chromium.org> wrote:

On Thu, Jan 24, 2019 at 11:06 PM Yoav Weiss <yo...@yoav.ws> wrote:

LGTM3

This seems like an important feature that would enable plugging this security hole.

At the same time:

* I'd love to see the V2 specs get merged to be part of the official repo. You could have them be there on a separate branch, and set up pr-preview to have a PR with diffs between V2 and V1.

FWIW we've successfully used the approach of incubating additions and major changes to a spec within it's existing repo in pointer events - both via a monkey patch and (for a contentious and hard to separate change) a branch. As long as the group is happy with that and it doesn't add process overhead, it seems less of a hassle to me than moving specs between different repos.

Procedural question: in the w3c repo, the master branch is automagically bikeshed-ed to populate the gh-pages branch, which is served up by github pages. There will be no such magic when I commit the V2 spec to a branch. How/where should I publish the formatted version of the spec?

[Rick Byers]

I haven't had to do that myself (PE is ReSpec so no build necessary, and I relied on rawgit.com links to share previews - but rawgit is now dead). But I imagine it would be OK to modify the script to publish to gh-pages under a separate filename as long as the spec is clearly indicated "Level 2 - Editors Draft" or something?

  

[Stefan Zager]

"The script" is operated by w3c.

[Philip Jägenstedt]

Something like https://w3c.github.io/IntersectionObserver/v2/ would be nice.

Having two branches publish to a single branch isn't entirely trivial though, because you deploy.sh will overwrite the existing content. You'll need to tweak deploy.sh to only add and never remove, and will still be left with the possibility that both branches are deploying at the same moment and one will fail. But it's doable.

[Stefan Zager]

Oh, wow, I never noticed that deploy.sh was in the repository. I think I can make it work, thanks. 

[Johnny Stenback]

It is true that the Blink launch process does not mandate a particular location (or set of locations) for specs for changes we ship through that process. I however feel it is reasonable as a minimum step to expect that Chromium contributions from those of us who are Google employees come with specs for things we ship through the Blink lunch process in a place that is well known and easily findable (e.g. WICG, or some other official and well known W3C/WHATWG/IETF/ECMA/etc location). Not only does that make various tooling (as Philip pointed out) easier, but it also makes the specs more easily findable, more likely we'll receive more feedback, and it gives not just us, but those who we expect and hope to get feedback from some level of IPR protection. I can't see any real downside to expecting that. And we would certainly appreciate the same from others who contribute, and maybe at some point it would make sense to actually bake that into the Blink process as well.

And if we for whatever reason find ourselves with a hard time finding good places for specs (i.e. WICG etc isn't working for us) then nothing stops us from creating a Chromium specific place for early stage specs. We could attached some level of IPR protection and publicize that location over time, but that I will not worry about unless there's a strong need to go there (I'm not aware of any current blockers with the set of locations where early stage spec work can happen today).

I'll add writing something up about this as a starting point to my todo list unless people strongly disagree with my thinking here.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY_5TmB3fAS2AafcLbNYLKEiEVsgDR0JrfazXFpkJ43O%2BA%40mail.gmail.com.

[Yoav Weiss]

On Wed, Jan 30, 2019 at 1:59 AM Johnny Stenback <jste...@chromium.org> wrote:

It is true that the Blink launch process does not mandate a particular location (or set of locations) for specs for changes we ship through that process. I however feel it is reasonable as a minimum step to expect that Chromium contributions from those of us who are Google employees come with specs for things we ship through the Blink lunch process in a place that is well known and easily findable (e.g. WICG, or some other official and well known W3C/WHATWG/IETF/ECMA/etc location). Not only does that make various tooling (as Philip pointed out) easier, but it also makes the specs more easily findable, more likely we'll receive more feedback, and it gives not just us, but those who we expect and hope to get feedback from some level of IPR protection. I can't see any real downside to expecting that. And we would certainly appreciate the same from others who contribute, and maybe at some point it would make sense to actually bake that into the Blink process as well.

Agree that it's significantly better for us to ship specs that are in well known venues whenever possible (for the IPR and visibility reasons you mentioned above).

In cases where it's not possible (e.g. no multi-vendor commitment for features destined for WHATWG specifications) we currently settled on publishing PRs against those specifications. This is not super convenient, but maybe we can improve that with tooling.

  

And if we for whatever reason find ourselves with a hard time finding good places for specs (i.e. WICG etc isn't working for us) then nothing stops us from creating a Chromium specific place for early stage specs. We could attached some level of IPR protection and publicize that location over time, but that I will not worry about unless there's a strong need to go there (I'm not aware of any current blockers with the set of locations where early stage spec work can happen today).

I don't see a reason to go there at the moment. For the WICG specifically, If there's any reason that incubation there is adding more friction than it should, please let me know and I'll do everything I can to reduce that friction.

[oj...@google.com]

Thanks for taking this on Johnny! I agree with your reasoning FWIW. :)

Yoav Weiss wrote:

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEhfpWN7y%3DAByouDg%2BfCW%3D-g6-yf0Q3OwoCik4mi-RQWjA%40mail.gmail.com.

[Stefan Zager]

I committed the V2 spec to the canonical spec repository on a branch:

https://github.com/w3c/IntersectionObserver/tree/v2

The rendered spec html is here:

http://w3c.github.io/IntersectionObserver/v2/

If you want to see the difference between V1 and V2, this copy of the V2 spec has highlights:

https://szager-chromium.github.io/IntersectionObserver/

I also created a CL to upstream the V2 tests to wpt:

https://chromium-review.googlesource.com/c/chromium/src/+/1447267

[Philip Jägenstedt]

Thanks Stefan, that all looks great!

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAHOQ7J_fk8A4UdieSht8ODjDDz-yt0ZPc%3DauEYnO564U%2BzSwuQ%40mail.gmail.com.

[Stefan Zager]

Update: we are un-shipping this feature from the M73. It caused an unacceptable performance regression on pages that don't use the feature. There's a fix in development for the regression, but the change is too significant to safely merge to the branch.

It's very likely that we'll be able to ship in M74.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAARdPYeVcYy5h2t4ciC3jQodV%2BhVWG5PZ8BL26ykpc8hcQurBw%40mail.gmail.com.