Intent to Ship: Layout Instability API
211 views
Skip to first unread message
Steve Kobes
Jul 17, 2019, 12:56:09 PM7/17/19
to blin...@chromium.org
Contact emails
Explainer
https://github.com/WICG/layout-instability/blob/master/README.md
Spec
https://wicg.github.io/layout-instability/
TAG review
https://github.com/w3ctag/design-reviews/issues/393
Summary
The Layout Instability API allows the developer to monitor the degree to which DOM elements have changed their on-screen position ("layout shifts") during the user's session.
Link to "Intent to Implement" blink-dev discussion
https://groups.google.com/a/chromium.org/d/msg/blink-dev/jF1-M8KWAMU/ubGV4Fx2BgAJ
Link to Origin Trial feedback summary
https://docs.google.com/document/d/1A9-iv-EMK8ml8tzJ_sdYbg5vZLk6Pz0TL7Z7-FYaO8A/edit
Is this feature supported on all six Blink platforms?
Yes
Demo link
https://output.jsbin.com/zajamil/quiet
Debuggability
DevTools supports visualizing layout shifts in realtime (More Tools > Rendering > Layout Shift Regions). Lighthouse will also report this metric.
Risks
Interoperability and Compatibility
The feature adds a new type of performance entry; we perceive no compatibility risk.
Edge: Mostly positive signals [1] [2] [3]
Firefox: Some positive signals in TPAC discussion
Safari: Some positive signals in TPAC discussion
Web / Framework developers: Positive signals
Ergonomics
This feature is built on top of PerformanceObserver, and reuses its existing facilities for buffering. It complements and aligns with LCP to present a holistic picture of user experience.
The internal measurement of layout instability is already shipping in Chrome, and has not significantly regressed performance.
Activation
No perceived risks.
Is this feature fully tested by web-platform-tests?
Partial WPT coverage: https://wpt.fyi/results/layout-instability
Some tests are not yet upstreamed (crbug.com/984109).
Entry on the feature dashboard
Joe Medley
Jul 17, 2019, 5:44:04 PM7/17/19
to Steve Kobes, blink-dev
Steve,
What version of Chrome are you aiming for?
Joe
If an API's not documented it doesn't exist.
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAjAU%3DZRt0vg2GqMnY0C21rnsnWgyce6ohmFMCNhe3y66LiEjg%40mail.gmail.com.
Steve Kobes
Jul 17, 2019, 5:48:26 PM7/17/19
to Joe Medley, blink-dev
We'd like to enable this API by default in Chrome 77.
It's currently behind an experimental flag: --enable-blink-features=LayoutInstabilityAPI
Daniel Bratell
Jul 18, 2019, 10:06:35 AM7/18/19
to Joe Medley, Steve Kobes, blink-dev
You got some sites to contribute to the feature in the origin trial, but I can't see if they in the end found the feature useful or not. Do you have any such stories/feedback?
/Daniel
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAjAU%3DbdieiOpOdC4K0NeybB6ZOjDo55DkGF9A3C4ARCb4Xqdw%40mail.gmail.com.
--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */
Steve Kobes
Jul 18, 2019, 3:23:24 PM7/18/19
to Daniel Bratell, Joe Medley, blink-dev
On Thu, 18 Jul 2019 at 06:06, Daniel Bratell <bra...@opera.com> wrote:
You got some sites to contribute to the feature in the origin trial, but I can't see if they in the end found the feature useful or not. Do you have any such stories/feedback?
There is some positive feedback from developers on the WICG discourse thread here:
Daniel Bratell
Jul 18, 2019, 7:52:01 PM7/18/19
to Steve Kobes, Joe Medley, blink-dev
Thanks! That is a good sign.
As I just wrote a rather long comment about performance analysis tools in the thread about Largest Contentful Paint, I should say that I have the same thought about this addition. We have a problem where web developers don't have the tools they need and want for understanding how users perceive their pages. We want more tools. How do we know this is a tool that will actually be used and be useful?
See it as a bigger question, how can the project as a whole be sure a certain feature will be used in a valuable way?
/Daniel
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAjAU%3DYCVBuEf5bh3fO67sv-VcjFeYV%3Dj6cQq%2BzNFynddn7M-w%40mail.gmail.com.
Florian Rivoal
Jul 20, 2019, 6:14:43 PM7/20/19
to Steve Kobes, blin...@chromium.org
This looks like an interesting concept, but it seems to me that the proposal is being a little loose with the terminology it uses, and that it causes ambiguities. Non exhaustive list of examples:
* Elements, boxes, and fragments are different things. Talking about the coordinates of an element is problematic. Some elements have no box (display:none, display:contents), some have several (list items, tables...), so it's not clear what the spec means when it says "the starting point" of "the element". Largely, the spec speaks of elements, as if they were things with coordinates, but it should probably be speaking of fragments instead.
* Also, the "visual representation" is defined for "block level elements" and for "inline level elements", and it defines these terms with a reference to HTML 4 (!). CSS-DISPLAY-3 would be a much better reference. And we should be speaking of block level boxes / inline level boxes, not elements. But there are also boxes which are neither block level nor inline level (e.g flex items, table rows..), and "visual representation" is undefined for these.
* fragmentation is taken into account for the visual representation of inline level boxes/elements, but not for block level ones, even though block level boxes can also be fragmented. If a block level element is inside a multicolumn, and gets fragmented into two columns, what is "the flow-relative offset in the coordinate space of the initial containing block of the current document measured in pixel units"? The one of the first fragment? of the second one? the smaller of the two? the larger of the two?
* "the geometric union of its box fragments", as used for the visual representation of inline level boxes, is not necessarily defined. Pagination can also induce fragmentation, and if two fragments are on different pages, it's unclear how to do a geometric union. Even if interactive paginated views are not a priority at the moment, designing a system that is incompatible with them is unfortunate.
* "the flow relative offset in the coordinate space of the initial containing block of the current document" seems problematic. First, I assume it's meant to be an offset in the block axis from the start of the document. If so, it should be stated (and if something else is meant, I curious as to what). But even then, I am unsure how that works out, for instance in the case of documents with orthogonal flows. If, in a generally horizontal document, a particular section is set to a vertical writing mode, and within that section, and asynchronously loading element shifts other boxes in the block axis. Is that a change if the offset? If so, how? If not, why not?
All in all, I think the kind of issues I mentioned above are unlikely to be fatal flaws in the system, but as currently written, there is a lot of ambiguities and room for interop problems. I think this should be sorted out before shipping.
Also, I am less sure about that, but I wonder how "evaluate the layout instability" copes with the compositor thread. In the middle of an animation or of a scroll, certain boxes may be moving relative to others off the main thread, and I wonder if getting the starting point of all elements in the document is something that can actually be done synchronously.
Even if that's not a problem, I wonder if it is actually a good idea to include the shift between elements caused by animations or scrolling. The goal of this API is to measure layout instability due to asynchronous loading, but if this metric is polluted by animations or user scrolling, it feels like it would become a lot less useful. Maybe there should be some amount of filtering to exclude animating / transitioning elements and maybe fixed positioned elements. And maybe we could measure the offset of elements with respect to the nearest ancestor scrollport, rather than to the Initial Containing block, to ignore the shits caused by scrolling.
I'll also note that this spec has not been presented to the CSS Working Group, even though it has a major dependency on layout. While the CSS-WG doesn't have an absolute right to see all layout related specs before they ship, it has expertise in the matter, and would likely have caught the type of problems I'm mentioning here.
—Florian
* Elements, boxes, and fragments are different things. Talking about the coordinates of an element is problematic. Some elements have no box (display:none, display:contents), some have several (list items, tables...), so it's not clear what the spec means when it says "the starting point" of "the element". Largely, the spec speaks of elements, as if they were things with coordinates, but it should probably be speaking of fragments instead.
* Also, the "visual representation" is defined for "block level elements" and for "inline level elements", and it defines these terms with a reference to HTML 4 (!). CSS-DISPLAY-3 would be a much better reference. And we should be speaking of block level boxes / inline level boxes, not elements. But there are also boxes which are neither block level nor inline level (e.g flex items, table rows..), and "visual representation" is undefined for these.
* fragmentation is taken into account for the visual representation of inline level boxes/elements, but not for block level ones, even though block level boxes can also be fragmented. If a block level element is inside a multicolumn, and gets fragmented into two columns, what is "the flow-relative offset in the coordinate space of the initial containing block of the current document measured in pixel units"? The one of the first fragment? of the second one? the smaller of the two? the larger of the two?
* "the geometric union of its box fragments", as used for the visual representation of inline level boxes, is not necessarily defined. Pagination can also induce fragmentation, and if two fragments are on different pages, it's unclear how to do a geometric union. Even if interactive paginated views are not a priority at the moment, designing a system that is incompatible with them is unfortunate.
* "the flow relative offset in the coordinate space of the initial containing block of the current document" seems problematic. First, I assume it's meant to be an offset in the block axis from the start of the document. If so, it should be stated (and if something else is meant, I curious as to what). But even then, I am unsure how that works out, for instance in the case of documents with orthogonal flows. If, in a generally horizontal document, a particular section is set to a vertical writing mode, and within that section, and asynchronously loading element shifts other boxes in the block axis. Is that a change if the offset? If so, how? If not, why not?
All in all, I think the kind of issues I mentioned above are unlikely to be fatal flaws in the system, but as currently written, there is a lot of ambiguities and room for interop problems. I think this should be sorted out before shipping.
Also, I am less sure about that, but I wonder how "evaluate the layout instability" copes with the compositor thread. In the middle of an animation or of a scroll, certain boxes may be moving relative to others off the main thread, and I wonder if getting the starting point of all elements in the document is something that can actually be done synchronously.
Even if that's not a problem, I wonder if it is actually a good idea to include the shift between elements caused by animations or scrolling. The goal of this API is to measure layout instability due to asynchronous loading, but if this metric is polluted by animations or user scrolling, it feels like it would become a lot less useful. Maybe there should be some amount of filtering to exclude animating / transitioning elements and maybe fixed positioned elements. And maybe we could measure the offset of elements with respect to the nearest ancestor scrollport, rather than to the Initial Containing block, to ignore the shits caused by scrolling.
I'll also note that this spec has not been presented to the CSS Working Group, even though it has a major dependency on layout. While the CSS-WG doesn't have an absolute right to see all layout related specs before they ship, it has expertise in the matter, and would likely have caught the type of problems I'm mentioning here.
—Florian
> --
> You received this message because you are subscribed to the Google Groups "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAjAU%3DZRt0vg2GqMnY0C21rnsnWgyce6ohmFMCNhe3y66LiEjg%40mail.gmail.com.
> You received this message because you are subscribed to the Google Groups "blink-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
Steve Kobes
Jul 22, 2019, 9:40:29 PM7/22/19
to Florian Rivoal, blin...@chromium.org
Florian, thanks for the feedback. I've filed bugs on https://github.com/WICG/layout-instability/issues for some of the spec updates you've suggested.
The intent is generally to track the visible visual movement of rendered elements. So for example a display:none element won't have any layout instability, because it isn't rendered. For a multi-column or paginated element, its starting point (for deciding if it has shifted) would be that of the first fragment, but all visible fragments would contribute to the impact region.
A layout shift can occur along either the block or inline axis, or both. The reference to "flow-relative offset" is meant to characterize "starting point" as the coordinate of a logical corner. For example, a box in a vertical-rl context would be shifted by a change in its top or right extents, but not its left or bottom extents.
The intent is also that user scrolling does not induce layout instability, unless of course the page makes some layout modification in response to the scroll event. The spec should be clearer about this. There is also an exclusion for transform animations (but not for animations of layout inducing properties).
Yoav Weiss
Jul 25, 2019, 12:58:17 PM7/25/19
to Daniel Bratell, Steve Kobes, Joe Medley, blink-dev
Hey Daniel! :)
On Thu, Jul 18, 2019 at 3:52 PM Daniel Bratell <bra...@opera.com> wrote:
Thanks! That is a good sign.As I just wrote a rather long comment about performance analysis tools in the thread about Largest Contentful Paint, I should say that I have the same thought about this addition. We have a problem where web developers don't have the tools they need and want for understanding how users perceive their pages. We want more tools. How do we know this is a tool that will actually be used and be useful?
See it as a bigger question, how can the project as a whole be sure a certain feature will be used in a valuable way?
takes API owner hat off
That's a great question. In many cases (this one included) the answer to that is an Origin Trial.
One of the results of the origin trial this feature went through is Wikimedia saying loud and clear that they want this shipped, as it enables them to catch layout-shift-related regressions before users report them, as they cannot test all their pages in the lab for all possible user devices.
puts WebPerfWG chair hat on
Another positive signal here would be that this was discussed at last year's TPAC F2F and had rather positive feedback from attendees (both browser vendors and industry folks), with no significant push back. A few quotes from the minutes:
Alex Christensen (Apple) - "seems very useful for developer tools. We could be motivated by a success story of a web site that improved via this"
Gilles Dubuc (Wikimedia) - "would be great to catch regressions with. Fundraising banners, etc"
Benoit Girard (Facebook) - "would be very useful for FB. we care about this, useful measure to track"
Mozilla folks were also in the room, provided feedback and as far as I remember were cautiously positive, even if it's not captured in the minutes.
--/DanielOn Thu, 18 Jul 2019 17:23:04 +0200, Steve Kobes <sko...@chromium.org> wrote:--On Thu, 18 Jul 2019 at 06:06, Daniel Bratell <bra...@opera.com> wrote:You got some sites to contribute to the feature in the origin trial, but I can't see if they in the end found the feature useful or not. Do you have any such stories/feedback?There is some positive feedback from developers on the WICG discourse thread here:
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAAjAU%3DYCVBuEf5bh3fO67sv-VcjFeYV%3Dj6cQq%2BzNFynddn7M-w%40mail.gmail.com.
--/* Opera Software, Linköping, Sweden: CEST (UTC+2) */
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/op.z442spilrbppqq%40cicero2.linkoping.osa.
Ojan Vafai
Jul 25, 2019, 7:47:04 PM7/25/19
to Yoav Weiss, Daniel Bratell, Steve Kobes, Joe Medley, blink-dev
API owners (rbyers, yoav, chrishtr, bratell, slightlyoff, ojan) discussed this with the team today. LGTMx3 to ship with caveats below.
This API has fairly positive feedback from other vendors and has direct feedback of real world impact in the wild from Wikipedia. The concerns expressed have been about spec precision, not behavior. Namely, looks good to ship assuming the spec issues get resolved quickly so we can disable it before shipping to stable if anything significantly breaking comes up as we refine the spec, e.g. changes the shape of the API.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CACj%3DBEgAwBt%2BGYEtXgktFB-gqDAn0x0a_r19KK0z%3DffvM_m0iA%40mail.gmail.com.
Florian Rivoal
Jul 26, 2019, 10:02:55 AM7/26/19
to Ojan Vafai, Yoav Weiss, Daniel Bratell, Steve Kobes, Joe Medley, blink-dev
I'll note that the TAG expressed support for my earlier comments: https://github.com/w3ctag/design-reviews/issues/393#issuecomment-514483032
I remain somewhat skeptical of shipping. Just because the spec issues can be fixed (which I believe they can) doesn't mean they are easy to fix.
In the meanwhile, another browser attempting to implement the same feature in order to be interoperable would have to spend non trivial effort in trying to reverse engineer Chrome's behavior, as the spec is insufficient to understand what Chrome does.
Furthermore, and number of the issues on the spec are likely to imply changes in behavior as they get fixed. One example would be ignoring scrolling or ongoing animations. Another is that a number of the "precision" issues on the spec are also more that just about using the right word instead of the wrong one. Switching from "element" to "box" or "fragment" also means deciding what to do when there isn't an 1 to 1 correspondance between these concepts. Exploring how to resolve these is likely to expose situations that hadn't been considered, and therefore possibly changing the behavior from what it is today.
To me, both these things mean it is premature to ship.
—Florian
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CANMdWTsjPaSMWzLYFuEzG5fwgn%3Dh2uBZbkSHLKJC30waYJfsvA%40mail.gmail.com.
Steve Kobes
Sep 11, 2019, 11:14:36 PM9/11/19
to Florian Rivoal, Ojan Vafai, Yoav Weiss, Daniel Bratell, Joe Medley, blink-dev
Update: the spec for the Layout Instability API has been refactored to express various concepts in a more precise manner, such as
- visual representation based on box fragments generated by an element or line boxes generated by a text node,
- starting point based on a flow-relative starting corner of the first fragment of the principal box, and
- allowance for scrolling and transform changes in the definition of "unstable node".
The explainer also has new guidance about the limitations of the API.
Please continue to file issues about anything you think could be improved.
Reply all
Reply to author
Forward
0 new messages