Intent to Ship: isInputPending

520 visualitzacions
Ves al primer missatge no llegit

Andrew Comminos

no llegida,
18 de set. 2020, 15:34:1218/9/20
a blink-dev

Contact emails

acom...@fb.com, pani...@chromium.org, tdre...@chromium.org


Explainer

https://github.com/WICG/is-input-pending


Specification

https://wicg.github.io/is-input-pending/


Design docs

https://github.com/WICG/is-input-pending


Summary

This API enables developers to query for the presence of pending input events while running JavaScript. This enables them to respond immediately to user input by yielding, while being able to complete their work as fast as possible.


Blink component

Blink>Scheduling


TAG review

https://github.com/w3ctag/design-reviews/issues/475


TAG review status

Issues addressed


Risks



Interoperability and Compatibility

Interoperability risk: Medium. If other browsers do not implement this API, then a similar (albeit less performant) outcome can be achieved by yielding more often after checking for the presence of navigator.scheduling.isInputPending. This API has been discussed at length at TPAC 2018, TPAC 2019, and in Web Performance Working Group calls.


Compatibility risk: Low. Existing behaviour is unaffected. The primary mechanism through which compatibility risk could be introduced is through the addition of new input event types later down the line, in which case the function may return true more or less often than expected. We attempt to mitigate this by defining a semantic dichotomy of events (continuous and discrete, rather than allow filtering on a per-event basis).


Gecko: No signal (https://github.com/mozilla/standards-positions/issues/155)


WebKit: No signal (https://lists.webkit.org/pipermail/webkit-dev/2020-September/031390.html)


Web developers: Strongly positive


  • A Google ads team ran an experiment demonstrating they could yield 15% of the time on certain domains with an average input delay improvement of 100ms. They stated that the isInputPending latency improvement has clear UX benefit.

  • The Google Slides team was able to see a substantial improvement in render time (about 25%) of the left-hand-side filmstrip UI for complex content by not yielding (which is only possible with isInputPending, or else there would be unacceptable input delay).

  • Facebook was able to reduce event latency at p95 by 100ms by integrating isInputPending in an origin trial, while retaining their scheduler’s throughput.


Activation

This API will typically be used by framework libraries, such as React with concurrent mode (although any site that manages its own work queue may benefit). React plans to integrate isInputPending into their concurrent mode scheduler immediately. For web applications that do not use frameworks leveraging isInputPending yet frequently perform long tasks, they may benefit from additional outreach.


Security

Implementers must take care to ensure that frames may not detect input events from other cross-origin frames. In Blink, this is achieved by performing hit testing on the compositor side for cross-origin frames within-process, and via viz for OOPIFs.



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes


Is this feature fully tested by web-platform-tests?

Yes (https://wpt.fyi/results/is-input-pending)


Tracking bug

https://bugs.chromium.org/p/chromium/issues/approval?id=930987


Sample links

https://wicg.github.io/is-input-pending/#examples-of-usage


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5719830432841728


Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/d/msg/blink-dev/eLq8t56CbaQ/KdbOY7ZHCwAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/d/msg/blink-dev/ItkbDBevOrs/yFffQlqTBQAJ



Yoav Weiss

no llegida,
21 de set. 2020, 3:22:5721/9/20
a Andrew Comminos,blink-dev
Thanks for working on this! I believe this API can unlock significant performance benefits.

On Fri, Sep 18, 2020 at 9:34 PM 'Andrew Comminos' via blink-dev <blin...@chromium.org> wrote:

Contact emails

acom...@fb.com, pani...@chromium.org, tdre...@chromium.org


Explainer

https://github.com/WICG/is-input-pending


Specification

https://wicg.github.io/is-input-pending/


Design docs

https://github.com/WICG/is-input-pending


Summary

This API enables developers to query for the presence of pending input events while running JavaScript. This enables them to respond immediately to user input by yielding, while being able to complete their work as fast as possible.


Blink component

Blink>Scheduling


TAG review

https://github.com/w3ctag/design-reviews/issues/475


TAG review status

Issues addressed


Risks



Interoperability and Compatibility

Interoperability risk: Medium. If other browsers do not implement this API, then a similar (albeit less performant) outcome can be achieved by yielding more often after checking for the presence of navigator.scheduling.isInputPending. This API has been discussed at length at TPAC 2018, TPAC 2019, and in Web Performance Working Group calls.


Compatibility risk: Low. Existing behaviour is unaffected. The primary mechanism through which compatibility risk could be introduced is through the addition of new input event types later down the line, in which case the function may return true more or less often than expected. We attempt to mitigate this by defining a semantic dichotomy of events (continuous and discrete, rather than allow filtering on a per-event basis).


Gecko: No signal (https://github.com/mozilla/standards-positions/issues/155)


WebKit: No signal (https://lists.webkit.org/pipermail/webkit-dev/2020-September/031390.html)


This was filed only recently, so it feels like we should give them a bit more time.
 

Web developers: Strongly positive


  • A Google ads team ran an experiment demonstrating they could yield 15% of the time on certain domains with an average input delay improvement of 100ms. They stated that the isInputPending latency improvement has clear UX benefit.

  • The Google Slides team was able to see a substantial improvement in render time (about 25%) of the left-hand-side filmstrip UI for complex content by not yielding (which is only possible with isInputPending, or else there would be unacceptable input delay).

  • Facebook was able to reduce event latency at p95 by 100ms by integrating isInputPending in an origin trial, while retaining their scheduler’s throughput.


I heard of other data-backed improvements from a large retailer (in the order of O(100) ms)
 

Activation

This API will typically be used by framework libraries, such as React with concurrent mode (although any site that manages its own work queue may benefit). React plans to integrate isInputPending into their concurrent mode scheduler immediately. For web applications that do not use frameworks leveraging isInputPending yet frequently perform long tasks, they may benefit from additional outreach.


From an API shape and ergonomics perspective, there are 2 open issues, which seems like something we should address before shipping: #35 and #15.
Both make some ergonomics tradeoff in favor of runtime performance, but it'd be good to determine: 
a) what the performance differences are? 
b) can we optimize those away or are they inherent to the API shape?

On top of that, it might be worthwhile to think through the interaction of this API and Long Tasks#37 


Security

Implementers must take care to ensure that frames may not detect input events from other cross-origin frames. In Blink, this is achieved by performing hit testing on the compositor side for cross-origin frames within-process, and via viz for OOPIFs.



Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes


Is this feature fully tested by web-platform-tests?

Yes (https://wpt.fyi/results/is-input-pending)


Tracking bug

https://bugs.chromium.org/p/chromium/issues/approval?id=930987


Sample links

https://wicg.github.io/is-input-pending/#examples-of-usage


Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5719830432841728


Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/d/msg/blink-dev/eLq8t56CbaQ/KdbOY7ZHCwAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/d/msg/blink-dev/ItkbDBevOrs/yFffQlqTBQAJ



--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/BYAPR15MB2728DBE16ED8A83933E33C7EC73F0%40BYAPR15MB2728.namprd15.prod.outlook.com.

matan genish

no llegida,
22 de set. 2020, 13:31:2622/9/20
a blink-dev,yo...@yoav.ws,blink-dev,Andrew Comminos
Really looking forward for this new API.
We have few ideas on how to leverage it for Excel Online.

Philip Walton

no llegida,
24 de set. 2020, 11:55:3824/9/20
a blink-dev,matan genish,yo...@yoav.ws,blink-dev,Andrew Comminos
We might want to consider holding off on shipping this until we resolve how this API interacts with the Long Tasks API. Specifically this issue:

Since my understanding is this API is planned to be included in React (and possibly other frameworks), the users of those frameworks might update to the latest version and test their performance in Lighthouse—only to see a regression due to the increased lengths of Long Tasks.

Kenneth Rohde Christiansen

no llegida,
24 de set. 2020, 12:46:4424/9/20
a Philip Walton,blink-dev,matan genish,yo...@yoav.ws,Andrew Comminos
Apple is skeptical of this new API as stated on webkit-dev:

https://www.mail-archive.com/webki...@lists.webkit.org/msg29681.html
> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/947e22f6-b811-4117-a9e5-9adad9015a61n%40chromium.org.



--
Kenneth Rohde Christiansen
Web Platform Architect, Intel Corporation.
Phone +45 4294 9458 ﹆﹆﹆

Daniel Bratell

no llegida,
24 de set. 2020, 15:50:0424/9/20
a blin...@chromium.org
I think Apple's response is reasonable from a theoretical point of view,
but on the other hand it seems like several applications have been able
to achieve substantial improvements using the isInputPending API.

From that point of view it is either isInputPending or some other
change that can result in similar improvements. I see that many people
have talked about this a lot, but since I'm not one of those, could
someone (acomminos?) recap the alternative approaches?

/Daniel

Andrew Comminos

no llegida,
24 de set. 2020, 18:37:0924/9/20
a blin...@chromium.org
Thanks for the feedback all! Responses inline.

On 9/24/20 12:49 PM, Daniel Bratell wrote:
> I think Apple's response is reasonable from a theoretical point of view,
> but on the other hand it seems like several applications have been able
> to achieve substantial improvements using the isInputPending API.
>
> From that point of view it is either isInputPending or some other
> change that can result in similar improvements. I see that many people
> have talked about this a lot, but since I'm not one of those, could
> someone (acomminos?) recap the alternative approaches?
>
> /Daniel

The alternative to isInputPending is having scheduling logic yield more
frequently, which can cause a significant throughput hit when performing
user-blocking work. This is the result of the scheduler being starved in
favour of lower-priority work by the UA or other script/frames, as well
as inherent task processing overhead.

A high level overview of the problem space is available here:

https://github.com/WICG/is-input-pending/blob/master/README.md#the-problem

> On 2020-09-24 18:46, Kenneth Rohde Christiansen wrote:
>> Apple is skeptical of this new API as stated on webkit-dev:
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mail-2Darchive.com_webkit-2Ddev-40lists.webkit.org_msg29681.html&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=eOKkJ4TL38BfPIxR7LVj1CE8Fxnyosruf0KSe6mjPAo&e=
>>
>> On Thu, Sep 24, 2020 at 5:55 PM 'Philip Walton' via blink-dev
>> <blin...@chromium.org> wrote:
>>> We might want to consider holding off on shipping this until we
>>> resolve how this API interacts with the Long Tasks API. Specifically
>>> this issue:
>>> https://github.com/WICG/is-input-pending/issues/37
>>>
>>> Since my understanding is this API is planned to be included in React
>>> (and possibly other frameworks), the users of those frameworks might
>>> update to the latest version and test their performance in
>>> Lighthouse—only to see a regression due to the increased lengths of
>>> Long Tasks.

I've left my thoughts in that issue, but it seems unlikely that the
activation energy is strong enough to require blocking on -- for the
React case you describe, developers would have to switch to concurrent
mode to leverage this (which is not available in a stable release yet).
Additionally, these numbers will be disrupted (positively) when moving
to concurrent mode regardless.

It's worth noting that frameworks with schedulers can't do _all_ of
their work in long tasks leveraging isInputPending, especially when they
want to paint responsively. It's typically beneficial during early load
when user input would change the result of paint.

>>> On Tuesday, September 22, 2020 at 10:31:26 AM UTC-7 matan genish wrote:
>>>> Really looking forward for this new API.
>>>> We have few ideas on how to leverage it for Excel Online.
>>>>
>>>> On Monday, September 21, 2020 at 10:22:57 AM UTC+3 yo...@yoav.ws wrote:
>>>>> Thanks for working on this! I believe this API can unlock
>>>>> significant performance benefits.
>>>>>
>>>>> On Fri, Sep 18, 2020 at 9:34 PM 'Andrew Comminos' via blink-dev
>>>>> <blin...@chromium.org> wrote:
>>>>>> Contact emails
>>>>>>
>>>>>> acom...@fb.com, pani...@chromium.org, tdre...@chromium.org
>>>>>>
>>>>>>
>>>>>> Explainer
>>>>>>
>>>>>> https://github.com/WICG/is-input-pending
>>>>>>
>>>>>>
>>>>>> Specification
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__wicg.github.io_is-2Dinput-2Dpending_&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=b3YSYz29rjXSHuUe2pJyXkTvSnH2AqyXX7uGWiPUow4&e=
>>>>>> (https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.webkit.org_pipermail_webkit-2Ddev_2020-2DSeptember_031390.html&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=wWr7C8mNAwpM5sWYboPX3XY7asp3o3-QyqX2nlK1QQM&e=
Thanks for the pings, I've gone ahead and aggregated some benchmarks for
those issues.

We arrived at a minor API change after looking more closely at #35.
However, the solution proposed in #15 constrains the API with the burden
of promise allocation (which, as measured, can be quite expensive in a
hot loop). I believe the concerns in #15 would be better addressed with
a separate yielding API, as discussed in the issue.

>>>>>> Security
>>>>>>
>>>>>> Implementers must take care to ensure that frames may not detect
>>>>>> input events from other cross-origin frames. In Blink, this is
>>>>>> achieved by performing hit testing on the compositor side for
>>>>>> cross-origin frames within-process, and via viz for OOPIFs.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Will this feature be supported on all six Blink platforms
>>>>>> (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
>>>>>>
>>>>>> Yes
>>>>>>
>>>>>>
>>>>>> Is this feature fully tested by web-platform-tests?
>>>>>>
>>>>>> Yes
>>>>>> (https://urldefense.proofpoint.com/v2/url?u=https-3A__wpt.fyi_results_is-2Dinput-2Dpending&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=z7fAmqCnne5B2fD1cp1rUZ5s1aZLfDMXOp5XSp30IE8&e=
>>>>>> )
>>>>>>
>>>>>>
>>>>>> Tracking bug
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.chromium.org_p_chromium_issues_approval-3Fid-3D930987&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=Gha8CTMM4S4iDosTNKibyyKhdZLjKm8cBN5MW2J1g_w&e=
>>>>>>
>>>>>>
>>>>>> Sample links
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__wicg.github.io_is-2Dinput-2Dpending_-23examples-2Dof-2Dusage&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=YlZfXWy0GPE4RWpjstlWpTkUneeufmrEr6V4IN1g2oA&e=
>>>>>>
>>>>>>
>>>>>> Link to entry on the Chrome Platform Status
>>>>>>
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__chromestatus.com_feature_5719830432841728&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=1DCrtFSVEN5tF9r07tYJABOpS0JBzSP4UJEk-YfPIOk&e=
>>>>>>
>>>>>>
>>>>>> Links to previous Intent discussions
>>>>>>
>>>>>> Intent to Prototype:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_a_chromium.org_d_msg_blink-2Ddev_eLq8t56CbaQ_KdbOY7ZHCwAJ&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=2nR0555YVbffpiE0Gp0kHtI8diB3IFQnaR-DNkJ4z6E&e=
>>>>>>
>>>>>> Intent to Experiment:
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_a_chromium.org_d_msg_blink-2Ddev_ItkbDBevOrs_yFffQlqTBQAJ&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=xoLz_YzejhQ8gIwnGXhL2s9FxBdgT-Y9f4TFlZ9docw&e=
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "blink-dev" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to blink-dev+...@chromium.org.
>>>>>> To view this discussion on the web visit
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_a_chromium.org_d_msgid_blink-2Ddev_BYAPR15MB2728DBE16ED8A83933E33C7EC73F0-2540BYAPR15MB2728.namprd15.prod.outlook.com&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=57ohRSn25tJakgEMM9gXlhD8hHl2R7mU8Tw0e8q5kGo&e=
>>>>>> .
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "blink-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it,
>>> send an email to blink-dev+...@chromium.org.
>>> To view this discussion on the web visit
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_a_chromium.org_d_msgid_blink-2Ddev_947e22f6-2Db811-2D4117-2Da9e5-2D9adad9015a61n-2540chromium.org&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=f4IqI-T64EhPzlPNDufP_w&m=G4UeKHarmKWvOL-jGqn6gyitV7KEb72pezmk4w95TlQ&s=7oG1S6VMYVXioZeflaaxmfFVS55Fqz8FtxBwjTzfEFw&e=
>>> .
>>
>>
>

Yoav Weiss

no llegida,
1 d’oct. 2020, 4:11:051/10/20
a Andrew Comminos,blink-dev
A few of us discussed the intent, the open issues and potential concerns regarding the API shape, which led me to reach the following conclusions:
  • Regarding issue 15:
    • The API doesn't currently allow yielding code to *know* that the reason for which it's yielding (e.g. specific pending input) is done
    • At the same time, applications already opportunistically yield today, assuming that the browser would handle high priority work before giving them back the main thread. The API would enable such apps to yield less when it's not necessary, resulting in significant performance benefits
    • `scheduler.yield()` is likely to be a complementary API to this, and can assist in cases where the caller needs to know the input was addressed. At the same time, it's not yet clear what exact shape it would take and when it would be ready.
  • Regarding issue 37, the intended use of the API should not incur more long tasks, as apps are expected to yield before long tasks occur. As such, integration with Long Tasks seem less critical, and not a blocker. (as RUM providers and synthetic testing tools should still warn developers of code that triggers LTs, even if it's calling isInputPending).
  • Issue 35 was addressed.
Given the above, LGTM1 to ship.



--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

Daniel Bratell

no llegida,
1 d’oct. 2020, 15:10:451/10/20
a blink-dev,yo...@yoav.ws,blink-dev,Andrew Comminos

LGTM2

/Daniel

Mike West

no llegida,
1 d’oct. 2020, 15:17:061/10/20
a blink-dev,Daniel Bratell,yo...@yoav.ws,blink-dev,Andrew Comminos
LGTM3.
Respon a tots
Respon a l'autor
Reenvia
0 missatges nous