Intent to Ship: Increased max nesting level for setTimeout(0)

[Etienne Pierre-doray]

Contact emails

etie...@chromium.org

Specification

https://html.spec.whatwg.org/multipage/timers-and-user-prompts.html

Design docs

https://docs.google.com/document/d/1boT0k8BQjl7mXXzvI9SdN4XJPSza27vE8T0CNxmMhCI

Summary

Increase the nesting threshold before which setTimeout(..., <4ms) start being clamped, from 5 to 15. setTimeout(..., 0) is commonly used to break down long Javascript tasks and let other internal tasks run, which prevents the browser from hanging. setTimeouts and setIntervals with an interval < 4ms are not clamped as aggressively as they were before. This improves short horizon performance, but websites abusing the API will still eventually have their set setTimeouts clamped

Blink component

Blink

TAG review

TAG review status

Not applicable

Risks

Interoperability and Compatibility

setTimeout is a well established and mature API. This change poses a risk of breaking websites and tests that rely on the current timing caused by clamping and the subtle task ordering that it entails. As an example, this change breaks assumptions about the ordering between setTimeout(0) and unrelated tasks in at least one case in Chrome tests (crbug.com/1302309). On the flip side, the implementation in Chrome is already non compliant (crbug.com/1108877). There's also a similar experiment on beta that is ongoing (crbug.com/1263190). Devs can use chrome://flags#unthrottled-nested-timeout to test their sites for compatibility issues.

Gecko: No signal

WebKit: Shipped/Shipping (https://sourcegraph.com/github.com/WebKit/WebKit/-/commit/786e3e0b252e38fb01c8db97a94d52cb0f57891e)

Web developers: No signals

Other signals:

WebView application risks

Does this intent deprecate or change behavior of existing APIs, such that it has potentially high risk for Android WebView-based applications?

Debuggability

setTimeout() and setInterval() have an associated trace event in DevTools. https://developer.chrome.com/docs/devtools/evaluate-performance/performance-reference/

Will this feature be supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

No

Is this feature fully tested by web-platform-tests?

No

Flag name

unthrottled-nested-timeout

Requires code in //chrome?

False

Tracking bug

https://crbug.com/1108877

Launch bug

https://launch.corp.google.com/launch/4201069

Estimated milestones

Chrome for desktop:	108
Chrome for Android:	108
Android Webview	108

Anticipated spec changes

The spec dictates a nesting threshold of 5 

https://html.spec.whatwg.org/multipage/timers-and-user-prompts.html "If nesting level is greater than 5, and timeout is less than 4, then set timeout to 4." Chrome has never respected the exact behavior (crbug.com/1108877), and safari recently updated the threshold to 10 (https://github.com/WebKit/WebKit/commit/786e3e0b252e38fb01c8db97a94d52cb0f57891e). A potential change to the spec is to make the threshold "implementation dependent" to match reality.

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5710690097561600

Links to previous Intent discussions

Ready for Trial: https://groups.google.com/a/chromium.org/g/blink-dev/c/-TjeYs7shTQ/m/FhJq0mQyDAAJ
Intent to Experiment: https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALoDvsZNwh5uBANxryWHCdgFVFCts6noKSU9FY1BcqYH0%3D55sg%40mail.gmail.com
Intent to Extend Experiment: https://mail.google.com/mail/u/0/#search/in%3Asent+settimeout/KtbxLzGLjTQPrFFjRfPrfQCtcCmwvTksJV

This was previously enabled through field-trial on Beta 50% and Stable 1% on M104-105, with a more aggressive nesting threshold = 100. No breakage was reported, but it showed small guiding metrics (startup) regressions on Android. I'm confident that having a lower threshold will eliminate the adverse effects. Ideally, I would conduct another round of field-trial, but I think we're better off with doing a waterfall roll-out, and experiment later on (1% stable) to confirm no regressions:

• Gradually rolling-out to each channel at 100% has more chances of teasing out potential brokerage and won't be perceived as flaky failures. I will loop back after 100% beta before we hit stable.
• This will involve fewer back and forth with blink-dev / API owners, and allow us to benefit from performance gains sooner.

This intent message was generated by Chrome Platform Status.

[Alex Russell]

This intent was the subject of a long discussion at API OWNERS today, and I'm wondering if we understand the constituent test in Speedometer (or the harness) that is favouring Safari's out-of-spec behaviour? 

Speedometer seems like the key motivator here, rather than public content, and winning it matters in the interim while Apple is gaming this for the purposes of benchmarketing. I'd support finching this on for Stable for some releases while we get resolution on fixing the benchmark.

Best,

Alex

[Etienne Pierre-doray]

Thanks for taking the time to discuss this.

I'm wondering if we understand the constituent test in Speedometer (or the harness) that is favouring Safari's out-of-spec behaviour? 

There's some context in crbug.com/1297550 and in speedometer2.1 release notes; Speedometer 2.1 hopefully fixes the benchmark to mitigate the impact of throttling setTimeout(0) (in local experiment, it does reduce improvements we can get with this change).

Speedometer seems like the key motivator here, rather than public content

Correct. I think Speedometer2.0 is the main motivator to shipping this in a timely manner. +sky@ who has been championing moving forward with this change. 

Ideally we can prove making this change has no negative impact on metrics we care about.

Another (long-term) benefit is perhaps to move away from the spec-mendated threshold (which is somewhat arbitrary) and hopefully take it away from the spec. A hard-to-prove benefit of removing the 4ms clamping is to match more closely the devs intent when they write setTimeout(0), and give the browser more leeway in implementing a throttling policy.

I'd support finching this on for Stable for some releases while we get resolution on fixing the benchmark.

I'm experimenting on M107 (with nesting threshold = 15) and will ramp up to 1% Stable soon. 

(We also experimented with Stable 1% on M104-105 for a different value (nesting = 100), which showed no regression on Windows / MacOs, but regressed startup time by 0.5% at the median on Android).

If finching for one milestone is enough to confirm no regression (from a metrics perspective, I believe it's enough to get statistically significant data), I'm hoping we can optimistically ship on M108 through a waterfall roll-out. Otherwise, maybe we can delay shipping 1+ milestone.

Another option we discussed is to ship as-is on desktop only (and figure out Android later), but I feel like this creates a more inconsistent platform.

[Alex Russell]

Thanks Etienne; questions inline:

On Thursday, October 20, 2022 at 8:04:19 AM UTC-7 Etienne Pierre-doray wrote:

Thanks for taking the time to discuss this.

I'm wondering if we understand the constituent test in Speedometer (or the harness) that is favouring Safari's out-of-spec behaviour? 

There's some context in crbug.com/1297550 and in speedometer2.1 release notes; Speedometer 2.1 hopefully fixes the benchmark to mitigate the impact of throttling setTimeout(0) (in local experiment, it does reduce improvements we can get with this change).

So if I go directly to browserbench.org and click on "speedometer", it takes me to:

https://browserbench.org/Speedometer2.1/

Is that a very new change? Is there a reason for us to continue to look at (or cite) 2.0 when 2.1 is live? 

 

Speedometer seems like the key motivator here, rather than public content

Correct. I think Speedometer2.0 is the main motivator to shipping this in a timely manner. +sky@ who has been championing moving forward with this change. 

What's the residual delta now that 2.1 is available to test against?

 

Ideally we can prove making this change has no negative impact on metrics we care about.

Another (long-term) benefit is perhaps to move away from the spec-mendated threshold (which is somewhat arbitrary) and hopefully take it away from the spec. A hard-to-prove benefit of removing the 4ms clamping is to match more closely the devs intent when they write setTimeout(0), and give the browser more leeway in implementing a throttling policy.

I'd support finching this on for Stable for some releases while we get resolution on fixing the benchmark.

I'm experimenting on M107 (with nesting threshold = 15) and will ramp up to 1% Stable soon. 

(We also experimented with Stable 1% on M104-105 for a different value (nesting = 100), which showed no regression on Windows / MacOs, but regressed startup time by 0.5% at the median on Android).

If finching for one milestone is enough to confirm no regression (from a metrics perspective, I believe it's enough to get statistically significant data), I'm hoping we can optimistically ship on M108 through a waterfall roll-out. Otherwise, maybe we can delay shipping 1+ milestone.

Presumably the downside of this change is in power/battery? Are there other impacts we're looking at?

 

Another option we discussed is to ship as-is on desktop only (and figure out Android later), but I feel like this creates a more inconsistent platform.

+1 to doing this in a more uniform way.

Best,

Alex

[Etienne Pierre-doray]

Is that a very new change? Is there a reason for us to continue to look at (or cite) 2.0 when 2.1 is live? 

I think that happened in August. sky@ or +hpayer@ might be able to answer this.

What's the residual delta now that 2.1 is available to test against?

 Measured on Canary MacBook pro M1, this gets lost in the noise.

 

Presumably the downside of this change is in power/battery? Are there other impacts we're looking at?

That's a relevant question. On one side, a website that loops setTimeout(0) indefinitely would use more CPU until it starts getting throttled. We have good CPU metrics on Mac and Android, which haven't regressed in the experiments so far (21 days 1% stable), which means this doesn't happen enough to make a dent. On the other side, this feature can also reduce CPU wakeups for a fixed amount of work (wakeups have an inherent cost on intel), although wake ups metrics haven't seen any significant shift in the wild.

[Hannes Payer]

On Fri, Oct 21, 2022 at 12:31 PM Etienne Pierre-doray <etie...@chromium.org> wrote:

Is that a very new change? Is there a reason for us to continue to look at (or cite) 2.0 when 2.1 is live? 

I think that happened in August. sky@ or +hpayer@ might be able to answer this.

2.1 is the new version. We can ignore 2.0.

--

 

Hannes Payer |

V8 |

Google Germany GmbH |

Erika-Mann Str. 33, 80636 München

Geschäftsführer: Paul Manicle, Liana Sebastian

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls Sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde. 

     

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Etienne Pierre-doray]

2.1 is the new version. We can ignore 2.0.

I suppose this doesn't need to be released on M108 then. I'll keep experimenting.

[Mike Taylor]

Sounds good - should we consider this Intent withdrawn for now?

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CALoDvsZaOO4ksGXnVxgw8Zv1_modDWQWYMurYjMfs8UX0BCUUA%40mail.gmail.com.

[Etienne Pierre-doray]

Sounds good - should we consider this Intent withdrawn for now?

Yes, let's drop this for now and maybe revisit when I have numbers.