Intent to Experiment (Origin Trial): Long Task Observer

[Shubhie Panicker]

Spec

Explainer: https://github.com/spanicker/longtasks

Discussed the API with W3C Web-Perf group at TPAC. Strong interest in this API from developers (eg. Facebook, msn.com), analytics / RUM vendors (eg. SOASTA) and ads vendors (Google Ads).

Summary

A new performance API to enable applications to detect presence of “long tasks” that monopolize the UI thread for extended periods of time and block other critical tasks from being executed - e.g. reacting to user input.

Preliminary data confirms that the API is very promising as an indicator of how performance-tuned the site is, and the user's experience on it. Running an Origin Trial is the next step here.

Link to “Intent to Implement” blink-dev discussion

https://groups.google.com/a/chromium.org/forum/#!searchin/blink-dev/intent$20long$20task/blink-dev/A_sM-fu6u50/ao9mnO8SAQAJ

Goals for experimentation

• Validate whether the API is a good problem signal for user responsiveness

• Validate whether the API is a good problem signal for (ir)responsible third-party content

• Inform discussion on types of attribution that should be surfaced for V2 API

Experimental timeline

Oct 6: Enabled Chrome 55 branch Dec 6: Chrome 55 Stable Jan 31st: Chrome 56 Stable: 7 weeks of 55 stable channel feedback Mar 14 2017 Chrome 57 Stable: 13 weeks of 55 stable channel feedback; 4 weeks of 56 stable channel feedback.

Alternately if approvals are slow:

Nov 4: Enabled Chrome 56

Jan 31st: Chrome 56 Stable

Mar 14 2017 Chrome 57 Stable: 4 weeks of 56 stable channel feedback.

Apr 25 2017 Chrome 58 Stable: 10 weeks of 56 stable channel feedback; 5 weeks of 57 stable channel feedback.

Any risks when the experiment finishes?

No, this is a monitoring API.

Reason this experiment is being extended

N/A

Ongoing technical constraints

None.

Will this feature be supported on all five Blink platforms supported by Origin Trials (Windows, Mac, Linux, Chrome OS, and Android)?

Yes.

OWP launch tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=651926

[Shubhie Panicker]

Contact emails:

pani...@google.com, igri...@google.com, nd...@google.com

[Chris Harrelson]

LGTM

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Philip Jägenstedt]

LGTM2

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[ben.m...@gmail.com]

Excited to hear about the origin trial here. Two quick questions:

1) What resolution did you guys come to around handling things other than scripts (eg style and layout)? Will this API at least expose the existence of such tasks (even if it is not able to attribute them). Put another way -- is this origin trial a substitution for the periodic heartbeat approach (which is able to detect any instance where the main thread is occupied). It might be helpful to document a bit about the current state of the evolution of the API to focus the feedback of any folks using the experiment.

2) Is there any way to configure a corporate chrome config to explicitly allow this feature for only FB origins? We'd like to be able to test this feature on a large percentage of FB employees, even if the experiment ends up getting throttled.

-b

[Rick Byers]

LGTM3

[Ian Clelland]

On Sat, Oct 1, 2016 at 6:57 PM, <ben.m...@gmail.com> wrote:

Excited to hear about the origin trial here. Two quick questions:

1) What resolution did you guys come to around handling things other than scripts (eg style and layout)? Will this API at least expose the existence of such tasks (even if it is not able to attribute them). Put another way -- is this origin trial a substitution for the periodic heartbeat approach (which is able to detect any instance where the main thread is occupied). It might be helpful to document a bit about the current state of the evolution of the API to focus the feedback of any folks using the experiment.

2) Is there any way to configure a corporate chrome config to explicitly allow this feature for only FB origins? We'd like to be able to test this feature on a large percentage of FB employees, even if the experiment ends up getting throttled.

For local use, where you can control all of the clients, this should be possible. Even without the origin trial running, there should be a setting in chrome://flags to enable the API for all origins.

Looking at https://www.chromium.org/administrators/policy-list-3, we might have to add explicit support for enabling experimental features -- akin to the EnableDeprecatedWebPlatformFeatures flag, but for proposed features, rather than deprecated ones.

Ian

[ben.m...@gmail.com]

Yeah, another challenge here is that in order for us to deploy this we'd really want to be able to limit domains to ones under our control. Presumably there is less privacy and security scrutiny for experimental web features that are behind a flag. Even for things gated by an origin trial we'd want to make sure that if the origin trial were to be disabled the feature would also be disabled. We'd be OK opening the feature to domains we own, but we wouldn't want FB employees to be exposed to a different set of features than the general population.

One other gating related challenge I noticed -- It seems like origin trials does not allow one to get an origin trial for a wildcard domain, such as *.facebook.com. This makes it difficult for us to do testing because each engineer's development version of FB has a unique domain, eg, www.bmaurer.sb.facebook.com. We also use subdomains for other purposes such as our @Work feature (mycorp.facebook.com is the Facebook version for mycorp). 

None of these are reasons not to experiment -- but in the long run hoping this feedback will make it easier to do testing on large sites such as Facebook.

[Shubhie Panicker]

chromestatus link: https://www.chromestatus.com/feature/5738471184400384

On Mon, Oct 3, 2016 at 8:36 AM, <ben.m...@gmail.com> wrote:

Yeah, another challenge here is that in order for us to deploy this we'd really want to be able to limit domains to ones under our control. Presumably there is less privacy and security scrutiny for experimental web features that are behind a flag. Even for things gated by an origin trial we'd want to make sure that if the origin trial were to be disabled the feature would also be disabled. We'd be OK opening the feature to domains we own, but we wouldn't want FB employees to be exposed to a different set of features than the general population.

One other gating related challenge I noticed -- It seems like origin trials does not allow one to get an origin trial for a wildcard domain, such as *.facebook.com. This makes it difficult for us to do testing because each engineer's development version of FB has a unique domain, eg, www.bmaurer.sb.facebook.com. We also use subdomains for other purposes such as our @Work feature (mycorp.facebook.com is the Facebook version for mycorp). 

None of these are reasons not to experiment -- but in the long run hoping this feedback will make it easier to do testing on large sites such as Facebook.

On Monday, October 3, 2016 at 7:15:32 AM UTC-7, Ian Clelland wrote:

On Sat, Oct 1, 2016 at 6:57 PM, <ben.m...@gmail.com> wrote:

Excited to hear about the origin trial here. Two quick questions:

1) What resolution did you guys come to around handling things other than scripts (eg style and layout)? Will this API at least expose the existence of such tasks (even if it is not able to attribute them). Put another way -- is this origin trial a substitution for the periodic heartbeat approach (which is able to detect any instance where the main thread is occupied). It might be helpful to document a bit about the current state of the evolution of the API to focus the feedback of any folks using the experiment.

Good question, I will update the explainer and add a new section on Origin Trial, and what we are trying to answer there.

The short answer is that this not a drop-in replacement for heartbeat approach for you at the moment, as we only show 50ms tasks currently. 

The longer term goal is to provide a replacement for such workarounds, however this requires further security review.

2) Is there any way to configure a corporate chrome config to explicitly allow this feature for only FB origins? We'd like to be able to test this feature on a large percentage of FB employees, even if the experiment ends up getting throttled. 

For local use, where you can control all of the clients, this should be possible. Even without the origin trial running, there should be a setting in chrome://flags to enable the API for all origins.

Looking at https://www.chromium.org/administrators/policy-list-3, we might have to add explicit support for enabling experimental features -- akin to the EnableDeprecatedWebPlatformFeatures flag, but for proposed features, rather than deprecated ones.

Thanks Ian for chiming is here.

I'll start a separate thread looping in appropriate folks on how to go about the Origin Trial / direct flag experiment. (Thanks for your interest in experimenting with this!)

The Origin Trials framework is quite new, so some speed bumps are expected and feedback is certainly appreciated.

  

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[ben.m...@gmail.com]

1) What resolution did you guys come to around handling things other than scripts (eg style and layout)? Will this API at least expose the existence of such tasks (even if it is not able to attribute them). Put another way -- is this origin trial a substitution for the periodic heartbeat approach (which is able to detect any instance where the main thread is occupied). It might be helpful to document a bit about the current state of the evolution of the API to focus the feedback of any folks using the experiment.

Good question, I will update the explainer and add a new section on Origin Trial, and what we are trying to answer there.

The short answer is that this not a drop-in replacement for heartbeat approach for you at the moment, as we only show 50ms tasks currently. 

The longer term goal is to provide a replacement for such workarounds, however this requires further security review.

Our heartbeat approach also ends up having a low resolution, in the ballpark of 50 ms, so we might actually be able to use it as a drop in. My main concern was if long tasks will allow visibility into all tasks on the main thread, even if they are things like style/layout or even another tab's JS. Obviously such tasks might not have a clear name, but heartbeat at least allows us to see the existence of such events. 

 

2) Is there any way to configure a corporate chrome config to explicitly allow this feature for only FB origins? We'd like to be able to test this feature on a large percentage of FB employees, even if the experiment ends up getting throttled. 

For local use, where you can control all of the clients, this should be possible. Even without the origin trial running, there should be a setting in chrome://flags to enable the API for all origins.

Looking at https://www.chromium.org/administrators/policy-list-3, we might have to add explicit support for enabling experimental features -- akin to the EnableDeprecatedWebPlatformFeatures flag, but for proposed features, rather than deprecated ones.

Thanks Ian for chiming is here.

I'll start a separate thread looping in appropriate folks on how to go about the Origin Trial / direct flag experiment. (Thanks for your interest in experimenting with this!)

The Origin Trials framework is quite new, so some speed bumps are expected and feedback is certainly appreciated.

Awesome, I'm more than happy to talk with folks about the challenges FB faces here.

-b 

[Shubhie Panicker]

On Tue, Oct 4, 2016 at 11:55 AM, <ben.m...@gmail.com> wrote:

1) What resolution did you guys come to around handling things other than scripts (eg style and layout)? Will this API at least expose the existence of such tasks (even if it is not able to attribute them). Put another way -- is this origin trial a substitution for the periodic heartbeat approach (which is able to detect any instance where the main thread is occupied). It might be helpful to document a bit about the current state of the evolution of the API to focus the feedback of any folks using the experiment.

Good question, I will update the explainer and add a new section on Origin Trial, and what we are trying to answer there.

The short answer is that this not a drop-in replacement for heartbeat approach for you at the moment, as we only show 50ms tasks currently. 

The longer term goal is to provide a replacement for such workarounds, however this requires further security review.

Our heartbeat approach also ends up having a low resolution, in the ballpark of 50 ms, so we might actually be able to use it as a drop in. My main concern was if long tasks will allow visibility into all tasks on the main thread, even if they are things like style/layout or even another tab's JS. Obviously such tasks might not have a clear name, but heartbeat at least allows us to see the existence of such events.

Yes long tasks will surface ALL long tasks - as we observe and report all tasks on the toplevel message loop.

That would be great if you could try replacing your heartbeat approach with this, and provide feedback.

 

 

2) Is there any way to configure a corporate chrome config to explicitly allow this feature for only FB origins? We'd like to be able to test this feature on a large percentage of FB employees, even if the experiment ends up getting throttled. 

For local use, where you can control all of the clients, this should be possible. Even without the origin trial running, there should be a setting in chrome://flags to enable the API for all origins.

Looking at https://www.chromium.org/administrators/policy-list-3, we might have to add explicit support for enabling experimental features -- akin to the EnableDeprecatedWebPlatformFeatures flag, but for proposed features, rather than deprecated ones.

Thanks Ian for chiming is here.

I'll start a separate thread looping in appropriate folks on how to go about the Origin Trial / direct flag experiment. (Thanks for your interest in experimenting with this!)

The Origin Trials framework is quite new, so some speed bumps are expected and feedback is certainly appreciated.

Awesome, I'm more than happy to talk with folks about the challenges FB faces here.

 

-b 

--

[Shubhie Panicker]

I'd like to request extending this origin trial through to M57 (i.e. until M58 hits stable).

We were not able to start the trial due to some issues:

- our partners: Google Ads & SOASTA were not able to enable the trial due to lack of support for 3rd parties to opt in (addressed in M57)

- also FB and Youtube had requested support for wildcard support (this was apparently fixed in M55, but I wasn't aware due to communication snafu).

Finally, we were receiving feedback from canary, and updated the API based on that - so running Origin Trial wasn't really preventing progress.

However Google Ads & SOASTA would like to run the Origin Trial in M57 and provide feedback, for Long Tasks V2 API (currently under exploration / design), so I'd like to request an extension for M57. (Note that Long Tasks V1 API is shipping in M58)

[Chris Harrelson]

Sounds reasonable. Thanks for the detailed justification.

LGTM

[Shubhie Panicker]

The Origin Trial team (Owen) has requested sending this update on our trial as part of their process.

As I've mentioned in this thread, our partners - Google Ads & SOASTA - were not able to enable the trial due to lack of support for 3rd parties to opt in. This was addressed in M57.

For this reason we haven't put effort into promoting it, and we have only had 4 sign ups.

Google Ads & SOASTA are expected to experiment in M57 and we expect to share more meaningful feedback at that point.

[Shubhie Panicker]

Ditto for M56. No updates due to above reason.

PS: Apologies for the spam, just following the Origin Trial mandatory process.

[owe...@google.com]

Is the plan to update the OT timeline to be enabled in 57, 58 and 59 since we only now believe it's really useful? Else are we still planning to disable in 58?

Additionally, since 57 is now in stable with Long Task Observer available, it'd be great to put together a blog post for developers.google.com/web so we can tell people to try it out. 

As a friendly reminder, we generally recommend feature teams publish a blog post at the Beta point so they can have developers signing up immediately, so when tokens expire in 6 weeks they can start looking at the feedback fairly fast. Of course that's not possible now, but worth bearing in mind in the future!

-b 

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[mbened...@gmail.com]

It's been 2 years, but I had to end this discussion by mentioning that since the release of Chrome 58, it's disabled by default, but can be shown by checking Verbose in the logs.

[Nicolás Peña]

Longtasks are observable via PerformanceObserver, see example in the spec.