Intent to Ship: JS Self-Profiling API

201 views
Skip to first unread message

Andrew Comminos

unread,
Jul 28, 2021, 2:25:00 PMJul 28
to blin...@chromium.org

Contact emails

acom...@fb.com, tdre...@chromium.org, n...@chromium.org


Explainer

https://www.github.com/WICG/js-self-profiling


Specification

https://wicg.github.io/js-self-profiling/


Summary

Adds a web-exposed sampling profiler for measuring client JavaScript execution time. Gathering JS profiles from real users can help developers debug slow observed performance without the need for invasive manual instrumentation.


Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?

Yes


Debuggability

No additional DevTools support is required for this feature.


Measurement

The Profiler constructor is gated behind a UseCounter to track adoption.


Blink component

Blink>PerformanceAPIs


Search tags

profiler, profiling, js


TAG review

https://github.com/w3ctag/design-reviews/issues/366


TAG review status

Issues addressed


Risks



Interoperability and Compatibility

As the API's purpose is to gather JS traces for analysis, the primary interoperability risk is that traces from different UAs cannot be treated interchangeably. The spec attempts to resolve this by leveraging the concept of function instance names (defined in ECMA-262) to label stack frames, ensuring consistency for analysis purposes.

Developers should still exercise caution when aggregating traces between different user agents, as they may support different sampling intervals or have significantly different performance characteristics because of VM differences.

This API has been discussed at length in the Web Performance Working Group during various calls, as well as TPAC 2018, 2019, and 2020. Additionally, the WebAppSec Working Group was consulted when considering gating the API behind a cross-origin isolation requirement.


Gecko: No signal (https://github.com/mozilla/standards-positions/issues/477)


WebKit: Negative (https://github.com/mozilla/standards-positions/issues/477#issuecomment-765106364)


Web developers: Strongly positive


Facebook has been able to debug and resolve a wide variety of performance issues during the origin trial period. We have also seen quite a bit of support from other large web properties:


Microsoft: https://github.com/WICG/js-self-profiling/issues/24

Elastic: https://github.com/WICG/js-self-profiling/issues/21

Boomerang: https://github.com/WICG/js-self-profiling/issues/14

Dropbox: https://github.com/WICG/js-self-profiling/issues/9


Ergonomics

This API is designed to be used in tandem with the performance timeline family of APIs -- sample timestamps in traces share a clock and time origin with HR-Time.

The largest concern with exposing a sampling profiler to the web is reducing the performance of the page if profiling is enabled liberally. As the UA has freedom to choose which sampling intervals it supports, the only mandatory overhead required is building the necessary code mapping for resolving unwinded stacks. Because the API is gated on document policy, any cost will only be incurred on pages that indicate they wish to profile by exposing the appropriate document policy header.



Activation

To leverage the tracing data effectively, a web service to ingest and aggregate traces is useful for analysis. This API would likely benefit from the publishing of open-source libraries as well as documentation to help develop such a service (although OT partners did not encounter significant challenges).



Security

The API was designed to provide no additional insight into the contents or execution characteristics of cross-origin scripts, nor the execution of cross-origin frames.

Contents of cross-origin opaque scripts are hidden by requiring all functions included via the spec's take a sample algorithm to be defined in a script served with CORS-same-origin through HTML’s muted errors property. Browser builtins (such as performance.now()) may still be included when invoked from cross-origin no-cors script (albeit with no information about the invoking script, line, or column number), although this is already made possible through manually overriding global objects. As a result, the API does not expose any new insight into the contents or execution characteristics of cross-origin script, beyond what is already possible through manual instrumentation (e.g. via Error.stack or runtime script modification).

Cross-origin execution contexts should not be observable by the API through the realm check in the spec's take a sample algorithm. Cross-origin iframes and other execution contexts that share an agent with a profiler will therefore not have their execution observable through this API.

Timing attacks remain a concern for any API that could introduce a new source of high-resolution timing information. As per a review (link) from the Edge security team, plus consultation with the WebAppSec WG, we do not believe this API enables any new attacks.



Is this feature fully tested by web-platform-tests?

Yes


Flag name

--enable-blink-features=ExperimentalJSProfiler


Requires code in //chrome?

False


Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=956688


Sample links

https://wicg.github.io/js-self-profiling/#examples


Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5170190448852992


Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/cl_WCx9OEcg/m/9b-6_7DPDAAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/GP0OlwtC1EQ/m/7Q8P3v8nAgAJ


Yoav Weiss

unread,
Jul 28, 2021, 3:54:20 PMJul 28
to Andrew Comminos, blin...@chromium.org
Thank you for meticulously pushing this!

I find that part surprising - while wrapping those built-ins is possible today, it's also possible to detect that they were wrapped (in most cases), and potentially code defensively against that.
You say that built-ins may be included - are they included in current implementation? What's the benefit of including them if the code around them is invisible?

As a result, the API does not expose any new insight into the contents or execution characteristics of cross-origin script, beyond what is already possible through manual instrumentation (e.g. via Error.stack or runtime script modification).

Cross-origin execution contexts should not be observable by the API through the realm check in the spec's take a sample algorithm. Cross-origin iframes and other execution contexts that share an agent with a profiler will therefore not have their execution observable through this API.

Timing attacks remain a concern for any API that could introduce a new source of high-resolution timing information. As per a review (link) from the Edge security team, plus consultation with the WebAppSec WG, we do not believe this API enables any new attacks.



Is this feature fully tested by web-platform-tests?

Yes


Flag name

--enable-blink-features=ExperimentalJSProfiler


Requires code in //chrome?

False


Tracking bug

https://bugs.chromium.org/p/chromium/issues/detail?id=956688


Sample links

https://wicg.github.io/js-self-profiling/#examples


Link to entry on the Chrome Platform Status

https://www.chromestatus.com/feature/5170190448852992


Links to previous Intent discussions

Intent to Prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/cl_WCx9OEcg/m/9b-6_7DPDAAJ

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/GP0OlwtC1EQ/m/7Q8P3v8nAgAJ


Any results from the origin trial?
 

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MN2PR15MB27330B61B9FCCE51908ABEA5C7EA9%40MN2PR15MB2733.namprd15.prod.outlook.com.

Andrew Comminos

unread,
Jul 28, 2021, 7:05:07 PMJul 28
to Yoav Weiss, blin...@chromium.org, Matan Genish
Thanks Yoav! Responses inline.

On 7/28/21 12:53 PM, Yoav Weiss wrote:
> I find that part surprising - while wrapping those built-ins is possible
> today, it's also possible to detect
> <https://www.slideshare.net/CharlesVazac/how-to-identify-bad-third-parties-on-your-page-midwest-js-public>
> that they were wrapped (in most cases), and potentially code defensively
> against that.

Indeed, it is possible to detect this instrumentation in many cases. I
don't believe any of the methods detailed in that deck are airtight --
e.g. the "trusted sanitizer" iframe approach could be worked around by
overriding createElement AFAICT. ES6 Proxy objects make this even harder
to mitigate through transparent virtualization. The profiling API
definitely makes built-in observation comparatively trivial, though.

> You say that built-ins may be included - are they included in current
> implementation? What's the benefit of including them if the code around
> them is invisible?

Currently, they are. The benefit would be primarily to determine whether
or not third-party code is behaving nicely within the environment of the
rest of the document, such as if it manages to force a synchronous
reflow by accessing layout metrics. Though as you point out, with
"invisible" frames surrounding them, this is of limited actionability.

I've gone ahead and filed an issue
(https://github.com/WICG/js-self-profiling/issues/51) to solicit
feedback on whether or not this is useful enough for folks to bother
keeping around.

> Any results from the origin trial?

We had some good successes in identifying and resolving issues on
Facebook properties in the origin trial. Here are some of the general
themes we observed:

- Developers found many situations where unexpected asynchronous code
was run on the critical path for loading. The profiling API helped
identify and defer lower-priority logic until after rendering was
complete, improving top-line loading metrics.
- Developers were able to identify situations where JS took
significantly longer to execute than expected in the wild than on their
machines. For instance, removing a simple main-thread animated loading
indicator saved hundreds of milliseconds of JS execution time on
lower-end devices that was imperceptible on developer machines.
- Teams were interested in using this dataset to provide budgeting and
ownership for JS execution time, eliminating the need for
instrumentation at every module barrier.

I've also cc'd Matan Genish to provide some detail on MSFT's experience
experimenting with the Edge origin trial :)

Andrew Comminos

unread,
Aug 5, 2021, 1:35:47 AMAug 5
to Yoav Weiss, blin...@chromium.org, Matan Genish
On 7/28/21 4:04 PM, Andrew Comminos wrote:
> On 7/28/21 12:53 PM, Yoav Weiss wrote:
>> You say that built-ins may be included - are they included in current
>> implementation? What's the benefit of including them if the code
>> around them is invisible?
>
> Currently, they are. The benefit would be primarily to determine whether
> or not third-party code is behaving nicely within the environment of the
> rest of the document, such as if it manages to force a synchronous
> reflow by accessing layout metrics. Though as you point out, with
> "invisible" frames surrounding them, this is of limited actionability.
>
> I've gone ahead and filed an issue
> (https://github.com/WICG/js-self-profiling/issues/51) to solicit
> feedback on whether or not this is useful enough for folks to bother
> keeping around.

Quick update on this -- the spec has been amended to only include
builtins called from script that is otherwise currently accessible
(CORS-same-origin). It's definitely a safer way to go given the
potential sniffing concerns you mentioned, and was not considered useful
by the developers that I reached out to.

I've posted a CL
(https://chromium-review.googlesource.com/c/chromium/src/+/3073352) to
make the Blink implementation consistent with this change as well.
Thanks again for the feedback!

Best,
Andrew

Alex Russell

unread,
Aug 5, 2021, 3:31:38 PMAug 5
to blink-dev, Andrew Comminos, blin...@chromium.org, Matan Genish, Yoav Weiss
Glad to see this moving forward; thanks for patiently shepherding this, Andrew. 

LGTM1.

Yoav Weiss

unread,
Aug 5, 2021, 4:58:58 PMAug 5
to Alex Russell, blink-dev, Andrew Comminos, Matan Genish
LGTM2

Thanks for removing the exposure of built-ins from non-CORS cross origin resources

Daniel Bratell

unread,
Aug 5, 2021, 6:05:38 PMAug 5
to Yoav Weiss, Alex Russell, blink-dev, Andrew Comminos, Matan Genish

LGTM3

I read through WebKit's concerns and while they do have a point when listing potential risks, I think they mostly fall in the foot gun category. It is possible to make a user's experience worse by enabling this API, especially if it's enabled on devices that have no performance margin. Still, that is not a reason to keep such a useful tool away from web developers.

/Daniel

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
Reply all
Reply to author
Forward
0 new messages