Intent to Ship: JS Self-Profiling API
Andrew Comminos
Contact emails
acom...@fb.com, tdre...@chromium.org, n...@chromium.org
Explainer
https://www.github.com/WICG/js-self-profiling
Specification
https://wicg.github.io/js-self-profiling/
Summary
Adds a web-exposed sampling profiler for measuring client JavaScript execution time. Gathering JS profiles from real users can help developers debug slow observed performance without the need for invasive manual instrumentation.
Is this feature supported on all six Blink platforms (Windows, Mac, Linux, Chrome OS, Android, and Android WebView)?
Yes
Debuggability
No additional DevTools support is required for this feature.
Measurement
The Profiler constructor is gated behind a UseCounter to track adoption.
Blink component
Search tags
profiler, profiling, js
TAG review
https://github.com/w3ctag/design-reviews/issues/366
TAG review status
Issues addressed
Risks
Interoperability and Compatibility
As the API's purpose is to gather JS traces for analysis, the primary interoperability risk is that traces from different UAs cannot be treated interchangeably. The spec attempts to resolve this by leveraging the concept of function instance names (defined in ECMA-262) to label stack frames, ensuring consistency for analysis purposes.
Developers should still exercise caution when aggregating traces between different user agents, as they may support different sampling intervals or have significantly different performance characteristics because of VM differences.
This API has been discussed at length in the Web Performance Working Group during various calls, as well as TPAC 2018, 2019, and 2020. Additionally, the WebAppSec Working Group was consulted when considering gating the API behind a cross-origin isolation requirement.
Gecko: No signal (https://github.com/mozilla/standards-positions/issues/477)
WebKit: Negative (https://github.com/mozilla/standards-positions/issues/477#issuecomment-765106364)
Web developers: Strongly positive
Facebook has been able to debug and resolve a wide variety of performance issues during the origin trial period. We have also seen quite a bit of support from other large web properties:
Microsoft: https://github.com/WICG/js-self-profiling/issues/24
Elastic: https://github.com/WICG/js-self-profiling/issues/21
Boomerang: https://github.com/WICG/js-self-profiling/issues/14
Dropbox: https://github.com/WICG/js-self-profiling/issues/9
Ergonomics
This API is designed to be used in tandem with the performance timeline family of APIs -- sample timestamps in traces share a clock and time origin with HR-Time.
The largest concern with exposing a sampling profiler to the web is reducing the performance of the page if profiling is enabled liberally. As the UA has freedom to choose which sampling intervals it supports, the only mandatory overhead required is building the necessary code mapping for resolving unwinded stacks. Because the API is gated on document policy, any cost will only be incurred on pages that indicate they wish to profile by exposing the appropriate document policy header.
Activation
To leverage the tracing data effectively, a web service to ingest and aggregate traces is useful for analysis. This API would likely benefit from the publishing of open-source libraries as well as documentation to help develop such a service (although OT partners did not encounter significant challenges).
Security
The API was designed to provide no additional insight into the contents or execution characteristics of cross-origin scripts, nor the execution of cross-origin frames.
Contents of cross-origin opaque scripts are hidden by requiring all functions included via the spec's take a sample algorithm to be defined in a script served with CORS-same-origin through HTML’s muted errors property. Browser builtins (such as performance.now()) may still be included when invoked from cross-origin no-cors script (albeit with no information about the invoking script, line, or column number), although this is already made possible through manually overriding global objects. As a result, the API does not expose any new insight into the contents or execution characteristics of cross-origin script, beyond what is already possible through manual instrumentation (e.g. via Error.stack or runtime script modification).
Cross-origin execution contexts should not be observable by the API through the realm check in the spec's take a sample algorithm. Cross-origin iframes and other execution contexts that share an agent with a profiler will therefore not have their execution observable through this API.
Timing attacks remain a concern for any API that could introduce a new source of high-resolution timing information. As per a review (link) from the Edge security team, plus consultation with the WebAppSec WG, we do not believe this API enables any new attacks.
Is this feature fully tested by web-platform-tests?
Flag name
--enable-blink-features=ExperimentalJSProfiler
Requires code in //chrome?
False
Tracking bug
https://bugs.chromium.org/p/chromium/issues/detail?id=956688
Sample links
https://wicg.github.io/js-self-profiling/#examples
Link to entry on the Chrome Platform Status
https://www.chromestatus.com/feature/5170190448852992
Links to previous Intent discussions
Intent to Prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/cl_WCx9OEcg/m/9b-6_7DPDAAJ
Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/GP0OlwtC1EQ/m/7Q8P3v8nAgAJ
Yoav Weiss
As a result, the API does not expose any new insight into the contents or execution characteristics of cross-origin script, beyond what is already possible through manual instrumentation (e.g. via Error.stack or runtime script modification).
Cross-origin execution contexts should not be observable by the API through the realm check in the spec's take a sample algorithm. Cross-origin iframes and other execution contexts that share an agent with a profiler will therefore not have their execution observable through this API.
Timing attacks remain a concern for any API that could introduce a new source of high-resolution timing information. As per a review (link) from the Edge security team, plus consultation with the WebAppSec WG, we do not believe this API enables any new attacks.
Is this feature fully tested by web-platform-tests?
Flag name
--enable-blink-features=ExperimentalJSProfiler
Requires code in //chrome?
False
Tracking bug
https://bugs.chromium.org/p/chromium/issues/detail?id=956688
Sample links
https://wicg.github.io/js-self-profiling/#examples
Link to entry on the Chrome Platform Status
https://www.chromestatus.com/feature/5170190448852992
Links to previous Intent discussions
Intent to Prototype: https://groups.google.com/a/chromium.org/g/blink-dev/c/cl_WCx9OEcg/m/9b-6_7DPDAAJ
Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/GP0OlwtC1EQ/m/7Q8P3v8nAgAJ
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/MN2PR15MB27330B61B9FCCE51908ABEA5C7EA9%40MN2PR15MB2733.namprd15.prod.outlook.com.
Andrew Comminos
On 7/28/21 12:53 PM, Yoav Weiss wrote:
> I find that part surprising - while wrapping those built-ins is possible
> today, it's also possible to detect
> against that.
don't believe any of the methods detailed in that deck are airtight --
e.g. the "trusted sanitizer" iframe approach could be worked around by
overriding createElement AFAICT. ES6 Proxy objects make this even harder
to mitigate through transparent virtualization. The profiling API
definitely makes built-in observation comparatively trivial, though.
> You say that built-ins may be included - are they included in current
> implementation? What's the benefit of including them if the code around
> them is invisible?
or not third-party code is behaving nicely within the environment of the
rest of the document, such as if it manages to force a synchronous
reflow by accessing layout metrics. Though as you point out, with
"invisible" frames surrounding them, this is of limited actionability.
I've gone ahead and filed an issue
(https://github.com/WICG/js-self-profiling/issues/51) to solicit
feedback on whether or not this is useful enough for folks to bother
keeping around.
> Any results from the origin trial?
Facebook properties in the origin trial. Here are some of the general
themes we observed:
- Developers found many situations where unexpected asynchronous code
was run on the critical path for loading. The profiling API helped
identify and defer lower-priority logic until after rendering was
complete, improving top-line loading metrics.
- Developers were able to identify situations where JS took
significantly longer to execute than expected in the wild than on their
machines. For instance, removing a simple main-thread animated loading
indicator saved hundreds of milliseconds of JS execution time on
lower-end devices that was imperceptible on developer machines.
- Teams were interested in using this dataset to provide budgeting and
ownership for JS execution time, eliminating the need for
instrumentation at every module barrier.
I've also cc'd Matan Genish to provide some detail on MSFT's experience
experimenting with the Edge origin trial :)
Andrew Comminos
> On 7/28/21 12:53 PM, Yoav Weiss wrote:
>> You say that built-ins may be included - are they included in current
>> implementation? What's the benefit of including them if the code
>> around them is invisible?
>
> Currently, they are. The benefit would be primarily to determine whether
> or not third-party code is behaving nicely within the environment of the
> rest of the document, such as if it manages to force a synchronous
> reflow by accessing layout metrics. Though as you point out, with
> "invisible" frames surrounding them, this is of limited actionability.
>
> I've gone ahead and filed an issue
> (https://github.com/WICG/js-self-profiling/issues/51) to solicit
> feedback on whether or not this is useful enough for folks to bother
> keeping around.
builtins called from script that is otherwise currently accessible
(CORS-same-origin). It's definitely a safer way to go given the
potential sniffing concerns you mentioned, and was not considered useful
by the developers that I reached out to.
I've posted a CL
(https://chromium-review.googlesource.com/c/chromium/src/+/3073352) to
make the Blink implementation consistent with this change as well.
Thanks again for the feedback!
Best,
Andrew
Alex Russell
Yoav Weiss
Daniel Bratell
LGTM3
I read through WebKit's concerns and while they do have a point when listing potential risks, I think they mostly fall in the foot gun category. It is possible to make a user's experience worse by enabling this API, especially if it's enabled on devices that have no performance margin. Still, that is not a reason to keep such a useful tool away from web developers.
/Daniel
--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAL5BFfU0KDb8FgZEoyYOdkEPjNW8Rn4mtGH-b_1rZvBjy4s7WA%40mail.gmail.com.