Intent to Ship: Same-origin prerendering triggered by the speculation rules API

[Angel Raposo]

Contact emails

noam.j.r...@gmail.com
dom...@chromium.org, jbr...@chromium.org, nhi...@chromium.org, navigat...@chromium.org

Explainer

This feature:

https://github.com/WICG/nav-speculation/blob/main/same-origin-explainer.md

Larger project: https://github.com/WICG/nav-speculation/blob/main/README.md

Specification

https://wicg.github.io/nav-speculation/prerendering.html

Design docs

https://docs.google.com/document/d/1P2VKCLpmnNm_cRAjUeE-bqLL0bslL_zKqiNeCzNom_w/edit?usp=sharing

Summary

Prerendering is “pre”-rendering, it’s about pre-loading and rendering a Web page before the user actually navigates to it. The main goal of prerendering is to make the next page navigation faster, or ideally nearly instant.

Sites can tell the user agents about which pages the user may likely visit, by asking to trigger a ‘prerendering’ for a particular URL (e.g. user is at page A and will likely navigate to page B next). Once the prerender is triggered, the browser pre-fetches the main resource, instantiates a hidden page, and processes the main resource to fetch and process more subresources.

We would like to ship same-origin speculation rules prerendering. With this feature, Chrome will start prerendering high-confidence URL suggestions provided by the page using speculation rules. During the prerendering process, a page will process and construct the full DOM tree, including the execution of scripts (this differs from the prefetching resources using No-state Prefetch)

This prerendering affects how the prerendered pages are processed. The website gets loaded before the navigation is committed, and the side-effects are delayed until activation (the actual navigation to the website was committed). The website can know that it is being prerendered (document.prerendering) and when it was activated (prerenderingchange event, performance.activationStart timing). These APIs have already been approved by the previous intent-to-ship for Omnibox prerendering.

Note that we are not shipping cross-origin speculation rules, which allows a web page to prerender another page under a different domain (external site). For details about our web exposed changes shipping plan, please consult this doc: Interoperability Roadmap for Shipping Prerender2 Incrementally.

Blink component

Internals>Preload>Prerender

TAG review

https://github.com/w3ctag/design-reviews/issues/667

TAG review status

Closed

Risks

Interoperability and Compatibility

Compatibility

There are some use cases that will need to know whether a page is being prerendered by the user agent or navigated by the user, e.g. Ads and analytics are likely examples of this.

To solve this challenge, this feature exposes `document.prerendering` which indicates if the page is prerendered or not. We understand that there’s always a risk of sites, that would benefit from using this API, not actually using it. 

We believe that this risk is tractable because: 

(1) triggering a prerender isn’t an entirely new behavior. E.g. Chrome used to have a prerendering feature a long while ago triggered by URLs entered in the navigation bar

(2) Similar behaviors are already present in other browsers, e.g. “Preload Top Hit” feature in Safari

To give more flexibility to web authors (site owners) who want to implement speculation rules, we are already including a header to network requests as `Purpose: prefetch` so that origin servers can identify requests for prerendered pages and decide how to handle them. A new header will be included to provide a CORS safelist alternative using  `Sec-purpose: prefetch;prerender` https://fetch.spec.whatwg.org/#cors-safelisted-request-header

https://chromestatus.com/feature/6247959677108224

There’s also a future risk of the speculation rules specification changing: including new rules, modifying previous ones, etc. We do not expect to make backward-incompatible changes to the speculation rules format, but if we did, the user agent wouldn’t be able to understand the rules (which would define them as null), opting out of the prerender which wouldn’t cause any visible effect (apart from a slower loading time for the next navigated page as it won’t be prerendered).

We are working closely on the tracker https://github.com/WICG/nav-speculation/issues which contains currently 8 non-blocking open discussions related with compatibility (1 of them covering cross-origin which we are not shipping with this I2S)

Interoperability

We believe that some browsers already have prerendering implementations which are not well-specified and may differ from each other. Our vision is to produce a specification that can help improve interoperability. There is a risk that other browsers do not converge on a prerendering standard. The danger here is that different browsers have different ways to trigger a prerendered page, and prerendered pages behave differently in different browsers.

Prerendering is a web-visible behavior, since it involves fetching the page and executing its scripts.

Prerendering can depend on UA-specific heuristics. For example, the browser might decide to act on a hint to prerender based on the system load, and the presence of other prerenders. We do not intend to codify heuristics in the specification. A conforming browser might simply ignore all hints to prerender a page.

Discussions around speculation rules and the broader prerender feature has already been initiated with different players:

Gecko: Some informal positive discussion with Gecko engineers on the HTML Standard issue tracker and in the HTML triage call; formal positions request here: https://github.com/mozilla/standards-positions/issues/613

WebKit: WebKit already ships URL-bar triggered prerendering, but not any APIs for letting pages know about it, and it's unclear what strategy they are using to prohibit disruptive behaviors for prerendered pages. We have reached out for a formal positions request here in the hopes of moving toward interoperability: https://lists.webkit.org/pipermail/webkit-dev/2022-February/032113.html

Web developers: We’ve already received positive feedback from initial web developers testing the feature. See https://github.com/WICG/proposals/issues/2 for positive sentiments on speculation rules triggered by different partners who are willing to implement the feature into their sites.

Other signals: There’s a public request for feedback published (https://web.dev/speculative-prerendering/#feedback-welcome) which is currently being managed through: https://github.com/WICG/nav-speculation/issues

Ergonomics

IT admins can disable Prerendering via the existing group policy "NetworkPredictionOptions"

This feature will be used in combination with the APIs which already shipped as part of omnibox prerendering: document.prerendering, prerenderingchange event, and performanceEntry.activationStart timing.

Activation

Developer needs to opt-in by defining the speculation rules on the page. Please notice that this doesn’t ensure the prerender of the next navigation as speculation rules are only used as suggestions to improve the heuristics of the user agent choosing which page to prerender. As hints, speculation rules may be followed by the browser (prerendering the page) or ignored completely (e.g. lowend device without resources to prerender the page).

Security

This feature is the first use of the Multiple-Page Architecture, which is a significant change to Chromium's internals. Both MPArch and this feature in particular underwent significant security review. See the design doc for more details.
From a web-exposed perspective, the security and privacy concerns are smaller, because this feature is restricted to the same-origin case only.

WebView application risks

This feature doesn’t deprecate or change behavior of existing APIs directly.

Debuggability

We are actively talking to the DevTools team about adding general Prerender support to it [metabug].
The current MVP is to reveal the prerendered page status in the panel so web developers can know if they finished successfully or not.

We also have a long term plan to show different debugging status messages in DevTools but there's no specific messages currently discussed that are focused only on speculation rules, only the "generic" ones at prerender feature level. Your thoughts are welcome if specific debugging logs regarding speculation rules would be required.

Is this feature fully tested by web-platform-tests?

We have written a lot of web platform tests using speculation rules as the trigger:

https://wpt.fyi/results/speculation-rules/prerender?label=experimental&label=master&aligned

(Currently Chromium tests this using VirtualTestSuites [cs] instead of EnableExperimentalWebPlatformFeatures, so the tests are not green on wpt.fyi.)

Flag name

Prerender2

Requires code in //chrome?

False

Tracking bug

https://crbug.com/1295143

Launch bug

https://crbug.com/1316557

Sample links

For a demo of prerendering generally, and its effect on websites, https://prerender2-specrules.glitch.me/ shows it triggered by speculation rules.

Estimated milestones

OriginTrial android last	102
OriginTrial android first	94
DevTrial on android	94

Anticipated spec changes

There are currently 8 open discussions which could potentially modify the API in the future. Having said that, half of them were opened in 2020/2021 which means they have been in discussions for a long term and we don’t expect immediate changes required in the short term.

We have reviewed all the current pending discussions and we believe they won’t cause compatibility issues as most of them will be resolved with the initial launch. We’ll keep working on the tracker and updating our roadmap doc if we detect any changes.

Link to entry on the Chrome Platform Status

https://chromestatus.com/feature/5355965538893824

Links to previous Intent discussions

Intent to Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/3JwGNnqH3QI/m/Ey103Q-yAQAJ

Intent to Extend Experiment: https://groups.google.com/a/chromium.org/g/blink-dev/c/Kpp6uJJRrqI/m/GTo_aF0qEQAJ

This intent message was generated by Chrome Platform Status.

--

Angel Raposo  |  TPM Prerender (20% role)  |  angel...@google.com  |   Google Japan G.K.  

This email may be confidential or privileged.  If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it went to the wrong person.  Thanks.

The above terms reflect a potential business arrangement, are provided solely as a basis for further discussion, and are not intended to be and do not constitute a legally binding obligation.  No legally binding obligations will be created, implied, or inferred until an agreement in final form is executed in writing by all parties involved.

もし、このメッセージが誤って貴殿に送信されたと思われる場合には、機密情報を含んでいる可能性もありますので、どなたにも転送せず、添付ファイルも含めて削除していただくとともに、発信者にその旨をお伝えいただきますようお願いいたします。

[Dave Tapuska]

This intent is for Android only, correct?

Will there be another Intent to Ship on Desktop?

Presumably desktop is blocked on extensions and the compatibility risk regarding those. (I recently discussed this at the WECG meeting).

dave.

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAA9vRHxK26yKNbkcL3uWGcGHGDEZpQBSBy3Avbp%2BqAV4iTpaOg%40mail.gmail.com.

[Hiroki Nakagawa]

Hi Dave,

Yes, this is for Android only. Sorry, we should have clarified that point. We're going to send separate I2P and I2S for the desktop.

Thanks,

Hiroki

[Yoav Weiss]

https://github.com/WICG/nav-speculation/issues/160 seems like something we'd want to settle before shipping?

Any other open issues that may impact the compat or interop risk here?

 

Also, is the opt-in part of this intent?

Compatibility

There are some use cases that will need to know whether a page is being prerendered by the user agent or navigated by the user, e.g. Ads and analytics are likely examples of this.

To solve this challenge, this feature exposes `document.prerendering` which indicates if the page is prerendered or not. We understand that there’s always a risk of sites, that would benefit from using this API, not actually using it. 

We believe that this risk is tractable because: 

(1) triggering a prerender isn’t an entirely new behavior. E.g. Chrome used to have a prerendering feature a long while ago triggered by URLs entered in the navigation bar

(2) Similar behaviors are already present in other browsers, e.g. “Preload Top Hit” feature in Safari

To give more flexibility to web authors (site owners) who want to implement speculation rules, we are already including a header to network requests as `Purpose: prefetch` so that origin servers can identify requests for prerendered pages and decide how to handle them. A new header will be included to provide a CORS safelist alternative using  `Sec-purpose: prefetch;prerender` https://fetch.spec.whatwg.org/#cors-safelisted-request-header

https://chromestatus.com/feature/6247959677108224

There’s also a future risk of the speculation rules specification changing: including new rules, modifying previous ones, etc. We do not expect to make backward-incompatible changes to the speculation rules format, but if we did, the user agent wouldn’t be able to understand the rules (which would define them as null), opting out of the prerender which wouldn’t cause any visible effect (apart from a slower loading time for the next navigated page as it won’t be prerendered).

We are working closely on the tracker https://github.com/WICG/nav-speculation/issues which contains currently 8 non-blocking open discussions related with compatibility (1 of them covering cross-origin which we are not shipping with this I2S)

Interoperability

We believe that some browsers already have prerendering implementations which are not well-specified and may differ from each other. Our vision is to produce a specification that can help improve interoperability. There is a risk that other browsers do not converge on a prerendering standard. The danger here is that different browsers have different ways to trigger a prerendered page, and prerendered pages behave differently in different browsers.

Prerendering is a web-visible behavior, since it involves fetching the page and executing its scripts.

Prerendering can depend on UA-specific heuristics. For example, the browser might decide to act on a hint to prerender based on the system load, and the presence of other prerenders. We do not intend to codify heuristics in the specification. A conforming browser might simply ignore all hints to prerender a page.

Discussions around speculation rules and the broader prerender feature has already been initiated with different players:

Gecko: Some informal positive discussion with Gecko engineers on the HTML Standard issue tracker and in the HTML triage call; formal positions request here: https://github.com/mozilla/standards-positions/issues/613

WebKit: WebKit already ships URL-bar triggered prerendering, but not any APIs for letting pages know about it, and it's unclear what strategy they are using to prohibit disruptive behaviors for prerendered pages. We have reached out for a formal positions request here in the hopes of moving toward interoperability: https://lists.webkit.org/pipermail/webkit-dev/2022-February/032113.html

Web developers: We’ve already received positive feedback from initial web developers testing the feature. See https://github.com/WICG/proposals/issues/2 for positive sentiments on speculation rules triggered by different partners who are willing to implement the feature into their sites.

It'd be great if you could share more details about the origin trial, feedback we received and if possible, user benefits from OT participants trying this out.

 

--

[Hiroki Nakagawa]

Hi Yoav, thanks for your comments!

I wonder if the issue is not blocking this I2S. Currently we support only the basic syntax like this, and such advanced syntax is out of the scope. I think we can incrementally support them later without breaking the compatibility.

 

Any other open issues that may impact the compat or interop risk here?

We audited open issues that could potentially affect compat or interop and concluded they have no risks. This document summarizes the results.

 

Also, is the opt-in part of this intent?

This I2S doesn't cover the opt-in mechanism.

 

Compatibility

There are some use cases that will need to know whether a page is being prerendered by the user agent or navigated by the user, e.g. Ads and analytics are likely examples of this.

To solve this challenge, this feature exposes `document.prerendering` which indicates if the page is prerendered or not. We understand that there’s always a risk of sites, that would benefit from using this API, not actually using it. 

We believe that this risk is tractable because: 

(1) triggering a prerender isn’t an entirely new behavior. E.g. Chrome used to have a prerendering feature a long while ago triggered by URLs entered in the navigation bar

(2) Similar behaviors are already present in other browsers, e.g. “Preload Top Hit” feature in Safari

To give more flexibility to web authors (site owners) who want to implement speculation rules, we are already including a header to network requests as `Purpose: prefetch` so that origin servers can identify requests for prerendered pages and decide how to handle them. A new header will be included to provide a CORS safelist alternative using  `Sec-purpose: prefetch;prerender` https://fetch.spec.whatwg.org/#cors-safelisted-request-header

https://chromestatus.com/feature/6247959677108224

There’s also a future risk of the speculation rules specification changing: including new rules, modifying previous ones, etc. We do not expect to make backward-incompatible changes to the speculation rules format, but if we did, the user agent wouldn’t be able to understand the rules (which would define them as null), opting out of the prerender which wouldn’t cause any visible effect (apart from a slower loading time for the next navigated page as it won’t be prerendered).

We are working closely on the tracker https://github.com/WICG/nav-speculation/issues which contains currently 8 non-blocking open discussions related with compatibility (1 of them covering cross-origin which we are not shipping with this I2S)

Interoperability

We believe that some browsers already have prerendering implementations which are not well-specified and may differ from each other. Our vision is to produce a specification that can help improve interoperability. There is a risk that other browsers do not converge on a prerendering standard. The danger here is that different browsers have different ways to trigger a prerendered page, and prerendered pages behave differently in different browsers.

Prerendering is a web-visible behavior, since it involves fetching the page and executing its scripts.

Prerendering can depend on UA-specific heuristics. For example, the browser might decide to act on a hint to prerender based on the system load, and the presence of other prerenders. We do not intend to codify heuristics in the specification. A conforming browser might simply ignore all hints to prerender a page.

Discussions around speculation rules and the broader prerender feature has already been initiated with different players:

Gecko: Some informal positive discussion with Gecko engineers on the HTML Standard issue tracker and in the HTML triage call; formal positions request here: https://github.com/mozilla/standards-positions/issues/613

WebKit: WebKit already ships URL-bar triggered prerendering, but not any APIs for letting pages know about it, and it's unclear what strategy they are using to prohibit disruptive behaviors for prerendered pages. We have reached out for a formal positions request here in the hopes of moving toward interoperability: https://lists.webkit.org/pipermail/webkit-dev/2022-February/032113.html

Web developers: We’ve already received positive feedback from initial web developers testing the feature. See https://github.com/WICG/proposals/issues/2 for positive sentiments on speculation rules triggered by different partners who are willing to implement the feature into their sites.

It'd be great if you could share more details about the origin trial, feedback we received and if possible, user benefits from OT participants trying this out.

We had experiments with several internal/external partners. One of the experiments showed that the feature can reduce the latency on their service by 45%. Other experiments also showed promising impacts on the CWV metrics. According to feedback from the partners, the lack of DevTools support for prerendering was challenging. As the "Debuggability" section on this I2S mentions, we've actively been developing the DevTools support, so the situation should be mitigated soon. Overall, they were supportive of the feature and interested in using it on production.

[Hiroki Nakagawa]

Let me share more details of the feedback.

We worked directly with 11 external partners; 8 of them valued the opportunity offered by Prerendering. 

• Origin Trial results: One of the external partners, Ameba News (https://news.ameba.jp/) has tested out same-origin prerendering via the Origin Trial during April 11 - 17 (1 week testing / around selected navigations / 2-3% of the traffic). While the site was already meeting CWV and good LCP, by adopting prerendering, the median LCP improved 2.7x (827ms to 302ms) and more importantly the P95 improved by 1.6 seconds [2843ms to 1195ms] (i.e. Prerendered pages have more consistency in the performance).
• Key challenges: 
• All participants identified "knowing what is safe to prerender / adapting a page to be safe" as a key challenge. We are exploring follow-ups to lower the bar.
  
• DevTools support. We are following up with the devtools team.
• 3P RUM tools to incorporate `activationStart` to correctly measure FCP / LCP. We will be guiding the RUMs for required changes.
• CrUX to incorporate the prerendering impact. We are following up with the CrUX team.

In addition, we worked with one internal partner who got great results (i.e. the 45% number shared earlier).

Finally, we received the following feedback from 6 OT participants:

• Difficulty: moderately easy (3), neither easy nor difficult (2), Slightly difficult (1)
• Keep using: extremely likely (2), moderately likely (3), can't say which way (1)
• Alternatives: none (2), fallback to prefetch (2)

Thanks,

Hiroki

[Yoav Weiss]

LGTM1

Thanks for the extra details. These results look amazing, so in my mind they significantly outweigh the risk of going first here.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.

[Daniel Bratell]

LGTM2

/Daniel

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/95894ef5-83cb-4906-a8c8-fad0b0e71f4fn%40chromium.org.

[Mike Taylor]

LGTM3

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/fc783e64-8244-a0d8-263a-9e53944ec093%40gmail.com.

[jmedley via Chromestatus]

In which version are you planning to ship?