Intent to implement: Prerendering API

[Roshan Vidyashankar]

Summary: The Prerendering API allows the rendering engine to start loading and rendering a web page without any visible UI until the page needs to be displayed to the user. It's a gecko API that consumers can choose to use. For now, we're not talking about exposing this to web pages yet, or even using it in any of our products, we're just working on the implementation.

Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=730101

Link to standard: At some point we are going to be contributing to a spec but there are a number of unknown things which we're hoping to find out through the implementation - like what APIs need to be blacklisted in prerendering (audio, window.open etc). Chrome and IE have shipped their own implementations of prerendering.

Platform coverage: All

Estimated or target release: Unknown

Preference behind which this will be implemented: dom.prerendering.enabled

Implementation Plan
There are two big parts to the implementation.

1. How do we handle the prerendering itself in a way that works both for desktop and b2g

For this, we're thinking about adding a boolean flag to nsDocShell which together with mIsActive will constitute a tri-state: "active", "inactive" and "prerendered". As far as the outside code is concerned, prerendered docshells are treated as inactive ones for the most part. This means that we'd basically need to not modify nsDocShell::GetIsActive, and audit all of its callers and find the potential consumers who would want to treat prerendered and inactive differently (I don't expect there to be any such consumers actually but we'll see). We'd use the prerendered state to reflect the correct status reflected through the page visibility API. The prerendered docshell masquerading as inactive will also mean that it won't get a rendering on b2g for example because that's how we hide mozbrowser iframes there.

In order for gaia and XUL based consumers to be able to control the prerendering, I think we'd need to use the nsIFrameLoader swapping facilities that we currently have for the <xul:browser> swapDocShells method and extend those to make them usable with the mozbrowser frameLoader as well. That way we can expose the prerendering API through both mozbrowser and <xul:browser> while sharing hopefully most of the code between these different consumers, and we'd get the ability to swap the frame loaders when we need to actually render a prerendered document.

2. What do we do with the APIs that have undesirable side-effects in prerendered pages (audio, window.open etc).

We first need to decide what we're going to do when a page calls into an API which has undesired side effects (and that is a blurry line -- I hope as part of this we come up with an actual list of what those actions are!). Chrome basically aborts the execution of scripts on the page and throws out the prerendered content as soon as any such APIs are called. IE's documentation claims that they "pause" a web page execution when that happens, but according to my testing, that behavior is really border-line non-deterministic. I think we basically have two options: either abort the execution of the script or throw an exception. The former breaks run-to-completion, which sucks. The latter is a bit better I think but then we need to detect when JS execution is "done" so that we can start to discard the prerendered content. I don't know if the latter is easily doable.

We are also thinking about adding a WebIDL annotation (let's call it [UnsafeForPrerendering] that we can add to stuff in the IDL and have it call into a BindingUtils helper which does either of the above.

[Jonas Sicking]

Very exited to see this happening.

Implementation issues aside, I have two comments:

* This is something that we really need on FirefoxOS. I hope that the
implementation strategy will work there too?
* A use-case that we came upon pretty quickly when we were looking at
using prerendering in FirefoxOS is that we might not know the exact
URL that the user is likely to navigate to, but we have a pretty good
guess about what template page it's going to be.

Consider for example bugzilla. After the user has done a search query,
they are likely to click on one of the bug links. Each bug has a
different URL, but all bugs share much of the page UI.

It would be really awesome if bugzilla could ask to prerender a URL
like "https://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=true".
Then when the user clicks a bugzilla URL like
"https://bugzilla.mozilla.org/show_bug.cgi?id=123456", we would enable
webpage logic to somehow force the prerendered page to be used, even
though the URLs don't match.

One way to do this would be to enable some way for the current page to
talk to the prerendered page. The current page could then tell the
prerendered page "the user just clicked bug 123456", at which point
the prerendered page could use replaceState to change its URL to
"https://bugzilla.mozilla.org/show_bug.cgi?id=123456", at which point
we would see that the URLs matched.

Another solution would be to enable the prerendered page to say "i'm
able to act as a prerender page for any URLs that match pattern
https://bugzilla.mozilla.org/show_bug.cgi?id=*".

Would be great if someone could bring up this use case with the
prerender editors and make sure it gets covered and that a specced
solution is defined.

/ Jonas

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[Karl Dubost]

Le 12 août 2014 à 07:03, Jonas Sicking <jo...@sicking.cc> a écrit :
> * A use-case that we came upon pretty quickly when we were looking at
> using prerendering in FirefoxOS is that we might not know the exact
> URL that the user is likely to navigate to, but we have a pretty good
> guess about what template page it's going to be.

If I remember bits of Google strategy, it was basically tied to their proxy servers where they basically know statically which page the users are most likely to click next.

There are also some logical next steps:

* Infinite loading through XHR
* Next items in a list of search results, gallery, etc. Anything sequential. (Next, Prev)

A bit more far fetched but maybe worth thinking for a version 2.0 heuristics. If it's only client-side, users could set up their browser to collect the browsing habits for the most common sites (*without ever sending it back to a server*). With this browsing habits sequence, it would be possible to know for the browser that most of the time the user follows a certain pattern and pretender the pages. Additional possible benefits: User understands his/her own patterns through a dashboard.

--
Karl Dubost, Mozilla
http://www.la-grange.net/karl/moz

[Patrick McManus]

an obvious tie in here is the network predictor (formerly 'seer') work Nick
Hurley has been doing. Basically already working on the "what to fetch
next" questions, but not the rendering parts.

[Jonas Sicking]

On Mon, Aug 11, 2014 at 3:40 PM, Karl Dubost <kdu...@mozilla.com> wrote:
>
> Le 12 août 2014 à 07:03, Jonas Sicking <jo...@sicking.cc> a écrit :
>> * A use-case that we came upon pretty quickly when we were looking at
>> using prerendering in FirefoxOS is that we might not know the exact
>> URL that the user is likely to navigate to, but we have a pretty good
>> guess about what template page it's going to be.
>
> If I remember bits of Google strategy, it was basically tied to their proxy servers where they basically know statically which page the users are most likely to click next.

In the Google search page use case (if that's the case you are talking
about?) the resulting pages generally don't share a "template". This
means that they have to make a best guess at which page to load, and
if they guess wrong the prerendering was for naught.

However in some cases we can do better than that by loading a template
page that can be used for multiple of the search results. And even
better, we can do it at a dramatically lower cost since we don't have
to worry about implementing the AI for guessing which result the user
is going to click.

Also note that this isn't just useful for search results. Where it
came up for FirefoxOS is in the contacts list, where it's likely that
the user is going to press one of the contacts, but guessing which one
is unlikely to be successful.

/ Jonas

[Karl Dubost]

Jonas,

Le 12 août 2014 à 08:33, Jonas Sicking <jo...@sicking.cc> a écrit :
> However in some cases we can do better than that by loading a template
> page that can be used for multiple of the search results.

This I'm not sure to understand how that would be working. Let's say be any page on the Web. How do you know in advance the template for the page? I might have missed something. For things owned by Mozilla. It would mean that you would send HTML templates fed with JSON data? Do you do partial rendering in this case? How does it work with injected data modifying the HTML template layout?

> Also note that this isn't just useful for search results.

I was not thinking about search results most of the time. Search results or any sequential web thingy (manual, gallery, etc. etc.) is just an obvious one.

> Where it came up for FirefoxOS is in the contacts list, where it's likely that
> the user is going to press one of the contacts, but guessing which one
> is unlikely to be successful.

hmmm… back button is about "caching the past". Easy to pre-render.
links and prefetch is about "caching the future". Harder to pre-render.

[Jonas Sicking]

On Mon, Aug 11, 2014 at 5:16 PM, Karl Dubost <kdu...@mozilla.com> wrote:
> Jonas,
>
> Le 12 août 2014 à 08:33, Jonas Sicking <jo...@sicking.cc> a écrit :
>> However in some cases we can do better than that by loading a template
>> page that can be used for multiple of the search results.
>
> This I'm not sure to understand how that would be working. Let's say be any page on the Web. How do you know in advance the template for the page? I might have missed something.

I think we are indeed talking past each other.

The goal of the prerendering API is to enable a webpage to include
something like

<link rel=prerender url="http://somewebsite.com/page.html">

This would tell the UA to load the above URL, as well as any resources
that is required to render the resulting page, and construct the DOM
and layout datastructures which are needed to display the page. All of
this is done in the hope that if the user navigates to
http://somewebsite.com/page.html, the UA can detect that a
prerendering has already been created and then display that page
*really* quickly.

This feature is useful in many scenarios. It's useful for the google
search page because they can use the above markup to point to whatever
search result they think the user is going to navigate to. It's useful
in a news article to point to the next page of the article. It's
useful for reddit because it can preload whatever link they think the
user is likely to click.

However, as the feature stands today, it can't be used by for example
the bugzilla search page.

After a user visits the bugzilla search page, they almost always visit
the search results page. However the exact URL of the search results
page vary greatly since it depends on the search parameters that were
entered. So the search page couldn't add a <link rel=prerender> in
order to prerender anything, since it would have no idea what URL to
stick in there.

This despite the fact that whichever search result page URL is loaded,
there are a lot of common parts. There's a common header and footer,
the same CSS files and JS files need to be loaded. The same styles are
applied to largely the same set of elements. Wouldn't it be great if
those pieces could be preloaded and prerendered, such that the only
thing that had to be loaded and rendered once the user pressed the
"search" button was the actual search results.

In this case the prerendered search results page acts as a "template
page" for any search result page that ends up being loaded. Of course,
if the user ends up not doing a search at all, then we'll have to drop
that prerendered page and instead do a new load from scratch. Just
like with any mispredicted <link rel=prerender>.

Similarly, the bugzilla search result page can't currently take
advantage of the prerender feature. Even though it's very likely that
the user is going to click a link to a bug, and even though all bug
pages share large parts of their resources and layout, there is no way
for the bugzilla search result page to load and prerender those common
parts.

Wouldn't it be great if the search result page could ask a generic bug
page to be loaded, such that if the user clicks a bug link, we could
just load the data that's specific to the clicked bug and insert that
data into the generic bug page.

I.e. the generic bug page would act as a template page for any bug
page. Of course, if the user doesn't find any of the search results
interesting, and goes somewhere different than a bug page, then the
prerendering was for naught and we will have to do a new load from
scratch.

I think the solution to these problems consist of two parts:
1. Make it possible for a page that contains a <link rel=prerender> to
initiate a postMessage-based conversation with the prerendered page.
2. Define that the prerendered page is used if the navigation
destination URL matches the prerendered page's URL (which can be
modified using replaceState), rather than if the URL in the <link
rel=prerender>. Redirects might make this a bit more complicated.

This way we can even do such things as tell the prerendered page to
start loading search results, using an XHR request, even before the
user has actually clicked "search". And update those search results
any time the user modifies any search parameters. If the user ends up
modifying the search parameters those results can simply be thrown
away without ever getting used, while still allowing the prerendered
search results template page to be used no matter which search ends up
being the final one.

Similarly the bugzilla serach results page could ask the prerendered
bug page to start preloading the relevant bug data as soon as a bug
link is hovered. Again, if the user ends up clicking something else,
the data can be thrown out and the correct data can be loaded.

All of this would be driven by webpage logic. The only part the
platform would provide is the prerendering feature and the two
additions listed above.

/ Jonas

[Ehsan Akhgari]

On 2014-08-11, 6:03 PM, Jonas Sicking wrote:
> Very exited to see this happening.
>
> Implementation issues aside, I have two comments:
>
> * This is something that we really need on FirefoxOS. I hope that the
> implementation strategy will work there too?

Yes, the majority of the code is going to be shared between
desktop/Android (where <xul:browser> is used for rendering content) and
Firefox OS (where mozbrowser iframes are used for rendering content.)

> * A use-case that we came upon pretty quickly when we were looking at
> using prerendering in FirefoxOS is that we might not know the exact
> URL that the user is likely to navigate to, but we have a pretty good
> guess about what template page it's going to be.
>

> Consider for example bugzilla. After the user has done a search query,
> they are likely to click on one of the bug links. Each bug has a
> different URL, but all bugs share much of the page UI.
>
> It would be really awesome if bugzilla could ask to prerender a URL
> like "https://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=true".
> Then when the user clicks a bugzilla URL like
> "https://bugzilla.mozilla.org/show_bug.cgi?id=123456", we would enable
> webpage logic to somehow force the prerendered page to be used, even
> though the URLs don't match.
>
> One way to do this would be to enable some way for the current page to
> talk to the prerendered page. The current page could then tell the
> prerendered page "the user just clicked bug 123456", at which point
> the prerendered page could use replaceState to change its URL to
> "https://bugzilla.mozilla.org/show_bug.cgi?id=123456", at which point
> we would see that the URLs matched.
>
> Another solution would be to enable the prerendered page to say "i'm
> able to act as a prerender page for any URLs that match pattern
> https://bugzilla.mozilla.org/show_bug.cgi?id=*".
>
> Would be great if someone could bring up this use case with the
> prerender editors and make sure it gets covered and that a specced
> solution is defined.

So I've been thinking about this use case. Here are a number of
thoughts that I have so far:

* The page which asks the UA to prerender the document is responsible
for telling the UA which URL patterns the prerendered page should
handle. As you note above, you typically want to render a different
page (e.g.,
https://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=true) which
knows how to handle the dynamic fetching of the resources it needs when
it's taken out of prerendering mode. But the controlling page also
knows what URLs on that page may be loaded into this prerendered
document. So far I've been thinking of an API like this:

var prerenderedDoc = new PrerenderedDocument(
// url to the document being loaded
"https://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=true",
// url scope for what it can handle
"https://bugzilla.mozilla.org/show_bug.cgi?id=");

* The two pages need to talk to each other. For that purpose, I think
we should add a postMessage API to PrerenderedDocument which will allow
the controlling document to either use it for single direction
communication, or post a MessagePort to the prerendered page for
bidirectional communication.

* The prerendered page needs to know when it's taken out of prerendering
mode. This is already possible through the visibilitychange event.
However the page also needs to know what URL ended up being loaded in
the controlling document. Because the prerendered page may get
activated as a result of a user initiated navigation, we cannot rely on
the controlling document having an opportunity to notify the prerendered
page. One option would be to extend the event object for
visibilitychange with a URL argument that is the URL that ends up being
loaded inside the page.

* The target page's URL actually needs to change in order to avoid
surprises as you navigate pages. I'm going back and forth between
expecting the prerendered page to call replaceState correctly, or
specifying that replaceState will get called after the visibilitychange
event gets dispatched. I haven't yet made up my mind on which approach
is the better one...

* Once we have the PrerenderedDocument interface, we can extend it with
a number of useful events, such as "load" which tells you when the
prerendered document's load event has been dispatched, and
"prerenderabort" to notify the controlling document if prerendering was
aborted (as a result of a call to let's say window.alert).

* The existing prerendering implementation in IE and Chrome is
opportunistic, i.e., the UA will be able to ignore the <link
rel=prerender> based on a number of heuristics (for example I think
Chrome prerenders at most one document at a time.) I think this is a
nice model to make sure that the UA has the option of avoiding
prerendering for memory consumption/responsive reasons, but I am not yet
sure if we need to introduce this notion on PrerenderedDocument...

* In order to simplify the implementation, there will be no direct DOM
access between the two pages even if they are loaded from the same
origin. This will help the UA to "kill" the prerendered page more
easily once it makes an API call which is blacklisted for example. All
communications need to happen by message passing.

These are still rough ideas, but any feedback is much appreciated. I'm
planning to propose something along these lines once the details have
been fleshed out more.

Cheers,
Ehsan

> On Mon, Aug 11, 2014 at 11:47 AM, Roshan Vidyashankar
> <rosh...@gmail.com> wrote:

[Jesper Kristensen]

This templated prerendereing sounds like a complicated API. Is there any
advantage of this over what is possible today with a single page
application using history.pushState?

[Ehsan Akhgari]

On 2014-08-12, 11:39 AM, Jesper Kristensen wrote:
> This templated prerendereing sounds like a complicated API. Is there any
> advantage of this over what is possible today with a single page
> application using history.pushState?

Yes, I think so. The idea of single page applications is nice, but it
doesn't eliminate the need for this API because:

1) That means rewriting large existing applications, and
2) Parts of this idea (for example for providing a way to communicate
with the prerendered content) would still be needed even fr single page
apps.

Cheers,
Ehsan

[Ilya Grigorik]

On Tuesday, August 12, 2014 8:39:11 AM UTC-7, Jesper Kristensen wrote:
> This templated prerendereing sounds like a complicated API. Is there any
> advantage of this over what is possible today with a single page
> application using history.pushState?

I'm still trying to wrap my head around the use case, but I'm with you on this so far...

In order to support the proposed behavior, you would need to:
(1) implement an endpoint that returns a base template page
(2) implement page logic to receive the message, fetch right data, and render it

If you have (1) and (2) implemented, you've effectively implemented the single-page app model.

It seems like there are two cases to consider here:
(a) You're on a page with the same base template as destination
(b) You're navigating to a page with a different base template

For (a), you're best off reusing the current context (it's already rendered, after all), and issuing a fetch for updated content and rendering it in place - with pushState, etc. Whereas for (b) you can also do it in place by prefetching the template and then issuing a data request for relevant data once you know the exact URL... Which, of course, is exactly how single-page apps work today.

In short, seems like this is inventing a derivative single-page app model for building pages/apps, and that makes me wonder because it doesn't seem to make it any easier. Everything described here can be achieved with current tools, but perhaps could be made better with some smart(er) prefetch/prerender strategies?

For example, prerender should not be limited to HTML:
https://igrigorik.github.io/resource-hints/#prerender

In your single-page app, you should be able to inject multiple prerender hints for images, CSS, JS, and let the UA fetch and process those ahead of time. Then, when the user triggers the action, just inject them into the doc and update the UI as necessary.

ig

[Peter Lepeska]

An alternative solution to templates for this use case is the following:

1) In the prerendered page's html, group preloaded resources (LINK rel=subresource currently) into "base" and "extended" hints. This can be accomplished by a new LINK attribute, for example.
2) The prerenderer can then fetch the html of the page and then only pre-download the subresources in the base group.

Given the above, the base group would include subresources associated with all bugs in bugzilla and so only these resources would be included in the prerender.

Peter

[Jonas Sicking]

On Tue, Aug 12, 2014 at 1:04 PM, Ilya Grigorik <igri...@gmail.com> wrote:
>
> I'm still trying to wrap my head around the use case, but I'm with you on this so far...

[snip]

> It seems like there are two cases to consider here:
> (a) You're on a page with the same base template as destination
> (b) You're navigating to a page with a different base template

Agreed.

> For (a), you're best off reusing the current context (it's already rendered, after all), and issuing a fetch for updated content and rendering it in place - with pushState, etc.

Yup.

> Whereas for (b) you can also do it in place by prefetching the template and then issuing a data request for relevant data once you know the exact URL... Which, of course, is exactly how single-page apps work today.

I don't understand what you are saying here.

The point of (b) is that the different base templates. So it'd be very
hard to use pushState and DOM mutations to morph the current page into
the destination page. The fact that the destination page is template
based doesn't make it any easier.

Consider for example google search and google calendar. The google
search page is very template based. I.e. while the contents of a
search result is varies, there's a lot of template content that is
shared between all search result pages.

If Google calendar included a "search the web" box, which when used
would navigate the user to a google search result page. It would be
very hard for google calendar to use pushState and DOM mutations to
transform a calendar page into a search result page.

Not only would the DOM need to be replaced. You'd also need to load
any scripts that google search depends on. And unload any script state
that google calendar had created and that might interfere with the
google search scripts.

This is essentially as hard as morphing the current page into a
completely different non-template-based webpage unless I'm missing
something.

It would be much more practically doable for calendar instead to
prerender a blank search result page, and then once the user commits
the search tell that prerendered page what the search parameters are
such that the page can load and render the appropriate search data.

/ Jonas

[Karl Dubost]

Jonas,
thanks for the explanations so far. It helps a lot to understand and sorry for the naive questions. There is still something I do not get:

Le 13 août 2014 à 07:17, Jonas Sicking <jo...@sicking.cc> a écrit :
> It would be much more practically doable for calendar instead to
> prerender a blank search result page, and then once the user commits
> the search tell that prerendered page what the search parameters are
> such that the page can load and render the appropriate search data.

Here you seem to talk about partial rendering.
I thought pre-render features were here to enable to show instantly a page when the user decides to follow a link. Aka the page is just ready to be shown. Here in your explanation you seem to say that the DOM is partially built but the painting is not done or at least has to be redone before displaying the page. Do we lose part of the benefits by doing that? Or maybe a more appropriate question is what is the gain in between a prefetch and a prerender in this case?



Another thought:
There is something very similar in between prefetch, prerender and the old next/prev values. That could revive those.
<link rel='next' href=''/>
<link rel='prev' href=''/>

[Jonas Sicking]

On Tue, Aug 12, 2014 at 4:18 PM, Karl Dubost <kdu...@mozilla.com> wrote:
> Jonas,
> thanks for the explanations so far. It helps a lot to understand and sorry for the naive questions. There is still something I do not get:
>
> Le 13 août 2014 à 07:17, Jonas Sicking <jo...@sicking.cc> a écrit :
>> It would be much more practically doable for calendar instead to
>> prerender a blank search result page, and then once the user commits
>> the search tell that prerendered page what the search parameters are
>> such that the page can load and render the appropriate search data.
>
> Here you seem to talk about partial rendering.
> I thought pre-render features were here to enable to show instantly a page when the user decides to follow a link. Aka the page is just ready to be shown.

The goal of prerender is to improve performance when navigating to a
new page. Right now it does so really well if the initial page is able
to correctly guess which URL the user will navigate to. But it does so
really poorly (i.e. not at all) if it guesses even slightly wrong.

Of course it's great if we can get the new page to render immediately.
But just because we can't get the new page to render immediately
doesn't mean that we don't care about loading the new page as fast as
possible anyway.

> Here in your explanation you seem to say that the DOM is partially built but the painting is not done or at least has to be redone before displaying the page. Do we lose part of the benefits by doing that? Or maybe a more appropriate question is what is the gain in between a prefetch and a prerender in this case?

I'm not talking about doing some form of partial parsing or partial rendering.

I'm talking about doing a full rendering of a "template page". I.e. a
normal webpage which the website can then use JavaScript to mutate
into the page that the user wanted to navigate to.

So for example when viewing a bugzilla search result, bugzilla could
request to prerender the URL
"http://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=1". When
this URL is loaded, bugzilla would return a page which contains the
same markup, the same CSS and the same JS as normal bug pages. But
which lacks any bug-specific data.

The browser would load this URL, just like any other. It would parse
all the markup, load all the CSS, JS and images that the page needs,
it would run any JS just like normal, and it would apply CSS just like
any other page.

When the user then clicks a link on the bugzilla search result page,
the search result page would use a click event listener and send a
message to the prerendered page letting it know that the user just
clicked the link for bug 123456.

The prerendered page would then use replaceState to change its URL to
http://bugzilla.mozilla.org/show_bug.cgi?id=123456. It would also fire
off an XHR request to fetch a JSON formatted description of all data
about bug 123456. Including bug title, bug dependencies, list of
attachment names and types and all bug comments.

Since the prerendered page's URL now match the URL of the page that
the user wanted to navigate to, the UA would decide to display the
prerendered page rather than load a new page. This would immediately
display the "template page" that was initially loaded.

If the template page still hasn't finished its XHR request (which is
likely given that switching to the prerendered page can be made very
quick), that means that the user will now see a page without the bug
data.

However this still looks like progress to the user, and very quick
progress. And the XHR request will very likely finish loading much
faster than it would have taken to load+process all the resources that
we currently load when the user clicks a bug link in the search
results.

I hope that makes it more clear.

So very little of this is handled by the UA. I.e. no magic partial
parsing, and no magic understanding of what parts of the template page
is headers vs. which parts will be filled in later.

> Another thought:
> There is something very similar in between prefetch, prerender and the old next/prev values. That could revive those.
> <link rel='next' href=''/>
> <link rel='prev' href=''/>

That could be, but I don't see how these could help in the scenarios
I've described in this thread?

/ Jonas

[Karl Dubost]

Le 13 août 2014 à 09:13, Jonas Sicking <jo...@sicking.cc> a écrit :
> The goal of prerender is to improve performance when navigating to a
> new page.

my understanding too.

> I'm talking about doing a full rendering of a "template page". I.e. a
> normal webpage which the website can then use JavaScript to mutate
> into the page that the user wanted to navigate to.

[cut the explanation]
So I guess I'm getting confused by the term 'prerender' and the scope. It looks like indeed more like a single page app (with all the caveats of client side generated pages) or a rel='template' more than a rel='prerender'.

What would be the markup for the scenario you are explaining?
And what would be the fallback for non JS user agents (cue accessibility and HTTP scripting here)?

If I do
<link rel='prerender' href='http://example.org/nextbug0001.html'/>

How the server knows it has to send the template or the full HTML page?
What is the HTTP caching story with regards to this URI http://example.org/nextbug0001.html?
And how does it help with http://example.org/nextbug0002.html in terms of caching?

[Andrew Sutherland]

On 08/12/2014 04:04 PM, Ilya Grigorik wrote:
> In short, seems like this is inventing a derivative single-page app model for building pages/apps, and that makes me wonder because it doesn't seem to make it any easier. Everything described here can be achieved with current tools, but perhaps could be made better with some smart(er) prefetch/prerender strategies?

:vingtetun can probably speak to it better than I can, but my
understanding was that for Firefox OS and the Haida UX effort
(https://wiki.mozilla.org/FirefoxOS/Haida) targeting mobile phones,
there were some practical benefits foreseen:

- Consistent in-app page/card transitions. Each Gaia app has a
custom/ad-hoc solution right now. The effort to make more reusable
components will likely address this, but then a consistent platform UX
still depends on apps effectively making themself "Firefox OS" specific
by using the Firefox building blocks/etc.

- Consistent support of/use of edge gestures inside the app.
https://wiki.mozilla.org/FirefoxOS/Haida#Edge_Gestures_Between_Open_Content

- Improved resource usage patterns by every page being an iframe with
only a little stuff in it. It's harder to accidentally leak memory or
get burnt by heap fragmentation if you're only doing a few things and
cleaned up entirely as you are superseded. However the assumption was
that these iframes would be talking to SharedWorkers which presumably
would have the same problem.

While Haida apps designed for the phone form factor would never be
usable on a desktop device as-is, it's neat that the pre-render
mechanism makes the same underlying implementation workable for both
cases, especially since it makes a "many simple pages" approach viable.
Right now if you want a performant implementation, you are inevitably
driven to the single-page app implementation approach.

Andrew

[Jonas Sicking]

On Tue, Aug 12, 2014 at 5:47 PM, Karl Dubost <kdu...@mozilla.com> wrote:
>
> Le 13 août 2014 à 09:13, Jonas Sicking <jo...@sicking.cc> a écrit :
>> The goal of prerender is to improve performance when navigating to a
>> new page.
>
> my understanding too.
>
>> I'm talking about doing a full rendering of a "template page". I.e. a
>> normal webpage which the website can then use JavaScript to mutate
>> into the page that the user wanted to navigate to.
>
> [cut the explanation]
> So I guess I'm getting confused by the term 'prerender' and the scope. It looks like indeed more like a single page app (with all the caveats of client side generated pages) or a rel='template' more than a rel='prerender'.

I don't understand why a separate rel value would be beneficial.

> What would be the markup for the scenario you are explaining?

Exactly the same as any other time when prerendering is used. Like I
said, no magic needed. So I believe the markup would be

<link rel=prerender href="...">

> And what would be the fallback for non JS user agents (cue accessibility and HTTP scripting here)?

The same as any other time when prerendering is used. The fallback
would simply be that the next page is loaded at normal speed, rather
than being prerendered. The only thing that the page would have to do
special here when the initial page attempts to send a message to the
prerendered page about which bug's data to load, if the API for
talking to the message channel is missing, don't do anything.

> If I do
> <link rel='prerender' href='http://example.org/nextbug0001.html'/>
>
> How the server knows it has to send the template or the full HTML page?

You include that information in the URL. See the examples I've used in
the explanations so far.

> What is the HTTP caching story with regards to this URI http://example.org/nextbug0001.html?
> And how does it help with http://example.org/nextbug0002.html in terms of caching?

The same as for any other time prerendering is used.

I'm definitely getting the impression that we're still talking past
each other. I guess maybe me or you are misunderstanding how the
prerendering spec is defined.

/ Jonas

[Karl Dubost]

Just to be clear I have *read*
http://code.google.com/chrome/whitepapers/prerender.html
So my understanding of pretender is based on this whitepaper. :)

Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
>> And what would be the fallback for non JS user agents (cue accessibility and HTTP scripting here)?
>
> The same as any other time when prerendering is used. The fallback
> would simply be that the next page is loaded at normal speed, rather
> than being prerendered. The only thing that the page would have to do
> special here when the initial page attempts to send a message to the
> prerendered page about which bug's data to load, if the API for
> talking to the message channel is missing, don't do anything.

<!doctype html>
<html>
<title>bug</title>
<link rel='prerender' href='http://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=1'/>
<a href="/bug0001">my next bug</a> and
<a href="/bug0002">my next next bug</a>
</html>

In your scenario, this would not work without scripting. I can see how useful it could be with Firefox OS and transitions between different states of an application. On the Web, it means that we get yet another dependency on JS for this to be working for capturing the click event and sending a XHR with Accept: application/json. On the other hand if no JS, the server receives the HTTP GET requests on the resource, but then without pre-rendering. :)
I guess it's why I'm asking questions to be sure how it would be working.

I was asking questions about the caching because once
http://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=1
is downloaded, rendered and cached. It could be reused for all subsequent requests with the same pattern. But in return it means we create an issue for Web developers for developing and testing, or just modifying the cache. I guess people can rely to cache busting techniques such as timestamp parameters in URIs.

> I'm definitely getting the impression that we're still talking past
> each other. I guess maybe me or you are misunderstanding how the
> prerendering spec is defined.

I understand what you are saying. :) But maybe we are assuming different things for the Web and/or have read a different spec. It's ok.

Thanks Jonas.
No more questions. :)

[Jonas Sicking]

On Tue, Aug 12, 2014 at 6:53 PM, Karl Dubost <kdu...@mozilla.com> wrote:
> Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
>>> And what would be the fallback for non JS user agents (cue accessibility and HTTP scripting here)?
>>
>> The same as any other time when prerendering is used. The fallback
>> would simply be that the next page is loaded at normal speed, rather
>> than being prerendered. The only thing that the page would have to do
>> special here when the initial page attempts to send a message to the
>> prerendered page about which bug's data to load, if the API for
>> talking to the message channel is missing, don't do anything.
>
> <!doctype html>
> <html>
> <title>bug</title>
> <link rel='prerender' href='http://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=1'/>
> <a href="/bug0001">my next bug</a> and
> <a href="/bug0002">my next next bug</a>
> </html>
>
> In your scenario, this would not work without scripting. I can see how useful it could be with Firefox OS and transitions between different states of an application.

Note that the example I used was bugzilla. So this is definitely
useful outside of FirefoxOS.

But yes, this depends on scripting. It sounds like between the two of
us we have three goals

* Enable fast navigation between webpages in websites like bugzilla.
* When a search result page is loaded, avoid loading and prerendering
a separate DOM for each bug.
* Avoid relying on JS.

I can only think of solutions which fulfill two of those goals at the
same time. Which two would be left up to the bugzilla web developers.

If you have ideas for how to solve all three at the same time I'm all ears.

> I was asking questions about the caching because once
> http://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=1
> is downloaded, rendered and cached.

Note that I don't think anyone is (yet) proposing caching of a
prerendered *page*. The individual resources that are loaded to make
up the prerendered page will be loaded through the normal http stack,
which means that those resources might get cached, or be loaded from
cache. But the only thing that's cached is the bytes coming from the
network. No parsed datastructures are cached. The parsed and rendered
DOM will either be used immediately, or thrown away completely.

> It could be reused for all subsequent requests with the same pattern.

For now the HTML resource will only be reused for other loads to
http://bugzilla.mozilla.org/show_bug.cgi?justLoadTemplate=1. Which
likely it mean it will only be used when a page is prerendered.

It will not be used if the user simply navigates to a bug page like
http://bugzilla.mozilla.org/show_bug.cgi?id=123456

/ Jonas

[Ilya Grigorik]

(reposting, got stuck in the queue)

On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Tue, Aug 12, 2014 at 1:04 PM, Ilya Grigorik <igri...@gmail.com>
> wrote:

> > Whereas for (b) you can also do it in place by prefetching the template
> and then issuing a data request for relevant data once you know the exact
> URL... Which, of course, is exactly how single-page apps work today.
>

> I don't understand what you are saying here.
>
> The point of (b) is that the different base templates. So it'd be very
> hard to use pushState and DOM mutations to morph the current page into
> the destination page. The fact that the destination page is template
> based doesn't make it any easier.
>

It's been done before, and with pretty good success:
- http://pjax.heroku.com/
- https://github.com/rails/turbolinks - derivative of pjax and in fairly
wide use, since its core to Rails.

As turbolinks repo notes, you get the most benefit from this technique when
you preserve the same CSS/JS... but the point is, full replace is not out
of scope either.

On Tue, Aug 12, 2014 at 7:19 PM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Tue, Aug 12, 2014 at 6:53 PM, Karl Dubost <kdu...@mozilla.com> wrote:
> > Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
> But yes, this depends on scripting. It sounds like between the two of
> us we have three goals
>
> * Enable fast navigation between webpages in websites like bugzilla.
> * When a search result page is loaded, avoid loading and prerendering
> a separate DOM for each bug.
> * Avoid relying on JS.
>

So, same origin; substantially different page template; single-page app.

It seems like the workflow you're after is something like:
<link rel="prerender" href="/some/other/template.html"
route="/other-thing/*" id="ABC">

When a navigation is initiated to "/other-thing/random", the above
prerender context is used:
- prerender context gets a notification that its visible (i.e. Page
Visibility API)
- prerender context inspects the actual target URL, extract right tokens
from it, does its thing...

Is there even a need for postMessage, etc? Seems like the necessary pieces
are already there - i.e. the URL.

ig

(Note: I'm still a bit skeptical of the use case, but... curious :))


On Wed, Aug 13, 2014 at 12:24 PM, Ilya Grigorik <igri...@gmail.com> wrote:

> On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>
>> On Tue, Aug 12, 2014 at 1:04 PM, Ilya Grigorik <igri...@gmail.com>
>> wrote:

>> > Whereas for (b) you can also do it in place by prefetching the template
>> and then issuing a data request for relevant data once you know the exact
>> URL... Which, of course, is exactly how single-page apps work today.
>>

>> I don't understand what you are saying here.
>>
>> The point of (b) is that the different base templates. So it'd be very
>> hard to use pushState and DOM mutations to morph the current page into
>> the destination page. The fact that the destination page is template
>> based doesn't make it any easier.
>>
>

> It's been done before, and with pretty good success:
> - http://pjax.heroku.com/
> - https://github.com/rails/turbolinks - derivative of pjax and in fairly
> wide use, since its core to Rails.
>
> As turbolinks repo notes, you get the most benefit from this technique
> when you preserve the same CSS/JS... but the point is, full replace is not
> out of scope either.

>
> On Tue, Aug 12, 2014 at 7:19 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>
>> On Tue, Aug 12, 2014 at 6:53 PM, Karl Dubost <kdu...@mozilla.com>
>> wrote:
>> > Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
>> But yes, this depends on scripting. It sounds like between the two of
>> us we have three goals
>>
>> * Enable fast navigation between webpages in websites like bugzilla.
>> * When a search result page is loaded, avoid loading and prerendering
>> a separate DOM for each bug.
>> * Avoid relying on JS.
>>
>

> So, same origin; substantially different page template; single-page app.
>
> It seems like the workflow you're after is something like:
> <link rel="prerender" href="/some/other/template.html"
> route="/other-thing/*" id="ABC">
>
> When a navigation is initiated to "/other-thing/random", the above
> prerender context is used:
> - prerender context gets a notification that its visible (i.e. Page
> Visibility API)
> - prerender context inspects the actual target URL, extract right tokens
> from it, does its thing...
>
> Is there even a need for postMessage, etc? Seems like the necessary pieces
> are already there - i.e. the URL.
>
> ig
>
> (Note: I'm still a bit skeptical of the use case, but... curious :))
>

[Ilya Grigorik]

On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Tue, Aug 12, 2014 at 1:04 PM, Ilya Grigorik <igri...@gmail.com>
> wrote:

> > Whereas for (b) you can also do it in place by prefetching the template
> and then issuing a data request for relevant data once you know the exact
> URL... Which, of course, is exactly how single-page apps work today.
>

[Ehsan Akhgari]

On 2014-08-13, 3:24 PM, Ilya Grigorik wrote:
> On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>
>> On Tue, Aug 12, 2014 at 1:04 PM, Ilya Grigorik <igri...@gmail.com>
>> wrote:

>>> Whereas for (b) you can also do it in place by prefetching the template
>> and then issuing a data request for relevant data once you know the exact
>> URL... Which, of course, is exactly how single-page apps work today.
>>

Well, firstly, this doesn't _need_ to be same origin as the controlling
document. Think of wikipedia where their application is spread across
tons of origins.

Also, with your proposal, how would the prerendered page know about the
actual URL? (We could borrow the bit from my proposal about extending
the event object.)

But most importantly, we need to think about the case where the
prerendered template page can't actually handle the URL that the
controlling document claims in their <link rel=prerender>. How would
the page signal this to the browser so that we know to throw away the
prerendered content and do a normal navigation?

Cheers,
Ehsan

[Jonas Sicking]

On Wed, Aug 13, 2014 at 12:24 PM, Ilya Grigorik <igri...@gmail.com> wrote:
> On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>> On Tue, Aug 12, 2014 at 1:04 PM, Ilya Grigorik <igri...@gmail.com>
>> wrote:

>> > Whereas for (b) you can also do it in place by prefetching the template
>> > and then issuing a data request for relevant data once you know the exact
>> > URL... Which, of course, is exactly how single-page apps work today.
>>

>> I don't understand what you are saying here.
>>
>> The point of (b) is that the different base templates. So it'd be very
>> hard to use pushState and DOM mutations to morph the current page into
>> the destination page. The fact that the destination page is template
>> based doesn't make it any easier.
>
>
> It's been done before, and with pretty good success:
> - http://pjax.heroku.com/
> - https://github.com/rails/turbolinks - derivative of pjax and in fairly
> wide use, since its core to Rails.
>
> As turbolinks repo notes, you get the most benefit from this technique when
> you preserve the same CSS/JS... but the point is, full replace is not out of
> scope either.

I don't understand why it makes a difference that the destination page
is "template based". If the current page and the destination page use
different CSS, JS and images, you are essentially morphing between two
entirely different pages.

If we think that's doable, then it sounds like we are arguing that
prerender is not useful for same-origin navigation? Or am I missing
something?

Do you really think that Google would be able to morph a google
calendar page into the google search result page?

> On Tue, Aug 12, 2014 at 7:19 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>> On Tue, Aug 12, 2014 at 6:53 PM, Karl Dubost <kdu...@mozilla.com> wrote:
>> > Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
>> But yes, this depends on scripting. It sounds like between the two of
>> us we have three goals
>>
>> * Enable fast navigation between webpages in websites like bugzilla.
>> * When a search result page is loaded, avoid loading and prerendering
>> a separate DOM for each bug.
>> * Avoid relying on JS.
>
>
> So, same origin; substantially different page template; single-page app.

I'm not sure why you say "single-page app"? The bugzilla search result
page and the page rendering individual bugs are decidedly different
pages right now. I.e. the DOM looks very different, they run different
scripts, they are generated from different server-side scripts and
they use different URLs, including the parts before the #.

Maybe I don't understand what your definition of "single-page app" is.

> It seems like the workflow you're after is something like:
> <link rel="prerender" href="/some/other/template.html"
> route="/other-thing/*" id="ABC">
>
> When a navigation is initiated to "/other-thing/random", the above prerender
> context is used:
> - prerender context gets a notification that its visible (i.e. Page
> Visibility API)
> - prerender context inspects the actual target URL, extract right tokens
> from it, does its thing...
>
> Is there even a need for postMessage, etc? Seems like the necessary pieces
> are already there - i.e. the URL.

The point of the postMessage was mainly to expose the actual target
URL. I.e. by enabling a message channel between the initiating page
and the prerendered page we wouldn't need to invent an API for
exposing the target URL. Nor would we need to add a route attribute or
define a URL pattern matching syntax.

A message channel mainly uses components that already exist in the web
platform. And it enables other features which would enable the
prerendered page to be even more up-to-date by the time it gets
rendered.

> (Note: I'm still a bit skeptical of the use case, but... curious :))

The use case here is any search interface, as well as any page that
contains links to database generated content. Both are incredibly
common on the web. And, like Eshan points out, it works even when
those navigate across origins.

So for example:
* Amazon's product search UI
* Amazon's search result UI
* Wikipedia's search UI
* Wikipedia's search result UI
* The search UI on NY Times
* The search results UI on NY Times
* Almost any product listing page
* A contacts manager which uses "real" URLs rather than fragment-based URLs.

I'm not sure why you are skeptical about that use case?

So it would dramatically improve the situations when you could use
prerendering. As it stands right now, you can't use prerendering at
all for search UI. And you can only use it for search result UIs

And to be clear, I'm not proposing making major changes or additions
to the current spec. The only suggested changes is to add support for
having a message channel, and that if the prerendered page calls
replaceState that we match against the new URL rather than the URL
that was originally loaded.

And technically we could probably get away without the message channel
since you can fake it using BroadcastChannel. But that's a bit error
prone.

/ Jonas

[Ilya Grigorik]

On Wed, Aug 13, 2014 at 5:16 PM, Jonas Sicking <jo...@sicking.cc> wrote:

> On Wed, Aug 13, 2014 at 12:24 PM, Ilya Grigorik <igri...@gmail.com>
> wrote:
> > On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>

> Do you really think that Google would be able to morph a google
> calendar page into the google search result page?
>

<aside>
While this is mostly orthogonal to our discussion, worth checking out if
you're not familiar with it:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/lWCtrTynaVk/Bvf4jIeEuPcJ

Obviously, the above does not make the page transition faster, but it does
provide a "morph facility".
</aside>

> > On Tue, Aug 12, 2014 at 7:19 PM, Jonas Sicking <jo...@sicking.cc> wrote:
> >>
> >> On Tue, Aug 12, 2014 at 6:53 PM, Karl Dubost <kdu...@mozilla.com>
> wrote:
> >> > Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
> >> But yes, this depends on scripting. It sounds like between the two of
> >> us we have three goals
> >>
> >> * Enable fast navigation between webpages in websites like bugzilla.
> >> * When a search result page is loaded, avoid loading and prerendering
> >> a separate DOM for each bug.
> >> * Avoid relying on JS.
> >
> >
> > So, same origin; substantially different page template; single-page app.
>
> I'm not sure why you say "single-page app"? The bugzilla search result
> page and the page rendering individual bugs are decidedly different
> pages right now. I.e. the DOM looks very different, they run different
> scripts, they are generated from different server-side scripts and
> they use different URLs, including the parts before the #.
>
> Maybe I don't understand what your definition of "single-page app" is.
>

In this context, an app that performs in-place updates, as opposed to full
page reloads, when transitioning between different views. The views can use
different JS, CSS, and so on. To achieve this, you have to build your app
in a very particular way. In the case of bugzilla, that would involve a
substantial rewrite of the app... you can't just drop in a few prerender
tags on the page and defer the rest to the browser.

> > It seems like the workflow you're after is something like:
> > <link rel="prerender" href="/some/other/template.html"
> > route="/other-thing/*" id="ABC">
> >
> > When a navigation is initiated to "/other-thing/random", the above
> prerender
> > context is used:
> > - prerender context gets a notification that its visible (i.e. Page
> > Visibility API)
> > - prerender context inspects the actual target URL, extract right tokens
> > from it, does its thing...
> >
> > Is there even a need for postMessage, etc? Seems like the necessary
> pieces
> > are already there - i.e. the URL.
>
> The point of the postMessage was mainly to expose the actual target
> URL. I.e. by enabling a message channel between the initiating page
> and the prerendered page we wouldn't need to invent an API for
> exposing the target URL. Nor would we need to add a route attribute or
> define a URL pattern matching syntax.
>

Why can't I just listen for visibilitychange event and then query
document.location in the handler? I'm intentionally handwaving my way
through the implementation details, but roughly, I'm picturing it as:
render this template, it'll match any of these routes; navigation is
invoked; matching prerender is found and is used to render the remainder of
the page content.

> (Note: I'm still a bit skeptical of the use case, but... curious :))
>
> The use case here is any search interface, as well as any page that
> contains links to database generated content. Both are incredibly
> common on the web. And, like Eshan points out, it works even when
> those navigate across origins.
>

... assuming all of those pages are rewritten to take advantage of this
technique, which is no small ask. To make matters worse, at least as
implemented today, prerender is a speculative thing that may or may not
happen. In which case, all of the sites would also have to implement a
fallback.

> So for example:
> * Amazon's product search UI
> * Amazon's search result UI
> * Wikipedia's search UI
> * Wikipedia's search result UI
> * The search UI on NY Times
> * The search results UI on NY Times
> * Almost any product listing page
> * A contacts manager which uses "real" URLs rather than fragment-based
> URLs.
>

> I'm not sure why you are skeptical about that use case?
>

All of these are plausible today with current infrastructure. I'm not
skeptical of the examples, I'm skeptical of the value add of this API on
top of what's out there today.

> And to be clear, I'm not proposing making major changes or additions
> to the current spec. The only suggested changes is to add support for
> having a message channel, and that if the prerendered page calls
> replaceState that we match against the new URL rather than the URL
> that was originally loaded.
>

You'd also need the ability to tell the UA to match against some URL
pattern, which we don't have today. Right?

ig

[Ehsan Akhgari]

On 2014-08-14, 1:44 AM, Ilya Grigorik wrote:
> On Wed, Aug 13, 2014 at 5:16 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>
>> On Wed, Aug 13, 2014 at 12:24 PM, Ilya Grigorik <igri...@gmail.com>
>> wrote:
>>> On Tue, Aug 12, 2014 at 3:17 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>
>> Do you really think that Google would be able to morph a google
>> calendar page into the google search result page?
>>
>
> <aside>
> While this is mostly orthogonal to our discussion, worth checking out if
> you're not familiar with it:
> https://groups.google.com/a/chromium.org/d/msg/blink-dev/lWCtrTynaVk/Bvf4jIeEuPcJ
>
> Obviously, the above does not make the page transition faster, but it does
> provide a "morph facility".
> </aside>

I don't understand how this is related to the current discussion. The
proposal there seems to be about some kind of IE4 style transition as
the navigation is happening
(http://msdn.microsoft.com/en-us/library/ms532847%28v=vs.85%29.aspx). I
don't think that is related to prerendering...

>>> On Tue, Aug 12, 2014 at 7:19 PM, Jonas Sicking <jo...@sicking.cc> wrote:
>>>>
>>>> On Tue, Aug 12, 2014 at 6:53 PM, Karl Dubost <kdu...@mozilla.com>
>> wrote:
>>>>> Le 13 août 2014 à 10:09, Jonas Sicking <jo...@sicking.cc> a écrit :
>>>> But yes, this depends on scripting. It sounds like between the two of
>>>> us we have three goals
>>>>
>>>> * Enable fast navigation between webpages in websites like bugzilla.
>>>> * When a search result page is loaded, avoid loading and prerendering
>>>> a separate DOM for each bug.
>>>> * Avoid relying on JS.
>>>
>>>
>>> So, same origin; substantially different page template; single-page app.
>>
>> I'm not sure why you say "single-page app"? The bugzilla search result
>> page and the page rendering individual bugs are decidedly different
>> pages right now. I.e. the DOM looks very different, they run different
>> scripts, they are generated from different server-side scripts and
>> they use different URLs, including the parts before the #.
>>
>> Maybe I don't understand what your definition of "single-page app" is.
>>
>
> In this context, an app that performs in-place updates, as opposed to full
> page reloads, when transitioning between different views. The views can use
> different JS, CSS, and so on. To achieve this, you have to build your app
> in a very particular way. In the case of bugzilla, that would involve a
> substantial rewrite of the app... you can't just drop in a few prerender
> tags on the page and defer the rest to the browser.

The point of what Jonas and I are talking about is to avoid having to
rewrite the app. The existing URLs and pages will be preserved for
normal (non-prerendered) navigation, and the app would need to implement
a separate template page which knows how to load the barebone
CSS/JS/images and then fill in the rest based on the information
received from the controlling document.

>>> It seems like the workflow you're after is something like:
>>> <link rel="prerender" href="/some/other/template.html"
>>> route="/other-thing/*" id="ABC">
>>>
>>> When a navigation is initiated to "/other-thing/random", the above
>> prerender
>>> context is used:
>>> - prerender context gets a notification that its visible (i.e. Page
>>> Visibility API)
>>> - prerender context inspects the actual target URL, extract right tokens
>>> from it, does its thing...
>>>
>>> Is there even a need for postMessage, etc? Seems like the necessary
>> pieces
>>> are already there - i.e. the URL.
>>
>> The point of the postMessage was mainly to expose the actual target
>> URL. I.e. by enabling a message channel between the initiating page
>> and the prerendered page we wouldn't need to invent an API for
>> exposing the target URL. Nor would we need to add a route attribute or
>> define a URL pattern matching syntax.
>>
>
> Why can't I just listen for visibilitychange event and then query
> document.location in the handler?

Because your document.location will reflect the URL of the loaded page,
not the URL of the navigation target. The issue is that the controlling
page will have several links such as http://app.com/result1,
http://app.com/result2, http://app.com/someotherresult etc. Before the
user decides which one they want to click on, neither the browser nor
the app know which URL to prerender.

> I'm intentionally handwaving my way
> through the implementation details, but roughly, I'm picturing it as:
> render this template, it'll match any of these routes; navigation is
> invoked; matching prerender is found and is used to render the remainder of
> the page content.

Like I asked before, what if the prerendered template page is unable to
handle the actual navigation target? In the example above, let's
imagine that result2 cannot be handled by the prerendered page for some
reason. It seems like with your proposal, the template page has no way
of indicating this to the browser, so if the user clicks on result2 then
the browser will render the prerendered template page which probably has
the wrong content on it.

>> (Note: I'm still a bit skeptical of the use case, but... curious :))
>>
>> The use case here is any search interface, as well as any page that
>> contains links to database generated content. Both are incredibly
>> common on the web. And, like Eshan points out, it works even when
>> those navigate across origins.
>>
>
> ... assuming all of those pages are rewritten to take advantage of this
> technique, which is no small ask.

There will be no massive rewrite.

> To make matters worse, at least as
> implemented today, prerender is a speculative thing that may or may not
> happen. In which case, all of the sites would also have to implement a
> fallback.

Well this is opt-in. If the controlling document wants to prerender a
template page like this without the target web site supporting that,
then we'll fall back to normal navigation.

>> So for example:
>> * Amazon's product search UI
>> * Amazon's search result UI
>> * Wikipedia's search UI
>> * Wikipedia's search result UI
>> * The search UI on NY Times
>> * The search results UI on NY Times
>> * Almost any product listing page
>> * A contacts manager which uses "real" URLs rather than fragment-based
>> URLs.
>>
>
>> I'm not sure why you are skeptical about that use case?
>>
>
> All of these are plausible today with current infrastructure. I'm not
> skeptical of the examples, I'm skeptical of the value add of this API on
> top of what's out there today.

The interesting thing about these use cases is that on each one of these
pages there is a number of different links that the user can click on,
and without knowing which one will be clicked beforehand, the website
can't prerender anything useful. The best it would be able to do with
the current API is to prerender the first result or some such.

>> And to be clear, I'm not proposing making major changes or additions
>> to the current spec. The only suggested changes is to add support for
>> having a message channel, and that if the prerendered page calls
>> replaceState that we match against the new URL rather than the URL
>> that was originally loaded.
>>
>
> You'd also need the ability to tell the UA to match against some URL
> pattern, which we don't have today. Right?

No, the matching would be exact. With the example above, let's say that
we have prerendered app.com/template. In the onclick handler for
result1, we'd send a message to the target page signaling it that
result1 has been picked, and the template page will replaceState its URL
to be app.com/result1, at which point the browser will do an exact match
between the prerendered doucument's URI and the navigation target, and
because they both match, the prerendered document will be rendered. If
result2 is clicked however, the onmessage handler in the template page
decides that it is unable to handle that, and doesn't replaceState, so
the browser will fail to match the URL to be loaded with any prerendered
content, so it will fall back to normal navigation.

Cheers,
Ehsan

[Gavin Sharp]

On Thu, Aug 14, 2014 at 8:32 AM, Ehsan Akhgari <ehsan....@gmail.com> wrote:
>> In this context, an app that performs in-place updates, as opposed to full
>> page reloads, when transitioning between different views. The views can
>> use
>> different JS, CSS, and so on. To achieve this, you have to build your app
>> in a very particular way. In the case of bugzilla, that would involve a
>> substantial rewrite of the app... you can't just drop in a few prerender
>> tags on the page and defer the rest to the browser.

> The point of what Jonas and I are talking about is to avoid having to
> rewrite the app.

[snip]

> the app would need to implement a separate template page which knows
> how to load the barebone CSS/JS/images and then fill in the rest based
> on the information received from the controlling document.

The argument being made is that what you describe here would be a
similar amount of work as "rewriting the app" (to be a single-page
app), at least for the specific page involved.

Gavin

[Jonas Sicking]

The last part is key. It's dramatically easier to rewrite just the
bug-page to enable transitioning between individual bug pages. Rather
than to rewrite all of bugzilla to enable transitioning between
arbitrary pages in bugzilla.

It's easier yet to write a bug-template-page that can transition
itself to be an arbitrary bugpage than to enable transitioning between
bug pages.

Another point here is that with ServiceWorkers developers can get
significant speedups prerendering aside if they create such a template
page that can transition itself to be a bugpage.

Finally, my argument is that with a relatively small addition to the
platform we enable web developers to do this work if they want to. If
they don't then it's no extra work for them. If we *don't* make this
small addition to the platform then developers simply can't get any of
the speedups that prerendering offers in these cases.

/ Jonas

[John Schoenick]

I don't understand how a new API is required to do this. Why couldn't
the bugzilla search results page create a hidden document that has a
search-result template, and then handle the navigation to any of the
result links by filling it in and switching to it?
>
> / Jonas

[Jonas Sicking]

That is exactly the goal yes. The only thing we need to ensure is that
the prerendered template page is used. Currently the prerendered page
is only used if the URL in the <link rel=prerender href=x> matches the
URL that the user navigates to.

/ Jonas

[John Schoenick]

Yes, but, without any prerendering API at all bugzilla could do this
today, couldn't it? Create a template iframe, and then switch to it when
a user clicks a link it can handle, rather than letting the navigation
proceed. Why is an entire pre-rendering API with <link> tags and defined
events needed?

The only thing that is missing in that scenario is the ability to do
e.g. |window.navigateTo(existingiframe)| to save boilerplate around
swapping contents with your iframe, and handling pushState() yourself.


> / Jonas

[Ehsan Akhgari]

How could bugzilla replace that iframe with its own window instead of a
navigation? (Not to mention that any trickery like that will not work
cross origin.)

> The only thing that is missing in that scenario is the ability to do
> e.g. |window.navigateTo(existingiframe)| to save boilerplate around
> swapping contents with your iframe, and handling pushState() yourself.

Well, that is more or less what prerendering is all about. :-)