Intent: 103 Early Hints for HTTP/1.1

[Hoang Tran]

Hi everyone,

For the research purpose, I am trying to implement Early Hints support
for HTTP/1.1.

As the first step, I played with preload Link header in Chrome. The
local setup uses nghttpx as the server since it supports Early Hints,
and I tried different scenarios:

A) Original Chrome: the server responses a 200 status code with a Link
preload header and full HTML body.
Chrome does fetching for the requested asset in
LinkLoader::LoadLinksFromHeader() in DocumentLoader, when it consumes
the response's body. Stack trace is here https://pastebin.com/ungRYe4s

B) Patched Chrome: in HttpStreamParser, I changed the parser to not
ignore 103 status response. (The patch is in the attachment)
With this change, Chrome treats 103 response as if it is a 200
status response.

Now, LinkLoader::LoadLinksFromHeader() is also called, when
URLLoader completed. Stack trace is here https://pastebin.com/zz5GuWGY
But here the Link header_value in the response object is empty, and
so the fetching for preload asset is never done.

So my question is, in scenario A, where Chrome stores the response's
Link header (which never happened in scenario B)?

And if I am able to make fetching works for scenario B, would the
preloaded assets are still relevant after getting main 200 OK response?

Chrome is new to me, so if you have any other suggestion, it will be
also very appreciated.

Thank you for all your help!

Regards,

Hoang

[Matt Menke]

Are the response headers making it to LinkLoader for the 103 response or for the subsequent 200 response?  If it's the 200 response, HttpNetworkTransaction also has some magic for 1xx responses.  You may be running into that.  Not sure what the cache layer would make of those responses, either.

The entire network stack and everything hanging off of it only expects one set of response headers per request (Redirect response headers follow another route, once they hit the URLRequestJob layer), so it's unclear what else will break if you violate that assumption.  There are a lot of things between HttpStreamParser and blink's loading code that are interested in response headers.

You may be best served by working on content_shell instead of chrome, since there's so much less random stuff there dangling precariously off of the network stack.

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+unsubscribe@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/f08d1de0-7dc9-470e-8da3-d13d7cff133f%40uclouvain.be.

[Charles Harrison]

chromium-dev -> bcc

+loading-dev

+yoav

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CAEK7mvpsrcpjdFNhkj7qf%3D66p621w5AW0zhtwV9SQWehVQRoEA%40mail.gmail.com.

[Hoang Tran]

Hello Matt,

Thank you for the response,

In scenario B, I do not put the Link header in the 200 response (this is for testing, I know it is not compliant with the Early Hints spec).
But on the wire, the 103 and 200 status headers are in the same TCP segment, due to a limitation of nghttpx-mruby, so Chrome may get confused. I am searching for a better control on server side, so any recommendation is also appreciated.

Thanks for the suggestion about using content_shell. I was not aware of it before but will give it a try.

Regards,
Hoang

[Matt Menke]

Reading the 103 and 200 headers together shouldn't be a problem, at least for the old code.  The test HttpNetworkTransactionTest.Ignores1xx tests just this case.

One thing I often find useful to do is to write unit tests to better understand what's going on.  Obviously if you're not planning on landing anything, unit tests aren't strictly necessary, but figuring out where things are going wrong can be simpler in a unit test environment, instead of in a full browser.  You may want to modify the aforementioned test (Or the one just above it, which simulates receiving the 1xx and 200 headers separately) to try and figure out if something's going wrong at the HttpStreamParser/HttpNetworkTransaction layers, or at a higher layer.

Another possibility is that socket reuse is causing some issues, so you could also try having your server send "Connection: Close" with each set of headers.

[Matt Menke]

chromium-dev -> bcc

[Hoang Tran]

Thank you Matt,

I found in HttpStreamParser/HttpNetworkTransaction two other places that ignore 1xx status and restart the parsing process.
After disabling them (as in the attachment), now I am able to drive the 103 status as if it is a 200 OK,
so new Document is created and committed in DocumentLoader.

Then, Chrome sends out requests for assets specified in 103 status, as step 3 below:

    Client                 Server

                 GET /
(1)          ------------------->

           103 + Link preload: style.css
(2)          <-------------------

            GET style.css
(3)         -------------------->

        200 OK + <style.css>
(4)          <-------------------

        200 OK +root index.html
(5)          <--------------------

Now the problem is all the headers in the main 200 response in step 5 is treated as the payload as below.
As you already told, this is because Chrome only processes a single set of header.

I think there will be several works need to be done.

Could you have some suggestion about where I can start from?

Thank you in advance.

Regards,
Hoang

[Matt Menke]

I recommend you start by reading https://chromium.googlesource.com/chromium/src/+/master/net/docs/life-of-a-url-request.md

Unfortunately, you're going to have to modify every major layer.  The network stack is basically pull based (Or at least net/) is.  content::ResourceLoader drives the request, and only calls the method that triggers reading headers (URLRequest::Start) once, before reading the body - you'll have to hack in a way to request headers twice between them, and you'll have to make similar, if hopefully simpler, changes to URLRequestHttpJob, HttpNetworkTransaction, and HttpStreamParser (HttpCache::Transaction, too, unless you disable the cache.  You can disable the cache by modifying the code that sets up the network stack, in content/shell/browser/shell_url_request_context_getter.cc).

You'll also have to modify the renderer side of the equation similarly, which I'm much less familiar with.  Alternatively, you could make the ResourceDispatcherHost handle the hints in the browser process and not dispatch the extra sets of headers to the renderer, though then you'd have to write new code to handle the hints in the browser process.  If resource hints needs CORS (Sorry, not too familiar with when CORS is needed), that potentially gets to be pretty hairy.  Even if that's not an issue, hooking up the renderer may still be simplest.

You'll also have to think about whether you want to handle 103s before redirects or not.  Handling them before redirects would change even more fundamental assumptions than a second set of response headers does, so I'd recommend not doing that.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/2e6423e3-59de-34bf-15d5-d7d2b92db857%40uclouvain.be.

[Bence Béky]

FYI there's a bug already filed for this which is currently Available: https://crbug.com/671310.  It is particularly interesting because many people outside Chrome commented on it and on https://crbug.com/662197 and https://crbug.com/669820.

On Fri, Jul 21, 2017 at 3:47 AM, Hoang Tran <hoang...@uclouvain.be> wrote:

[Hoang Tran]

Hi Matt,

Thank you for your suggestions. I think this will be useful for anyone want to try with Informational Responses.

Since I don't have any more free time for this work, I summarized the ongoing work in this slides:

https://www.slideshare.net/VietHoangTran/implementing-early-hints-in-chrome

And from my discussion with the author Kazuho Oku, he pointed out another issue with my approach is that the RenderFrame/displayed page would switch too early and go blank, before the final 200 response arrived. This may irritate the end users. A better approach might be removing the dependency of preload on Document Commit. But this is also not easy I think.

Regards,

Hoang

[Yoav Weiss]

Apologies for not chiming in earlier (was on vacation when this thread started and failed to notice it on my return... :/)

I'm afraid that using Early Hints for link preload is not feasible, as you noticed. Preloads rely on Document to be there, in order to know which CSP policy to apply, referrer policy, etc. All that information is not available before the 200 response headers are fully returned, even if you were to recreate all the related renderer logic at part of the net stack (which you probably shouldn't). So the problem is not the implementation reliance on Document Commit, but the fact that you have don't have all the data you need at the time the 103s arrive.

At the same time, implementing 103 support in the net stack could be interesting for <link rel=preconnect>, which do not need all the above. So maybe you can pivot your work towards that goal?

chromium-dev -> bcc

To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.

To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/f08d1de0-7dc9-470e-8da3-d13d7cff133f%40uclouvain.be.

--
You received this message because you are subscribed to the Google Groups "net-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.

To post to this group, send email to net...@chromium.org.