Reasons chrome might not re-use an HTTP2 connection

449 views
Skip to first unread message

Ben Maurer

unread,
Mar 9, 2016, 11:17:32 PM3/9/16
to net-dev
Hey,

We're running an experiment at Facebook where we are loading our javascript and css from www.facebook.com so that the client can re-use the HTTP2 socket they have open for downloading stuff from our CDN. We're only serving this experiment to users who we know are coming to us over HTTP2. One thing I've noticed in some of our traces is that there appear to be some chrome users where we see non-zero times for DNS Lookup and Connection establishment for these resources.

Are there any scenerios in which Chrome would not re-use the http2 connection established for www.facebook.com upon seeing something of the form:

<html><head><script src="/path/to/foo.js" />

-b


Matt Menke

unread,
Mar 9, 2016, 11:28:40 PM3/9/16
to Ben Maurer, net-dev
Disclaimer:  I'm not an expert on our HTTP/2 code.

Unless it ran into some error / timeout with the old socket, a network change, or entering / leaving suspend mode, the only two cases I can think of are when one is a "privacy mode" request and the other is not (A privacy mode socket means no cookies sent/received, no channel ID - I'm not an expert on what request end up in that bucket).  Another case is when two different instances of the network stack are in use - like one was made using incognito mode, which doesn't seem that likely.

There's also the case where we run out of sockets - there's a per-network-stack instance limit of 256 connections (32 over proxies).  If we hit this limit, we'll try and close idle connections.  HTTP/2 sessions are generally the last to be shut down in that case, though, because of how things are wired up.

WebSocket requests also use their own connection, but I assume we're not talking WebSockets.

--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/6a420c53-205c-4330-8fb3-ea8aadff5d6c%40chromium.org.

Ryan Sleevi

unread,
Mar 9, 2016, 11:43:03 PM3/9/16
to Matt Menke, Ben Maurer, net-dev
On Wed, Mar 9, 2016 at 8:28 PM, Matt Menke <mme...@chromium.org> wrote:
the only two cases I can think of are when one is a "privacy mode" request and the other is not (A privacy mode socket means no cookies sent/received, no channel ID - I'm not an expert on what request end up in that bucket). 

For web-facing aspect, "privacy mode" = CORS Anonymous. As per Fetch spec, if there are different CORS flags on it, you end up with different sockets in the socket pool.

Given that we're talking about Javascript & CSS, I suspect the CORS behaviour / Fetch spec logic is to blame (which is to say, it's WAI / WASpecced)

Ben Maurer

unread,
Mar 9, 2016, 11:47:54 PM3/9/16
to Ryan Sleevi, Matt Menke, net-dev
So we have crossorigin=anonymous on our scripts. However, they are the same origin as the document (ie, the document is www.facebook.com and the scripts are specified as a relative path). In my testing locally I'm seeing the www.facebook.com socket be re-used in this situation.

Randy Smith

unread,
Mar 10, 2016, 10:50:07 AM3/10/16
to Ben Maurer, Ryan Sleevi, Matt Menke, net-dev
I recently did some debugging on an issue like this in connecting to the facebook CDN, and eventually found that it was indeeds the anonymous issue.  For me it was loading www.facebook.com (no login); the resources a-ZN6WoEOje.png and zcgpzWDXjE7.js were being loaded over different connections due to one being loaded with network flags DO_NOT_USE_EMBEDDED_IDENTITY (which didn't force privacy mode) and one being loaded with (DO_NOT_SAVE_COOKIES | DO_NOT_SEND_AUTH_DATA | DO_NOT_SEND_COOKIES | VERIFY_EV_CERT) (which did).  Grabbing a net-internals dump will tell you pretty quickly if that's what's going on; take a look at the load flags in the URL request and that HTTP2_SESSION each HTTP_STREAM_JOB is bound to.

-- Randy


--
You received this message because you are subscribed to the Google Groups "net-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to net-dev+u...@chromium.org.
To post to this group, send email to net...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/net-dev/CABgOVaL_8ssEH2gYg1da7zhMUzgovNwQFUdQ57p29rhzDZLKWw%40mail.gmail.com.

Ben Maurer

unread,
Mar 10, 2016, 1:13:36 PM3/10/16
to Randy Smith, Ryan Sleevi, Matt Menke, net-dev
Hey,

Keep in mind that most users (yourself included) are seeing the traditional method which uses a CDN on a different origin. In this case I do see the privacy mode flags that you mention.

However in the experimental configuration, these flags are not set, despite the crossorigin=anonymous flag since the request is not cross origin. Below is a paste from my net-internals.

-b

t=8967 [st= 997]   +URL_REQUEST_START_JOB  [dt=0]
                    --> load_flags = 33024 (MAYBE_USER_GESTURE | VERIFY_EV_CERT)
                    --> method = "GET"
                    --> priority = "LOWEST"
                    --> url = "https://www.facebook.com/rsrc.php/v2/yo/r/XtyuVDKK3xt.js"
t=8967 [st= 997]      SERVICE_WORKER_START_REQUEST
t=8967 [st= 997]      SERVICE_WORKER_FALLBACK_RESPONSE
t=8967 [st= 997]   -URL_REQUEST_START_JOB
t=8967 [st= 997]   +URL_REQUEST_START_JOB  [dt=143]
                    --> load_flags = 33024 (MAYBE_USER_GESTURE | VERIFY_EV_CERT)
                    --> method = "GET"
                    --> priority = "LOWEST"
                    --> url = "https://www.facebook.com/rsrc.php/v2/yo/r/XtyuVDKK3xt.js"
t=8967 [st= 997]      URL_REQUEST_DELEGATE  [dt=1]
t=8968 [st= 998]      HTTP_CACHE_GET_BACKEND  [dt=0]
t=8968 [st= 998]      HTTP_CACHE_OPEN_ENTRY  [dt=0]
                      --> net_error = -2 (ERR_FAILED)
t=8968 [st= 998]      HTTP_CACHE_CREATE_ENTRY  [dt=0]
t=8968 [st= 998]      HTTP_CACHE_ADD_TO_ENTRY  [dt=0]
t=8968 [st= 998]      URL_REQUEST_DELEGATE  [dt=0]
t=8968 [st= 998]     +HTTP_STREAM_REQUEST  [dt=0]
t=8968 [st= 998]        HTTP_STREAM_REQUEST_STARTED_JOB
                        --> source_dependency = 1477793 (HTTP_STREAM_JOB)
t=8968 [st= 998]        HTTP_STREAM_REQUEST_BOUND_TO_JOB
                        --> source_dependency = 1477793 (HTTP_STREAM_JOB)
t=8968 [st= 998]     -HTTP_STREAM_REQUEST
t=8968 [st= 998]     +HTTP_TRANSACTION_SEND_REQUEST  [dt=0]
t=8968 [st= 998]        HTTP_TRANSACTION_HTTP2_SEND_REQUEST_HEADERS
                        --> :authority: www.facebook.com
                            :method: GET
                            :path: /rsrc.php/v2/yo/r/XtyuVDKK3xt.js
                            :scheme: https
                            accept: */*
                            accept-encoding: gzip, deflate, sdch
                            accept-language: en-US,en;q=0.8
                            cookie: [945 bytes were stripped]
                            origin: https://www.facebook.com
                            referer: https://www.facebook.com/
                            user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.75 Safari/537.36
t=8968 [st= 998]     -HTTP_TRANSACTION_SEND_REQUEST

Randy Smith

unread,
Mar 10, 2016, 1:15:43 PM3/10/16
to Ben Maurer, Ryan Sleevi, Matt Menke, net-dev
Is it possible to send the entire net-internals? Or at least the
URL_REQUEST->HTTP_STREAM_JOB for two requests that end up being bound
to two different HTTP2_SESSIONs?

-- Randy

Ben Maurer

unread,
Mar 10, 2016, 1:18:03 PM3/10/16
to Randy Smith, Ryan Sleevi, Matt Menke, net-dev
Locally I've found that the session is reused. However I have a bunch of resource timing traces collected from the field that show connection establishment for users who are in the experimental condition. I'm not sure what I can tweek about my local tests that might trigger the conditions i'm seeing from live data.

Ryan Sleevi

unread,
Mar 10, 2016, 1:27:05 PM3/10/16
to Ben Maurer, Randy Smith, Ryan Sleevi, Matt Menke, net-dev
Just brainstorming, is there any possibility the users are behind proxies (e.g. transparent proxies such as anti-virus solutions or corporate proxies)? Perhaps they have issues with Connection keep-alives?

To the extent possible, do you have a way to make sure these users ended up in the HTTP/2 bucket and not the HTTP/1 bucket? For example, is your experiment only enabled for H2 connections, or is it enabled if a user came over on H2 (meaning that if they, say, were a corporate user who goes back on the corporate network, would end up mediating through the proxy again and end up as H1)?

Ben Maurer

unread,
Mar 10, 2016, 1:40:39 PM3/10/16
to Ryan Sleevi, Randy Smith, Matt Menke, net-dev
On Thu, Mar 10, 2016 at 10:26 AM, Ryan Sleevi <rsl...@chromium.org> wrote:
Just brainstorming, is there any possibility the users are behind proxies (e.g. transparent proxies such as anti-virus solutions or corporate proxies)? Perhaps they have issues with Connection keep-alives?

To the extent possible, do you have a way to make sure these users ended up in the HTTP/2 bucket and not the HTTP/1 bucket? For example, is your experiment only enabled for H2 connections, or is it enabled if a user came over on H2 (meaning that if they, say, were a corporate user who goes back on the corporate network, would end up mediating through the proxy again and end up as H1)?

Our experiment is enabled if the connection to www.facebook.com for the current page the viewer is looking at came in over an HTTP2 connections. I have no idea if any major proxies have implemented http2 -- seems a bit unlikely though.

Matt Menke

unread,
Mar 10, 2016, 2:22:35 PM3/10/16
to Ben Maurer, Ryan Sleevi, Randy Smith, net-dev
Are we sending all the right cookies with all requests?  Be great if we could figure out if the extra sockets were or were not privacy mode sockets.
Reply all
Reply to author
Forward
0 new messages