Detecting HTTP pipelining in Network tests

[Guypo]

Hey,

While HTTP pipelining has been dormant in desktop, it's very much
alive in mobile browsers. Android and Opera support it (accounting for
40% of total mobile browsing), and there are indications more browsers
will be adding support soon.

WDYT about adding HTTP pipelining detection in Browserscope?
It should be fairly easy to detect - simply create an iframe going to
a test page with ~40 resource on the same host, and track on the
server how they were requested. That said, it may get tricky if the
web server hides the fact the requests were piped. Ideally you can
capture the network traffic going to that client at that time, and
reach conclusions from that (that's how I was doing it manually).

I would suggest capturing these variables:
1) HTTP pipelining support in general.
2) Max piped requests per connection
3) Server Support Detection (Per Server, Per Connection)
4) Request Distribution Algorithm

The 1st is obvious.
The 2nd and 3rd are explained in this blog post:
http://www.blaze.io/mobile/http-pipelining-big-in-mobile/
The 4th is explained in this blog post:
http://www.blaze.io/technical/http-pipelining-request-distribution-algorithms/

Just throwing the idea out there... I'm afraid I can't volunteer dev
time to go along with it right now.

WDYT?

Cheers,
Guypo

[Steve Souders]

Hi, Guypo.

This would be nice to do. It won't be much of a differentiator now
since so few browsers support it, but it could help push future
browser development in this area.

One complication is the Network category runs on Google App Engine
which dramatically limits what you can do on the server wrt inspecting
and setting HTTP headers. Can you expand on the logic to "track on the
server how they were requested" in order to measure the variables you
suggested?

Thanks.

-Steve

On Aug 13, 2:29 pm, Guypo <guy...@gmail.com> wrote:
> Hey,
>
> While HTTP pipelining has been dormant in desktop, it's very much
> alive in mobile browsers. Android and Opera support it (accounting for
> 40% of total mobile browsing), and there are indications more browsers
> will be adding support soon.
>
> WDYT about adding HTTP pipelining detection in Browserscope?
> It should be fairly easy to detect - simply create an iframe going to
> a test page with ~40 resource on the same host, and track on the
> server how they were requested. That said, it may get tricky if the
> web server hides the fact the requests were piped. Ideally you can
> capture the network traffic going to that client at that time, and
> reach conclusions from that (that's how I was doing it manually).
>
> I would suggest capturing these variables:
> 1) HTTP pipelining support in general.
> 2) Max piped requests per connection
> 3) Server Support Detection (Per Server, Per Connection)
> 4) Request Distribution Algorithm
>
> The 1st is obvious.
> The 2nd and 3rd are explained in this blog post:http://www.blaze.io/mobile/http-pipelining-big-in-mobile/

> The 4th is explained in this blog post:http://www.blaze.io/technical/http-pipelining-request-distribution-al...

[Brian Pane]

Here's an idea for detecting pipelining inside an app that doesn't
have low-level access to the TCP streams:

1. Attempt to measure the round trip time between the client and the
server. To do this, get the browser to request a URI X, send back a
302 with "Connection: Keep-Alive" and "Location: Y," where Y is a new
URI that has the server's current clock value encoded into it. When
the browser requests Y, measure the time delta. Repeat several times
and keep the smallest value found.

2. Ask the browser to fetch a lot of very small resources. As each
resource is requested, make a note of the connection (based on the
client's IP address and TCP port number) and the current time.

3. If the resulting data set contains successive requests on the same
connection that are separated in time by less than the round trip time
measured in step 1, you've found request pipelining.

-Brian

> --
> You received this message because you are subscribed to the Google Groups "Browserscope" group.
> To post to this group, send email to browse...@googlegroups.com.
> To unsubscribe from this group, send email to browserscope...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/browserscope?hl=en.
>
>

[Guypo]

Steve,

Pipelining is already reasonably supported in Mobile, and there are
various indications it's growing there, so IMO this won't be too
premature :)
In fact, since we haven't tested for it, we may know that even more
browsers support it (amongst the less mainstream browsers)

Brian,

Very creative thinking, and I can see it working.
It relies on the roundtrip time being much bigger than the server
processing time, but that's probably a fair assumption.
It also relies on the servlet knowing at least the source IP, and
ideally the connection ID - do you know if those details are available
to a Google App Engine Servlet?

I think Browserscope actually already measures latency at an earlier
network test, so that might be good enough instead of step #2.

Seems like this technique also helps you determine the other
pipelining aspects too, since not only can you detect pipelining, but
also how many requests were piped on each connection. Thus you can
determine max piped requests per connection, the server-support
detection model (per connection or per server), and if we number the
resources also the fill-first vs distribute-first methodology.

Very cool!

Cheers,
Guypo