Comparison of page timings

[niharika varshney]

Hi,

I intend to use the proxy to record page timings and then use them for future performance comparisons.

Before venturing out to start automation, I did a brief comparison of the firebug addon NetExport (which we were using manually) captured HAR and the BMP captured HAR, to understand if there are some major differences.

Based on whatever comparisons I have done till now, I see that the blocking time and the connect time seem to be wide apart, based on their comparison in the har viewer (www.softwareishard.com/har/viewer/) 

NetExport shows time in Blocking mode, whereas, Browsermob proxy doesn't

And vice versa for the connect time. 

Here's a chart of 3 sites, whose homepage, I selected for comparison and ran the tests 3 times(on each site) in a matter of minutes from the same machines, clearing history between all the runs. 

Used the net export plugin to export data from NetExport and used browsermob (2.6) integrated Webdriver code to get the BrowserMob data.

1. Does such a comparison make sense?  

2. Has anyone done similar study?

3. Why is there a noticeable difference in the timings( I realize there may be many factors), is the proxy adding a bit to the delay?

Regards,

Niharika

[Patrick Lightbody]

This is a great writeup! Thank you for sharing.

The comparison makes total sense, and I think we should do *more* of this. The biggest reason you'll see a difference around Blocking time is due to how BMP "fakes" it. Technically, it's not possible for BMP to really know what the blocking time is, since Blocking is really an in-browser measurement that is supposed to tell you how much time the browser waited to begin an HTTP request but couldn't because it was blocked by the rules around # of concurrent HTTP requests.

BMP has no insight in to what is happening in the browser, so instead a way to work around it is to configure the browser to have a very large limit of concurrent requests to the proxy (way higher than a regular user), effectively causing no blocking to happen in the browser. Instead, you can then have the *proxy* enforce rules about # of connections per host and therefore attempt to record a close approximation of Blocking time.

So there may be a few things going on here:

1) BMP is currently set to allow SIX connections per host, but that might be worth revisiting. Can you check what Firefox 11 allows?

2) There is no API to modify #1, and there should be since each browser has it's own rules and so we should try to emulate it. It might even be good to try to automatically emulate it based on the User-Agent.

3) Your browser must be configured to allow a bunch of concurrent requests to the proxy. By default Firefox only allows 8 but can be configured to allow more by tweaking network.http.max-persistent-connections-per-proxy. Other browsers may operate differently.

I hope that helps - we definitely need to do more research here and there may very well be bugs in the code too! Please let me know how you continue investigating this.

Patrick

--

Patrick Lightbody

+1 (415) 830-5488

On Mar 25, 2012, at 11:47 PM, niharika varshney wrote:

Hi,

I intend to use the proxy to record page timings and then use them for future performance comparisons.

Before venturing out to start automation, I did a brief comparison of the firebug addon NetExport (which we were using manually) captured HAR and the BMP captured HAR, to understand if there are some major differences.

Based on whatever comparisons I have done till now, I see that the blocking time and the connect time seem to be wide apart, based on their comparison in the har viewer (www.softwareishard.com/har/viewer/) 

NetExport shows time in Blocking mode, whereas, Browsermob proxy doesn't

And vice versa for the connect time. 

Here's a chart of 3 sites, whose homepage, I selected for comparison and ran the tests 3 times(on each site) in a matter of minutes from the same machines, clearing history between all the runs. 

Used the net export plugin to export data from NetExport and used browsermob (2.6) integrated Webdriver code to get the BrowserMob data.

<image.png>

[niharika varshney]

Sorry for not mentioning this earlier.  But I am running my tests on FF 3.6,

network.http.max-persistent-connections-per-proxy : 8

network.http.max-persistent-connections-per-server :6

network.http.max-connections-per-server;15

network.http.max-connections;30

Checked for FF11 as well.  Only the max-connections have been changed to 256 from 30.

Thanks for the explanation on the Blocking time.  But, pardon me if I have not really understood this, but since the setting in the proxy is 6 , which is similar to what the browser has, shouldn't the blocking time be effectively simulated? 

Were you suggesting to increase the max-persistent-connections-per-proxy to , let's say 25 and then check the data for the blocking time, while running the automated case, since then there should not be any blocking time at the browser end and whatever the blocking is ..is at the proxy.

Can you also kindly explain whether we can pin down a reason for the difference in the connect time?  There can be bugs in NetExport as well, can there? :)

Regards,

Niharika

[Pavel Paulau]

In short, does it mean that connection model with BMP enabled dramatically differs from default proxy-less configuration?

And how to deal with browsers where it is not possible to tweak network settings?

[Patrick Lightbody]

Pavel,

We don't know what the timing differences are. I'm hoping that Niharika's testing can help us shed light on that.

If there aren't APIs to change the network settings in WebDriver for the browser of your choice, you'll have to find another option. For example, you could pre-configure a profile and make sure the browser launches with that profile. If you're having issues, try asking the selenium-users group.

Patrick

--

Patrick Lightbody

+1 (415) 830-5488

[Patrick Lightbody]

Niharika,

Apologies for taking so long to respond:

Thanks for the explanation on the Blocking time.  But, pardon me if I have not really understood this, but since the setting in the proxy is 6 , which is similar to what the browser has, shouldn't the blocking time be effectively simulated? 

Yes - in theory :)

Were you suggesting to increase the max-persistent-connections-per-proxy to , let's say 25 and then check the data for the blocking time, while running the automated case, since then there should not be any blocking time at the browser end and whatever the blocking is ..is at the proxy.

Yes, the idea would be that there would be no blocking time in the browser and all the blocking time would happen in the proxy, where we can measure it.

Can you also kindly explain whether we can pin down a reason for the difference in the connect time?  There can be bugs in NetExport as well, can there? :)

You're right, there could be. I recommend comparing measurements to http://webpagetest.org, which is considered the gold standard. I would be interested to hear about any further research you do.

Patrick

--

Patrick Lightbody

+1 (415) 830-5488

[saif sm]

 Hi Niharika,

Can you please help me in setting up BMP to capture HAR using selenium.

thanks

Saif

[Krishnan Mahadevan]

Have you taken a look at the bmp documentation? It has everything that is needed to get you started. 

Thanks & regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

My Scribblings @ http://wakened-cognition.blogspot.com/
My Technical Scribbings @ http://rationaleemotions.wordpress.com/

--

---
You received this message because you are subscribed to the Google Groups "BrowserMob Proxy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to browsermob-pro...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Xiaoming Hu]

Any difference of onLoad timing between BPM and firebug+netExport?