Har data acquisition tools.

[Tim Hawkins]

Im looking for a tool that i can mate up with harstorage to acquire har data. 

We have a network of websites (about 50), which we would like to get an hourly scan of certain key pages and store the results for historical performance tracking/SLA management.  so about 400 samples an hour

So I need something that will run a bunch of URL's on a browser, and drop the har data into harstorage.

Preferably something we can run on a linux server 

Any Ideas?

[Matt Welsh]

Why not httparchive.org?

On Sep 4, 2012 11:27 PM, "Tim Hawkins" <tim.th...@gmail.com> wrote:

[Tim Hawkins]

Thats ok for the public sites, but we have a pile of staging and dev sites that are inside our firewall, that we wont be able to expose. I really need something that can run on an internal server. 

Sent from my iPad

On Sep 5, 2012, at 9:41 PM, Matt Welsh <m...@google.com> wrote:

Why not httparchive.org?

On Sep 4, 2012 11:27 PM, "Tim Hawkins" <tim.th...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google
Groups "HTTP Archive Specification" group.
To post to this group, send email to
http-archive-...@googlegroups.com
To unsubscribe from this group, send email to
http-archive-specif...@googlegroups.com
For more options, visit this group at
https://groups.google.com/forum/#!forum/http-archive-specification

[Steven Roussey]

Use phantomjs. One of its examples will create a hat file if I remember correctly. 

--
Steven Roussey

[Tim Hawkins]

Noted, will take a look. 

Sent from my iPad

[Steve Souders]

HTTP Archive is open source. You can run a private instance of HA along with WebPagetest and gather HAR files (& video etc.) and trend them over time. There are several companies doing this already including Zillow. Caveat: there are not A LOT of people doing this so the documentation for private instances is limited. I can happily answer questions and would appreciate contributions back to the code base & documentation that make private instances easier.

-Steve

[Jan Honza Odvarko]

On Sep 5, 3:54 pm, Tim Hawkins <tim.thawk...@gmail.com> wrote:
> Thats ok for the public sites, but we have a pile of staging and dev sites that
> are inside our firewall, that we wont be able to expose. I really need something
> that can run on an internal server.

You could use Firebug + NetExport + Selenium

Take a look at
http://www.softwareishard.com/blog/firebug/automate-page-load-performance-testing-with-firebug-and-selenium/

The solution is able to send collected HAR data to a web server.

Honza

[Tim Hawkins]

Thanks folks, phantomjs seems to work great, i have created a small scheduler in php to poll an hourly list of urls, and retrieve the har data, seems to work fine.

Only minor issue is it does not seem to provide the domloaded/pageloaded data, but I'm tracking that down. Im using the examples/netsniff.js example as a starting point.

Will push to github and post url once i have it fully sorted out.

Sent from my iPad