Some new developments from the Make the Web Faster Group

[Lisa]

Making the web go faster

Question:

We are looking for a tool that will allow us to accurately measure page rendering time for multiple browser types.  Tools like HTTPWatch 
don't seem to capture JavaScript time.  A tool like DynaTrace AJAX looks promising but we need something that can be used across multiple 
browser types. Any suggestions? 

Answer One

Compuware Gomez has a commercial tool that will let you do this. http://www.gomez.com. If you are going the build-your-own route, 
and are open to RUM testing, the Boomerang project from Yahoo is quite interesting and would allow you to do what you are trying to do.
http://yahoo.github.com/boomerang/doc/ 

Answer Two

As you've seen from the responses, there isn't really a good solution yet.  If you are just interested in the load time then Boomerang works pretty well (and even better as browsers start to implement the Web Timing spec).  There are a bunch of solutions that are browser-specific (Firebug + Net Export, Pagetest/WebPagetest, Dynatrace Ajax Edition) and have different feature sets. 

We started up a Google group to standardize the interfaces and components (http://groups.google.com/group/web-testing-framework) but there isn't anything that exists today (at least that I'm aware of).  Hopefully that will start to change over the next few months

This is what I had expected to see:

Answer Three

In newer browsers are the start render can also be collected by javascript instrumentation.  http://dvcs.w3.org/hg/webperf/raw-file/tip/specs/NavigationTiming/Overview.html. This is available(in different namespaces) on chrome 6+, ie9. Unfortunately didn't make it to Firefox 4 before feature freeze.. 

Answer Four

Can I clarify what data you want (the title says rendering time?) and what you want to use the monitoring data for e.g. KPI reporting as that has a big impact on what you select...

• If you want repeatable, accurate MONITORING of your application performance so you know when (and are alerted) when your backend slows down or fails - I would use active, synthetic monitoring e.g. www.siteconfidence.com or keynote or gomez or alertsite or webmetrics etc ad nauseum. 
• If you want to constantly MEASURE the real user experience - I would use a proper RUM tool either tag-based like WebTuna www.webtuna.com or network based like Pion www.atomiclabs.com or Coradiant or Tealeaf etc 
• if you want to TEST the performance of your application (including client-side measurements like render start time) I would automate it, either externally like http://blog.siteconfidence.com/2010/11/performance-analyser-is-live-s... as part of your continuous integration process internally like this http://blog.dynatrace.com/2010/11/17/dynatrace-ajax-edition-2-0-is-re... 

Best of all:

Answer 5

Current customers of Gomez ActualXF include Pizza Hut, Apple, and Toys R Us, and many others.  Real User Monitoring is a category of performance products that is rapidly growing.  Just within the past year, in addition to the Gomez product, there have been a flurry of start-ups and open source targeting this area.  The W3C Web Timing APIs will also make browser-based RUM products more standardized when they are consistently implemented.