IE9 or Chrome?

[Steve Souders]

Right now the HTTP Archive crawls are done using IE9. There are some features in WebPagetest that only work on Chrome. I'd like to get a list of the pros & cons of switching from IE9 to Chrome. The fuller list of alternatives is:

1. IE9
2. Chrome
3. some other version of IE (ie, IE10 or IE11)
4. wait until we can do both IE *and* Chrome! (probably 6-12 months)

Here are some tradeoffs I'm aware of:

IE:
    - has PageSpeed
    - IE is probably the most used browser *family*.

Chrome:
    - Changing makes trend comparisons apples-to-organges.
    - latest version is probably the most used browser *version*
    - If we allow Chrome to update then the "baseline" is constantly changing.
    - can do the RWD test that Guypo wrote about.

Please reply with other tradeoffs that should be considered.

Thanks.

-Steve

[Charlie Clark]

Am .12.2014, 20:19 Uhr, schrieb Steve Souders <steveso...@gmail.com>:

> - IE is probably the most used browser *family*.

I'd have to take issue with that based on the stats I have access to:
Chrome overtook IE in all versions this year (Akamai probably has the most
representative figures http://www.akamai.com/html/io/index1.html but I'm
seeing this for corporate sites as well). IE still has a lead on corporate
networks.

I'd be happy with a move to IE 11 from 2015-01-01 to provide some degree
of continuity while fully rendering sites: Flexbox, components, etc. are
all getting take up and some of them are very difficult to do proper
fallbacks for.

For consistency's sake there should be no move to Chrome until we have
some historical data that allows us to compare the same site with the
different browsers.

Charlie
--
Charlie Clark
Managing Director
Clark Consulting & Research
German Office
Kronenstr. 27a
Düsseldorf
D- 40217
Tel: +49-211-600-3657
Mobile: +49-178-782-6226

[Patrick Meenan]

PageSpeed is the biggest thing that is available in IE that you can't get from any of the other browsers but it's a fairly old version of the checks at this point and doesn't remotely match what the online Page Speed Insights tool checks.

With Chrome we could also get some stats from the timeline (amount of time spent running javascript for example).

Both IE 11 and Chrome would let us measure the adoption of SPDY/HTTP2, Newer image formats (webp/jpegxr).

Being able to trend historically is going to be a really big problem though and not just with the initial transition.  Chrome updating every 6 weeks will change the results in ways that may not represent changes in the sites (though it will reflect how users are consuming the sites).  Even IE 11 pushes new functionality with Windows Update these days though the changes are probably not going to be as large as what you will see with Chrome.

When we make the transition we're definitely going to want to run both in parallel for a period (or at least a subset on the new browser) so we can test both the processing and what the impact on the stats is going to be.  One thing we might be able to do is instead of always doing 3 runs we could configure it to do "up to X runs" until it gets a successful result which might buy us enough headroom to run both in parallel now.

My vote would be to transition to Chrome for desktop but to do it slowly (soon, before we ramp up to 1M URLs and cause even more headaches) and work through the "moving target" issues.  When we have more capacity we can consider adding IE back (or even IE and Firefox).

--
You received this message because you are subscribed to the Google Groups "HTTP Archive" group.
To unsubscribe from this group and stop receiving emails from it, send an email to httparchive+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steve Souders]

There's validity in the argument to do IE and Chrome for 6+ months before switching in order to maintain trending history. However, if we have enough capacity to run them in parallel for that long, there's no reason to stop. It's not like we're going to return the hardware or use the capacity for something else. We *could* but it's very unlikely. 

Let's move forward with the plan to run *both* IE and Chrome indefinitely. In order to do that we need to bulk up on hardware (and perhaps other shortcuts per Pat's suggestion). So the rollout of Chrome is gated on increasing capacity. 

A separate discussion is *which* version of IE to use. For now let's stick with IE9, and revisit that as appropriate.

-Steve

On Monday, December 15, 2014 12:55:26 PM UTC-8, Patrick Meenan wrote:

PageSpeed is the biggest thing that is available in IE that you can't get from any of the other browsers but it's a fairly old version of the checks at this point and doesn't remotely match what the online Page Speed Insights tool checks.

With Chrome we could also get some stats from the timeline (amount of time spent running javascript for example).

Both IE 11 and Chrome would let us measure the adoption of SPDY/HTTP2, Newer image formats (webp/jpegxr).

Being able to trend historically is going to be a really big problem though and not just with the initial transition.  Chrome updating every 6 weeks will change the results in ways that may not represent changes in the sites (though it will reflect how users are consuming the sites).  Even IE 11 pushes new functionality with Windows Update these days though the changes are probably not going to be as large as what you will see with Chrome.

When we make the transition we're definitely going to want to run both in parallel for a period (or at least a subset on the new browser) so we can test both the processing and what the impact on the stats is going to be.  One thing we might be able to do is instead of always doing 3 runs we could configure it to do "up to X runs" until it gets a successful result which might buy us enough headroom to run both in parallel now.

My vote would be to transition to Chrome for desktop but to do it slowly (soon, before we ramp up to 1M URLs and cause even more headaches) and work through the "moving target" issues.  When we have more capacity we can consider adding IE back (or even IE and Firefox).

On Mon, Dec 15, 2014 at 3:33 PM, Charlie Clark <charlie.clark@clark-consulting.eu> wrote:

[Ilya Grigorik]

On Thu, Dec 18, 2014 at 1:13 PM, Steve Souders <steveso...@gmail.com> wrote:

Let's move forward with the plan to run *both* IE and Chrome indefinitely. In order to do that we need to bulk up on hardware (and perhaps other shortcuts per Pat's suggestion). So the rollout of Chrome is gated on increasing capacity. 

What is the bottleneck here? Funding? Developer time?

Personally, I'm not concerned with the update cycle: that's a feature not a bug; that's how users experience the web; all UAs are converging on this model.. so at best we're postponing the inevitable. We just need to make sure that we log the UA version number in each run.

Having access to latest web features and additional timeline perf metrics would be a huge win. Big +1 to enabling Chrome. In fact, personally I'd prioritize support for multiple browsers over reaching deeper into the tail.

ig

[Steve Souders]

Developer time is the bottleneck.

-Steve

On Thursday, December 18, 2014 4:52:28 PM UTC-8, Ilya Grigorik wrote:

[Ilya Grigorik]

On Thu, Dec 18, 2014 at 5:58 PM, Steve Souders <steveso...@gmail.com> wrote:

Developer time is the bottleneck.

Figures. I was afraid you'd say that.. Funding is easier to fix :)