Perf Comparison, Opera/IE/Netscape

[David Hyatt]

I ran the page load tests on my 1.5ghz machine and got the following
results. This should give you an idea of where the current trunk stands
relative to other browsers on a fast Win32 box. K-Meleon 0.6 is used as
a reference to indicate how much the trunk has improved just in the last
few weeks! :)

K-Meleon 0.6 - 519ms
Opera 5 - 469ms
Mozilla 11/08 - 463ms
IE 6 - 321ms

If you're interested in the Mozilla/IE ratio, it comes out to about
1.44. To highlight just how far we've come, Netscape 6.0 is more than 4
times slower than IE6.

dave
(hy...@netscape.com)

[Stuart Ballard]

David Hyatt wrote:
>
> K-Meleon 0.6 - 519ms
> Opera 5 - 469ms
> Mozilla 11/08 - 463ms
> IE 6 - 321ms
>
> If you're interested in the Mozilla/IE ratio, it comes out to about
> 1.44. To highlight just how far we've come, Netscape 6.0 is more than 4
> times slower than IE6.

Cool!

How about NS4? Have we caught that up yet?

Stuart.

[DeMoN LaG]

Stuart Ballard <sbal...@netreach.com> wrote in
news:3BEC45F9...@netreach.com, on 09 Nov 2001:

Heh, forget NS4. These times show Mozilla faster than Opera. I mean,
Opera is touted as a blazing fast web browser, if we are just as fast as
they are I think that everyone on the Mozilla team deserves a great big
hug :)

--
ICQ: N/A (temporarily)
AIM: FlyersR1 9
email: de_on-lag@ho_e.co_
_ = m

[David Hyatt]

I didn't think that was even worth posting. :)

Netscape 4.78 - 1557ms

Dave
(hy...@netscape.com)

[David Hyatt]

Sigh. I'm a tool. I just noticed that my NS6 had its memory cache
boosted. I'll have to rerun with the default settings. This is why IMO
the default memory cache setting should be a percentage of the machine's
physical RAM.

With the default settings, the # will probably be more like 563ms, not
463ms.

Dave
(hy...@netscape.com)

[David Hyatt]

Ok, good news. Default settings on this same machine yielded a result
of 534, so the degradation was not as bad as I feared.

dave
(hy...@netscape.com)

[Gollum]

DeMoN LaG wrote:
> [...] These times show Mozilla faster than Opera. I mean,

> Opera is touted as a blazing fast web browser, if we are just as fast as
> they are I think that everyone on the Mozilla team deserves a great big
> hug :)

I'd like to see such a comparison for an old (<200MHz) Pentium with
less than 64 MB RAM. I wonder how Mozilla compares to Opera then...

[Ian Hickson]

David Hyatt wrote:

> Sigh. I'm a tool. I just noticed that my NS6 had its memory cache
> boosted.

From what to what?

--
Ian Hickson

[Yatsu]

Gollum wrote:

Agreed. NS4 and Opera and undeniably much, much faster than Mozilla
pageloading-wise on low end machines. Even on those with more than enough
RAM to run Mozilla, Mozilla is dog slow.

Mozilla seems to always lag a second or two before actually doing anything,
where as netscape just loads the darn page (in this case www.google.com.
i'd say netscape is atleast twice, if not three times as fast as mozilla
loading this page.).

While i never expect Mozilla to be as responsive as other programs on low
end machines, i _do_ expect it's pageload times to be comparable with other
browsers. Mozilla has come a long way, yes. But there's still alot to be
done :)

[markus hübner]

>Mozilla seems to always lag a second or two before actually doing
>anything,

Ya, noticed this too. Any clue for this so far?
Where are the page load test to run on my machine as well?

:: markus

[David Hyatt]

This is an issue with timers, namely that they don't fire when they're
supposed to. The slower the machine, the more off the timer is, and so
the page ends up staying invisible for far too long.

dave
(hy...@netscape.com)

[Yatsu]

David Hyatt wrote:

> This is an issue with timers, namely that they don't fire when they're
> supposed to. The slower the machine, the more off the timer is, and so
> the page ends up staying invisible for far too long.
>
> dave
> (hy...@netscape.com)

is there a bug # for this?

[Puffin Moses]

in article 3BEC7335...@netscape.com, David Hyatt at hy...@netscape.com
wrote on 01-11-10 01.22:
<snip>

> Ok, good news. Default settings on this same machine yielded a result
> of 534, so the degradation was not as bad as I feared.

<remodel>
>>>>> Mozilla 11/08 - 534ms

>>>>> K-Meleon 0.6 - 519ms
>>>>> Opera 5 - 469ms

>>>>> IE 6 - 321ms

So in other words, mozilla turned out to be the slowest :(
But its a very little difference, and with the steady progress of mozilla it
will soon be the undisputed king i guess!

Now that the page rendering seems to work at good speed, it would be nice to
see some focus on drop-down menus, UI and overall responsiveness that is the
one thing stopping me from using mozilla, i get a feeling like its going to
crash or something, it feels sluggish.

But since you guys obviously can make great efforts (see page load) it would
surely be fixed if you turned your attention to it!

[Daniela Lindenthaler]

Any URL to get the software to run the tests on my computer as well?

David Hyatt schrieb:

[John Morrison]

Daniela Lindenthaler wrote:
>
> Any URL to get the software to run the tests on my computer as well?

This is where it is checked into the mozilla tree, with info on how to
set it up.

http://lxr.mozilla.org/mozilla/source/tools/page-loader/README.txt

John

[dman84]

so maybe networking code is better in Netscape 4.x in some certain
situations? because Page load /rendering is way faster in ns6.2 than
netscape 4.x especially when it comes to CSS and more standard stuff
that 4.x doesn't have. It really depends on page content more than
anything. IE6 maybe faster, but has crap for compliance to standards,
so most likely IE is using the system cache and all that too, since its
integrated so far in the OS.

-dman84

[Yatsu]

dman84 wrote:

> so maybe networking code is better in Netscape 4.x in some certain
> situations?

no no, this is _every_ situation.

> because Page load /rendering is way faster in ns6.2 than
> netscape 4.x especially when it comes to CSS and more standard stuff
> that 4.x doesn't have.

it doesn't matter if the actual pageloading is a million times that of
NS4.x. If nothing is displayed for 1-2 seconds _every_ _single_ _page_ you
go to, who cares about pageloading times?

[markus hübner]

David Hyatt wrote:

David could you please compare the result with the latest Opera 6 Beta 1 ?

[David Hyatt]

Opera 6 Beta 1 in SDI (*cough* tabbed browsing mode *cough*) scored a
484. I went into the prefs and noticed that Opera has a painfully small
disk cache (only 2 megs!). I'm going to try bumping it up to the same
size as Mozilla's default (50 megs) and see what kind of # I get.
Still, since most people don't change these sorts of settings, the
defaults are key.

dave
(hy...@netscape.com)

[David Hyatt]

With the disk cache set to 50 megs, I got a score of 481.

dave

[Sam Emrick]

The 'problem' (maybe 'attribute' is the better word) of such a 'single
number', based on single benchmark, is... well.... just that... its one
number, when in fact the relative performance of mozilla to ns4 is quite
varied depending on the testcase, depending on document content and what
functions are most heavily exercised by the testcase. What we use here for
OS/2 Mozilla (internally to us known as Quint) is a set of 13 homegrown
benchmark tests, each test designed to focus on particular sort of content.
So, from table below, what we see is, depending on the testcase, a range
from over 3 times slower than ns4 ... up to... over 9 times faster than ns4
for embedded table (ns4 was just awful for embedded table).

Completion time, Quint (.9.4.1) / Netscape 4.61
299.0% Large text HTML
155.0% Backgd Image
172.9% Forms
321.3% HotLinks
184.7% Large Table
101.1% Images
134.0% JavaScript
94.3% Large jpg
82.7% 30 Dukes
105.6% Mix
285.7% Nested UL
185.6% 300 Text Flds
11.4% Embed Tables
130.6% Geo Mean

My personal overall summary would be as such: Mozilla is approaching
acceptable performance for pages which are small and image intensive. Such
content is more or less typical 'internet web page'. However, Mozilla
continues to need significant performance improvement in handling large
documents, large lists, pages with large number of styles / style changes.
This kind of content is more typical of html-based document publishing,
on-line product documentation, 'not-news' newsgroup/discussion boards, and
the like.

The performance effort thus far has often focused on 'responsiveness' (fast
initial display of partial content). That was and is 'goodness. More effort
now needs to be made, I believe, in completion time in handling really big
docs. One problem being, we both know, the goals of 'fast initial response'
and 'fast overall completion time' are.... in a variety of
ways/places/code... conflicting goals. I'd propose, as a general approach,
some sort of 'large doc load detection' and some places implementing
bimodal code behavior based on detecting a large document load..

Any thoughts about this?
Sam

[Chris Hofmann]

can we get these tests and integrate them into regular reporting cycles
on "testerbox"?

testerbox is going to be our new way of reporting on a large number of
preformance and functionally tests on continuous build cycles...

We have a dream to one day make testerbox report on hundreds of
test case and characteristics...

chris h.

[Sam Emrick]

>can we get these tests and integrate them into regular reporting cycles
>on "testerbox"?

I think no problem that (as is, zippo documentation, unsupported,
yadayada).
Send you a zip? or what's process?
Sam

[markus hübner]

David Hyatt wrote:

Thank you very much David!
What mid-term speed improvements do you think Mozilla can achieve?

:: markus

[Henrik Gemal]

David Hyatt wrote:

Dave: could you please post the numbers once more. In the thread there's
some new numbers and some revised one. Please post a new posting with
the complete numbers...

--
Henrik Gemal
Mozilla Evangelist

Get the latest and greatest Mozilla web browser at:
http://gemal.dk/mozilla/