Talos Improvement! SVG NoChrome decrease 0.993% on Android 2.2 mobile

Nobody

unread,

Aug 19, 2011, 4:27:49 PM8/19/11

to dev-tree-...@lists.mozilla.org

Improvement! SVG NoChrome decrease 0.993% on Android 2.2 mobile
---------------------------------------------------------------
Previous: avg 3451.576 stddev 4.442 of 30 runs up to revision 6181ba4693f9
New : avg 3417.292 stddev 4.577 of 5 runs since revision 1881f9b5f8b5
Change : -34.284 (0.993% / z=7.718)
Graph : http://mzl.la/nBR3gW

Changeset range: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=6181ba4693f9&tochange=1881f9b5f8b5

Changesets:
* http://hg.mozilla.org/mozilla-central/rev/805cd41a9b9f
: Markus Stange <mst...@themasta.com> - Bug 679717 - Add "mac-lion-theme" system metric in order to allow for Lion-specific CSS rules without the need for per-file manifest overrides. r=dbaron
: http://bugzilla.mozilla.org/show_bug.cgi?id=679717

* http://hg.mozilla.org/mozilla-central/rev/63b8853b1d67
: Jonas Sicking <jo...@sicking.cc> - Bug 667612: Make addEventListener not throw if passed a null listener. r=smaug
: http://bugzilla.mozilla.org/show_bug.cgi?id=667612

* http://hg.mozilla.org/mozilla-central/rev/5046e7177feb
: Matheus Kerschbaum <mat...@gmail.com> - Bug 678792 - Remove --disable-necko-disk-cache. r=khuey,jduell
: http://bugzilla.mozilla.org/show_bug.cgi?id=678792

* http://hg.mozilla.org/mozilla-central/rev/8fbc35ff4d7c
: Matheus Kerschbaum <mat...@gmail.com> - Bug 678794 - Remove NECKO_OFFLINE_CACHE checks. r=jduell
: http://bugzilla.mozilla.org/show_bug.cgi?id=678794

* http://hg.mozilla.org/mozilla-central/rev/acf10d96fa73
: flod <francesc...@mozillaitalia.org> - Bug 649266 - Certificate window is broken with long text labels. r=kaie
: http://bugzilla.mozilla.org/show_bug.cgi?id=649266

* http://hg.mozilla.org/mozilla-central/rev/9e6e62e4fbc1
: Matheus Kerschbaum <mat...@gmail.com> - Bug 679617 - Don't define TOPIC_FRECENCY_UPDATED twice. r=mak
: http://bugzilla.mozilla.org/show_bug.cgi?id=679617

* http://hg.mozilla.org/mozilla-central/rev/6f55c978504a
: Matheus Kerschbaum <mat...@gmail.com> - Bug 679616 - Stop whitelisting dead mozilla.widget.render-mode pref in about:support. r=bas
: http://bugzilla.mozilla.org/show_bug.cgi?id=679616

* http://hg.mozilla.org/mozilla-central/rev/a98208ca4e14
: Ed Morley <b...@edmorley.co.uk> - Bug 659234 - Remove unused variable |rv| from test_file_perms.cpp; r=sdwilsh
: http://bugzilla.mozilla.org/show_bug.cgi?id=659234

* http://hg.mozilla.org/mozilla-central/rev/0f8e88bb5bc8
: Matheus Kerschbaum <mat...@gmail.com> - Bug 568971 - Nuke nsIGlobalHistory3. r=mak sr=bz
: http://bugzilla.mozilla.org/show_bug.cgi?id=568971

* http://hg.mozilla.org/mozilla-central/rev/a67e4bd575b4
: Marco Castelluccio <mar.cast...@studenti.unina.it> - Bug 674014 - Remove "check whether mmap() sees write()s". r=glandium
: http://bugzilla.mozilla.org/show_bug.cgi?id=674014

* http://hg.mozilla.org/mozilla-central/rev/690b443ea689
: Oleg Romashin <rom...@gmail.com> - Bug 679236 Add Maemo6 geolocation provider (Qt Mobility based). r=dougt
: http://bugzilla.mozilla.org/show_bug.cgi?id=679236

* http://hg.mozilla.org/mozilla-central/rev/58b7de3f9b79
: Ed Morley <b...@edmorley.co.uk> - Bug 670461 - Fix conversion from 'double' to 'PRUint32' build warning in gfx/src/nsRenderingContext.h(73); r=joedrew
: http://bugzilla.mozilla.org/show_bug.cgi?id=670461

* http://hg.mozilla.org/mozilla-central/rev/6309d93cd072
: Neil Deakin <ne...@mozilla.com> - Bug 673301, crash using non-string data with text/plain drags, r=josh
: http://bugzilla.mozilla.org/show_bug.cgi?id=673301

* http://hg.mozilla.org/mozilla-central/rev/e8cb0687737a
: Matheus Kerschbaum <mat...@gmail.com> - Bug 678155 - Remove NS_COM. r=bsmedberg
: http://bugzilla.mozilla.org/show_bug.cgi?id=678155

* http://hg.mozilla.org/mozilla-central/rev/46dc0fb0b1cb
: Ehsan Akhgari <eh...@mozilla.com> - Bug 674770 - contenteditable breaks middle-click to open links when middlemouse.paste=true; r=roc
: http://bugzilla.mozilla.org/show_bug.cgi?id=674770

* and 33 more

Nobody

unread,

Aug 19, 2011, 5:46:27 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Codesighs increase 0.0768% on MacOSX 10.6.2 Firefox
-----------------------------------------------------------------
Previous: avg 43024456.667 stddev 13838.419 of 30 runs up to revision 6181ba4693f9
New : avg 43057500.000 stddev 0.000 of 5 runs since revision 1881f9b5f8b5
Change : +33043.333 (0.0768% / z=2.388)
Graph : http://mzl.la/q8bepn

Nobody

unread,

Aug 19, 2011, 6:27:27 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( SunSpider increase 3.37% on Win7 Firefox
------------------------------------------------------
Previous: avg 11.941 stddev 0.101 of 30 runs up to revision 6181ba4693f9
New : avg 12.344 stddev 0.073 of 5 runs since revision 1881f9b5f8b5
Change : +0.403 (3.37% / z=3.991)
Graph : http://mzl.la/mYcN4a

Nobody

unread,

Aug 19, 2011, 6:48:01 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Dromaeo (CSS) decrease 3.98% on Win7 Firefox
----------------------------------------------------------
Previous: avg 3161.255 stddev 30.054 of 30 runs up to revision 6181ba4693f9
New : avg 3035.376 stddev 17.219 of 5 runs since revision 1881f9b5f8b5
Change : -125.879 (3.98% / z=4.188)
Graph : http://mzl.la/o7szor

Nobody

unread,

Aug 19, 2011, 6:48:13 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Dromaeo (String/Array/Eval/Regex) decrease 3.05% on Win7 Firefox
------------------------------------------------------------------------------
Previous: avg 723.351 stddev 4.979 of 30 runs up to revision 6181ba4693f9
New : avg 701.278 stddev 4.769 of 5 runs since revision 1881f9b5f8b5
Change : -22.073 (3.05% / z=4.433)
Graph : http://mzl.la/nevpXL

Nobody

unread,

Aug 19, 2011, 6:48:09 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Dromaeo (SunSpider) decrease 6.92% on Win7 Firefox
----------------------------------------------------------------
Previous: avg 797.710 stddev 9.760 of 30 runs up to revision 6181ba4693f9
New : avg 742.489 stddev 1.187 of 5 runs since revision 1881f9b5f8b5
Change : -55.221 (6.92% / z=5.658)
Graph : http://mzl.la/nrAE3o

Nobody

unread,

Aug 19, 2011, 6:48:17 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Dromaeo (DOM) decrease 3.33% on Win7 Firefox
----------------------------------------------------------
Previous: avg 237.989 stddev 1.968 of 30 runs up to revision 6181ba4693f9
New : avg 230.055 stddev 1.621 of 5 runs since revision 1881f9b5f8b5
Change : -7.934 (3.33% / z=4.032)
Graph : http://mzl.la/o0zylc

Nobody

unread,

Aug 19, 2011, 6:48:54 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Dromaeo (CSS) decrease 4.06% on XP Firefox
--------------------------------------------------------
Previous: avg 3207.017 stddev 31.869 of 30 runs up to revision 6181ba4693f9
New : avg 3076.906 stddev 22.806 of 5 runs since revision 1881f9b5f8b5
Change : -130.111 (4.06% / z=4.083)
Graph : http://mzl.la/qUPHsS

Nobody

unread,

Aug 19, 2011, 6:48:57 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( Dromaeo (SunSpider) decrease 6.91% on XP Firefox
--------------------------------------------------------------
Previous: avg 801.156 stddev 4.263 of 30 runs up to revision 6181ba4693f9
New : avg 745.756 stddev 5.243 of 5 runs since revision 1881f9b5f8b5
Change : -55.400 (6.91% / z=12.996)
Graph : http://mzl.la/pCewEA

Nobody

unread,

Aug 19, 2011, 6:49:35 PM8/19/11

to dev-tree-...@lists.mozilla.org

Regression :( DHTML NoChrome increase 4.92% on Android 2.2 mobile
-----------------------------------------------------------------
Previous: avg 721.838 stddev 6.495 of 30 runs up to revision 6181ba4693f9
New : avg 757.341 stddev 3.254 of 5 runs since revision d963476cbf67
Change : +35.503 (4.92% / z=5.466)
Graph : http://mzl.la/ppKjE2

Changeset range: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=6181ba4693f9&tochange=d963476cbf67

* and 36 more

* and 25 more

Matt Brubeck

unread,

Aug 19, 2011, 7:08:41 PM8/19/11

to

On 08/19/2011 03:27 PM, Nobody wrote:
> Regression :( SunSpider increase 3.37% on Win7 Firefox
> ------------------------------------------------------
> Previous: avg 11.941 stddev 0.101 of 30 runs up to revision 6181ba4693f9
> New : avg 12.344 stddev 0.073 of 5 runs since revision 1881f9b5f8b5
> Change : +0.403 (3.37% / z=3.991)
> Graph : http://mzl.la/mYcN4a
>
> Changeset range: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=6181ba4693f9&tochange=1881f9b5f8b5

The changeset range I was able to get for mozilla-inbound isn't much
more useful. Windows builds were failing on inbound for much of the range:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?tochange=54a15fc04437&fromchange=1dccbc1234f4

Matt Brubeck

unread,

Aug 19, 2011, 7:11:41 PM8/19/11

to

On 08/19/2011 03:49 PM, Nobody wrote:
> Regression :( DHTML NoChrome increase 4.92% on Android 2.2 mobile
> -----------------------------------------------------------------
> Previous: avg 721.838 stddev 6.495 of 30 runs up to revision 6181ba4693f9
> New : avg 757.341 stddev 3.254 of 5 runs since revision d963476cbf67
> Change : +35.503 (4.92% / z=5.466)
> Graph : http://mzl.la/ppKjE2
>
> Changeset range: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=6181ba4693f9&tochange=d963476cbf67

From the mozilla-inbound graph, it's clear this was caused by
46b3d16fc427 (Bug 610834), which was expected to cause Talos
regressions. See the comments for details:
https://bugzilla.mozilla.org/show_bug.cgi?id=610834#c13

Matt Brubeck

unread,

Aug 19, 2011, 7:17:32 PM8/19/11

to Luke Wagner

The only JS changes I spotted in the regression range are from this push
by Luke:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=73aafae10571

Luke Wagner

unread,

Aug 19, 2011, 10:18:05 PM8/19/11

to Matt Brubeck, dev-tree-...@lists.mozilla.org

I ran sunspider bef/aft the whole merge (which includes my csets but
also 3 others in js/src) both in the shell and in the browser (built
from m-c) on Win7. Both showed low variance and produced identical
results before/after. I'm not sure what talos is measuring if the
result is "12"; SunSpider is measured in ms, usually 160-200 of them.
Thus, I think this can be ignored.

Matt Brubeck

unread,

Aug 19, 2011, 10:54:36 PM8/19/11

to Luke Wagner

On 08/19/2011 07:18 PM, Luke Wagner wrote:
>>> The changeset range I was able to get for mozilla-inbound isn't much
>>> more useful. Windows builds were failing on inbound for much of the
>>> range:
>>>
>>> https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?tochange=54a15fc04437&fromchange=1dccbc1234f4
>>
>> The only JS changes I spotted in the regression range are from this
>> push by Luke:
>>
>> https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=73aafae10571
>>
> I ran sunspider bef/aft the whole merge (which includes my csets but
> also 3 others in js/src) both in the shell and in the browser (built
> from m-c) on Win7. Both showed low variance and produced identical
> results before/after. I'm not sure what talos is measuring if the result
> is "12"; SunSpider is measured in ms, usually 160-200 of them. Thus, I
> think this can be ignored.

A number of other benchmarks also showed regressions in the same range,
all both Windows 7 and Windows XP (and no other platforms, at least as
far as the regression finder has reported). These regressions may or
may not all be caused by the same changeset:

Ts, Paint
Dromaeo (CSS)
Dromaeo (Sunspider)
Dromaeo (jslib)
Dromaeo (String/Array/Eval/Regex)
Dromaeo (DOM)

(Also, if Talos "SunSpider" is not measuring the right thing, we should
figure out why.)

Mike Shaver

unread,

Aug 19, 2011, 11:06:22 PM8/19/11

to lu...@mozilla.com, dev-tree-...@lists.mozilla.org, Matt Brubeck

On Fri, Aug 19, 2011 at 10:18 PM, Luke Wagner <lu...@mozilla.com> wrote:
> I'm not sure what talos is measuring if the result is "12"; SunSpider is
> measured in ms, usually 160-200 of them. Thus, I think this can be ignored.

Talos reports the mean of the SunSpider tests, IIRC. You have to
multiply it by...22? to get the number you're used to.

Mike

Luke Wagner

unread,

Aug 22, 2011, 12:18:19 PM8/22/11

to Matt Brubeck, dev-tree-...@lists.mozilla.org, Ehsan Akhgari

Backing out all four JS csets on try didn't fix the regression.
Instead, try points to a preceding cset:

http://hg.mozilla.org/mozilla-central/rev/a8199bd4af83
Ehsan Akhgari: Bug 680277 - Debug builds should always be compiled with
frame pointers; r=khuey

For reference, here are try results for the two preceding csets:
http://tbpl.allizom.org/?tree=Try&usebuildbot=1&rev=31c28753b800
http://tbpl.allizom.org/?tree=Try&usebuildbot=1&rev=3ec5f3c78f79

Here is with a8199bd4af83:
http://tbpl.allizom.org/?tree=Try&usebuildbot=1&rev=f42fb96afc49

Luke Wagner

unread,

Aug 20, 2011, 6:35:37 PM8/20/11

to Mike Shaver, dev-tree-...@lists.mozilla.org

Ahhh, thanks for clearing that up; that's great to hear! I should have read the logs. There is a persistent rumor (or perhaps past truth) that Talos measured SunSpider via page load time which would have been ridiculous.

On Aug 19, 2011, at 8:06 PM, Mike Shaver wrote:

> On Fri, Aug 19, 2011 at 10:18 PM, Luke Wagner <lu...@mozilla.com> wrote:

>> I'm not sure what talos is measuring if the result is "12"; SunSpider is
>> measured in ms, usually 160-200 of them. Thus, I think this can be ignored.
>