Talos Regression: Txul increase 3.56% on XP 1.9.1

[cat...@mozilla.com]

Regression: Txul increase from 100.882 to 104.474 (3.56%) on Tue Apr 21 10:36:00 2009. XP 1.9.1 build 20090421073654
http://graphs.mozilla.org/#show=2420846,2419200,2417613&sel=1240142361,1240487961
http://hg.mozilla.org/releases/mozilla-1.9.1/pushloghtml?fromchange=cfb24602548b&tochange=970bee033ed0

[Shawn Wilsher]

This is https://bugzilla.mozilla.org/show_bug.cgi?id=466379

> _______________________________________________
> dev-tree-management mailing list
> dev-tree-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tree-management

[Daniel Holbert]

On 04/21/2009 04:21 PM, Shawn Wilsher wrote:
> This is https://bugzilla.mozilla.org/show_bug.cgi?id=466379

However, roc replied on that bug, saying there's no chance its checkin
was guilty... (Its only changes that affect Windows were some
s/nsPoint/nsIntPoint/ replacements.)

https://bugzilla.mozilla.org/show_bug.cgi?id=466379#c29

But looking at the graphs, this definitely seems to be a significant
regression -- all three WinXP boxes simultaneously spiked, each one
hitting a higher value than its maximum over the past 7 days, and each
one staying that high for 2-4 cycles (bringing us up to the present).

Is it possible that the talos-regression-catching-bot had too small of a
regression range? (Maybe the true culprit landed slightly earlier or
later?)

~Daniel

[Johnathan Nightingale]

This is why, much as it pains me, the tinderbox waterfall is still a
useful tool.

I checked the waterfall for qm-pxp-talos03's jump from 94.95 to 106.16.
It gives me this range:

http://hg.mozilla.org/releases/mozilla-1.9.1/pushloghtml?fromchange=c11afdb243c8&tochange=cfb24602548b

Which includes Peter V's bug 481677 and Dao's bug 488255.

Note that catlee's script offered a range that used cfb24602548b as
fromchange (meaning it wasn't included in the window), whereas on this
particular box, cfb24602548b is the first build *with* the regression,
and hence is included in the window.

So 2 questions:

- Are either of those bugs likely culprits? I notice that Peter V's
is credited by beltzner with an XP Tp win
- Catlee - is it possible your script is assuming ranges are inclusive
of fromchange, and needs to be adjusted?

Cheers,

J

[Chris AtLee]

On 22/04/09 09:00 AM, Johnathan Nightingale wrote:
> This is why, much as it pains me, the tinderbox waterfall is still a
> useful tool.
>
> I checked the waterfall for qm-pxp-talos03's jump from 94.95 to 106.16.
> It gives me this range:
>
> http://hg.mozilla.org/releases/mozilla-1.9.1/pushloghtml?fromchange=c11afdb243c8&tochange=cfb24602548b
>
>
> Which includes Peter V's bug 481677 and Dao's bug 488255.
>
> Note that catlee's script offered a range that used cfb24602548b as
> fromchange (meaning it wasn't included in the window), whereas on this
> particular box, cfb24602548b is the first build *with* the regression,
> and hence is included in the window.
>
> So 2 questions:
>
> - Are either of those bugs likely culprits? I notice that Peter V's is
> credited by beltzner with an XP Tp win
> - Catlee - is it possible your script is assuming ranges are inclusive
> of fromchange, and needs to be adjusted?

The ranges should be exclusive of fromchange, since that's supposed to
be the revision of the "last good build".

What's probably going on is that the script saw a build of cfb24602548b
with results of 100.88, which it considered to be within the noise
tolerance, and so that build was marked as the "last good build".

Currently the script doesn't try and do anything special with multiple
builds from the same changeset. It just sorts the data, looks for
points that are out of place, and then reports what the last good and
first failing build were.

[Justin Dolske]

On 4/22/09 6:00 AM, Johnathan Nightingale wrote:
> This is why, much as it pains me, the tinderbox waterfall is still a
> useful tool.
>
> I checked the waterfall for qm-pxp-talos03's jump from 94.95 to 106.16.
> It gives me this range:

Looking at the specific results from -talos01 and -talos03, it seems
pretty clear the jump is associated with Peter V's patch.

talos01 ran the changeset before his checkin at 97.84, and then his
changeset at 104.95. talos03 was the first to report a jump, running
his changeset at 106.16.

I'll try backing it out to see what happens.

Justin