help me identify perf regressions - webkit-gardening

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion help me identify perf regressions
Received-SPF: pass (google.com: domain of sulli...@google.com designates 209.85.214.52 as permitted sender) client-ip=209.85.214.52;
MIME-Version: 1.0
In-Reply-To: <CAEoffTAr0tCBy=4ftPue3V_eJL+0c2aQYgN244=3LLW574H...@mail.gmail.com>
References: <CANMdWTt76D0Czxo4Gp_kur-AYUW+iiYDtk3yXjvkYMR2yj4...@mail.gmail.com>
	<CAEoffTAr0tCBy=4ftPue3V_eJL+0c2aQYgN244=3LLW574H...@mail.gmail.com>
Date: Tue, 30 Oct 2012 21:01:03 -0400
Message-ID: <CAJB_qsm+YYY=hquLpM139gh4GTaXOF6LUTOsjbgi8Tgpv5E...@mail.gmail.com>
Subject: Re: help me identify perf regressions
From: Annie Sullivan <sulli...@google.com>
To: Dirk Pranke <dpra...@chromium.org>
Cc: Ojan Vafai <o...@chromium.org>, webkit-gardening <webkit-garden...@chromium.org>
Content-Type: multipart/alternative; boundary=0015174bde04ca693d04cd506fef

--0015174bde04ca693d04cd506fef
Content-Type: text/plain; charset=ISO-8859-1

FYI, I think we can probably manage to reduce the false positives if we get
some feedback on why they're not helpful. Some things that could possibly
be problems:

* Monitoring too many graphs
* Alerts too sensitive
* Getting emails on improvements as well as regressions
* % change calculation incorrect
* Actual problems with GASP

Any Googler who wants to take a look at the alerts can see a full archive
at
https://groups.google.com/a/google.com/forum/?fromgroups#!forum/webkit-gasper

If anybody has time to make a list of good/bad alerts, it would be helpful
to have the "Debug this alert in Gasper" link, the rev the reported
regression happens at, and a quick explanation of why it's not helpful. You
don't need to look at all the alerts; a small sample would be useful. I
made a doc here for feedback: http://go/webkit-gasper-feedback

Thanks!
-Annie

On Tue, Oct 30, 2012 at 6:46 PM, Dirk Pranke <dpra...@chromium.org> wrote:

> This sounds great!
>
> -- Dirk
>
> On Tue, Oct 30, 2012 at 3:04 PM, Ojan Vafai <o...@chromium.org> wrote:
> > I'm testing out gasper alerts for the webkit canary bots. So far today
> it's
> > been ~40 false positives + 1 true regression (that got rolled back).
> That's
> > actually a good enough hit rate that I think it's much easier than
> manually
> > monitoring the perf graphs.
> >
> > That said, I have 0 insight into whether it's missing any actual
> > regressions. Over the next few weeks, could the active gardeners just
> > ping/cc me anytime they encounter a perf regression?
> >
> > If there are no missing regressions, then I think we could make it the
> > gardeners job to monitor the gasper emails instead of having to look at
> the
> > graphs. It would be considerably less work and would catch perf
> regressions
> > much faster.
> >
> > Thanks,
> > Ojan
> >
> > --
> >
> >
>

--0015174bde04ca693d04cd506fef
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div>FYI, I think we can probably manage to reduce the false positives if w=
e get some feedback on why they&#39;re not helpful. Some things that could =
possibly be problems:<br></div><div><br></div><div>* Monitoring too many gr=
aphs</div>
<div>* Alerts too sensitive</div><div>* Getting emails on improvements as w=
ell as regressions</div><div>* % change calculation incorrect</div><div>* A=
ctual problems with GASP</div><div><br></div><div>Any Googler who wants to =
take a look at the alerts can see a full archive at=A0<a href=3D"https://gr=
oups.google.com/a/google.com/forum/?fromgroups#!forum/webkit-gasper">https:=
//groups.google.com/a/google.com/forum/?fromgroups#!forum/webkit-gasper</a>=
</div>
<div><br></div><div>If anybody has time to make a list of good/bad alerts, =
it would be helpful to have the &quot;Debug this alert in Gasper&quot; link=
, the rev the reported regression happens at, and a quick explanation of wh=
y it&#39;s not helpful. You don&#39;t need to look at all the alerts; a sma=
ll sample would be useful. I made a doc here for feedback: <a href=3D"http:=
//go/webkit-gasper-feedback">http://go/webkit-gasper-feedback</a></div>
<div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Thanks!</di=
v><div class=3D"gmail_extra">-Annie</div><div class=3D"gmail_extra"><br><di=
v class=3D"gmail_quote">On Tue, Oct 30, 2012 at 6:46 PM, Dirk Pranke <span =
dir=3D"ltr">&lt;<a href=3D"mailto:dpra...@chromium.org" target=3D"_blank">d=
pra...@chromium.org</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">This sounds great!<br>
<span class=3D"HOEnZb"><font color=3D"#888888"><br>
-- Dirk<br>
</font></span><div><div class=3D"h5"><br>
On Tue, Oct 30, 2012 at 3:04 PM, Ojan Vafai &lt;<a href=3D"mailto:ojan@chro=
mium.org">o...@chromium.org</a>&gt; wrote:<br>
&gt; I&#39;m testing out gasper alerts for the webkit canary bots. So far t=
oday it&#39;s<br>
&gt; been ~40 false positives + 1 true regression (that got rolled back). T=
hat&#39;s<br>
&gt; actually a good enough hit rate that I think it&#39;s much easier than=
 manually<br>
&gt; monitoring the perf graphs.<br>
&gt;<br>
&gt; That said, I have 0 insight into whether it&#39;s missing any actual<b=
r>
&gt; regressions. Over the next few weeks, could the active gardeners just<=
br>
&gt; ping/cc me anytime they encounter a perf regression?<br>
&gt;<br>
&gt; If there are no missing regressions, then I think we could make it the=
<br>
&gt; gardeners job to monitor the gasper emails instead of having to look a=
t the<br>
&gt; graphs. It would be considerably less work and would catch perf regres=
sions<br>
&gt; much faster.<br>
&gt;<br>
&gt; Thanks,<br>
&gt; Ojan<br>
&gt;<br>
</div></div>&gt; --<br>
&gt;<br>
&gt;<br>
</blockquote></div><br></div>

--0015174bde04ca693d04cd506fef--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
