Various memory test regressions
L. David Baron
over the past few days. I don't have the time to track them all down:
* Lk, RLk, and MH regressed on 1.8 branch between 20:13 and 21:41 on
2006-05-05 (although the RLk and part of the Lk regression seem to be
fixed at 11:09 on 2006-05-06
* Lk, RLk, and MH regressed on 1.8 branch between 14:16 and 17:35 on
2006-05-05
* Lk, RLk, and MH regressed on trunk between 2006-05-05 09:28 and
2006-05-06 09:23. (Yes, that huge window is what happens when people
check in on orange.)
There have also been performance test regressions:
* a large Windows-only Ts regression on 2006-05-02 on the trunk.
* a cross-platform Ts regression sometime (midday-ish) on 2006-05-05 on
the trunk.
* pair of Ts regressions on the 1.8 branch late on 2006-05-05.
I haven't looked at any performance numbers other than Ts.
It seems like it might be wise to close the trees until these
regressions are at least better identified (or are they, and I just
didn't hear about it?).
-David
--
L. David Baron <URL: http://dbaron.org/ >
Technical Lead, Layout & CSS, Mozilla Corporation
Gavin Sharp
> There have been a number of regressions in memory tests (leaks, bloat)
> over the past few days. I don't have the time to track them all down:
>
Here are some bonsai links, this should help sort out causes:
> * Lk, RLk, and MH regressed on 1.8 branch between 20:13 and 21:41 on
> 2006-05-05 (although the RLk and part of the Lk regression seem to be
> fixed at 11:09 on 2006-05-06
>
> * Lk, RLk, and MH regressed on 1.8 branch between 14:16 and 17:35 on
> 2006-05-05
>
> * Lk, RLk, and MH regressed on trunk between 2006-05-05 09:28 and
> 2006-05-06 09:23. (Yes, that huge window is what happens when people
> check in on orange.)
>
> There have also been performance test regressions:
>
> * a large Windows-only Ts regression on 2006-05-02 on the trunk.
>
> * a cross-platform Ts regression sometime (midday-ish) on 2006-05-05 on
> the trunk.
>
> * pair of Ts regressions on the 1.8 branch late on 2006-05-05.
>
http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=MOZILLA_1_8_BRANCH&branchtype=match&date=explicit&mindate=2006-05-05+15%3A41%3A00&maxdate=2006-05-05+16%3A08%3A00
and
http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=MOZILLA_1_8_BRANCH&branchtype=match&date=explicit&mindate=2006-05-05+21%3A24%3A00&maxdate=2006-05-05+21%3A51%3A00
I used the raw data logs from sparky (1.8) and pacifica (trunk) for the
Ts ranges.
Gavin
Gavin Sharp
>> * a large Windows-only Ts regression on 2006-05-02 on the trunk.
>>
> http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=HEAD&branchtype=match&date=explicit&mindate=2006-05-02+10%3A15%3A00&maxdate=2006-05-02+11%3A58%3A00
>
Bug 76111 has a Ts fix that looks like it's ready to land. Bug 334407
looks like it could potentially affect Ts.
>> * a cross-platform Ts regression sometime (midday-ish) on 2006-05-05 on
>> the trunk.
>>
> http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=HEAD&branchtype=match&date=explicit&mindate=2006-05-05+09%3A44%3A00&maxdate=2006-05-05+12%3A03%3A00
>
This looks like it would be caused by the session restore changes (bug
328159).
>> * pair of Ts regressions on the 1.8 branch late on 2006-05-05.
>>
> http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=MOZILLA_1_8_BRANCH&branchtype=match&date=explicit&mindate=2006-05-05+15%3A41%3A00&maxdate=2006-05-05+16%3A08%3A00
> and
> http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=MOZILLA_1_8_BRANCH&branchtype=match&date=explicit&mindate=2006-05-05+21%3A24%3A00&maxdate=2006-05-05+21%3A51%3A00
>
The former points to bug 334471 (microsummaries), which looks like it
was potentially fixed in bug 336833. The second points to bug 328159
(session restore) which I assume is the same increase seen on the branch
above.
steve....@gmail.com
This is Bug 336403
Boris Zbarsky
L. David Baron wrote:
> * Lk, RLk, and MH regressed on trunk between 2006-05-05 09:28 and
> 2006-05-06 09:23. (Yes, that huge window is what happens when people
> check in on orange.)
I've closed the trunk tree for this, at least until we've identified the
culprit. Is there a bug filed on this yet?
> * a large Windows-only Ts regression on 2006-05-02 on the trunk.
Almost certainly bug 76111; there's a patch in that bug to resolve this.
> * a cross-platform Ts regression sometime (midday-ish) on 2006-05-05 on
> the trunk.
As gavin said, this is probably fallout from bug 328159, unless the
XMLHttpRequest changes matter here (which they might, since we do XMLHttpRequest
during startup). Would be nice to get a bug filed on this so we can track it,
if there isn't one yet.
-Boris
Dietrich Ayala
I'm currently working on fixing the regressions related to session-restore.
Boris Zbarsky
> * Lk, RLk, and MH regressed on 1.8 branch between 20:13 and 21:41 on
> 2006-05-05 (although the RLk and part of the Lk regression seem to be
> fixed at 11:09 on 2006-05-06
>
> * Lk, RLk, and MH regressed on 1.8 branch between 14:16 and 17:35 on
> 2006-05-05
We should get bugs filed on these, I guess. Nothing jumps out at me. I've gone
ahead and made a comment on the 1.8 branch tinderbox to the effect that the
branch is closed except for regression fixes for the test regressions.
Note that the microsummaries service is known to leak; perhaps some of these
other leaks are just entrained by it?
-Boris
Boris Zbarsky
>> 2006-05-06 09:23. (Yes, that huge window is what happens when people
>> check in on orange.)
>
> http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&branch=HEAD&branchtype=match&date=explicit&mindate=2006-05-05+09%3A28&maxdate=2006-05-06+09%3A23
Given that and these parts of the NEW-LEAKS output:
nsAnnotationService 56 -
mozStorageService 20 -
mozStorageStatement 416 -
mozStorageConnection
this looks like a likely regression from bug 334471. The patch in bug 336922
might fix it, I hope.
-Boris
Boris Zbarsky
1) We still have a large branch leak regression that I'm having trouble
reproducing.
2) We still have a small trunk leak regression that I'm having trouble
reproducing.
3) We still have a 0.3-0.4MB MH regression. I'm not quite sure what to do with
it.
4) We still have a trunk cross-platform Ts regression.
5) We still have a branch cross-plagform Ts regression.
Details follow:
L. David Baron wrote:
> * Lk, RLk, and MH regressed on 1.8 branch between 20:13 and 21:41 on
> 2006-05-05 (although the RLk and part of the Lk regression seem to be
> fixed at 11:09 on 2006-05-06
This seems to be fixed. We're back down to about the level we were at as of
2006-05-05 17:30. Note, however, that this level is itself a regression from
2006-05-05 14:00 or so. Just as a point of comparison, the Rlk number is
currently (ignoring the latest blip due to people enabling safe-browsing) at
24.6KB. On 2006-05-05 14:00 it was 1.54KB.
Now the problem I have is that I haven't been able to reproduce the leaks
locally; my numbers locally looks a lot more like the 1.54KB number. I'm
pulling an updated branch and will try again....
If someone wants to help debug, the approximate bonsai range for that first
branch leak regression is
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=MOZILLA_1_8_BRANCH&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2006-05-05+14&maxdate=2006-05-05+18&cvsroot=%2Fcvsroot
> * Lk, RLk, and MH regressed on 1.8 branch between 14:16 and 17:35 on
> 2006-05-05
This is fixed.
> * Lk, RLk, and MH regressed on trunk between 2006-05-05 09:28 and
> 2006-05-06 09:23. (Yes, that huge window is what happens when people
> check in on orange.)
This is mostly fixed. We're currently at 2.22KB for RLk as opposed to 1.56KB on
2006-05-05 09:28. Lk is back to the old level. MH is still about 0.3MB higher
than it used to be, but that could just be because we have more objects in
flight? Not sure....
> * a large Windows-only Ts regression on 2006-05-02 on the trunk.
This is fixed.
> * a cross-platform Ts regression sometime (midday-ish) on 2006-05-05 on
> the trunk.
This is still an issue.
> * pair of Ts regressions on the 1.8 branch late on 2006-05-05.
This seems to still be a problem. See the sparky and pacifica graphs.
-Boris
Boris Zbarsky
> 1) We still have a large branch leak regression that I'm having trouble
> reproducing.
This seems to have gone away when the storage stuff got backed out. I couldnt'
reproduce it with a local build that DID have storage in it, so I don't know
what's up here, but in any case it's gone now.
> 2) We still have a small trunk leak regression that I'm having trouble
> reproducing.
This is now down into the noise. We're not quite sure _why_, but since I
couldn't reproduce it to start with....
> 3) We still have a 0.3-0.4MB MH regression. I'm not quite sure what to do
> with it.
No changes here.
> 4) We still have a trunk cross-platform Ts regression.
> 5) We still have a branch cross-plagform Ts regression.
No changes here (though it seems to be almost gone on Linux trunk, the mac and
windows boxes are showing it still). We're going to try disabling session
restore and seeing whether that has an effect.
-Boris
Dietrich Ayala
>> 4) We still have a trunk cross-platform Ts regression.
>> 5) We still have a branch cross-plagform Ts regression.
>
> No changes here (though it seems to be almost gone on Linux trunk, the
> mac and windows boxes are showing it still). We're going to try
> disabling session restore and seeing whether that has an effect.
>
> -Boris
The disabling of session-restore on the trunk resulted in a 2% decrease on Atlantia. Pacifica didn't
move, and the others were all noise. ("disabling" meant that init() just returned; the component
loaded, but never actually did anything)
The complete removal of the session-restore code from the build seemed to put most boxes back to
pre-session-restore Ts levels.