Next Tuesday I am going to ask for approval to finish up moving the
unit tests from the builders to the minis.
The last standing platform was Win2003 for debug unit tests which
would move to Win7 debug unit tests.
2 out of the 3 perma-oranges have been fixed today.

philor pointed out that xpcshell takes quite long on Windows 7 on the
minis compared to the runs on the IX/VMs combination for xpcshell.

I am writing this so we can all be aware of it when we bring this up
on Tuesday. For a small summary keep on reading or you can jump to the
full post with all the data and charts.

See you on Tuesday,
Armen

[1] http://armenzg.blogspot.com/2010/12/xpcshell-case-on-windows.html

PS = My apologies if my I used the averages for only one day as it was
the easiest data I could collect.

###############

The numbers:
* xpcshell debug on Win2003 (average in mins) - 46 mins
* xpcshell debug on Win7 (average in mins)       - 106 mins

It would have not been as bad if it didn't become the new longest test
suite to run but it has become.
The previous longest pole was:
* xpcshell opt on Win7 x64 (average in mins) - 80 mins

On my blog post I mention that the world would be happier if we
managed to run xpcshell in less time or break it into pieces. If
xpcshell on windows did not take that long, Window's longest poll
would instead be "mochitest-other" or "dromaeo" which would take
around 60 mins which is comparable to "mochitest-4 debug for
Fedora" (72 mins).

It sounds to me like the problem is "xpcshell tests take a long time
to run on Windows 7", since even the opt builds take extremely long to
run. We should try tracking down the root cause here, this is not
reasonable. I suspect there's some bug or bad interaction with our
xpcshell tests on Windows 7 that's making them take entirely too long
to run. I don't think we should look at splitting up the suite or
anything until we've determined why it's taking so long.

I happen to know that there are a lot of developers with Windows 7
machines nowadays, so a good first step would be to have some people
download builds+tests and try running the same commands we run on the
test slaves locally and see how long they take.

-Ted

On Dec 4, 10:28 am, Ted Mielczarek <t...@mielczarek.org> wrote:
Guess what? You are right.

Not just the hardware but maybe the OS (win7) might be at fault.
http://armenzg.blogspot.com/2010/12/test-suites-on-windows.html
xpcshell and reftests run worst. All other test suites are unaffected
by the switch (neither hardware nor OS nor graphics bound).

I will post the steps that we run for xpcshell so people can try to
reproduce/debug.

Cheers,
Armen

I believe this should do the trick:

#!/bin/sh.exe
FTP="http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds"
BRANCH=mozilla-central
OS=win32
TYPE=debug
BINARY=firefox-4.0b8pre.en-US.win32.zip
TESTS_PACKAGE=firefox-4.0b8pre.en-US.win32.tests.zip

if [[ ("$#" != 1 || ! ($1 =~ ^[0-9]+$) ) ]]; then
   echo "One argument required in the form of a unix timestamp"
   echo "We want to build an ftp tinderbox-builds URL:"
   echo "$FTP/$BRANCH-$OS-$TYPE/\$UNIX_TIMESTAMP"
   exit 1
fi
UNIX_TIMESTAMP=$1

wget --progress=dot:mega -N $FTP/$BRANCH-$OS-$TYPE/$UNIX_TIMESTAMP/
$BINARY
unzip -o $BINARY
wget --progress=dot:mega -N $FTP/$BRANCH-$OS-$TYPE/$UNIX_TIMESTAMP/
$TESTS_PACKAGE
unzip -o $TESTS_PACKAGE bin* certs* xpcshell*
if [ ! -d firefox/plugins ]; then mkdir firefox/plugins; fi && cp bin/
xpcshell.exe firefox && cp -R bin/components/* firefox/components/ &&
cp -R bin/plugins/* firefox/plugins/ && python -u xpcshell/
runxpcshelltests.py --symbols-path=symbols --manifest=xpcshell/tests/
all-test-dirs.list firefox/xpcshell.exe

On 10-12-06 11:20 AM, armenzg wrote:

Is it possible that the difference would be in our startup/shutdown
times?  I noticed that we're about 7% slower in our startup time on
Windows 7 compared to Windows XP <http://is.gd/in0q9>, and about 50%
slower in our shutdown times on the same OSes <http://is.gd/in0I9>.

(Note that the single most unusual thing about the xpcshell tests is
that we basically startup and shutdown Gecko for every test, compared to
once per suite).

If this is really the case, we should probably have blocking bugs on
file for investigating the issue...

Cheers,
Ehsan

On 12/7/2010 7:00 PM, Ehsan Akhgari wrote:

Ehsan, you're absolutely right.  I filed bug 617503 [1], on this
yesterday and started trying to profile it.  There is ongoing analysis
in that bug (many many thanks to Jimm), but yes, we're spending all our
time in startup/shutdown.

Clint
[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=617503