Windows 64-bit

[Armen Zambrano Gasparnian]

Hi all,
We are currently setting up a Windows 64-bit pool of slaves and have
been filing bugs to get things green. [1]

I would like to keep things green as much as possible by unhidding those
jobs that are already green. [2]
If there are no objections I will unhide them for mozilla-central
tomorrow morning.

This is all preparatory work to fully support Windows 64-bit as a tier-1
platform.

Read below for status, bugs (volunteers needed) and tracking bug.

cheers,
Armen

##############

Status:
- we are cloning more build machines
- we don't have leak builds (see bugs below)
- mochitest-1 is the only suite with perma-oranges
- we don't have capacity for all branches but will eventually do

Bugs (Help needed):
- bug 669384 - make buildsymbols fails for leak test builds
- bug 670915 - make package fails for leak test builds
- bug 670697 - sporadic make check failures
- bug 671567 - WebGL failures for Windows 7 64-bit (mochitest-1)

Tracking bug:
- Bug 558448 - [Tracking bug] officially support Windows 64-bit builds

###############

[1]
http://armenzg.blogspot.com/2011/07/firefox-and-windows-64-bit-builds.html
[2] http://tbpl.mozilla.org/?tree=Firefox&noignore=1&jobname=WINNT%206.1

[armenzg]

These green jobs have been unhidden for *mozilla-central*.
When we have more machines we will add them to other branches as well
(WIP)
Note that there are no leak builds (see list of bugs at the bottom).
The opt builds sometimes go orange because of make check (bug 670697 -
sporadic make check failures)

cheers,
Armen

On Jul 14, 2:16 pm, Armen Zambrano Gasparnian <arme...@mozilla.com>
wrote:

> [1]http://armenzg.blogspot.com/2011/07/firefox-and-windows-64-bit-builds...
> [2]http://tbpl.mozilla.org/?tree=Firefox&noignore=1&jobname=WINNT%206.1

[armenzg]

On Jul 15, 9:47 am, armenzg <arme...@gmail.com> wrote:
> These green jobs have been unhidden for *mozilla-central*.
> When we have more machines we will add them to other branches as well
> (WIP)

No one would break my heart if they get hidden when they go orange
since there is no coverage on other branches yet.
Nevertheless I would thank you greatly for keeping them green while we
get full coverage! ;)

[Nicholas Nethercote]

On Fri, Jul 15, 2011 at 11:47 PM, armenzg <arm...@gmail.com> wrote:
> These green jobs have been unhidden for *mozilla-central*.

Why are there only opt Win64 builds, but no debug builds? (Sorry if
that was answered elsewhere.)

Nick

[Armen Zambrano Gasparnian]

Mentioned on the first and second email:

- bug 669384 - make buildsymbols fails for leak test builds
- bug 670915 - make package fails for leak test builds

cheers,
Armen

[Philip Ringnalda]

On 7/14/11 11:16 AM, Armen Zambrano Gasparnian wrote:
> If there are no objections I will unhide them for mozilla-central
> tomorrow morning.

Objection. Loud, strident, red-faced shouting objection.

Things visible on mozilla-central are things which, if you break them,
you must back out. Simply, unequivocally.

After you back out, then you should fix your failures on try, and vow
next time to use try before you push. If you pushed a merge from an
integration branch, then you should either use your branch better, or
shout about how your branch has been handicapped, because it isn't
running everything it needs to run to tell you whether or not you are in
shape to merge.

If it isn't running on try, and it wasn't just an oversight that it's
not running on some integration branch, then it is absolutely without
question not ready to be visible on mozilla-central.

I understand that that makes the "get it green, making sure none of the
failures require changes on slaves" process more difficult; I've watched
&noignore=1 and filed the bugs on lots of previous platform changes and
additions. I know it's hard and frustrating. That doesn't change the
fact that mozilla-central is very much not "where you can make something
visible to get developer attention."

I've rehidden all the win64 tests. Sorry :(

[armenzg]

We have win64 enabled for most branches.
The builds are now compiling again. You should see test and talos jobs
being triggered (very bad wait times).
I will be adding hopefully this week a lot of Try slaves and enable
things there as well.

Please have some mercy as I am trying to enable things as fast as I
can.

From releng's priorities we have two:
* Try capacity
* Win7x64 testers. We currently don't have enough capacity. I will
explain more in a following post.

cheers,
Armen

Yesterday's status update (today the builds actually compile):
[1] http://armenzg.blogspot.com/2011/08/win64-builds-status-update.html

[armenzg]

We now have try support for win64.
I will unhide the builds tomorrow when the orange builds are surpassed
by several green builds.

https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64&rev=5ea8d7c61095
https://tbpl.mozilla.org/?tree=Firefox&usebuildbot=1noignore=1&jobname=WINNT%206.1%20x86-64&rev=787ebf02a8c1

On Aug 23, 12:13 pm, armenzg <arme...@mozilla.com> wrote:
> We havewin64enabled for most branches.

[Phil Ringnalda]

On Wed, Sep 7, 2011 at 1:10 PM, armenzg <arm...@mozilla.com> wrote:
> We now have try support for win64.
> I will unhide the builds tomorrow when the orange builds are surpassed
> by several green builds.
>
> https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64&rev=5ea8d7c61095
> https://tbpl.mozilla.org/?tree=Firefox&usebuildbot=1noignore=1&jobname=WINNT%206.1%20x86-64&rev=787ebf02a8c1

https://tbpl.mozilla.org/?tree=Mozilla-Inbound&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64

[Kyle Huey]

Do we have enough builders for this? I saw yesterday morning (after the
holiday in NA) that try had a decent backlog of Win64 builds ...

- Kyle

> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
>

[Armen Zambrano Gasparnian]

We should have enough. I added the slaves midday yesterday and they
started picking up all the jobs. Let's see tomorrow's wait times email.

On 11-09-07 4:47 PM, Phil Ringnalda wrote:

> On Wed, Sep 7, 2011 at 1:10 PM, armenzg<arm...@mozilla.com> wrote:
>> We now have try support for win64.
>> I will unhide the builds tomorrow when the orange builds are surpassed
>> by several green builds.
>>
>>
https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64&rev=5ea8d7c61095
>>
https://tbpl.mozilla.org/?tree=Firefox&usebuildbot=1noignore=1&jobname=WINNT%206.1%20x86-64&rev=787ebf02a8c1
>
>

https://tbpl.mozilla.org/?tree=Mozilla-Inbound&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64

I disabled win64 coverage with all branches that have nightly builds
until I could manage to get green builds on mozilla-central (See bug
681158). We will re-enable it on the next few days until all project
branches have imported from mozilla-central.
The problem was that win64 was permanently orange and that was
preventing the nightly builds to be generated (we have a logic to take
the last green for all platforms; win64 was orange). Not sure why this
had not affected mozilla-central at all.

Let me know if you have any further questions.

cheers,
Armen

[Armen Zambrano Gasparnian]

We should have enough. I added the slaves midday yesterday and they
started picking up all the jobs. Let's see tomorrow's wait times email.

On 11-09-07 4:47 PM, Phil Ringnalda wrote:

> On Wed, Sep 7, 2011 at 1:10 PM, armenzg<arm...@mozilla.com> wrote:
>> We now have try support for win64.
>> I will unhide the builds tomorrow when the orange builds are surpassed
>> by several green builds.
>>
>>
https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64&rev=5ea8d7c61095
>>
https://tbpl.mozilla.org/?tree=Firefox&usebuildbot=1noignore=1&jobname=WINNT%206.1%20x86-64&rev=787ebf02a8c1
>
>

https://tbpl.mozilla.org/?tree=Mozilla-Inbound&usebuildbot=1&noignore=1&jobname=WINNT%206.1%20x86-64

I disabled win64 coverage with all branches that have nightly builds
until I could manage to get green builds on mozilla-central (See bug
681158). We will re-enable it on the next few days until all project
branches have imported from mozilla-central.
The problem was that win64 was permanently orange and that was
preventing the nightly builds to be generated (we have a logic to take
the last green for all platforms; win64 was orange). Not sure why this
had not affected mozilla-central at all.

Let me know if you have any further questions.

cheers,
Armen

On 11-09-07 4:53 PM, Kyle Huey wrote:

[Andrew McCreight]

I think the problem there may have been that Win64 was added to the -all option before there was actually anything running. I put up two try runs over the weekend, and they just sat there until last night waiting for Win64 to happen.

[Armen Zambrano Gasparnian]

There were no win64 slaves attached to the trypool from Thursday until
Tuesday since I went on PTO on Friday and Monday was a holiday.

Everything should be good now.

cheers,
Armen

[Armen Zambrano Gasparnian]

There were no win64 slaves attached to the trypool from Thursday until
Tuesday since I went on PTO on Friday and Monday was a holiday.

Everything should be good now.

cheers,
Armen

On 11-09-07 5:12 PM, Andrew McCreight wrote:

[Armen Zambrano Gasparnian]

I have made a blog post with the status update:
http://armenzg.blogspot.com/2011/09/win64-status-update.html

I enable this morning builds for win64 on the remaining branches and
triggered builds for them.

I disable unit tests and talos for all branches except mozilla-central.
We just couldn't keep up with 5 testers. I hope to have a bigger pool by
the end of the month.

cheers,
Armen