Pymake enabled on Win32 builders

Showing 1-8 of 8 messages
Pymake enabled on Win32 builders Siddharth Agarwal 9/1/12 5:37 AM
[crossposting to mozilla.dev.builds; please continue the discussion on
mozilla.dev.platform]

As of yesterday, Pymake has been enabled on all Firefox Win32 builders.
This includes mozilla-central, mozilla-inbound, try, and all project
branches (except elm [1]). Clobber build times (this includes Try
builds!) should go down by 45 minutes to an hour, resulting in faster
turnaround for developers and lower load on the infrastructure.

Pymake has now become the official way to build Firefox 18 and above on
Windows. If you’re on Windows and you haven't moved to Pymake yet, you
should: it's really simple [2]. I recommend setting up an alias to the
in-tree Pymake in your MSYS profile.

GNU Make will still work for now, but we won't have tier 1 continuous
integration for it (at least for Firefox — Thunderbird's still on GNU
Make, but I’m hoping to switch those builds to Pymake soon [3]). Win64
builds are also on GNU Make right now [4], but those aren't tier 1.

A few notes:

- If your project branch is seeing build failures on Windows:
   * make sure you're tracking an up-to-date mozilla-central: the last
couple of fixes landed earlier this week and more fixes will be coming
down the line
   * try setting a clobber for Win32 builds on your branch.
   * ask in #developers or #pymake -- perhaps someone might be able to
help you out
   * if all else fails, file a bug with releng similar to [1].

- While try builds are much faster now, we also lose the ability to
build Firefox 15-17 on them. I spent a bit of time investigating
workarounds but didn't get anywhere, unfortunately.

- This also means that you need to pushing an up-to-date mozilla-central
to try, otherwise you *will* see failures on Windows.

- One out of every 20 builds or so is failing with a strange error in
dom/bindings/test -- this seems to be corruption caused by a race
condition that isn't being properly handled somewhere but honestly
shouldn't be happening in the first place. If you see an error in
dom/bindings similar to [5], please retrigger that build. The error's
being tracked in [6].

Thanks to everyone who helped push it over the finish line: ted, khuey,
coop, gps, glandium, catlee, bsmedberg, bhearsum, and anyone else if I
missed them (sorry!).

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=787563
[2] https://developer.mozilla.org/en-US/docs/pymake
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=786516
[4] https://bugzilla.mozilla.org/show_bug.cgi?id=787655
[5]
https://tbpl.mozilla.org/php/getParsedLog.php?id=14886515&tree=Mozilla-Inbound&full=1
[6] https://bugzilla.mozilla.org/show_bug.cgi?id=787658
Re: Pymake enabled on Win32 builders Siddharth Agarwal 9/1/12 7:05 AM
(please use mozilla.dev.platform)

On 01-09-2012 19:24, Ehsan Akhgari wrote:
> On 12-09-01 8:37 AM, Siddharth Agarwal wrote:
>> - While try builds are much faster now, we also lose the ability to
>> build Firefox 15-17 on them. I spent a bit of time investigating
>> workarounds but didn't get anywhere, unfortunately.
>
> That is unfortunate.  Can we backport the necessary changes to those
> branches?

17: perhaps. 16 and below: quite unlikely.

One workaround I just thought of is to have a patch people can apply
that makes $topsrcdir/build/pymake/make.py a stub around GNU make. Devs
pushing Firefox 15-17 to try would have to add this patch to their patch
queue. Would that be an acceptable workaround?
Re: Pymake enabled on Win32 builders Benjamin Smedberg 9/1/12 11:44 AM
On 9/1/2012 8:37 AM, Siddharth Agarwal wrote:
>
>
> GNU Make will still work for now, but we won't have tier 1 continuous
> integration for it (at least for Firefox — Thunderbird's still on GNU
> Make, but I’m hoping to switch those builds to Pymake soon [3]). Win64
> builds are also on GNU Make right now [4], but those aren't tier 1.
I do not expect gmake to be a supported build configuration on windows
for long. As soon as we have the internal builds sorted out, I expect
that we will require building with pymake instead of gmake on Windows.

--BDS

Re: Pymake enabled on Win32 builders Siddharth Agarwal 9/1/12 11:56 AM
On 01-09-2012 19:35, Siddharth Agarwal wrote:
> (please use mozilla.dev.platform)
>
>
> 17: perhaps. 16 and below: quite unlikely.
>
> One workaround I just thought of is to have a patch people can apply
> that makes $topsrcdir/build/pymake/make.py a stub around GNU make. Devs
> pushing Firefox 15-17 to try would have to add this patch to their patch
> queue. Would that be an acceptable workaround?

Well, here's the patch: https://gist.github.com/3582912

I've added a link to this to the Try server wiki page:
https://wiki.mozilla.org/ReleaseEngineering/TryServer#Gecko_15-17_note

I pushed aurora tip + this patch to try and it worked perfectly.
Re: Pymake enabled on Win32 builders Mike Hommey 9/1/12 12:22 PM
Note that at the moment gmake is still required to build nss. There are
bugs in pymake that need to be fixed, and changes needed to nss before
we can say we don't need gmake at all.

Mike
Re: Pymake enabled on Win32 builders Axel Hecht 9/2/12 1:13 AM
Please don't forget to update the l10n builders before unsupporting gmake.

Axel
Re: Pymake enabled on Win32 builders Siddharth Agarwal 9/2/12 2:35 AM
On 02-09-2012 13:43, Axel Hecht wrote:
>>
>
> Please don't forget to update the l10n builders before unsupporting gmake.

The main thing stopping us from switching l10n repacks to Pymake is the
fact that they're on Python 2.5. Pymake requires 2.6 or 2.7.
Re: Pymake enabled on Win32 builders Siddharth Agarwal 9/5/12 4:00 AM
On 01-09-2012 18:07, Siddharth Agarwal wrote:
> - One out of every 20 builds or so is failing with a strange error in
> dom/bindings/test -- this seems to be corruption caused by a race
> condition that isn't being properly handled somewhere but honestly
> shouldn't be happening in the first place. If you see an error in
> dom/bindings similar to [5], please retrigger that build. The error's
> being tracked in [6].

This has now been fixed on mozilla-central. If you're seeing these on
your project branch or locally, please update mozilla-central.