Faster builds, now ; on windows, too.

[Mike Hommey]

Hi,

Episode 1 was the "You want faster builds, don't you" thread.
Episode 2 was the "Faster builds, now" thread.
Here comes episode 3.

I'm sure fellow developers building on Windows felt sad that they were
left out on the recent build improvements. Rejoice at last, as we are
now bringing those to you.

As of current mozilla-inbound, and I'm sure mozilla-central soon, it
only takes a few steps:

- Download the mozmake binary from
http://people.mozilla.org/~mhommey/mozmake.exe and place it in some
location in your $PATH. c:\mozilla-build\msys\local\bin is probably
good for most people. See bug 927213 if you want to know how it was
built. It will eventually be shipped in next release of MozillaBuild.
- Add the following to your mozconfig:
export MOZ_PSEUDO_DERECURSE=no-pymake
(Note this should become the default next week)
- Build with:
./mach build

After you built once, you can do edit-compile-edit-compile cycles with:
./mach build binaries

Enjoy the faster build times.

Cheers,

Mike

PS: mozmake is also being tested on the birch branch, and it is showing
promising turnaround times on Windows build slaves (-35 minutes on opt
builds, -1 hour on debug builds ; most of which is, surprisingly, is
gained on make check, see
https://bugzilla.mozilla.org/show_bug.cgi?id=925605#c13 )

[Chris Pearce]

Thanks for putting this together, and thanks to everybody working on
making the build faster and thus making us all more productive. This
sped up clobber build times on Windows for me by 6 minutes, around 22%,
which is great. Some of us can't switch to a *nix based platform in
order to get faster builds, so thanks very much.

Cheers,
Chris P.

[Nicholas Nethercote]

On Wed, Oct 16, 2013 at 6:43 AM, Mike Hommey <m...@glandium.org> wrote:
>
> I'm sure fellow developers building on Windows felt sad that they were
> left out on the recent build improvements. Rejoice at last, as we are
> now bringing those to you.

In case you're interested how this happened... AIUI, these
improvements are because make 4.0 came out and it actually implements
-jN properly on Windows, and with -jN working it's faster than pymake.

(Well, almost properly, which is why glandium had to fix some things.
Presumably/hopefully his fixes will end up in make 4.01 soon.)

Nick

[Mike Hommey]

(FYI, FWIW)

-jN itself works properly. I just had to patch for unrelated issues,
namely:
- a typo that makes $(info), $(warning) and $(error) unreliable at
best, or crashy at worst.
- attempt to execute msys-path programs (e.g. "/usr/bin/install") with
CreateProcess() (yeah, that doesn't work well)
- enable a builtin workaround for sh.exe not working very very well
with quotes in a "sh -c ...." command.

The latter is a #define in config.h that exists for that purpose but is
not enabled by default, the former two are submitted upstream.

Mike

[Avi Hal]

On Wednesday, October 16, 2013 4:43:03 PM UTC+3, Mike Hommey wrote:
...

> - Build with:
>
> ./mach build
>
>
> After you built once, you can do edit-compile-edit-compile cycles with:
>
> ./mach build binaries

So what's the difference between |./mach build| and |./mach build binaries|? would such difference exist also after updating mozillabuild with the new mozmake (or the new make)?

[Gregory Szorc]

https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/build-targets.html
answers the first part.

In addition, mozmake should be faster than pymake in almost all
circumstances.

[David Rajchenbach-Teller]

Wouldn't it be interesting to also have a
./mach build frontend
that repackages XUL and js code?

On 10/21/13 6:53 PM, Gregory Szorc wrote:
>> So what's the difference between |./mach build| and |./mach build binaries|? would such difference exist also after updating mozillabuild with the new mozmake (or the new make)?
>

> https://ci.mozilla.org/job/mozilla-central-docs/Build_Documentation/build-targets.html
> answers the first part.
>
> In addition, mozmake should be faster than pymake in almost all
> circumstances.
>

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>


--
David Rajchenbach-Teller, PhD
Performance Team, Mozilla

[Gregory Szorc]

On the Q4 goals list. Bug 929147.

On 10/21/2013 2:47 PM, David Rajchenbach-Teller wrote:
> Wouldn't it be interesting to also have a
> ./mach build frontend
> that repackages XUL and js code?
>
>
> On 10/21/13 6:53 PM, Gregory Szorc wrote:

>>> So what's the difference between |./mach build| and |./mach build binaries|? would such difference exist also after updating mozillabuild with the new mozmake (or the new make)?
>>

[Gijs Kruitbosch]

I tend to use something like

./mach build browser/base browser/components browser/themes
browser/locales browser/devtools

(obviously including only the directories where I changed stuff)

Which is fast and works.

~ Gijs

[Gijs Kruitbosch]

I tend to use something like

./mach build browser/base browser/components browser/themes
browser/locales browser/devtools

(obviously including only the directories where I changed stuff)

Which is fast and works.

~ Gijs

On 21/10/13 23:47 , David Rajchenbach-Teller wrote:

[Neil]

David Rajchenbach-Teller wrote:

>Wouldn't it be interesting to also have a
> ./mach build frontend
>that repackages XUL and js code?
>
>

Does ./mach build chrome work? (I don't think it's parallelised though.)
Hopefully a combination of bug 929147 with bug 921003 will speed it up.

--
Warning: May contain traces of nuts.

[Avi Hal]

(On win7, i7 @3.2GHz) Clobber build from 29 mins down to 24, no-op build from some minutes to 16s! \o/

[Mike Hommey]

On Tue, Oct 22, 2013 at 10:06:13AM +0100, Neil wrote:
> David Rajchenbach-Teller wrote:
>
> >Wouldn't it be interesting to also have a
> > ./mach build frontend
> >that repackages XUL and js code?
> >
> Does ./mach build chrome work? (I don't think it's parallelised
> though.) Hopefully a combination of bug 929147 with bug 921003 will
> speed it up.

make chrome/mach build chrome doesn't do everything that is not code,
sadly. We will move towards that, though, maybe in another target
(chrome is probably not the right name for that).

Mike

[Neil]

Mike Hommey wrote:

>On Tue, Oct 22, 2013 at 10:06:13AM +0100, Neil wrote:
>
>
>>David Rajchenbach-Teller wrote:
>>
>>>Wouldn't it be interesting to also have a ./mach build frontend that repackages XUL and js code?
>>>
>>Does ./mach build chrome work?
>>
>>

>make chrome/mach build chrome doesn't do everything that is not code, sadly.
>

I guess it depends on which js code; from the mention of XUL I was
assuming anything with a chrome: URL.