Build System Project - Update from the last 2 weeks

[David Burns]

Below is a highlight of all work the build peers have done in the last 2
weeks as part of their work to modernise the build infrastructure.

Since the last report[1] a large number of improvements have landed in
Mozilla Central.

The build system now lazily installs test files. Before, the build copied
tens of thousands of test and support files. This could take dozens of
seconds on Windows or machines with slow I/O. Now, the build system defers
installing test files until they are needed there (e.g. when running tests
or creating test packages). Furthermore, only the test files relevant to
the action performed are installed. Mach commands running tests should be
significantly faster, as they no longer examine the state of tens of
thousands of files on every invocation.

After upgrading build machines to use VS2015, we have seen a decrease in
build times[2] for PGO on Windows by around 100 minutes. This brings PGO
times on Windows in line with that of PGO(Strictly speaking this is LTO)
times on Linux.

This work, coupled with the build promotion work[3] have reduced the time
it takes automation to release Firefox on Windows by over 10 hours.

We are continuing to remove configure, m4 code, and Makefiles from
mozilla-central. As mentioned in the previous status email, this will allow
us to replace the build backend with a more performant tool.

David

[1]
https://groups.google.com/d/msg/mozilla.dev.platform/IKRdGCjdN_Y/sK2QbXqmCAAJ

[2]
https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=%5Bmozilla-inbound,04b9f1fd5577b40a555696555084e68a4ed2c28f,1%5D&series=%5Bmozilla-inbound,c0018285639940579da345da71bb7131d372c41e,1%5D&series=%5Bmozilla-inbound,65e0ddb3dc085864cbee77ab034dead6323a1ce6,1%5D

[3] https://rail.merail.ca/posts/release-build-promotion-overview.html

[Mike Hommey]

On Tue, Apr 05, 2016 at 09:02:09PM +0100, David Burns wrote:
> Below is a highlight of all work the build peers have done in the last 2
> weeks as part of their work to modernise the build infrastructure.
>
> Since the last report[1] a large number of improvements have landed in
> Mozilla Central.
>
> The build system now lazily installs test files. Before, the build copied
> tens of thousands of test and support files. This could take dozens of
> seconds on Windows or machines with slow I/O. Now, the build system defers
> installing test files until they are needed there (e.g. when running tests
> or creating test packages). Furthermore, only the test files relevant to
> the action performed are installed. Mach commands running tests should be
> significantly faster, as they no longer examine the state of tens of
> thousands of files on every invocation.
>
> After upgrading build machines to use VS2015, we have seen a decrease in
> build times[2] for PGO on Windows by around 100 minutes. This brings PGO
> times on Windows in line with that of PGO(Strictly speaking this is LTO)
> times on Linux.

Just a nit: strictly speaking Windows builds do PGO+LTO, Linux builds do
PGO, but not LTO.

Mike

[Jeff Gilbert]

I thought Linux did LTO but not PGO?

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

[Ted Mielczarek]

No. GCC *has* an LTO optimizer, but we're not using it. We're just doing
a PGO build. MSVC requires enabling LTO to use their PGO, so the
resulting build has both.

-Ted

[Nick Fitzgerald]

Is there a reason why we aren't using LTO?

[Mike Hommey]

On Fri, Apr 08, 2016 at 04:43:29PM -0700, Nick Fitzgerald wrote:
> Is there a reason why we aren't using LTO?

Last time it was tried, LTO regressed performance compared to the PGO
builds we produce (and yes, both LTO and LTO+PGO were tried).

We recently upgraded GCC to 4.8, so we can give it another spin, but I
don't think 4.8 would make a significant difference here. I want to
update to 5.3 next cycle (but keeping the minimum supported version at
4.8), maybe that one would make a difference.

Mike