Build System Project - Update from the last 2 weeks

[David Burns]

Below is a highlight of all work the build peers have done in the last 2 weeks as part of their work to modernise the build infrastructure.

Since the last report[1] a large number of improvements have landed in Mozilla Central.

The build system now lazily installs test files. Before, the build copied tens of thousands of test and support files. This could take dozens of seconds on Windows or machines with slow I/O. Now, the build system defers installing test files until they are needed there (e.g. when running tests or creating test packages). Furthermore, only the test files relevant to the action performed are installed. Mach commands running tests should be significantly faster, as they no longer examine the state of tens of thousands of files on every invocation.

After upgrading build machines to use VS2015, we have seen a decrease in build times[2] for PGO on Windows by around 100 minutes. This brings PGO times on Windows in line with that of PGO(Strictly speaking this is LTO) times on Linux.

This work, coupled with the build promotion work[3] have reduced the time it takes automation to release Firefox on Windows by over 10 hours.

We are continuing to remove configure, m4 code, and Makefiles from mozilla-central. As mentioned in the previous status email, this will allow us to replace the build backend with a more performant tool.

David

[1] https://groups.google.com/d/msg/mozilla.dev.platform/IKRdGCjdN_Y/sK2QbXqmCAAJ

[2] https://treeherder.mozilla.org/perf.html#/graphs?timerange=2592000&series=%5Bmozilla-inbound,04b9f1fd5577b40a555696555084e68a4ed2c28f,1%5D&series=%5Bmozilla-inbound,c0018285639940579da345da71bb7131d372c41e,1%5D&series=%5Bmozilla-inbound,65e0ddb3dc085864cbee77ab034dead6323a1ce6,1%5D

[3] https://rail.merail.ca/posts/release-build-promotion-overview.html

[Mike Hommey]

On Tue, Apr 05, 2016 at 09:02:09PM +0100, David Burns wrote:
> Below is a highlight of all work the build peers have done in the last 2
> weeks as part of their work to modernise the build infrastructure.
>
> Since the last report[1] a large number of improvements have landed in
> Mozilla Central.
>
> The build system now lazily installs test files. Before, the build copied
> tens of thousands of test and support files. This could take dozens of
> seconds on Windows or machines with slow I/O. Now, the build system defers
> installing test files until they are needed there (e.g. when running tests
> or creating test packages). Furthermore, only the test files relevant to
> the action performed are installed. Mach commands running tests should be
> significantly faster, as they no longer examine the state of tens of
> thousands of files on every invocation.
>
> After upgrading build machines to use VS2015, we have seen a decrease in
> build times[2] for PGO on Windows by around 100 minutes. This brings PGO
> times on Windows in line with that of PGO(Strictly speaking this is LTO)
> times on Linux.

Just a nit: strictly speaking Windows builds do PGO+LTO, Linux builds do
PGO, but not LTO.

Mike
_______________________________________________
firefox-dev mailing list
firef...@mozilla.org
https://mail.mozilla.org/listinfo/firefox-dev

[Jeff Gilbert]

I thought Linux did LTO but not PGO?

> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform