Any tips on how to speed up linking the official build

[Stephen Cheng]

On Windows, linking the official build is fairly slow and takes a lot of memory. When chrome_child.dll and chrome.dll are being linked at the same time, the two linking process can take about 10G memory. It can take 10 to 15 minutes to complete the final linking. I already have an quad-core i7 with 16GB memory and I do the linking on SSD. Is there any other tips I am not aware of that can speed up the linking?

Thanks.

[Brett Wilson]

You can try setting the GYP_LINK_CONCURRENCY environment variable to
only link one at a time. I think by default it does one link per 4 GB
of RAM it detects (see tools/gyp/pylib/gyp/generators/ninja.py and
search for the env var).

Brett

> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev

[Marc-Antoine Ruel]

"Linking" on Windows with WPO enabled really means running the optimizer pass on all the code during link time. So 10-15m isn't too bad in practice. Sadly adding more cores won't help. You can disable WPO if you are in a hurry.

M-A

--
M-A

[Bruce]

Actually more cores does help with WPO/LTCG build performance. Starting with VS 2013 the code-gen phase is multi-threaded. It defaults to using four cores, if available, and this can (IIRC) roughly double or triple build times. This can be configured (as of Update 2 or 3?) with /cgthreads:n

http://msdn.microsoft.com/en-us/library/dn655038.aspx

I haven't verified that this is working as designed on Chrome builds, but it should be.

So, having lots of memory (enough for the working set of the linkers plus lots of disk cache), plus an SSD plus 8+ cores is needed for optimal parallel linking, I believe.

[Bruce Dawson]

While doing some official (LTCG) builds on Windows I did some more research to find out why LTCG builds are so slow. I tested with pdf.dll which takes ~162 to ~163 s to LTCG link on my machine. I then tried adding /cgthreads:8 which tells VS 2013 to use eight threads instead of the default of four. This saved a measly ~13 s. I will add /cgthreads:8 to the Official build settings, but the improvement will be barely noticeable.

The single-threaded time was much longer than I'd seen on previous projects so I profiled the early section of the link, before "Generating code" is printed, as this accounts for roughly two thirds of the total time. In this ~98 s time period the linker/compiler spends 90% of its time in c2.dll!SsrFree, mostly in two tiny loops. That seems like a bug so I filed this.

On Fri, Oct 24, 2014 at 10:49 AM, Bruce <bruce...@chromium.org> wrote:

Actually more cores does help with WPO/LTCG build performance. Starting with VS 2013 the code-gen phase is multi-threaded. It defaults to using four cores, if available, and this can (IIRC) roughly double or triple build times (compared to VS 2010). This can be configured (as of Update 2 or 3?) with /cgthreads:n