Jumbo update

[Daniel Bratell]

It's been a long while since I sent a public jumbo project update so here goes.

Background: Jumbo is a unity build system aimed at dramatically lowering the multi hour build times when working with chromium. A unity build system works by compiling many cc files in the same translation unit. The speedup varies between 6x (v8) and 25x (blink bindings). It is used officially by some companies and by individual engineers at many more companies and lowers the threshold for external contributors to join the project.

Documentation: https://chromium.googlesource.com/chromium/src/+/master/docs/jumbo.md

Result/status:
So far jumbo compilation is supported in chrome/browser, content, blink, v8 and in a number of smaller targets. Most targets don't yet support jumbo compilation. The current total effect is about 2.7x faster full builds for non-goma (1 hour instead of 3 hours) with use_jumbo_build=true (was 1.9x last mail I think).

Sadly sometimes code changes are necessary to support jumbo compilations so it won't just work by itself (which clearly would have been too good to be true). This means that jumbo support sometimes breaks on master, like it's broken by a v8 change right now (waiting for a v8 roll to include the fix). The plan is to get support in CQ to prevent this from happening but lack of resources keep delaying that to "RSN".

The lack of CQ support has meant that someone has had to fix problems on master, often several times per week, sometimes more than once in a day. That CQ does not test jumbo has slowed down, or nearly stopped, further expansion of jumbo support. Despite that de facto pause the measurable value of jumbo keeps increasing as normal builds become slower every week (+1% per week) in a way that jumbo builds do not. See attached graph.

Beyond unintentional symbol name clashes jumbo keeps detecting duplicate code. Most of the time shorter functions that should be in some utility library but are instead copied.

Some people from Opera and Vewd have been looking at adding clang features to prevent certain clashes that appear when you combine cc files. It is not possible to do in a plugin but there are promising results with minor changes to core clang code.

I plan to talk about jumbo and other ways of getting builds faster at BlinkOn9. Hope to see you there.

/Daniel

--

/* Opera Software, Linköping, Sweden: CET (UTC+1) */

[pdknsk]

Just want to give feedback on this. It's tremendously good. It had become frustrating to compile Chrome without it, as it was getting slower and slower. What I'd like is some additional guidance (or measurements) on jumbo_file_merge_limit. The docs only mention 4-core CPUs. What would you recommend for 32 vCPU (or 96 vCPU even)? I've used 32, as I figure there isn't much difference in either direction either way with so many cores.

[Daniel Bratell]

I'd recommend 50 since that is what is tested and actively maintained, but it's a slightly arbitrary number. I would have preferred a higher number if 4 cores/8 threads full builds had been the only thing that mattered. By having the default at 50 the single-file compilation cycles will not be too long and computers with many more cores will have a higher chance to use them all.

For goma which is effectively something between 100 and 1000 virtual CPU, brucedawson found 8 to be a suitable compromise between efficiency and parallelism.

A large part of the win is going from 1 (no jumbo) to 10. Another large win is going from 10 to 50. Then full time compilations keep improving all the way up to ~400 while single-file editing cycles become slower (and linking faster, several dimensions to this).

Any change from 8/50 will change what files are compiled together and there is always the risk that a certain combo won't compile though so choosing another number will increase the risk you have to spend time fixing your local build. So if someone asks, I recommend 50.

/Daniel

On Fri, 30 Mar 2018 01:04:51 +0200, pdknsk <pdk...@gmail.com> wrote:

Just want to give feedback on this. It's tremendously good. It had become frustrating to compile Chrome without it, as it was getting slower and slower. What I'd like is some additional guidance (or measurements) on jumbo_file_merge_limit. The docs only mention 4-core CPUs. What would you recommend for 32 vCPU (or 96 vCPU even)? I've used 32, as I figure there isn't much difference in either direction either way with so many cores.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/75608daa-059f-4879-bffd-ee21e898a58f%40chromium.org.

--

/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[pdknsk]

I only do full builds, so I may try a higher number, particularly if linking is accelerated. Does this also affect the last step, linking chrome? Which is now by far the slowest step, taking minutes.

[Bruce Dawson]

What platform are you building on?

There are some instructions for Windows on how to get faster builds, with some special instructions for getting faster links:

https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md#Faster-builds

I set use_lld = true. Using jumbo builds will also improve link times somewhat.