Jumbo: Fully supported in Blink + Some requests

[Daniel Bratell]

Since jumbo was added to the main parts of blink, we have added it to the
rest as well with a few exceptions so I want to give a small project
update.

Quick background
----------------
Jumbo is a unity build implementation in Chromium, merging source files
before compiling them for a huge build time speedup. This is necessary
because the build time for non-goma has quickly increased to unsustainable
levels the last 1-2 years.
Doc: https://chromium.googlesource.com/chromium/src/+/master/docs/jumbo.md

Blink Success!
--------------
Time to compile Blink depends on the actual hardware but measured in CPU
hours, it's dropped from 13.2 CPU hours to 1.1 CPU hours.
That is, Blink compiles 13 times faster! (and links slightly faster)
For non-goma builds on "ordinary" hardware this means the full Chrome
compilation time has been reduced by 40%, i.e. by more than an hour. This
is a great step for everyone wanting to work on Chromium or Blink outside
goma, whether it's a student, a hobby programmer, or a company like Opera,
Vewd, Yandex or any of the others.

Requests
--------
I think Jumbo has broken on master about once or twice per week because of
code changes. Next big step will be to have a jumbo compilation among the
cq try bots. dpranke is investigating that right now.

Until then, a few requests:

* When tempted to copy a utility function/class/macro somewhere, in a test
or production code, please rename, give a custom namespace or find a
shared place instead.

* Macros are global. Please undef them after use, leaving the macro name
free for other things.


Near future
-----------
With Blink and v8 jumbified, chrome/browser and content/browser are the
single slowest code chunks to compile. There is a question whether to add
more before there is a cq support though. We're looking at low risk/high
effect changes until then.

/Daniel


--
/* Opera Software, Linköping, Sweden: CEST (UTC+2) */

[Oystein Eftevaag]

Do we have any data on how often incremental builds are done during a normal developer workflow vs. full builds, and what the total gains look like when adjusted for that?

Anecdotally, having jumbo'ified a large codebase in the past (gaming industry), most developers ended up turning the feature off as:

* Single/few-file incremental builds were much more commonly done than full builds.

* Incremental builds were more on the "critical path" of developer workflow: People tended to grab lunch after grabbing tip-of-tree and needing to do a full rebuild (or start one before leaving for the day), whereas for incremental builds you sit and wait.

* Jumbo builds would frequently break in subtle ways (less applicable for Chrome due to better bot/test coverage).

* The compilation speedups were mainly present on non-PCH enabled toolchains, specifically we didn't see a lot of speedup for MSVC which was the most common developer toolchain (unsure how much this applies to Chrome)

This was a project without any sort of distributed compilation toolchain and we'd already spent a substantial amount of time bringing compile times down by slimming down header includes (plus obviously a much smaller codebase) and so the same situation might not apply, but I'm still I'm a little worried that if this becomes default behavior for goma-enabled developer builds, we'll gain time when it matters less at the expense of slowing things down when it matters more.

[Bruce Dawson]

I think that normal developer builds are significantly biased towards component builds, so it is important that jumbo builds help with that scenario as well. How much they help depends a lot on what files you are touching, but ensuring that jumbo builds are almost faster should just be a matter of adjusting the jumbo_file_merge_limit - this is why I've argued for having this value default to as low a value as possible. That is, if jumbo_file_merge_limit is small enough then the costs of jumbo builds for incremental builds should be close to zero.

A separate benefit is that jumbo builds push us back from some of VC++'s hard-coded limits. For instance, a release-x86-non-component-goma build with symbol_level = 2 generates a chrome_child.dll.pdb that is more than 4 GiB - so the build fails. I hit this yesterday. Turning on use_jumbo_build shrunk the PDB to about 2.3 GiB. Similarly, jumbo builds will presumably shrink ILK files which will let us do incremental linking more frequently. This will make incremental builds *much* faster since a non-incremental link can take many minutes, compared to an incremental link taking just a few seconds.

I certainly do enough full builds (different configurations often) that I want every build-speed advantage I can get.

[Daniel Bratell]

I have seen no such data, but when jumbo in it's current form is 1 hour faster for a full build (non-goma, 4c/8t machine) and 2 hours faster with non-committed changes, you will have to do an awful lot of incremental compilations that are seconds slower for it to end up as a net loss.

Anecdotally I have noticed that I've totally changed how I work now when I can change branches in the middle of the workday without deep regret. I can also test things that I would previously have skipped because it would have required multiple full recompilations.

Still, I want the critical path to be as quick as possible, as I imagine everyone wants. That is why the size of the jumbo units is configurable (and will end up with a smaller default than currently on master), and why it's possible to exclude your favourite build target from jumbo with the jumbo_build_excluded setting.

One thing you mention is really critical though: Can't have jumbo builds be unreliable. I hope they can get full CQ testing soon, but presently they are broken (compilation, not runtime) 1-2 times per week. Someone (I or someone else) fix them within a few hours but this is with "just" Blink enabled. It will be much more frequently if all the code compiles the same way.

Whether it will be used by default together with goma or not I don't know. I'd love for it to be an improvement for everyone, but it's not goma users that are in critical need of it.

/Daniel - notices that compilation times have increased a further 17% the last 3 months

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/239ece30-33a2-4420-8662-0766e2fd34c8%40chromium.org.

[Dirk Pranke]

I think there's a decent way to go in tuning how we want jumbo builds to work before we enable them by default for developers; we may in fact never do that, even if there are clear reasons to support it for people (or bots) that do want to make the tradeoffs.

-- Dirk 

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/239ece30-33a2-4420-8662-0766e2fd34c8%40chromium.org.

[Daniel Bratell]

On Tue, 17 Oct 2017 23:27:46 +0200, Bruce Dawson <bruce...@chromium.org> wrote:

I think that normal developer builds are significantly biased towards component builds, so it is important that jumbo builds help with that scenario as well. How much they help depends a lot on what files you are touching, but ensuring that jumbo builds are almost faster should just be a matter of adjusting the jumbo_file_merge_limit - this is why I've argued for having this value default to as low a value as possible. That is, if jumbo_file_merge_limit is small enough then the costs of jumbo builds for incremental builds should be close to zero.

Yes, the current default (100) is too high, but I've kept it there because if it compiles with a large chunk size it has a reasonable chance to work with smaller sizes, while the opposite is not true. Unfortunately it doesn't work with 50 right now (v8 unittest thing) or I would have landed that reduction now when it has some attention.

Eventually we could have 10 or 20 for people with many cores, 50 for people with few cores, 100 or infinity for single core/full compilation. It's mostly a question of keeping the configurations compiling while people do thousands of changes to the source per week.

A separate benefit is that jumbo builds push us back from some of VC++'s hard-coded limits. For instance, a release-x86-non-component-goma build with symbol_level = 2 generates a chrome_child.dll.pdb that is more than 4 GiB - so the build fails. I hit this yesterday. Turning on use_jumbo_build shrunk the PDB to about 2.3 GiB. Similarly, jumbo builds will presumably shrink ILK files which will let us do incremental linking more frequently. This will make incremental builds *much* faster since a non-incremental link can take many minutes, compared to an incremental link taking just a few seconds.

Another thing that might possibly be retired is the "split_static_library" workaround. It solves problems that don't exists for jumbo builds for the reasons you list above.

[WoodpeckerTablet]

I want to install Google Chrome in my laptop!

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAE5mQiOUnWOjMwkr0r3KaYVqQaxd3p_j5ybak%2BJA2ANd%2Bjmr-A%40mail.gmail.com.

--
@page { margin: 2.01cm } P { margin-bottom: 0.2cm } P.western { so-language: en-US } -->

YouTube    : http://bit.ly/WoodpeckerYT

Twitter      : http://twitter.com/thewoodpecker1

Website    : http://woodpeckermc152test.playat.ch/

[PhistucK]

Go to https://www.google.com/chrome/ and click on "Download Chrome".

Please, refrain from posting random messages to unrelated threads and especially refrain from posting off topic messages to the group. This is not a user support forum.

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev+unsubscribe@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAE5mQiOUnWOjMwkr0r3KaYVqQaxd3p_j5ybak%2BJA2ANd%2Bjmr-A%40mail.gmail.com.

--
@page { margin: 2.01cm } P { margin-bottom: 0.2cm } P.western { so-language: en-US } -->

YouTube    : http://bit.ly/WoodpeckerYT

Twitter      : http://twitter.com/thewoodpecker1

Website    : http://woodpeckermc152test.playat.ch/

--
You received this message because you are subscribed to the Google Groups "blink-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscribe@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/59F5FBB5.9090802%40gmail.com.