Why not use clang with PGO to reduce build time for about 20%

[Matthew Larionov]

Hi! I've got a question have you ever tried to use clang with PGO to build chromium on try bots/developers machines?

Here're my results(Linux):

 

clang-pgo:
real    128m59.857s
user    968m7.005s
sys     48m28.458s

vs

clang-nopgo:
real    155m10.938s
user    1177m15.533s
sys     50m44.323s

 

What I've done:

1. Built clang from revision specified in tools/clang/scripts/update.py [1] and enabled flags: CFLAGS=-fprofile-instr-generate LDFLAGS=-fprofile-instr-generate

2. Build chromium using this clang with a little hacks in toolchain.ninja to pass LLVM_PROFILE_FILE= to clang-based commands

3. Merged profile data with llvm-profdata merge -o merged_profile $outdir/*.profraw

4. Built new clang with CFLAGS=-fprofile-instr-use=$(realpath merged_profile) -Wno-profile-instr-unprofiled -Wno-profile-instr-out-of-date

 LDFLAGS=-fprofile-instr-use=$(realpath merged_profile)

5. Then built browser using clang from runhooks vs just crafted one(results are above)

 

So why chromium doesn't use a PGO-based version of clang? Is there any reason for this?

 

P. S. GN args used: is_component_build=true is_debug=false. ninja target: all

 

1. https://cs.chromium.org/chromium/src/tools/clang/scripts/update.py?q=update.py&sq=package:chromium&l=30

[Gabriel Charette]

+infr...@chromium.org 

That's a pretty cool idea actually (and even more so if we can make it work on Windows :-)). But I'm far from being the expert here so I'll defer to those who are!

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Nico Weber]

The reason is mostly "we haven't done it yet". We build clang packages with tools/clang/scripts/update.py (see also the upload_revision.py in the same dir). Maybe you want to send a patch to make it do that? :-)

(There's also cl...@chromium.org for clang-specific discussions.)

Nico

--

[Matthew Larionov]

If there're no objections tomorrow I'll post an issue and start work on CL to bundle clangs with PGO.

[Nico Weber]

Sounds great, thanks!

Since you have the setup locally: how do build times compare for pgo and non pgo for build configs different from the one you used to collect profiles? (Say, in a debug build, a static library build, a build targeting arm/android.)

--
You received this message because you are subscribed to the Google Groups "Clang maintainers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clang+un...@chromium.org.

[Matthew Larionov]

Actually I did the investigation several weeks ago, just had no time to post a question here.

I've tried debug build & static build - same 20% boost, but never tried arm/android build, so no results for this one. But expect same numbers.

I'll run several builds for arm/android while working on task and post the results.

I've also made a test against source changes - gathered profile for ToT and then ran 2

builds(clang-nopgo vs clang-pgo) on release branch that was about 2-3 weeks past master - still 20% speedup :)

[Nico Weber]

Nice :-)

Upstream clang has cmake cache files (to be used with cmake's -C flag) for various build configs, including PGO (added here http://llvm.org/viewvc/llvm-project?view=revision&revision=278987). These files collect profiles compiling clang itself instead of Chromium – I was hoping that that would provide "representative enough" profiles, and I was hoping that we could replace much of update.py with an upstream Chromium.cmake cache file eventually. This doesn't affect you much (other than that you could try collecting profiles during a clang build instead of a Chromium build and see if that helps about the same amount, if you want), but it's in a somewhat related area, so I thought I'd mention it. (This was also discussed in https://llvmdevelopersmeetingbay2016.sched.org/event/8YzZ/developing-and-shipping-clang-with-cmake ; slides for that talk should become available in a week or two.)

(re-sending with correct address, sorry about double email)