LTO-support libraries for tool chain on Linux?

[Vitali Lovich]

Hi,

I’m trying to turn on LTO for our project that embeds v8. We utilize the tool chain that v8 downloads to make everything consistent. Out of the box the build didn’t work because the LTO support libraries on my system were LLVM-11. I hacked together a prototype that turns on LTO and we see an about 10% performance benefit on synthetic benchmarks so it’s compelling.

However, to enable that I needed to side load the LLVMgold, libLLVM, and libLTO libraries from Arch into the lib/ folder of the downloaded tool chain (+ libstdc++6 because the rpath in libLLVM had a relative path I think).

I’m wondering if there’s a way to alter the package to grab those libraries when the tool chain is built or to generate a new package containing them for those looking to enable LTO support. Alternatively, I could build it by hand. My concern with that are that Chromium seems to track the tip of an LLVM release and will be version bumped at arbitrary points meaning it can be tricky to orchestrate that works correctly at all times. Similarly, if we don’t track as frequently I’m wondering if the LTO path is carefully managed to bump a version number on every incompatible change or just on “major” releases for safety (ie I can maybe live with broken builds but certainly not erroneous codegen).

Appreciate any tips/ solutions anyone has.

Thanks!

[Reid Kleckner]

It sounds like you are already using clang to build V8 if you are considering LLVM LTO. I think the easiest way to use LTO is probably to use the LLD linker instead of ld.bfd or ld.gold. I'm not sure how you have packaged LLVM, but LLD doesn't use a plugin architecture, so as long as the loader's library search paths work out, LTO should work.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/9c05132a-050b-40be-95df-b2fd05163276n%40chromium.org.

[Vitali Lovich]

Hmm... getting closer but boringssl is still failing to build. It looks like the root cause is that the toolchain is missing llvm-ranlib & the system ranlib is expecting LLVMgold.so to add the LTO support?

deps/v8/third_party/llvm-build/Release+Asserts/bin/llvm-ar qc libtest_support_lib.a CMakeFiles/test_support_lib.dir/abi_test.cc.o CMakeFiles/test_support_lib.dir/file_test.cc.o CMakeFiles/test_support_lib.dir/malloc.cc.o CMakeFiles/test_support_lib
.dir/test_util.cc.o CMakeFiles/test_support_lib.dir/wycheproof_util.cc.o
/usr/bin/ranlib libtest_support_lib.a
bfd plugin: LLVM gold plugin has failed to create LTO module: Unknown attribute kind (70) (Producer: 'LLVM13.0.0git' Reader: 'LLVM 11.1.0')
bfd plugin: LLVM gold plugin has failed to create LTO module: Unknown attribute kind (70) (Producer: 'LLVM13.0.0git' Reader: 'LLVM 11.1.0')
bfd plugin: LLVM gold plugin has failed to create LTO module: Unknown attribute kind (70) (Producer: 'LLVM13.0.0git' Reader: 'LLVM 11.1.0')
bfd plugin: LLVM gold plugin has failed to create LTO module: Unknown attribute kind (70) (Producer: 'LLVM13.0.0git' Reader: 'LLVM 11.1.0')
bfd plugin: LLVM gold plugin has failed to create LTO module: Invalid record

And then later the linkage fails:

clang: warning: argument unused during compilation: '-nostdinc++' [-Wunused-command-line-argument]
ld.lld: error: undefined symbol: testing::internal::IsTrue(bool)
>>> referenced by gtest-internal.h:529 (deps/boringssl/third_party/googletest/include/gtest/internal/gtest-internal.h:529)
>>>               lto.tmp:(testing::internal::SuiteApiResolver<testing::Test>::GetSetUpCaseOrSuite(char const*, int))
>>> referenced by gtest-internal.h:550 (deps/boringssl/third_party/googletest/include/gtest/internal/gtest-internal.h:550)
>>>               lto.tmp:(testing::internal::SuiteApiResolver<testing::Test>::GetTearDownCaseOrSuite(char const*, int))
>>> referenced by gtest.h:1869 (deps/boringssl/third_party/googletest/include/gtest/gtest.h:1869)
>>>               lto.tmp:(testing::WithParamInterface<Base64TestVector>::GetParam())
>>> referenced 65 more times