[llvm-dev] (Thin)LTO llvm build
Carsten Mattner via llvm-dev
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc's lto support is
not the same as llvm's lto or thinlto, but I only fell back after it
failed to build that way, and I wasn't aware of gcc6 providing almost
5x large code than without lto.
Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Mehdi Amini via llvm-dev
> On Sep 10, 2016, at 3:03 AM, Carsten Mattner via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> I tried building llvm, clang, lld, lldb from the 3.9 svn release
> branch with LTO, and some of the results were unexpected.
>
> I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
> providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
> fell back to building with -DLLVM_ENABLE_LTO=On and using the system
> CC/CXX (gcc 6.1).
>
> The resulting installed build is many times bigger than the non-LTO
> version. Is this to be expected? I thought LTO would reduce size by
> detecting more unused code.
LTO also inline more, thus increasing the size. But usually it is quite slight.
If you’re concerned about binary size, disabling clang plugin may help:
cmake -DCLANG_PLUGIN_SUPPORT=OFF
>
> The bindist .tar.xz archives are as follows:
>
> LTO=Off: 195MB
> LTO=On : 953MB
>
> Now, I am of course aware that falling back to gcc's lto support is
> not the same as llvm's lto or thinlto, but I only fell back after it
> failed to build that way, and I wasn't aware of gcc6 providing almost
> 5x large code than without lto.
GCC may be using “fat” intermediate objects by default, i.e. it generates both the binary and the IR in the intermediates files and the static archives.
What is the size of the final clang binary itself?
>
> Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
> how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
> supported, or does it require the use of some libXX too?
You need to have Gold installed, but that is the same requirement as for LTO I believe.
Then if you’re using 3.9 as a host compiler, it should “just work”.
—
Mehdi
Carsten Mattner via llvm-dev
>
>> On Sep 10, 2016, at 3:03 AM, Carsten Mattner via llvm-dev <llvm...@lists.llvm.org> wrote:
>>
>> I tried building llvm, clang, lld, lldb from the 3.9 svn release
>> branch with LTO, and some of the results were unexpected.
>>
>> I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
>> providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
>> fell back to building with -DLLVM_ENABLE_LTO=On and using the system
>> CC/CXX (gcc 6.1).
>>
>> The resulting installed build is many times bigger than the non-LTO
>> version. Is this to be expected? I thought LTO would reduce size by
>> detecting more unused code.
>
> LTO also inline more, thus increasing the size. But usually it is quite slight.
> If you’re concerned about binary size, disabling clang plugin may help:
>
> cmake -DCLANG_PLUGIN_SUPPORT=OFF
Curious to know why this would result in smaller binaries.
>> The bindist .tar.xz archives are as follows:
>>
>> LTO=Off: 195MB
>> LTO=On : 953MB
>>
>> Now, I am of course aware that falling back to gcc's lto support is
>> not the same as llvm's lto or thinlto, but I only fell back after it
>> failed to build that way, and I wasn't aware of gcc6 providing almost
>> 5x large code than without lto.
>
> GCC may be using “fat” intermediate objects by default, i.e. it
> generates both the binary and the IR in the intermediates files
> and the static archives.
Are you saying I should strip files?
> What is the size of the final clang binary itself?
Interestingly the non-LTO clang-3.9 is 49MB while the LTO one if 58MB.
>> Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
>> how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
>> supported, or does it require the use of some libXX too?
>
> You need to have Gold installed, but that is the same requirement as for LTO I believe.
> Then if you’re using 3.9 as a host compiler, it should “just work”.
$ c++ -v
gcc version 6.2.1 20160830 (GCC)
$ ld.gold --version
GNU gold (GNU Binutils 2.27) 1.12
$ clang -v
clang version 3.9.0 (branches/release_39 279763)
Mehdi Amini via llvm-dev
On Sep 10, 2016, at 2:20 PM, Carsten Mattner <carsten...@gmail.com> wrote:On Sat, Sep 10, 2016 at 6:25 PM, Mehdi Amini <mehdi...@apple.com> wrote:On Sep 10, 2016, at 3:03 AM, Carsten Mattner via llvm-dev <llvm...@lists.llvm.org> wrote:
I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
LTO also inline more, thus increasing the size. But usually it is quite slight.
If you’re concerned about binary size, disabling clang plugin may help:
cmake -DCLANG_PLUGIN_SUPPORT=OFF
Curious to know why this would result in smaller binaries.
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc's lto support is
not the same as llvm's lto or thinlto, but I only fell back after it
failed to build that way, and I wasn't aware of gcc6 providing almost
5x large code than without lto.
GCC may be using “fat” intermediate objects by default, i.e. it
generates both the binary and the IR in the intermediates files
and the static archives.
Are you saying I should strip files?
What is the size of the final clang binary itself?
Interestingly the non-LTO clang-3.9 is 49MB while the LTO one if 58MB.Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
You need to have Gold installed, but that is the same requirement as for LTO I believe.
Then if you’re using 3.9 as a host compiler, it should “just work”.
$ c++ -v
gcc version 6.2.1 20160830 (GCC)
$ ld.gold --version
GNU gold (GNU Binutils 2.27) 1.12
$ clang -v
clang version 3.9.0 (branches/release_39 279763)
Mehdi Amini via llvm-dev
Teresa Johnson via llvm-dev
I tried building llvm, clang, lld, lldb from the 3.9 svn release
branch with LTO, and some of the results were unexpected.
I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
fell back to building with -DLLVM_ENABLE_LTO=On and using the system
CC/CXX (gcc 6.1).
The resulting installed build is many times bigger than the non-LTO
version. Is this to be expected? I thought LTO would reduce size by
detecting more unused code.
The bindist .tar.xz archives are as follows:
LTO=Off: 195MB
LTO=On : 953MB
Now, I am of course aware that falling back to gcc's lto support is
not the same as llvm's lto or thinlto, but I only fell back after it
failed to build that way, and I wasn't aware of gcc6 providing almost
5x large code than without lto.
Also, given a fresh 3.9 install, on an otherwise gcc/libstdc++ system,
how can I configure and build with -DLLVM_ENABLE_LTO=Thin? Is this
supported, or does it require the use of some libXX too?
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Carsten Mattner via llvm-dev
>
>
>
> On Sat, Sep 10, 2016 at 3:03 AM, Carsten Mattner via llvm-dev <llvm...@lists.llvm.org> wrote:
>>
>> I tried building llvm, clang, lld, lldb from the 3.9 svn release
>> branch with LTO, and some of the results were unexpected.
>>
>> I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
>> providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
>
>
> I've been able to do a ThinLTO build of clang for some time with gold.
> I'm not sure if there was a regression when the 3.9 release was cut?
> Can you send me the error?
I'll share the result and any reproduceable error (if there are any), after
the next build tests.
Carsten Mattner via llvm-dev
> You may need to use llvm-ar/llvm-ranlib though:
> cmake -DCMAKE_AR=path/to/llvm-ar -DCMAKE_RANLIB=path/to/llvm-ranlib
Anything else I should override, when I'm already using more than CC and CXX
from my llvm install?
Carsten Mattner via llvm-dev
> Supporting Plugin implies that every function has to available, even if
> inlined everywhere they can’t be dead-stripped. Because the plugin may call
> into it.
> Disabling plugin allows to remove all this dead code.
That makes sense, thanks.
> I’m just saying, it depends what you’re measuring: what matters is the final
> binary usually, not the size of the intermediate files.
I had only measured the size of the installed tree and assumed any
stripping that might happen would be the same as with LTO=OFF.
Let's see if I can get ThinLTO+No_PLUGINS to build and the results of
that. Might take a while.
Teresa Johnson via llvm-dev
Hi Teresa,
I've just attempted a THIN_LTO build again, and the
error I ran into is as follows, and happens during
cmake. I'm explicitly telling cmake to us llvm-ar and llvm-ranlib
as suggested.
[...]
-- Performing Test COMPILER_RT_HAS_STATIC_FLAG - Success
-- Performing Test COMPILER_RT_SUPPORTS_ATOMIC_KEYWORD
-- Performing Test COMPILER_RT_SUPPORTS_ATOMIC_KEYWORD - Success
-- Builtin supported architectures:
-- ISL version: isl-0.17.1-164-gcbba1b6
-- Performing Test HAS_ATTRIBUTE_WARN_UNUSED_RESULT
-- Performing Test HAS_ATTRIBUTE_WARN_UNUSED_RESULT - Failed
-- Performing Test HAVE___ATTRIBUTE__
-- Performing Test HAVE___ATTRIBUTE__ - Failed
-- Performing Test HAVE_DECL_FFS
-- Performing Test HAVE_DECL_FFS - Failed
-- Performing Test HAVE_DECL___BUILTIN_FFS
-- Performing Test HAVE_DECL___BUILTIN_FFS - Failed
-- Performing Test HAVE_DECL__BITSCANFORWARD
-- Performing Test HAVE_DECL__BITSCANFORWARD - Failed
CMake Error at tools/polly/lib/External/CMakeLists.txt:90 (message):
No ffs implementation found
[...]
Carsten Mattner via llvm-dev
>
> Cc'ing llvm-dev again in case someone knows what is going on.
>
> That error is coming from a polly configure script, which I don't
> have cloned into my own tree. Not sure why polly would behave
> differently when configuring for ThinLTO. Does polly configure ok
> when configuring the same way, but for regular LTO?
Yes it does, and I don't remember when I've added it, meaning it's
been working when bootstrapping llvm with the host gcc toolchain, for
a long time.
Mehdi Amini via llvm-dev
> On Sep 16, 2016, at 1:48 PM, Carsten Mattner via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> On Fri, Sep 16, 2016 at 10:18 PM, Teresa Johnson <tejo...@google.com> wrote:
>>
>> Cc'ing llvm-dev again in case someone knows what is going on.
>>
>> That error is coming from a polly configure script, which I don't
>> have cloned into my own tree. Not sure why polly would behave
>> differently when configuring for ThinLTO. Does polly configure ok
>> when configuring the same way, but for regular LTO?
>
> Yes it does, and I don't remember when I've added it, meaning it's
> been working when bootstrapping llvm with the host gcc toolchain, for
> a long time.
>
Can you look for the specific errors in the CMake log and error files?
CMake is very good at not presenting the real issue…
Here it is trying to build very simple programs to check features availability. Like:
#include <strings.h>
int main() { ffs(0); return 0; }
—
Mehdi
Teresa Johnson via llvm-dev
On Fri, Sep 16, 2016 at 10:18 PM, Teresa Johnson <tejo...@google.com> wrote:
>
> Cc'ing llvm-dev again in case someone knows what is going on.
>
> That error is coming from a polly configure script, which I don't
> have cloned into my own tree. Not sure why polly would behave
> differently when configuring for ThinLTO. Does polly configure ok
> when configuring the same way, but for regular LTO?
Yes it does, and I don't remember when I've added it, meaning it's
been working when bootstrapping llvm with the host gcc toolchain, for
a long time.
Carsten Mattner via llvm-dev
> Can you look for the specific errors in the CMake log and error files?
>
> CMake is very good at not presenting the real issue…
> Here it is trying to build very simple programs to check features
> availability. Like:
>
> #include <strings.h>
> int main() { ffs(0); return 0; }
In CMakeError.log there are errors like:
CheckIncludeFile.c:1:10: fatal error: 'malloc/malloc.h' file not found
#include <malloc/malloc.h>
...
CheckIncludeFile.c:1:10: fatal error: 'ndir.h' file not found
#include <ndir.h>
...
CheckIncludeFile.c:1:10: fatal error: 'sys/ndir.h' file not found
#include <sys/ndir.h>
...
fatal error: 'intrin.h' file not found
These seem like tests that are expected to fail.
...
The interesting test
int main() { __builtin_ffs(0); return 0; }
fails like this:
/usr/bin/ld[...]llvm/bin/../lib/LLVMgold.so:
cannot open shared object file: No such file[...]
I have two questions here:
Is there anything I might have missed when configuring the gcc-built
llvm install for it to be incomplete in the linker department? I mean,
the system linker is used (not lld) and that via CC=clang is looking
for the gold linker plugin, which seems to be missing. Probably
something got lost in my transition from autoconf to cmake?
Anything else to pass to cmake so that it, in addition to llvm-ar and
llvm-ranlib, also used lld? Or isn't that a requirement for building
with ThinLTO?
Carsten Mattner via llvm-dev
>
> On Fri, Sep 16, 2016 at 1:48 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>>
>> On Fri, Sep 16, 2016 at 10:18 PM, Teresa Johnson <tejo...@google.com> wrote:
>> >
>> > Cc'ing llvm-dev again in case someone knows what is going on.
>> >
>> > That error is coming from a polly configure script, which I don't
>> > have cloned into my own tree. Not sure why polly would behave
>> > differently when configuring for ThinLTO. Does polly configure ok
>> > when configuring the same way, but for regular LTO?
>>
>> Yes it does, and I don't remember when I've added it, meaning it's
>> been working when bootstrapping llvm with the host gcc toolchain, for
>> a long time.
>
>
> You mentioned in your original email:
> > I first tried to rebuild llvm with llvm-3.9, which has ThinLTO, by
> > providing -DLLVM_ENABLE_LTO=Thin, but that failed very quickly, so I
> > fell back to building with -DLLVM_ENABLE_LTO=On and using the system
> > CC/CXX (gcc 6.1).
>
> I assume this error when you try to bootstrap using llvm-3.9 and ThinLTO then?
> What about bootstrapping with llvm-3.9 and -DLLVM_ENABLE_LTO=On?
>
> If the latter works, I'll have to clone polly and see if I can reproduce it and
> see what is different between a -DLLVM_ENABLE_LTO=Thin and
> -DLLVM_ENABLE_LTO=On build then.
Same error, and as mentioned in reply to Mehdi, I'll have to verify it there
isn't an underlying issue of my local llvm install.
However, I would certainly welcome a success report and binary size info.
Mehdi Amini via llvm-dev
I assume that "/usr/bin/ld[...]llvm/bin” is the path where clang is installed right?
It seems that when you built the stage1 clang, it didn’t find Gold installed, and then didn’t build the plugin.
So you need to go back to the first build (not the ThinLTO one, the one with the system compiler), and make sure that CMake finds gold.
>
> I have two questions here:
>
> Is there anything I might have missed when configuring the gcc-built
> llvm install for it to be incomplete in the linker department? I mean,
> the system linker is used (not lld) and that via CC=clang is looking
> for the gold linker plugin, which seems to be missing. Probably
> something got lost in my transition from autoconf to cmake?
>
> Anything else to pass to cmake so that it, in addition to llvm-ar and
> llvm-ranlib, also used lld? Or isn't that a requirement for building
> with ThinLTO?
No requirement for lld.
—
Mehdi
Mehdi Amini via llvm-dev
Carsten Mattner via llvm-dev
> You probably missed -DLLVM_BINUTILS_INCDIR.
>
> See: http://llvm.org/docs/GoldPlugin.html
plugin-api.h is in /usr/include, so I'd expect it to be found, but I
can explicitly set BINUTILS_INCDIR and re-bootstrap with gcc 6.2.1.
I have ld.gold, but I'm not sure if /usr/bin/ld uses it, though I'd expect
it to since it's been in for a couple releases now.
$ ld -v
GNU ld (GNU Binutils) 2.27
$ ld.bfd -v
GNU ld (GNU Binutils) 2.27
$ ld.gold -v
GNU gold (GNU Binutils 2.27) 1.12
Carsten Mattner via llvm-dev
>
> I assume that "/usr/bin/ld[...]llvm/bin” is the path where clang is installed right?
Correct.
> It seems that when you built the stage1 clang, it didn’t find Gold installed,
> and then didn’t build the plugin.
> So you need to go back to the first build (not the ThinLTO one, the one with
> the system compiler), and make sure that CMake finds gold.
Going to re-bootstrap since the cmake transition somehow dropped that here.
Teresa Johnson via llvm-dev
On Fri, Sep 16, 2016 at 11:28 PM, Mehdi Amini <mehdi...@apple.com> wrote:
> You probably missed -DLLVM_BINUTILS_INCDIR.
>
> See: http://llvm.org/docs/GoldPlugin.html
plugin-api.h is in /usr/include, so I'd expect it to be found, but I
can explicitly set BINUTILS_INCDIR and re-bootstrap with gcc 6.2.1.
I have ld.gold, but I'm not sure if /usr/bin/ld uses it, though I'd expect
it to since it's been in for a couple releases now.
$ ld -v
GNU ld (GNU Binutils) 2.27
$ ld.bfd -v
GNU ld (GNU Binutils) 2.27
$ ld.gold -v
GNU gold (GNU Binutils 2.27) 1.12
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Carsten Mattner via llvm-dev
>
> On Fri, Sep 16, 2016 at 2:54 PM, Carsten Mattner via llvm-dev <llvm...@lists.llvm.org> wrote:
>>
>> On Fri, Sep 16, 2016 at 11:28 PM, Mehdi Amini <mehdi...@apple.com> wrote:
>>
>> > You probably missed -DLLVM_BINUTILS_INCDIR.
>> >
>> > See: http://llvm.org/docs/GoldPlugin.html
>>
>> plugin-api.h is in /usr/include, so I'd expect it to be found, but I
>> can explicitly set BINUTILS_INCDIR and re-bootstrap with gcc 6.2.1.
>>
>> I have ld.gold, but I'm not sure if /usr/bin/ld uses it, though I'd expect
>> it to since it's been in for a couple releases now.
>>
>> $ ld -v
>> GNU ld (GNU Binutils) 2.27
>> $ ld.bfd -v
>> GNU ld (GNU Binutils) 2.27
>> $ ld.gold -v
>> GNU gold (GNU Binutils 2.27) 1.12
>
>
> Looks like your default ld is GNU ld.bfd not ld.gold. You can either change your
> /usr/bin/ld (which probably is a link to /usr/bin/ld.bfd) to point instead to
> /usr/bin/ld.gold, or if you prefer, set your PATH before the stage1 compile to a
> location that has ld linked to ld.gold.
I can look into and check why Arch Linux has it configured like that.
In the meantime, Mehdi's suggestion to explicitly pass BINUTILS_INCDIR
restored the previous configure behavior, and the new llvm build has
lib/LLVMgold.so. Thanks to both of you for pointing out the missing cmake
flag.
I've checked the configure step and it didn't fail as it did before, but before
I try to build in ThinLTO mode: since the configure step checks for the gold
plugin, is it safe to assume that I don't have to change the default system
ld to gold and ThinLTO will work, or is that a build requirement for
bootstrapping llvm in ThinLTO mode?
Mehdi Amini via llvm-dev
Mehdi Amini via llvm-dev
Teresa Johnson via llvm-dev
Try to build llvm-tblgen, you’ll know quite quickly :)Also, you should limit the number of parallel link jobs: cmake -DLLVM_PARALLEL_LINK_JOBS=1And use ninja if you don’t already: cmake -GNinja
—Mehdi
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Carsten Mattner via llvm-dev
<llvm...@lists.llvm.org> wrote:
> Yes and to add on - the ThinLTO backend by default will
> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
Is it just me or does that sound not very intuitive or at least a
little unexpected?
It's good that it uses the resources eagerly, but in terms of build systems this
is certainly surprising if there's no control of that parameter via
make/ninja/xcode.
> too much for machines with hyperthreading. If that ends up being an issue I can
> give you a workaround (I've been struggling to find a way that works on various
> OS and arches to compute the max number of physical cores to fix this in the source).
I've been using ninja -jN so far. I suppose when building with ThinLTO I should
run ninja -j1. Would that
What's the workaround?
Mehdi Amini via llvm-dev
> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
> <llvm...@lists.llvm.org> wrote:
>
>> Yes and to add on - the ThinLTO backend by default will
>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>
> Is it just me or does that sound not very intuitive or at least a
> little unexpected?
> It's good that it uses the resources eagerly, but in terms of build systems this
> is certainly surprising if there's no control of that parameter via
> make/ninja/xcode.
You can control the parallelism used by the linker, but the option is linker dependent
(On MacOS: -Wl,-mllvm,-threads=1)
>
>> too much for machines with hyperthreading. If that ends up being an issue I can
>> give you a workaround (I've been struggling to find a way that works on various
>> OS and arches to compute the max number of physical cores to fix this in the source).
>
> I've been using ninja -jN so far. I suppose when building with ThinLTO I should
> run ninja -j1. Would that
>
> What's the workaround?
Seems like you missed my previous email: : cmake -DLLVM_PARALLEL_LINK_JOBS=1
Also, ninja is parallel by default, so no need to pass -j.
This way you get nice parallelism during the compile phase, and ninja will issue only one link job at a time.
—
Mehdi
Mehdi Amini via llvm-dev
> On Sep 16, 2016, at 6:17 PM, Mehdi Amini <mehdi...@apple.com> wrote:
>
>
>> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>>
>> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
>> <llvm...@lists.llvm.org> wrote:
>>
>>> Yes and to add on - the ThinLTO backend by default will
>>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>>
>> Is it just me or does that sound not very intuitive or at least a
>> little unexpected?
I should add: this is an issue for LLVM because we link *a lot* of various binaries.
However, I don’t believe it is the case of most projects, which is why we have this default (which matches ninja’s default).
—
Mehdi
>> It's good that it uses the resources eagerly, but in terms of build systems this
>> is certainly surprising if there's no control of that parameter via
>> make/ninja/xcode.
>
> You can control the parallelism used by the linker, but the option is linker dependent
> (On MacOS: -Wl,-mllvm,-threads=1)
>
>>
>>> too much for machines with hyperthreading. If that ends up being an issue I can
>>> give you a workaround (I've been struggling to find a way that works on various
>>> OS and arches to compute the max number of physical cores to fix this in the source).
>>
>> I've been using ninja -jN so far. I suppose when building with ThinLTO I should
>> run ninja -j1. Would that
>>
>> What's the workaround?
>
> Seems like you missed my previous email: : cmake -DLLVM_PARALLEL_LINK_JOBS=1
> Also, ninja is parallel by default, so no need to pass -j.
>
> This way you get nice parallelism during the compile phase, and ninja will issue only one link job at a time.
_______________________________________________
Carsten Mattner via llvm-dev
>
>> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>>
>> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
>> <llvm...@lists.llvm.org> wrote:
>>
>>> Yes and to add on - the ThinLTO backend by default will
>>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>>
>> Is it just me or does that sound not very intuitive or at least a
>> little unexpected?
>> It's good that it uses the resources eagerly, but in terms of build systems this
>> is certainly surprising if there's no control of that parameter via
>> make/ninja/xcode.
>
> You can control the parallelism used by the linker, but the option is linker dependent
> (On MacOS: -Wl,-mllvm,-threads=1)
That's what I meant. Maybe lld can gain support for that and allow us to use
the same ld pass-through via the compile driver so that it works on Linux and
BSD too.
>>> too much for machines with hyperthreading. If that ends up being an issue I can
>>> give you a workaround (I've been struggling to find a way that works on various
>>> OS and arches to compute the max number of physical cores to fix this in the source).
>>
>> I've been using ninja -jN so far. I suppose when building with ThinLTO I should
>> run ninja -j1. Would that
>>
>> What's the workaround?
>
> Seems like you missed my previous email: : cmake -DLLVM_PARALLEL_LINK_JOBS=1
I didn't miss that and it will hopefully help with limiting LTO link phase
resource use, but I do wonder if it means it's either linking one binary
or compiling objects and not both at parallel.
> Also, ninja is parallel by default, so no need to pass -j.
>
> This way you get nice parallelism during the compile phase,
> and ninja will issue only one link job at a time.
I know, but I use ninja for C++ project because those are the most
frequent CMake user, and compiling C++ often requires limited it
to less than NUM_CORES.
Mehdi Amini via llvm-dev
Also, ninja is parallel by default, so no need to pass -j.This way you get nice parallelism during the compile phase,and ninja will issue only one link job at a time.
I know, but I use ninja for C++ project because those are the most
frequent CMake user, and compiling C++ often requires limited it
to less than NUM_CORES.
Teresa Johnson via llvm-dev
> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
> <llvm...@lists.llvm.org> wrote:
>
>> Yes and to add on - the ThinLTO backend by default will
>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>
> Is it just me or does that sound not very intuitive or at least a
> little unexpected?
> It's good that it uses the resources eagerly, but in terms of build systems this
> is certainly surprising if there's no control of that parameter via
> make/ninja/xcode.
You can control the parallelism used by the linker, but the option is linker dependent
(On MacOS: -Wl,-mllvm,-threads=1)
>
>> too much for machines with hyperthreading. If that ends up being an issue I can
>> give you a workaround (I've been struggling to find a way that works on various
>> OS and arches to compute the max number of physical cores to fix this in the source).
>
> I've been using ninja -jN so far. I suppose when building with ThinLTO I should
> run ninja -j1. Would that
>
> What's the workaround?
Seems like you missed my previous email: : cmake -DLLVM_PARALLEL_LINK_JOBS=1
Also, ninja is parallel by default, so no need to pass -j.
This way you get nice parallelism during the compile phase, and ninja will issue only one link job at a time.
—
Mehdi
Mehdi Amini via llvm-dev
On Fri, Sep 16, 2016 at 6:17 PM, Mehdi Amini <mehdi...@apple.com> wrote:
> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
> <llvm...@lists.llvm.org> wrote:
>
>> Yes and to add on - the ThinLTO backend by default will
>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>
> Is it just me or does that sound not very intuitive or at least a
> little unexpected?
> It's good that it uses the resources eagerly, but in terms of build systems this
> is certainly surprising if there's no control of that parameter via
> make/ninja/xcode.
You can control the parallelism used by the linker, but the option is linker dependent
(On MacOS: -Wl,-mllvm,-threads=1)Wait - this is to control the ThinLTO backend parallelism, right? In which case you wouldn't want to use 1, but rather the number of physical cores.
When using gold the option is -Wl,-plugin-opt,jobs=N, where N is the amount of backend parallel ThinLTO jobs that will be issued in parallel. So you could try with the default, but if you have HT on then you might want to try with the number of physical cores instead.
Teresa Johnson via llvm-dev
On Fri, Sep 16, 2016 at 6:17 PM, Mehdi Amini <mehdi...@apple.com> wrote:
> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
> <llvm...@lists.llvm.org> wrote:
>
>> Yes and to add on - the ThinLTO backend by default will
>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>
> Is it just me or does that sound not very intuitive or at least a
> little unexpected?
> It's good that it uses the resources eagerly, but in terms of build systems this
> is certainly surprising if there's no control of that parameter via
> make/ninja/xcode.
You can control the parallelism used by the linker, but the option is linker dependent
(On MacOS: -Wl,-mllvm,-threads=1)Wait - this is to control the ThinLTO backend parallelism, right? In which case you wouldn't want to use 1, but rather the number of physical cores.Well it depends what behavior you want :)I should have used N to match ninja -jN.When using gold the option is -Wl,-plugin-opt,jobs=N, where N is the amount of backend parallel ThinLTO jobs that will be issued in parallel. So you could try with the default, but if you have HT on then you might want to try with the number of physical cores instead.How does it affects parallel LTO backends?(I hope it doesn't)
--Mehdi
>
>> too much for machines with hyperthreading. If that ends up being an issue I can
>> give you a workaround (I've been struggling to find a way that works on various
>> OS and arches to compute the max number of physical cores to fix this in the source).
>
> I've been using ninja -jN so far. I suppose when building with ThinLTO I should
> run ninja -j1. Would that
>
> What's the workaround?
Seems like you missed my previous email: : cmake -DLLVM_PARALLEL_LINK_JOBS=1
Also, ninja is parallel by default, so no need to pass -j.
This way you get nice parallelism during the compile phase, and ninja will issue only one link job at a time.
—
Mehdi
--
Teresa Johnson | Software Engineer | tejo...@google.com | 408-460-2413
Mehdi Amini via llvm-dev
On Sep 16, 2016, at 7:48 PM, Teresa Johnson <tejo...@google.com> wrote:On Fri, Sep 16, 2016 at 7:40 PM, Mehdi Amini <mehdi...@apple.com> wrote:On Fri, Sep 16, 2016 at 6:17 PM, Mehdi Amini <mehdi...@apple.com> wrote:
> On Sep 16, 2016, at 6:13 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Sat, Sep 17, 2016 at 2:07 AM, Teresa Johnson via llvm-dev
> <llvm...@lists.llvm.org> wrote:
>
>> Yes and to add on - the ThinLTO backend by default will
>> kick off std::thread::hardware_concurrency # of threads, which I'm finding is
>
> Is it just me or does that sound not very intuitive or at least a
> little unexpected?
> It's good that it uses the resources eagerly, but in terms of build systems this
> is certainly surprising if there's no control of that parameter via
> make/ninja/xcode.
You can control the parallelism used by the linker, but the option is linker dependent
(On MacOS: -Wl,-mllvm,-threads=1)Wait - this is to control the ThinLTO backend parallelism, right? In which case you wouldn't want to use 1, but rather the number of physical cores.Well it depends what behavior you want :)I should have used N to match ninja -jN.When using gold the option is -Wl,-plugin-opt,jobs=N, where N is the amount of backend parallel ThinLTO jobs that will be issued in parallel. So you could try with the default, but if you have HT on then you might want to try with the number of physical cores instead.How does it affects parallel LTO backends?(I hope it doesn't)In regular LTO mode, the option will also affect parallel LTO codegen, which is off by default. Is that what you meant?
Carsten Mattner via llvm-dev
should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
and run ninja without -j or -jNUM_PHYS_CORES?
Carsten Mattner via llvm-dev
> ThinLTO is quite lean on the memory side. What's you bottleneck to throttle you -j?
It's an acquired behavior from previous compiles of heavy users of C++ templates
or some other C++ feature that blows up space use, having to be careful not
to swap. I usually try -jNUM_VIRT_CORES, but could also run -j of course.
gcc's space overhead has been optimized significantly for OpenOffice and Mozilla
a couple releases back, so it's probably less of an issue these days.
Mehdi Amini via llvm-dev
> On Sep 17, 2016, at 3:19 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> So, when I embark on the next ThinLTO try build, probably this Sunday,
> should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
> and run ninja without -j or -jNUM_PHYS_CORES?
ThinLTO is fairly lean on memory: It should not consume more memory per thread than if you launch the same number of clang process in parallel to process C++ files.
For example when linking the clang binary itself, without debug info it consumes 0.6GB with 8 threads, 0.9GB with 16 threads, and 1.4GB with 32 threads.
With full debug info, we still have room for improvement, right now it consumes 2.3GB with 8 threads, 3.5GB with 16 threads, and 6.5GB with 32 threads.
So I believe that configuring with -DDLLVM_PARALLEL_LINK_JOBS=1 should be enough without other constrains, but your mileage may vary.
—
Mehdi
Carsten Mattner via llvm-dev
>
>> On Sep 17, 2016, at 3:19 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>>
>> So, when I embark on the next ThinLTO try build, probably this Sunday,
>> should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
>> and run ninja without -j or -jNUM_PHYS_CORES?
>
>
> ThinLTO is fairly lean on memory: It should not consume more memory per thread than if you launch the same number of clang process in parallel to process C++ files.
>
> For example when linking the clang binary itself, without debug info it consumes 0.6GB with 8 threads, 0.9GB with 16 threads, and 1.4GB with 32 threads.
> With full debug info, we still have room for improvement, right now it consumes 2.3GB with 8 threads, 3.5GB with 16 threads, and 6.5GB with 32 threads.
>
> So I believe that configuring with -DDLLVM_PARALLEL_LINK_JOBS=1 should be enough without other constrains, but your mileage may vary.
Sure, I'll try that to not introduce too many variables into the
configure changes,
though I have to ask if using lld would make it possible to have a common -Wl
that works across platforms, being able to ignore if it's binutils.
If I really wanted to pass that to cmake, overriding LDFLAGS would work, right?
Xinliang David Li via llvm-dev
Mehdi Amini via llvm-dev
> On Sep 17, 2016, at 4:03 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Sun, Sep 18, 2016 at 12:32 AM, Mehdi Amini <mehdi...@apple.com> wrote:
>>
>>> On Sep 17, 2016, at 3:19 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>>>
>>> So, when I embark on the next ThinLTO try build, probably this Sunday,
>>> should I append -Wl,-plugin-opt,jobs=NUM_PHYS_CORES to LDFLAGS
>>> and run ninja without -j or -jNUM_PHYS_CORES?
>>
>>
>> ThinLTO is fairly lean on memory: It should not consume more memory per thread than if you launch the same number of clang process in parallel to process C++ files.
>>
>> For example when linking the clang binary itself, without debug info it consumes 0.6GB with 8 threads, 0.9GB with 16 threads, and 1.4GB with 32 threads.
>> With full debug info, we still have room for improvement, right now it consumes 2.3GB with 8 threads, 3.5GB with 16 threads, and 6.5GB with 32 threads.
>>
>> So I believe that configuring with -DDLLVM_PARALLEL_LINK_JOBS=1 should be enough without other constrains, but your mileage may vary.
>
> Sure, I'll try that to not introduce too many variables into the
> configure changes,
> though I have to ask if using lld would make it possible to have a common -Wl
> that works across platforms, being able to ignore if it's binutils.
I’m not sure I understand the question about lld.
Lld will be a different linker, with its own set of option.
Actually we usually rely on the clang driver to hide platform specific option and provide a common interface to the user.
>
> If I really wanted to pass that to cmake, overriding LDFLAGS would work, right?
I don’t believe LDFLAGS is a valid cmake flag. You need to define both CMAKE_EXE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS.
—
Mehdi
Carsten Mattner via llvm-dev
> I’m not sure I understand the question about lld. Lld will be a
> different linker, with its own set of option. Actually we usually
> rely on the clang driver to hide platform specific option and
> provide a common interface to the user.
I was thinking if I force lld, then the -Wl param will be the same
across platforms, and I wouldn't have to accommodate for different
linkers.
> I don’t believe LDFLAGS is a valid cmake flag. You need to define
> both CMAKE_EXE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS.
It respects it, as it should, or otherwise packagers would have to
replicate CFLAGS, CXXFLAGS, etc. via CMAKE_*_FLAGS in package build
descriptions.
Carsten Mattner via llvm-dev
> As Mehdi mentioned, thinLTO backend processes use very little memory, you
> may get away without any additional flags (neither -Wl,--plugin-opt=jobs=..,
> nor -Dxxx for cmake to limit link parallesm) if your build machine has
> enough memory. Here is some build time data of parallel linking (with
> ThinLTO) 52 binaries in clang build (linking parallelism equals ninja
> parallelism). The machine has 32 logical cores and 64GB memory.
>
> 1) Using the default ninja parallelism, the peak 1min load-average is 537.
> The total elapse time is 9m43s
> 2) Using ninja -j16, the peak load is 411. The elapse time is 8m26s
> 3) ninja -j8 : elapse time is 8m34s
> 4) ninja -j4 : elapse time is 8m50s
> 5) ninja -j2 : elapse time is 9m54s
> 6) ninja -j1 : elapse time is 12m3s
>
> As you can see, doing serial thinLTO linking across multiple binaries do not
> give you the best performance. The build performance peaked at j16 in this
> configuration. You may need to find your best LLVM_PARALLEL_LINK_JOBS
> value.
What did you set LLVM_PARALLEL_LINK_JOBS to?
Maybe I should first try to leave it unset and see if it fits within
my machine's
hardware limits.
> Having said that, there is definitely room for ThinLTO usability
> improvement so that ThinLTO parallel backend can coordinate well with the
> build system's parallelism so that user does not need to figure out the
> sweet spot.
Definitely. If parallelism can be controlled on multiple layers, an
outer layer's
setting ought to influence it in a reasonable way to make it more intuitive
to use.
Teresa Johnson via llvm-dev
On Sun, Sep 18, 2016 at 6:30 AM, Mehdi Amini <mehdi...@apple.com> wrote:
> I’m not sure I understand the question about lld. Lld will be a
> different linker, with its own set of option. Actually we usually
> rely on the clang driver to hide platform specific option and
> provide a common interface to the user.
I was thinking if I force lld, then the -Wl param will be the same
across platforms, and I wouldn't have to accommodate for different
linkers.
> I don’t believe LDFLAGS is a valid cmake flag. You need to define
> both CMAKE_EXE_LINKER_FLAGS and CMAKE_SHARED_LINKER_FLAGS.
It respects it, as it should, or otherwise packagers would have to
replicate CFLAGS, CXXFLAGS, etc. via CMAKE_*_FLAGS in package build
descriptions.
Awanish via llvm-dev
./configure --disable-shared CC="llvm-gcc -flto -use-gold-plugin -Wl,-plugin-opt=also-emit-llvm" CFLAGS="-g" RANLIB="ar --plugin /home/awanish/llvm-2.9/llvm-gcc-4.2-2.9.source/libexec/gcc/x86_64-unknown-linux-gnu/4.2.1/LLVMgold.so -s" AR_FLAGS="--plugin /home/awanish/llvm-2.9/llvm-gcc-4.2-2.9.source/libexec/gcc/x86_64-unknown-linux-gnu/4.2.1/LLVMgold.so -cru"
but I am getting an error which states that
checking for gcc... llvm-gcc -flto -use-gold-plugin -Wl,-plugin-opt=also-emit-llvm
checking whether the C compiler works... no
configure: error: in `/home/awanish/PHD/benchmark/httpd-2.2.16/myBuild/srclib/apr':
configure: error: C compiler cannot create executables
I got reference for configuring like this from "https://dslabredmine.epfl.ch/embedded/cloud9/user/CompilingLLVM.html".
Can anyone please tell me where I am doing wrong and what is correct procedure for generating .bc for for httpd which can be run on klee?
-- Thanks and Regards Awanish Pandey PhD, CSE IIT Kanpur
Xinliang David Li via llvm-dev
On Sun, Sep 18, 2016 at 5:45 AM, Xinliang David Li <xinli...@gmail.com> wrote:
> As Mehdi mentioned, thinLTO backend processes use very little memory, you
> may get away without any additional flags (neither -Wl,--plugin-opt=jobs=..,
> nor -Dxxx for cmake to limit link parallesm) if your build machine has
> enough memory. Here is some build time data of parallel linking (with
> ThinLTO) 52 binaries in clang build (linking parallelism equals ninja
> parallelism). The machine has 32 logical cores and 64GB memory.
>
> 1) Using the default ninja parallelism, the peak 1min load-average is 537.
> The total elapse time is 9m43s
> 2) Using ninja -j16, the peak load is 411. The elapse time is 8m26s
> 3) ninja -j8 : elapse time is 8m34s
> 4) ninja -j4 : elapse time is 8m50s
> 5) ninja -j2 : elapse time is 9m54s
> 6) ninja -j1 : elapse time is 12m3s
>
> As you can see, doing serial thinLTO linking across multiple binaries do not
> give you the best performance. The build performance peaked at j16 in this
> configuration. You may need to find your best LLVM_PARALLEL_LINK_JOBS
> value.
What did you set LLVM_PARALLEL_LINK_JOBS to?
Maybe I should first try to leave it unset and see if it fits within
my machine's
hardware limits.
Michael Kruse via llvm-dev
the problem on my system with this error message:
/usr/bin/ld: /usr/lib/llvm-3.8/bin/../lib/LLVMgold.so: error loading
plugin: /usr/lib/llvm-3.8/bin/../lib/LLVMgold.so: cannot open shared
object file: No such file or directory
Sure, that file doesn't exist. I'd expect Ubuntu 16.04 to have
installed it for the installed version of clang, but `apt-file seach
LLVMgold` returns:
llvm-3.5-dev: /usr/lib/llvm-3.5/lib/LLVMgold.so
llvm-3.6-dev: /usr/lib/llvm-3.6/lib/LLVMgold.so
llvm-3.7-dev: /usr/lib/llvm-3.7/lib/LLVMgold.so
llvm-dev: /usr/lib/LLVMgold.so
/usr/lib/LLVMgold.so is a symlink to llvm-3.8/lib/LLVMgold.so, which
doesn't exist. This looks like a mistake in the llvm-3.8-dev package.
This bug might be related:
https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-snapshot/+bug/1254970
The tests use CMake's standard mechanism check_c{xx}_source_compiles()
to check whether a source file compiles and links. I assume it is just
the first project in the configure chain to make use of it.
Michael
2016-09-16 23:18 GMT+02:00 Carsten Mattner via llvm-dev
<llvm...@lists.llvm.org>:
> On Fri, Sep 16, 2016 at 10:54 PM, Mehdi Amini <mehdi...@apple.com> wrote:
>
>> Can you look for the specific errors in the CMake log and error files?
>>
>> CMake is very good at not presenting the real issue…
>> Here it is trying to build very simple programs to check features
>> availability. Like:
>>
>> #include <strings.h>
>> int main() { ffs(0); return 0; }
>
> In CMakeError.log there are errors like:
>
> CheckIncludeFile.c:1:10: fatal error: 'malloc/malloc.h' file not found
> #include <malloc/malloc.h>
>
> ...
>
> CheckIncludeFile.c:1:10: fatal error: 'ndir.h' file not found
> #include <ndir.h>
>
> ...
>
> CheckIncludeFile.c:1:10: fatal error: 'sys/ndir.h' file not found
> #include <sys/ndir.h>
>
> ...
>
> fatal error: 'intrin.h' file not found
>
> These seem like tests that are expected to fail.
>
> ...
>
> The interesting test
>
> int main() { __builtin_ffs(0); return 0; }
>
> fails like this:
>
> /usr/bin/ld[...]llvm/bin/../lib/LLVMgold.so:
> cannot open shared object file: No such file[...]
>
> I have two questions here:
>
> Is there anything I might have missed when configuring the gcc-built
> llvm install for it to be incomplete in the linker department? I mean,
> the system linker is used (not lld) and that via CC=clang is looking
> for the gold linker plugin, which seems to be missing. Probably
> something got lost in my transition from autoconf to cmake?
>
> Anything else to pass to cmake so that it, in addition to llvm-ar and
> llvm-ranlib, also used lld? Or isn't that a requirement for building
> with ThinLTO?
Mehdi Amini via llvm-dev
—
Mehid
Carsten Mattner via llvm-dev
The results so far are negative, so I must have done something wrong.
$ export CXX=clang CC=clang
$ cmake \
-G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_BINUTILS_INCDIR=/usr/include \
-DCMAKE_INSTALL_PREFIX=$PREFIX \
-DLLVM_TARGETS_TO_BUILD="X86" \
-DLLVM_ENABLE_BACKTRACES=OFF \
-DLLVM_BUILD_EXAMPLES=OFF \
-DLLVM_INCLUDE_EXAMPLES=OFF \
-DLLVM_BUILD_TESTS=OFF \
-DLLVM_INCLUDE_TESTS=OFF \
-DLLVM_BUILD_DOCS=OFF \
-DLLVM_INCLUDE_DOCS=OFF \
-DLLVM_ENABLE_DOXYGEN=OFF \
-DLLVM_ENABLE_SPHINX=OFF \
-DLLDB_DISABLE_PYTHON=ON \
-DCLANG_PLUGIN_SUPPORT=OFF \
-DLLVM_ENABLE_LTO=Thin \
-DCMAKE_AR=$PREFIX/bin/llvm-ar \
-DCMAKE_RANLIB=$PREFIX/bin/llvm-ranlib
$ ninja -j4
/usr/bin/ld: error: Could not create temporary file: Too many open files
clang-3.9: error: unable to execute command: Segmentation fault (core dumped)
clang-3.9: error: linker command failed due to signal (use -v to see invocation)
[1289/3332] Linking CXX shared module lib/LLVMgold.so
$ ninja -j2
/usr/bin/ld: error: Could not create temporary file: Too many open files
clang-3.9: error: unable to execute command: Segmentation fault (core dumped)
clang-3.9: error: linker command failed due to signal (use -v to see invocation)
[1289/3332] Linking CXX shared module lib/LLVMgold.so
$ ninja -j1
/usr/bin/ld: error: Could not create temporary file: Too many open files
clang-3.9: error: unable to execute command: Segmentation fault (core dumped)
clang-3.9: error: linker command failed due to signal (use -v to see invocation)
ninja: build stopped: subcommand failed.
Command exited with non-zero status 1
$ ls -l /tmp/lto*|wc -l
1310
Carsten Mattner via llvm-dev
<carsten...@gmail.com> wrote:
> I finally got around to trying to build with LTO=Thin as discussed earlier.
>
> The results so far are negative, so I must have done something wrong.
>
> $ export CXX=clang CC=clang
Typo: I did of course CXX=clang++.
Carsten Mattner via llvm-dev
$ ls -l /tmp/lto*|wc -l
$ ls -1 /tmp/lto*|wc -l
Not that it matters in this case.
Teresa Johnson via llvm-dev
clang-3.9: error: unable to execute command: Segmentation fault (core dumped)
clang-3.9: error: linker command failed due to signal (use -v to see invocation)
[1289/3332] Linking CXX shared module lib/LLVMgold.so
$ ninja -j2
/usr/bin/ld: error: Could not create temporary file: Too many open files
clang-3.9: error: unable to execute command: Segmentation fault (core dumped)
clang-3.9: error: linker command failed due to signal (use -v to see invocation)
[1289/3332] Linking CXX shared module lib/LLVMgold.so
$ ninja -j1
/usr/bin/ld: error: Could not create temporary file: Too many open files
clang-3.9: error: unable to execute command: Segmentation fault (core dumped)
clang-3.9: error: linker command failed due to signal (use -v to see invocation)
ninja: build stopped: subcommand failed.
Command exited with non-zero status 1
$ ls -l /tmp/lto*|wc -l
1310
Carsten Mattner via llvm-dev
>
> ThinLTO needs to create as many temporary files as there are input
> modules to the link. From your 'ls' below, it doesn't look like there is
> an incredibly huge amount already created, can you check your limits?
> What is the output of:
>
> $ ulimit -n
>
> and
>
> $ sysctl fs.file-max
Sorry for the noise, forgot to verify because I had assumed it was
still set to 4096 from a similar change a couple days ago.
Alas, it's just 1024 in this session. Given the # of object files,
2048 looks more than enough. Going to re-run with that.
Teresa Johnson via llvm-dev
No worries, thanks for the update. Teresa
Carsten Mattner via llvm-dev
> No worries, thanks for the update. Teresa
2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left.
Once I've been successful with this, I might try building a ThinLTO Firefox
or maybe Chromium, although browsers usually have PGO profiles one
can run for feeding back real-world exercise. Not sure if PGO can be
combined with LTO, or if it makes sense at all...
What would be more fun is combining ThinLTO for C code with MLton
for SML code when building Ur/Web.
Teresa Johnson via llvm-dev
On Mon, Sep 26, 2016 at 4:25 PM, Teresa Johnson <tejo...@google.com> wrote:
> No worries, thanks for the update. Teresa
2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left.
Once I've been successful with this, I might try building a ThinLTO Firefox
or maybe Chromium,
although browsers usually have PGO profiles one
can run for feeding back real-world exercise. Not sure if PGO can be
combined with LTO, or if it makes sense at all...
What would be more fun is combining ThinLTO for C code with MLton
for SML code when building Ur/Web.
> On Mon, Sep 26, 2016, 7:16 AM Carsten Mattner <carsten...@gmail.com>
> wrote:
>>
>> On Mon, Sep 26, 2016 at 3:52 PM, Teresa Johnson <tejo...@google.com>
>> wrote:
>> >
>> > ThinLTO needs to create as many temporary files as there are input
>> > modules to the link. From your 'ls' below, it doesn't look like there is
>> > an incredibly huge amount already created, can you check your limits?
>> > What is the output of:
>> >
>> > $ ulimit -n
>> >
>> > and
>> >
>> > $ sysctl fs.file-max
>>
>> Sorry for the noise, forgot to verify because I had assumed it was
>> still set to 4096 from a similar change a couple days ago.
>> Alas, it's just 1024 in this session. Given the # of object files,
>> 2048 looks more than enough. Going to re-run with that.
Carsten Mattner via llvm-dev
...
[1223/1315] Linking CXX executable bin/lldb-3.9.1
FAILED: bin/lldb-3.9.1
[long list of object files as part of clang++ invocation command]
/usr/bin/ld: bin/lldb-3.9.1: hidden symbol `__morestack' in
/usr/lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o)
is referenced by DSO
/usr/bin/ld: final link failed: Bad value
...
27428.01user 218.31system 1:59:40elapsed 385%CPU (0avgtext+0avgdata
980356maxresident)k
931232inputs+4172128outputs (1390major+36787899minor)pagefaults 0swaps
Failed to link lldb, while less then 100 ninja targets were left.
Same if I retry with ninja -j1.
Any suggestions?
Carsten Mattner via llvm-dev
>
>
>
> On Mon, Sep 26, 2016 at 8:08 AM, Carsten Mattner <carsten...@gmail.com> wrote:
>>
>> On Mon, Sep 26, 2016 at 4:25 PM, Teresa Johnson <tejo...@google.com> wrote:
>> > No worries, thanks for the update. Teresa
>>
>> 2048 wasn't enough. Bumped to 4096. Only 1300 ninja targets left.
>>
>> Once I've been successful with this, I might try building a ThinLTO Firefox
>> or maybe Chromium,
>
>
> I have built both of those successfully with ThinLTO, so they should work.
Sounds good.
>> although browsers usually have PGO profiles one
>> can run for feeding back real-world exercise. Not sure if PGO can be
>> combined with LTO, or if it makes sense at all...
>
>
> Yes, absolutely! ThinLTO and PGO are largely orthogonal, but PGO will help enable
> more indirect call promotion with ThinLTO, and there is also a patch out for review (D24638)
> to use FDO to guide ThinLTO function importing heuristics.
Also sounds good.
>> What would be more fun is combining ThinLTO for C code with MLton
>> for SML code when building Ur/Web.
>
>
> Interesting. I am not at all familiar with SML code/building, but theoretically switching the C code to
> use -flto=thin shouldn't affect the interaction between those two compiles. Presumably one or both
> are compiled into a library and linked in to create a final binary?
Ur/Web consists of C and SML code, and I'd have to take a deep dive before
I could say how it's linked, but basically yes,the finaly piece is a combination
of object code from C and SML sources. I thought given MLton's whole program
optimizer, ThinLTO might be natural extension of the build process.
Just because :).
Teresa Johnson via llvm-dev
$ /usr/bin/time ninja -j4
...
[1223/1315] Linking CXX executable bin/lldb-3.9.1
FAILED: bin/lldb-3.9.1
[long list of object files as part of clang++ invocation command]
/usr/bin/ld: bin/lldb-3.9.1: hidden symbol `__morestack' in
/usr/lib64/gcc/x86_64-pc-linux-gnu/6.2.1/libgcc.a(morestack.o)
is referenced by DSO
/usr/bin/ld: final link failed: Bad value
...
27428.01user 218.31system 1:59:40elapsed 385%CPU (0avgtext+0avgdata
980356maxresident)k
931232inputs+4172128outputs (1390major+36787899minor)pagefaults 0swaps
Failed to link lldb, while less then 100 ninja targets were left.
Same if I retry with ninja -j1.
Any suggestions?
Carsten Mattner via llvm-dev
> I'll either need to get a reproducer from you and/or try to repro it myself.
Assuming I run into this with BUILD_SHARED_LIBS=OFF, which seems unlikely,
what kind of reproducer data would you need from me?
> I usually don't check out and build lldb, so perhaps I will start with that.
I must say I'm surprised a ThinLTO build of all of LLVM svn isn't part of
the periodic builders.
> As a workaround you could try -DBUILD_SHARED_LIBS=OFF.
Thanks, will do, but probably not before Wednesday because it looks like
this requires a full rebuild, if my interpretation of the cmake warning/error
is correct.
Mehdi Amini via llvm-dev
> On Sep 27, 2016, at 2:18 AM, Carsten Mattner <carsten...@gmail.com> wrote:
>
>> On Mon, Sep 26, 2016 at 11:02 PM, Teresa Johnson <tejo...@google.com> wrote:
>> I'll either need to get a reproducer from you and/or try to repro it myself.
>
> Assuming I run into this with BUILD_SHARED_LIBS=OFF, which seems unlikely,
> what kind of reproducer data would you need from me?
>
>> I usually don't check out and build lldb, so perhaps I will start with that.
>
> I must say I'm surprised a ThinLTO build of all of LLVM svn isn't part of
> the periodic builders.
I have one, it just does include LLDB.
Mehdi
Teresa Johnson via llvm-dev
> On Sep 27, 2016, at 2:18 AM, Carsten Mattner <carsten...@gmail.com> wrote:
>
>> On Mon, Sep 26, 2016 at 11:02 PM, Teresa Johnson <tejo...@google.com> wrote:
>> I'll either need to get a reproducer from you and/or try to repro it myself.
>
> Assuming I run into this with BUILD_SHARED_LIBS=OFF, which seems unlikely,
> what kind of reproducer data would you need from me?
>
>> I usually don't check out and build lldb, so perhaps I will start with that.
>
> I must say I'm surprised a ThinLTO build of all of LLVM svn isn't part of
> the periodic builders.
I have one, it just does include LLDB.
Mehdi
>
>> As a workaround you could try -DBUILD_SHARED_LIBS=OFF.
>
> Thanks, will do, but probably not before Wednesday because it looks like
> this requires a full rebuild, if my interpretation of the cmake warning/error
> is correct.
Mehdi Amini via llvm-dev
On Sep 27, 2016, at 7:00 AM, Teresa Johnson <tejo...@google.com> wrote:On Tue, Sep 27, 2016 at 6:53 AM, Mehdi Amini <mehdi...@apple.com> wrote:
> On Sep 27, 2016, at 2:18 AM, Carsten Mattner <carsten...@gmail.com> wrote:
>
>> On Mon, Sep 26, 2016 at 11:02 PM, Teresa Johnson <tejo...@google.com> wrote:
>> I'll either need to get a reproducer from you and/or try to repro it myself.
>
> Assuming I run into this with BUILD_SHARED_LIBS=OFF, which seems unlikely,
> what kind of reproducer data would you need from me?Let me see if I can reproduce it first. Probably a zip of all the inputs to the link and the link command line. Another option would be to save the temp files, but it would require some work to find the one producing the error and I might need all the inputs anyway...>
>> I usually don't check out and build lldb, so perhaps I will start with that.
>
> I must say I'm surprised a ThinLTO build of all of LLVM svn isn't part of
> the periodic builders.
I have one, it just does include LLDB.Great, I was going to ask how to get one set up. Is it an official build bot?
And I assume you meant it just does "not" include LLDB?
Teresa Johnson via llvm-dev
Carsten Mattner via llvm-dev
>
> I can't reproduce the failure. I am building with a clang built
> Release from recent source as my stage-1 bootstrap compiler:
> clang version 4.0.0 (trunk 282322) (llvm/trunk 282341)
The clang I use was built from the 3.9 release branch:
clang version 3.9.1 (branches/release_39 281767)
Is that too old for it to build itself with ThinLTO?
> I am configuring the stage-2 bootstrap compiler with the following
> command, which largely is copied from your earlier email with your
> cmake command (changes noted below):
>
> lto=Thin
> type=Release
> cflgs=-w
> INSTALL_PREFIX=$HOME/llvm/${build_dir_name}/clang-install
> BUILD_COMPILER=$HOME/llvm/llvm_10_release_build
> SRC=$HOME/llvm/llvm_head
> BINUTILS=$HOME/binutils/binutils
> env DYLD_LIBRARY_PATH=$BUILD_COMPILER/lib/ \
> cmake \
> -G Ninja \
> -DCMAKE_BUILD_TYPE=${type} \
> -DLLVM_BINUTILS_INCDIR=$BINUTILS/include \
[...]
> -DCMAKE_C_FLAGS=${cflgs} \
> -DCMAKE_CXX_FLAGS=${cflgs} \
> -DCMAKE_EXE_LINKER_FLAGS=${linkflgs} \
> $SRC
Is BUILD_COMPILER/bin the result of a previous non-LTO build of trunk?
If any of you have suggestions what to omit from the above flags, I'm
open to suggestions. This is the culmination of trying to replicate
the previous autoconf flags. It's not an exact mapping, but almost.
> The changes from your config:
> 1) Used llvm-ar and llvm-nm from the build compiler. I noticed that
> you are specifying those as coming from the same directory used for
> CMAKE_INSTALL_PREFIX, which seems wrong - that is where the compiler
> built here will be installed.
That's because I'm reusing the previous built 3.9_release (without
LTO) and I move it away after the build before running ninja install
to the same location after extending PATH with the locally moved tree
of the previously installed PREFIX for llvm.
Is that a problem?
> 2) Used my own binutils checkout as I don't have plugin-api.h
> installed in /usr/include
Curious, is your host binutils too old for that to exist in
/usr/include?
> 3) Set BUILD_SHARED_LIBS=ON (which I don't see in your earlier
> config, did you add that later?)
What I emailed earlier was the full set of flags passed to cmake, and
I had just added SHARED_LIBS=OFF as suggested for the next rebuild.
Should I use ON or OFF?
> 4) Set LLVM_PARALLEL_LINK_JOBS=1 (probably not needed)
I'll try this next time, just to see how much of a difference it makes
when I run with with -j4.
> 5) Build with -w, to avoid occasional issues with warnings only
> emitted by clang (probably not needed)
> 6) Specified CMAKE_C_COMPILER and CMAKE_CXX_COMPILER explicitly (you
> might have had the right versions in your path?)
I've exported CXX and CC to clang++ and clang, which happen to be the
locally build 3.9 release branch versions (non-LTO variant).
> I did an nm on the .so libraries to see where __morestack was
> referenced and its linkage. No surprise, it is coming from
> libLLVMRuntimeDyld.so, and I do see it as weak as expected:
>
> $ nm lib/libLLVMRuntimeDyld.so | grep morestack
> w __morestack
>
> Can you do the above nm on your lib/libLLVMRuntimeDyld.so and send
> me the output?
In the unfinished build dir I only have lib/libLLVMRuntimedyld.a.
Teresa Johnson via llvm-dev
On Tue, Sep 27, 2016 at 6:33 PM, Teresa Johnson <tejo...@google.com> wrote:
>
> I can't reproduce the failure. I am building with a clang built
> Release from recent source as my stage-1 bootstrap compiler:
> clang version 4.0.0 (trunk 282322) (llvm/trunk 282341)
The clang I use was built from the 3.9 release branch:
clang version 3.9.1 (branches/release_39 281767)
Is that too old for it to build itself with ThinLTO?
> I am configuring the stage-2 bootstrap compiler with the following
> command, which largely is copied from your earlier email with your
> cmake command (changes noted below):
>
> lto=Thin
> type=Release
> cflgs=-w
> INSTALL_PREFIX=$HOME/llvm/${build_dir_name}/clang-install
> BUILD_COMPILER=$HOME/llvm/llvm_10_release_build
> SRC=$HOME/llvm/llvm_head
> BINUTILS=$HOME/binutils/binutils
> env DYLD_LIBRARY_PATH=$BUILD_COMPILER/lib/ \
> cmake \
> -G Ninja \
> -DCMAKE_BUILD_TYPE=${type} \
> -DLLVM_BINUTILS_INCDIR=$BINUTILS/include \
[...]
> -DCMAKE_C_FLAGS=${cflgs} \
> -DCMAKE_CXX_FLAGS=${cflgs} \
> -DCMAKE_EXE_LINKER_FLAGS=${linkflgs} \
> $SRC
Is BUILD_COMPILER/bin the result of a previous non-LTO build of trunk?
If any of you have suggestions what to omit from the above flags, I'm
open to suggestions. This is the culmination of trying to replicate
the previous autoconf flags. It's not an exact mapping, but almost.
> The changes from your config:
> 1) Used llvm-ar and llvm-nm from the build compiler. I noticed that
> you are specifying those as coming from the same directory used for
> CMAKE_INSTALL_PREFIX, which seems wrong - that is where the compiler
> built here will be installed.
That's because I'm reusing the previous built 3.9_release (without
LTO) and I move it away after the build before running ninja install
to the same location after extending PATH with the locally moved tree
of the previously installed PREFIX for llvm.
Is that a problem?
> 2) Used my own binutils checkout as I don't have plugin-api.h
> installed in /usr/include
Curious, is your host binutils too old for that to exist in
/usr/include?
> 3) Set BUILD_SHARED_LIBS=ON (which I don't see in your earlier
> config, did you add that later?)
What I emailed earlier was the full set of flags passed to cmake, and
I had just added SHARED_LIBS=OFF as suggested for the next rebuild.
Should I use ON or OFF?
> 4) Set LLVM_PARALLEL_LINK_JOBS=1 (probably not needed)
I'll try this next time, just to see how much of a difference it makes
when I run with with -j4.
> 5) Build with -w, to avoid occasional issues with warnings only
> emitted by clang (probably not needed)
> 6) Specified CMAKE_C_COMPILER and CMAKE_CXX_COMPILER explicitly (you
> might have had the right versions in your path?)
I've exported CXX and CC to clang++ and clang, which happen to be the
locally build 3.9 release branch versions (non-LTO variant).
> I did an nm on the .so libraries to see where __morestack was
> referenced and its linkage. No surprise, it is coming from
> libLLVMRuntimeDyld.so, and I do see it as weak as expected:
>
> $ nm lib/libLLVMRuntimeDyld.so | grep morestack
> w __morestack
>
> Can you do the above nm on your lib/libLLVMRuntimeDyld.so and send
> me the output?
In the unfinished build dir I only have lib/libLLVMRuntimedyld.a.
Carsten Mattner via llvm-dev
Just in case it's important, I'm using Arch Linux (and most Linux
distros these days) CFLAGS/CPPFLAGS/LDFLAGS, which are as follows:
$ grep FLAGS /etc/makepkg.conf
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"
This is the same way Fedora or Arch linux would build clang, or any
other usual package for that matter.
The only uncommon thing I set is the macro -D_GLIBCXX_USE_CXX11_ABI=0,
which I use to avoid C++11 ABI chaos since g++5.x, although it's
probably safe to drop by now, especially when CXX=clang++. That's
g++-specific, so is irrelevant to the issue at hand.
> I wonder if there is a bug that was fixed. I have bootstrapped with
> ThinLTO with the 3.9 compiler, but not with LLDB and not with
> SHARED_LIBS=ON. I can try with an older 3.9 version.
As you seem to have faster hardware, I would appreciate a validation
before I embark on another >=3h build endavour.
> Not sure I followed that, but in essence you just want to make sure that
> llvm-ranlib and llvm-ar match the build compiler.
Sorry for the confusing description.
I build llvm and install it locally in my home dir somewhere
($PREFIX). After building llvm, I move that installation out of the
way before installing the new build in a now empty $PREFIX, but not
before prepending the moved-to PREFIX in $PATH, in case I need the old
toolchain during 'ninja install'.
Make more sense?
> I used ON to try to reproduce your issue, the OFF was a suggestion
> to avoid it as a workaround. I'm surprised your build defaults this
> to ON, according to http://llvm.org/docs/CMake.html it defaults to
> OFF.
Hmm, I will explicitly set it to OFF in the next build.
Is it expected for the .so files listed below to be created with
SHARED_LIB=OFF?
> This is from your SHARED_LIBS=OFF build I assume? With the build
> containing the DSO error I'm assuming BUILD_SHARED_LIBS=ON, which
> would mean this should be a .so.
One would expect so, but in fact it's with the flags I posted
yesterday, so I'm equally confused as to why there's no .so, but the
error
These are the DSO file in lib/:
$ ls lib/*.so
lib/BugpointPasses.so
lib/libclang.so
lib/libgomp.so
lib/libiomp5.so
lib/liblldb.so
lib/libLTO.so
lib/libomp.so
lib/LLVMgold.so
lib/LLVMHello.so
lib/LLVMPolly.so
I'll assume, given the existence of the various .so files, that
SHARED_LIBS=ON is the default and what's been configured.
Unless cmake's configure step somehow is responsible for nuking
libLLVMRUntimedyld.so, I don't know if it's supposed to exist yet in
the build plan (failed) so far.
> If this archive is from the build getting the DSO error then I'm
> confused about which DSO is giving the error. In that case what does
> the link line look like?
For easier reading, I've newlined the parameters.
[1/90] Linking CXX executable bin/lldb-3.9.1
FAILED: bin/lldb-3.9.1
: && $PREFIX/bin/clang++
-D_GLIBCXX_USE_CXX11_ABI=0
-O2
-pipe
-fstack-protector-strong
--param=ssp-buffer-size=4
-march=x86-64
-mtune=generic
-fPIC
-fvisibility-inlines-hidden
-Wall
-W
-Wno-unused-parameter
-Wwrite-strings
-Wcast-qual
-Wmissing-field-initializers
-pedantic
-Wno-long-long
-Wcovered-switch-default
-Wnon-virtual-dtor
-Wdelete-non-virtual-dtor
-Werror=date-time
-std=c++11
-fcolor-diagnostics
-ffunction-sections
-fdata-sections
-flto=thin
-Wno-deprecated-declarations
-Wno-unknown-pragmas
-Wno-strict-aliasing
-Wno-deprecated-register
-Wno-vla-extension
-O3
-DNDEBUG
-Wl,-O1,--sort-common,--as-needed,-z,relro,--hash-style=gnu
-flto=thin
-Wl,-allow-shlib-undefined
-Wl,-O3
-Wl,--gc-sections \
tools/lldb/tools/driver/CMakeFiles/lldb.dir/Driver.cpp.o \
tools/lldb/tools/driver/CMakeFiles/lldb.dir/Platform.cpp.o
-o bin/lldb-3.9.1
-lpthread
lib/liblldb.so.3.9.1
-Wl,--start-group
lib/liblldbBase.a
lib/liblldbBreakpoint.a
lib/liblldbCommands.a
lib/liblldbDataFormatters.a
lib/liblldbHost.a
lib/liblldbCore.a
lib/liblldbExpression.a
lib/liblldbInitialization.a
lib/liblldbInterpreter.a
lib/liblldbSymbol.a
lib/liblldbTarget.a
lib/liblldbUtility.a
lib/liblldbPluginDisassemblerLLVM.a
lib/liblldbPluginSymbolFileDWARF.a
lib/liblldbPluginSymbolFilePDB.a
lib/libLLVMDebugInfoPDB.a
lib/libLLVMDebugInfoCodeView.a
lib/liblldbPluginSymbolFileSymtab.a
lib/liblldbPluginDynamicLoaderStatic.a
lib/liblldbPluginDynamicLoaderPosixDYLD.a
lib/liblldbPluginDynamicLoaderHexagonDYLD.a
lib/liblldbPluginDynamicLoaderWindowsDYLD.a
lib/liblldbPluginCPlusPlusLanguage.a
lib/liblldbPluginGoLanguage.a
lib/liblldbPluginJavaLanguage.a
lib/liblldbPluginObjCLanguage.a
lib/liblldbPluginObjCPlusPlusLanguage.a
lib/liblldbPluginObjectFileELF.a
lib/liblldbPluginObjectFileJIT.a
lib/liblldbPluginSymbolVendorELF.a
lib/liblldbPluginObjectContainerBSDArchive.a
lib/liblldbPluginObjectContainerMachOArchive.a
lib/liblldbPluginProcessGDBRemote.a
lib/liblldbPluginProcessUtility.a
lib/liblldbPluginPlatformAndroid.a
lib/liblldbPluginPlatformGDB.a
lib/liblldbPluginPlatformFreeBSD.a
lib/liblldbPluginPlatformKalimba.a
lib/liblldbPluginPlatformLinux.a
lib/liblldbPluginPlatformNetBSD.a
lib/liblldbPluginPlatformPOSIX.a
lib/liblldbPluginPlatformWindows.a
lib/liblldbPluginPlatformMacOSX.a
lib/liblldbPluginDynamicLoaderMacOSXDYLD.a
lib/liblldbPluginUnwindAssemblyInstEmulation.a
lib/liblldbPluginUnwindAssemblyX86.a
lib/liblldbPluginAppleObjCRuntime.a
lib/liblldbPluginRenderScriptRuntime.a
lib/liblldbPluginLanguageRuntimeGo.a
lib/liblldbPluginLanguageRuntimeJava.a
lib/liblldbPluginCXXItaniumABI.a
lib/liblldbPluginABIMacOSX_arm.a
lib/liblldbPluginABIMacOSX_arm64.a
lib/liblldbPluginABIMacOSX_i386.a
lib/liblldbPluginABISysV_arm.a
lib/liblldbPluginABISysV_arm64.a
lib/liblldbPluginABISysV_i386.a
lib/liblldbPluginABISysV_x86_64.a
lib/liblldbPluginABISysV_hexagon.a
lib/liblldbPluginABISysV_ppc.a
lib/liblldbPluginABISysV_ppc64.a
lib/liblldbPluginABISysV_mips.a
lib/liblldbPluginABISysV_mips64.a
lib/liblldbPluginABISysV_s390x.a
lib/liblldbPluginInstructionARM.a
lib/liblldbPluginInstructionARM64.a
lib/liblldbPluginInstructionMIPS.a
lib/liblldbPluginInstructionMIPS64.a
lib/liblldbPluginObjectFilePECOFF.a
lib/liblldbPluginOSGo.a
lib/liblldbPluginOSPython.a
lib/liblldbPluginMemoryHistoryASan.a
lib/liblldbPluginInstrumentationRuntimeAddressSanitizer.a
lib/liblldbPluginInstrumentationRuntimeThreadSanitizer.a
lib/liblldbPluginSystemRuntimeMacOSX.a
lib/liblldbPluginProcessElfCore.a
lib/liblldbPluginJITLoaderGDB.a
lib/liblldbPluginExpressionParserClang.a
lib/liblldbPluginExpressionParserGo.a
lib/liblldbPluginProcessLinux.a
lib/liblldbPluginProcessPOSIX.a
-Wl,--end-group
lib/libclangCodeGen.a
lib/libLLVMBitWriter.a
lib/libLLVMCoverage.a
lib/libLLVMipo.a
lib/libLLVMVectorize.a
lib/libLLVMIRReader.a
lib/libLLVMAsmParser.a
lib/libLLVMInstrumentation.a
lib/libLLVMLinker.a
lib/libLLVMObjCARCOpts.a
lib/libLLVMObject.a
lib/libLLVMScalarOpts.a
lib/libLLVMInstCombine.a
lib/libLLVMTarget.a
lib/libLLVMTransformUtils.a
lib/libLLVMAnalysis.a
lib/libclangRewriteFrontend.a
lib/libclangFrontend.a
lib/libclangDriver.a
lib/libclangParse.a
lib/libLLVMMCParser.a
lib/libLLVMProfileData.a
lib/libLLVMOption.a
lib/libclangRewrite.a
lib/libclangSerialization.a
lib/libclangSema.a
lib/libclangAnalysis.a
lib/libclangEdit.a
lib/libclangAST.a
lib/libclangLex.a
lib/libclangBasic.a
lib/libLLVMMC.a
lib/libLLVMBitReader.a
lib/libLLVMCore.a
lib/libLLVMSupport.a
-lrt
-ldl
-lcurses
-lpthread
-lz
-lm
-Wl,-rpath,"\$ORIGIN/../lib
Teresa Johnson via llvm-dev
On Tue, Sep 27, 2016 at 8:37 PM, Teresa Johnson <tejo...@google.com> wrote:
Just in case it's important, I'm using Arch Linux (and most Linux
distros these days) CFLAGS/CPPFLAGS/LDFLAGS, which are as follows:
$ grep FLAGS /etc/makepkg.conf
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"
This is the same way Fedora or Arch linux would build clang, or any
other usual package for that matter.
The only uncommon thing I set is the macro -D_GLIBCXX_USE_CXX11_ABI=0,
which I use to avoid C++11 ABI chaos since g++5.x, although it's
probably safe to drop by now, especially when CXX=clang++. That's
g++-specific, so is irrelevant to the issue at hand.
> I wonder if there is a bug that was fixed. I have bootstrapped with
> ThinLTO with the 3.9 compiler, but not with LLDB and not with
> SHARED_LIBS=ON. I can try with an older 3.9 version.
As you seem to have faster hardware, I would appreciate a validation
before I embark on another >=3h build endavour.
> Not sure I followed that, but in essence you just want to make sure that
> llvm-ranlib and llvm-ar match the build compiler.
Sorry for the confusing description.
I build llvm and install it locally in my home dir somewhere
($PREFIX). After building llvm, I move that installation out of the
way before installing the new build in a now empty $PREFIX, but not
before prepending the moved-to PREFIX in $PATH, in case I need the old
toolchain during 'ninja install'.
Make more sense?
> I used ON to try to reproduce your issue, the OFF was a suggestion
> to avoid it as a workaround. I'm surprised your build defaults this
> to ON, according to http://llvm.org/docs/CMake.html it defaults to
> OFF.
Hmm, I will explicitly set it to OFF in the next build.
Is it expected for the .so files listed below to be created with
SHARED_LIB=OFF?
Carsten Mattner via llvm-dev
> Sure, I will try this and let you know. Unfortunately, though, I
> have another big work commitment that is going to eat up most of my
> time through Thu, although I may be able to find some time to try
> it.
No worries, if I get around it before you do, I will :).
> I think so - what is confusing me is that your cmake command is
> specifying the same $PREFIX for both the install destination and the
> source of the llvm-ar/llvm-ranlib that will be used in the new
> build. Should the llvm-ar/llvm-ranlib in the cmake command use your
> moved-to PREFIX instead?
Only if 'ninja install` would reference the two executables. If that's
case, and I haven't gotten that far yet due to ThinLTO not finishing,
I'll consider copying those to the local build tree first.
> Can you nm the .so files in your lib dir and see if any reference
> __morestack?
$ for so in lib/*.so; do echo $so ; nm $so|grep __morestack;done
lib/BugpointPasses.so
lib/libclang.so
lib/libgomp.so
lib/libiomp5.so
lib/liblldb.so
U __morestack
lib/libLTO.so
lib/libomp.so
lib/LLVMgold.so
lib/LLVMHello.so
lib/LLVMPolly.so
Looks like liblldb.so does. Does that explain anything?
Teresa Johnson via llvm-dev
Xinliang David Li via llvm-dev
Teresa Johnson via llvm-dev
Teresa Johnson via llvm-dev
I just built a stage-1 compiler from the 3.9 release bits and built the lldb from head sources which worked fine. Let me try again using 3.9 build compiler to build 3.9 bits. Teresa
Mehdi Amini via llvm-dev
On Sep 30, 2016, at 9:55 AM, Teresa Johnson <tejo...@google.com> wrote:On Fri, Sep 30, 2016 at 9:35 AM, Teresa Johnson <tejo...@google.com> wrote:I just built a stage-1 compiler from the 3.9 release bits and built the lldb from head sources which worked fine. Let me try again using 3.9 build compiler to build 3.9 bits. TeresaNevermind - it looks like I didn't actually get the 3.9 bits. Anyone know how to get 3.9 via git or git-svn? Typically I use git svn like so to get trunk bits:
Teresa Johnson via llvm-dev
That worked, let me retry everything. Thanks!
Carsten Mattner via llvm-dev
> I just built a stage-1 compiler from the 3.9 release bits and built
> the lldb from head sources which worked fine. Let me try again using
> 3.9 build compiler to build 3.9 bits.
I had also started a build a few hours ago, what a coincidence. I had
to ditch both build and source trees because of cmake failing to
reconfigure due to some file it surprisingly had generated in the
source (not build) tree.
So, with -DBUILD_SHARED_LIBS=OFF, and a fresh checkout of the 3.9
release branch, I've run into the same error, this time only 39 (not a
typo, and not 3.9) ninja targets left.
Some parts finished, so here's a surprising size difference for clang-3.9.
LTO=OFF stripped clang-3.9 is 44MB
LTO=Thin stripped clang-3.9 is 50MB
I would have expected more aggressive pruning or at least not an
increase, though I don't know if it can be caused by
-DBUILD_SHARED_LIBS=OFF.
FYI, -DLLVM_PARALLEL_LINK_JOBS=1 didn't seem to make any difference,
seeing how each LD stop was using all available cores.
Mehdi Amini via llvm-dev
> On Sep 30, 2016, at 12:57 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Fri, Sep 30, 2016 at 6:35 PM, Teresa Johnson <tejo...@google.com> wrote:
>> I just built a stage-1 compiler from the 3.9 release bits and built
>> the lldb from head sources which worked fine. Let me try again using
>> 3.9 build compiler to build 3.9 bits.
>
> I had also started a build a few hours ago, what a coincidence. I had
> to ditch both build and source trees because of cmake failing to
> reconfigure due to some file it surprisingly had generated in the
> source (not build) tree.
>
> So, with -DBUILD_SHARED_LIBS=OFF, and a fresh checkout of the 3.9
> release branch, I've run into the same error, this time only 39 (not a
> typo, and not 3.9) ninja targets left.
>
> Some parts finished, so here's a surprising size difference for clang-3.9.
>
> LTO=OFF stripped clang-3.9 is 44MB
> LTO=Thin stripped clang-3.9 is 50MB
That’s expected: there’s more inlining and other optimizations happening.
Also we rely more on linker dead stripping with ThinLTO. On MacOS it works well with the atom model, on ELF I’d expect the LTO equivalent of -ffunction-sections to be used (I don’t know if Gold and the gold-plugin have such an option though).
Finally, there are few other things that we need to do in ThinLTO to gain a few more percents on this aspect.
>
> I would have expected more aggressive pruning or at least not an
> increase, though I don't know if it can be caused by
> -DBUILD_SHARED_LIBS=OFF.
BUILD_SHARED_LIBS is a non-sense from a performance point of view. It is really not advised to use this.
>
> FYI, -DLLVM_PARALLEL_LINK_JOBS=1 didn't seem to make any difference,
> seeing how each LD stop was using all available cores.
It only helps limiting oversubscribing the number of cores, and reduces the peak memory for the link.
—
Mehdi
Teresa Johnson via llvm-dev
> On Sep 30, 2016, at 12:57 PM, Carsten Mattner <carsten...@gmail.com> wrote:
>
> On Fri, Sep 30, 2016 at 6:35 PM, Teresa Johnson <tejo...@google.com> wrote:
>> I just built a stage-1 compiler from the 3.9 release bits and built
>> the lldb from head sources which worked fine. Let me try again using
>> 3.9 build compiler to build 3.9 bits.
>
> I had also started a build a few hours ago, what a coincidence. I had
> to ditch both build and source trees because of cmake failing to
> reconfigure due to some file it surprisingly had generated in the
> source (not build) tree.
>
> So, with -DBUILD_SHARED_LIBS=OFF, and a fresh checkout of the 3.9
> release branch, I've run into the same error, this time only 39 (not a
> typo, and not 3.9) ninja targets left.
>
> Some parts finished, so here's a surprising size difference for clang-3.9.
>
> LTO=OFF stripped clang-3.9 is 44MB
> LTO=Thin stripped clang-3.9 is 50MB
That’s expected: there’s more inlining and other optimizations happening.
Also we rely more on linker dead stripping with ThinLTO. On MacOS it works well with the atom model, on ELF I’d expect the LTO equivalent of -ffunction-sections to be used (I don’t know if Gold and the gold-plugin have such an option though).
Finally, there are few other things that we need to do in ThinLTO to gain a few more percents on this aspect.
>
> I would have expected more aggressive pruning or at least not an
> increase, though I don't know if it can be caused by
> -DBUILD_SHARED_LIBS=OFF.
BUILD_SHARED_LIBS is a non-sense from a performance point of view. It is really not advised to use this.
>
> FYI, -DLLVM_PARALLEL_LINK_JOBS=1 didn't seem to make any difference,
> seeing how each LD stop was using all available cores.
It only helps limiting oversubscribing the number of cores, and reduces the peak memory for the link.
—
Mehdi
Carsten Mattner via llvm-dev
>
>
> On Fri, Sep 30, 2016 at 1:11 PM, Mehdi Amini <mehdi...@apple.com> wrote:
> >
> >
> > > On Sep 30, 2016, at 12:57 PM, Carsten Mattner <carsten...@gmail.com>
> > > wrote:
> > >
> > > On Fri, Sep 30, 2016 at 6:35 PM, Teresa Johnson <tejo...@google.com>
> > > wrote:
> > > > I just built a stage-1 compiler from the 3.9 release bits and
> > > > built the lldb from head sources which worked fine. Let me try
> > > > again using 3.9 build compiler to build 3.9 bits.
> > >
> > > I had also started a build a few hours ago, what a coincidence.
> > > I had to ditch both build and source trees because of cmake
> > > failing to reconfigure due to some file it surprisingly had
> > > generated in the source (not build) tree.
> > >
> > > So, with -DBUILD_SHARED_LIBS=OFF, and a fresh checkout of the
> > > 3.9 release branch, I've run into the same error, this time only
> > > 39 (not a typo, and not 3.9) ninja targets left.
> > >
> > > Some parts finished, so here's a surprising size difference for
> > > clang-3.9.
> > >
> > > LTO=OFF stripped clang-3.9 is 44MB
> > > LTO=Thin stripped clang-3.9 is 50MB
> >
> > That’s expected: there’s more inlining and other optimizations happening.
Right, though usually the chance to do complete whole program optimization
provides enough opportunities for it to go the other way. Of course, it can
also increase.
> > Also we rely more on linker dead stripping with ThinLTO. On MacOS
> > it works well with the atom model, on ELF I’d expect the LTO
> > equivalent of -ffunction-sections to be used (I don’t know if Gold
> > and the gold-plugin have such an option though).
>
>
> It does, but I found you have to pass the options to the plugin to get them
> to kick in for any *LTO compile. See D24644 for my proposed fix. To
> workaround, pass them to the plugin-opt:
> -Wl,--gc-sections -Wl,-plugin-opt,-function-sections
> -Wl,-plugin-opt,-data-sections
> Not sure if there is any benefit to passing them also on the command line
> (-ffunction-sections -fdata-sections), but probably a good idea.
I'll extend LDFLAGS with that before running cmake the next time.
Thank you.
> > Finally, there are few other things that we need to do in ThinLTO
> > to gain a few more percents on this aspect.
> >
> > > I would have expected more aggressive pruning or at least not an
> > > increase, though I don't know if it can be caused by
> > > -DBUILD_SHARED_LIBS=OFF.
> >
> > BUILD_SHARED_LIBS is a non-sense from a performance point of view.
> > It is really not advised to use this.
>
>
> I actually think Carsten was not getting BUILD_SHARED_LIBS=ON as I
> had originally thought given the DSO error: His cmake command didn't
> contain it, and from his email earlier this week, the __morestack
> reference is coming from lib/liblldb.so, not an llvm library built
> shared as I had assumed. Which also explains why he is still getting
> it when configuring with -DBUILD_SHARED_LIBS=OFF.
Correct.
> I am having a bunch of trouble getting a release-built 3.9 compiler,
> lots of warnings when building with clang that are being turned into
> errors from -Werror. Need to try again with -w...
Did it ever build warning-free, or why have you enabled -Werror? I
mean, I see warnings here, so it would fail if promoted to errors.
Teresa Johnson via llvm-dev
Teresa Johnson via llvm-dev
> > Finally, there are few other things that we need to do in ThinLTO
> > to gain a few more percents on this aspect.
> >
> > > I would have expected more aggressive pruning or at least not an
> > > increase, though I don't know if it can be caused by
> > > -DBUILD_SHARED_LIBS=OFF.
> >
> > BUILD_SHARED_LIBS is a non-sense from a performance point of view.
> > It is really not advised to use this.
>
>
> I actually think Carsten was not getting BUILD_SHARED_LIBS=ON as I
> had originally thought given the DSO error: His cmake command didn't
> contain it, and from his email earlier this week, the __morestack
> reference is coming from lib/liblldb.so, not an llvm library built
> shared as I had assumed. Which also explains why he is still getting
> it when configuring with -DBUILD_SHARED_LIBS=OFF.
Correct.
> I am having a bunch of trouble getting a release-built 3.9 compiler,
> lots of warnings when building with clang that are being turned into
> errors from -Werror. Need to try again with -w...
Did it ever build warning-free, or why have you enabled -Werror? I
mean, I see warnings here, so it would fail if promoted to errors.
Carsten Mattner via llvm-dev
> I completely forgot to add -Wl,--gc-sections. The *-sections options enables
> gc-sections to work effectively. (edit: oh wait, it looks like you already have
> that one on your link command, so you should be set there!)
>
> Unfortunately, I just can't reproduce the DSO link error you are getting.
> I've tried building head bits with a 3.9 compiler, and 3.9 bits with the
> 3.9 compiler, to no avail.
Thanks for looking into this. I'm chasing a more urgent bug this weekend,
so may not get around to another test for a day or two.
> Can you zip up all of the inputs to your link and make it available to me?
With my upload bandwidth it would be very impractical, sorry.
How about I share my scripts I use to checkout, configure and build?
Should I try building trunk in ThinLTO mode using 3.9? I mean, is trunk
considered stable enough for building and later use as occasional
CC/CXX? I might try the build regardles, just to check...
Carsten Mattner via llvm-dev
Carsten Mattner via llvm-dev
clang -O2 -pipe -fstack-protector-strong
--param=ssp-buffer-size=4 -march=x86-64 -mtune=generic
-Wl,-O1,--sort-common,--as-needed,-z,relro,--hash-style=gnu
-Wl,--gc-sections -Wl,-plugin-opt,-function-sections
-Wl,-plugin-opt,-data-sections CMakeFiles/cmTC_358cb.dir/testCCompiler.c.o
-o cmTC_358cb && :
/usr/bin/ld: bad -plugin-opt option
clang-3.9: error: linker command failed with exit code 1 (use -v to see
invocation)
ninja: build stopped: subcommand failed.
Xinliang David Li via llvm-dev
Carsten Mattner via llvm-dev
> -plugin-opt is a gold linker option. Turn on gold linker with -fuse-ld=gold
Thanks, I'll add that to CFLAGS and CXXFLAGS on next try, although it seems
to be gcc-specific.
Xinliang David Li via llvm-dev
Teresa Johnson via llvm-dev
On Sat, Oct 1, 2016 at 4:40 PM, Teresa Johnson <tejo...@google.com> wrote:
> I completely forgot to add -Wl,--gc-sections. The *-sections options enables
> gc-sections to work effectively. (edit: oh wait, it looks like you already have
> that one on your link command, so you should be set there!)
>
> Unfortunately, I just can't reproduce the DSO link error you are getting.
> I've tried building head bits with a 3.9 compiler, and 3.9 bits with the
> 3.9 compiler, to no avail.
Thanks for looking into this. I'm chasing a more urgent bug this weekend,
so may not get around to another test for a day or two.
> Can you zip up all of the inputs to your link and make it available to me?
With my upload bandwidth it would be very impractical, sorry.
How about I share my scripts I use to checkout, configure and build?
Should I try building trunk in ThinLTO mode using 3.9? I mean, is trunk
considered stable enough for building and later use as occasional
CC/CXX? I might try the build regardles, just to check...
Teresa Johnson via llvm-dev
-plugin-opt is a gold linker option. Turn on gold linker with -fuse-ld=gold
Carsten Mattner via llvm-dev
> But something is odd then - if the builds aren't using gold then I don't
> see how ThinLTO could have been kicking in?
I'm not an expert on LLVM's build process, but it's important to note that the
failure was immediately upon trying to run cmake with the extended LDFLAGS,
which now includes the extra flags to enable more pruning.
My understaning was that ld.gold is required for the plugin to do ThinLTO,
and given my observation before/after enabling llvm-ar/llvm-ranlib to make
LTO work, and how long the ld processes took for executables and DSOs,
I'm under the impression that ld.gold was actually used.
Carsten Mattner via llvm-dev
>
> On Sat, Oct 1, 2016 at 11:43 AM, Carsten Mattner <carsten...@gmail.com> wrote:
> >
> > On Sat, Oct 1, 2016 at 4:40 PM, Teresa Johnson <tejo...@google.com> wrote:
> >
> > > I completely forgot to add -Wl,--gc-sections. The *-sections
> > > options enables gc-sections to work effectively. (edit: oh wait,
> > > it looks like you already have that one on your link command, so
> > > you should be set there!)
> > >
> > > Unfortunately, I just can't reproduce the DSO link error you are
> > > getting. I've tried building head bits with a 3.9 compiler, and
> > > 3.9 bits with the 3.9 compiler, to no avail.
> >
> > Thanks for looking into this. I'm chasing a more urgent bug this
> > weekend, so may not get around to another test for a day or two.
>
> Oh sure.
The chase ended successfully yesterday with a correction now in wider
testing. No urgent matter this Sunday.
> > > Can you zip up all of the inputs to your link and make it available to me?
> >
> > With my upload bandwidth it would be very impractical, sorry.
> > How about I share my scripts I use to checkout, configure and build?
>
>
> Ok
Will try to get to get today.
> > Should I try building trunk in ThinLTO mode using 3.9? I mean, is
> > trunk considered stable enough for building and later use as
> > occasional CC/CXX? I might try the build regardles, just to
> > check...
>
> I use trunk, but it depends on how close to the bleeding edge you
> are comfortable with. But like I said, I also tried bootstrapping
> with 3.9 (both trunk as well as 3.9 sources) and couldn't reproduce.
Hmm, so you're saying neither fails for you, right?
Xinliang David Li via llvm-dev
Carsten Mattner via llvm-dev
> bfd linker 2.26 works fine with LLVMgold.so. As I mentioned in a previous
> email, 2.26 ar and ranlib also works fine as long as LLVMgold.so is put in a
> path binutils know about : $(bindir)/../lib/bfd-plugins/
I don't want to put a copy or symlink my user LLVMgold.so in
the system owned /usr.
My system has binutils 2.27.
AFAIU, if I explicitly tell cmake to use llvm-ar/llvm-ranlib, then it will
do the right thing for ThinLTO and make LLVMgold plugin will be
Xinliang David Li via llvm-dev
On Sun, Oct 2, 2016 at 5:45 PM, Xinliang David Li <xinli...@gmail.com> wrote:
> bfd linker 2.26 works fine with LLVMgold.so. As I mentioned in a previous
> email, 2.26 ar and ranlib also works fine as long as LLVMgold.so is put in a
> path binutils know about : $(bindir)/../lib/bfd-plugins/
I don't want to put a copy or symlink my user LLVMgold.so in
the system owned /usr.
My system has binutils 2.27.
AFAIU, if I explicitly tell cmake to use llvm-ar/llvm-ranlib, then it will
do the right thing for ThinLTO and make LLVMgold plugin will be
used.
Teresa Johnson via llvm-dev
Teresa Johnson via llvm-dev
bfd linker 2.26 works fine with LLVMgold.so. As I mentioned in a previous email, 2.26 ar and ranlib also works fine as long as LLVMgold.so is put in a path binutils know about : $(bindir)/../lib/bfd-plugins/
DavidOn Sun, Oct 2, 2016 at 3:59 AM, Carsten Mattner <carsten...@gmail.com> wrote:On Sun, Oct 2, 2016 at 6:52 AM, Teresa Johnson <tejo...@google.com> wrote:
> But something is odd then - if the builds aren't using gold then I don't
> see how ThinLTO could have been kicking in?
I'm not an expert on LLVM's build process, but it's important to note that the
failure was immediately upon trying to run cmake with the extended LDFLAGS,
which now includes the extra flags to enable more pruning.
My understaning was that ld.gold is required for the plugin to do ThinLTO,
and given my observation before/after enabling llvm-ar/llvm-ranlib to make
LTO work, and how long the ld processes took for executables and DSOs,
I'm under the impression that ld.gold was actually used.
Carsten Mattner via llvm-dev
Okay. Do you mind focusing on the 3.9 branch? It's less of a moving target
and lends itself more to figuring out what's failing for me.
As soon as I get around to it, I'll send you my full configure and build
steps as a shell script. Need to streamline it.
Teresa Johnson via llvm-dev
On Mon, Oct 3, 2016 at 3:50 PM, Teresa Johnson <tejo...@google.com> wrote:
>
> On Sun, Oct 2, 2016 at 4:02 AM, Carsten Mattner <carsten...@gmail.com> wrote:
>>
>> On Sun, Oct 2, 2016 at 6:41 AM, Teresa Johnson <tejo...@google.com> wrote:
> >
> > > I use trunk, but it depends on how close to the bleeding edge you
> > > are comfortable with. But like I said, I also tried bootstrapping
> > > with 3.9 (both trunk as well as 3.9 sources) and couldn't reproduce.
>>
>> Hmm, so you're saying neither fails for you, right?
>
> Yes
Okay. Do you mind focusing on the 3.9 branch? It's less of a moving target
and lends itself more to figuring out what's failing for me.
As soon as I get around to it, I'll send you my full configure and build
steps as a shell script. Need to streamline it.
Xinliang David Li via llvm-dev
Teresa Johnson via llvm-dev
Is -fsplit-stack option used anywhere? My wild guess is that with ld.bfd, the thinLTO link for the DSO does not bring in morestack.o from libgcc.a, but the hidden symbol is defined in lldb binary.