[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution
David Demelier via llvm-dev
I'm trying to create a pure LLVM toolchain (that will not depend on GNU
and produce GNU-free code too) on a musl based distribution.
For now, I use gcc to bootstrap and build all LLVM components. I do it
individually because I was running out of space and memory trying to
build all using LLVM_ENABLE_PROJECTS. Also, I don't want to create a
all-in-one package. Then, once I'm able to build program with clang,
I'll rebuild all using clang instead of gcc.
# LLVM
I've built LLVM using the following configuration:
-DCMAKE_BUILD_TYPE=Release
-DBUILD_SHARED_LIBS=On
-DCMAKE_INSTALL_PREFIX=/usr
-DLLVM_HOST_TRIPLE=x86_64-linux-musl
-DLLVM_ENABLE_RTTI=On
# compiler-rt
I've needed to disable xray/sanitizers otherwise it didn't build on musl.
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
-DCOMPILER_RT_BUILD_BUILTINS=On
-DCOMPILER_RT_BUILD_SANITIZERS=Off
-DCOMPILER_RT_BUILD_XRAY=Off
-DCOMPILER_RT_USE_BUILTINS_LIBRARY=On
# libc++
-DBUILD_SHARED_LIBS=On
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
-DLIBCXX_HAS_MUSL_LIBC=On
-DLIBCXX_HAS_GCC_S_LIB=Off
-DLIBCXX_ENABLE_STATIC=Off
# libc++abi
-DBUILD_SHARED_LIBS=On
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
-DLIBCXXABI_ENABLE_STATIC=Off
# libunwind
-DBUILD_SHARED_LIBS=On
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
-DLIBUNWIND_ENABLE_STATIC=Off
-DLIBUNWIND_USE_COMPILER_RT=On
# clang
And finally clang:
-DBUILD_SHARED_LIBS=On
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=/usr
-DCLANG_DEFAULT_CXX_STDLIB=libc++
-DCLANG_DEFAULT_LINKER=lld
-DCLANG_DEFAULT_RTLIB=compiler-rt
-DCLANG_VENDOR="Vanilla Linux"
Everything built fine, now the problem is the location of compiler-rt
libraries which are not the correct place where clang/ld.lld searches.
As the following output shows:
clang -v /tmp/test.c
Vanilla Linux clang version 8.0.0 (tags/RELEASE_800/final) (based on
LLVM 8.0.0)
Target: x86_64-unknown-linux-musl
Thread model: posix
InstalledDir: /usr/bin
"/usr/bin/clang-8" -cc1 -triple x86_64-unknown-linux-musl -emit-obj
-mrelax-all -disable-free -disable-llvm-verifier -discard-value-names
-main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie
-mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose
-mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu
x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir
/usr/lib/clang/8.0.0 -internal-isystem /usr/local/include
-internal-isystem /usr/lib/clang/8.0.0/include -internal-externc-isystem
/include -internal-externc-isystem /usr/include -fdebug-compilation-dir
/usr/src/vanilla -ferror-limit 19 -fmessage-length 184
-fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/test-1d99ce.o -x c
/tmp/test.c -faddrsig
clang -cc1 version 8.0.0 based upon LLVM 8.0.0 default target
x86_64-linux-musl
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/usr/lib/clang/8.0.0/include
/usr/include
End of search list.
"/usr/bin/ld.lld" -pie --eh-frame-hdr -m elf_x86_64 -dynamic-linker
/lib/ld-musl-x86_64.so.1 -o a.out /usr/bin/../lib/Scrt1.o
/usr/bin/../lib/crti.o crtbeginS.o -L/usr/bin/../lib -L/lib -L/usr/lib
/tmp/test-1d99ce.o
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lc
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a crtendS.o
/usr/bin/../lib/crtn.o
ld.lld: error: cannot open crtbeginS.o: No such file or directory
ld.lld: error: cannot open
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such
file or directory
ld.lld: error: cannot open
/usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such
file or directory
ld.lld: error: cannot open crtendS.o: No such file or directory
clang-8: error: linker command failed with exit code 1 (use -v to see
invocation)
compiler-rt installed files under the following directories hierarchy
instead:
usr/lib/linux/libclang_rt.builtins-x86_64.a
usr/lib/linux/libclang_rt.fuzzer-x86_64.a
usr/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a
usr/lib/linux/libclang_rt.profile-x86_64.a
What did I misconfigured? Is it clang/lld or compiler-rt? Also do I need
to do something else regarding crtbeginS.o and crtendS.o that are not
provided by musl?
Regards,
--
David
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Peter Smith via llvm-dev
I don't know much about the specifics of Musl, so I'm responding generally.
As I understand it, clang expects to find the compiler-rt libraries
relative to the resource directory, which you can find out the
location of with clang --print-resource-dir . By default it is
lib/clang/9.0.0 assuming you are building from master. I think that
-DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that
you would either need to take out the CMAKE_INSTALL_PREFIX or change
the location of the resource directory, which I think that you can
alter at build time.
I don't think that there is a way of preventing clang adding
crtbeginS.o and crtendS.o without --nostdlib or --nostartfiles but
using these options will also not add the other crt*.o files that you
may be expecting.
There is a line in lib/Driver/Toolchains/Gnu.cpp which the Linux driver uses:
const bool HasCRTBeginEndFiles =
ToolChain.getTriple().hasEnvironment() ||
(ToolChain.getTriple().getVendor() != llvm::Triple::MipsTechnologies);
It looks like Musl may need to be included there?
Peter
David Demelier via llvm-dev
> Hello David,
>
> I don't know much about the specifics of Musl, so I'm responding generally.
>
> As I understand it, clang expects to find the compiler-rt libraries
> relative to the resource directory, which you can find out the
> location of with clang --print-resource-dir . By default it is
> lib/clang/9.0.0 assuming you are building from master. I think that
> -DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that
> you would either need to take out the CMAKE_INSTALL_PREFIX or change
> the location of the resource directory, which I think that you can
> alter at build time.
>
You're right, I've just checked both packages in Alpine and Arch and
they indeed move the /usr/lib/linux directory under the clang's one.
I've fixed that it worked, now I still have to find the solution
regarding crtbeginS.so and crtendS.so.
Thanks :)
> I don't think that there is a way of preventing clang adding
> crtbeginS.o and crtendS.o without --nostdlib or --nostartfiles but
> using these options will also not add the other crt*.o files that you
> may be expecting.
>
> There is a line in lib/Driver/Toolchains/Gnu.cpp which the Linux driver uses:
> const bool HasCRTBeginEndFiles =
> ToolChain.getTriple().hasEnvironment() ||
> (ToolChain.getTriple().getVendor() != llvm::Triple::MipsTechnologies);
> It looks like Musl may need to be included there?
I'll have a look, I don't know much the lowest part of the toolchain
build process so I don't understand very much what's going on there.
Hopefully someone that has better low-level knowledges and use musl will
be able to guide me for this.
What can I tell is that musl provide those files:
usr/lib/Scrt1.o
usr/lib/crt1.o
usr/lib/crti.o
usr/lib/crtn.o
usr/lib/ld-musl-x86_64.so.1
Petr Hosek via llvm-dev
Le 25/03/2019 à 14:41, Peter Smith a écrit :
> Hello David,
>
> I don't know much about the specifics of Musl, so I'm responding generally.
>
> As I understand it, clang expects to find the compiler-rt libraries
> relative to the resource directory, which you can find out the
> location of with clang --print-resource-dir . By default it is
> lib/clang/9.0.0 assuming you are building from master. I think that
> -DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that
> you would either need to take out the CMAKE_INSTALL_PREFIX or change
> the location of the resource directory, which I think that you can
> alter at build time.
>
You're right, I've just checked both packages in Alpine and Arch and
they indeed move the /usr/lib/linux directory under the clang's one.
I've fixed that it worked, now I still have to find the solution
regarding crtbeginS.so and crtendS.so.
David Demelier via llvm-dev
> On Mon, Mar 25, 2019 at 7:05 AM David Demelier via llvm-dev <
> llvm...@lists.llvm.org> wrote:
>
> > Le 25/03/2019 à 14:41, Peter Smith a écrit :
> > > Hello David,
> > >
> > > I don't know much about the specifics of Musl, so I'm responding
> > generally.
> > >
> > > As I understand it, clang expects to find the compiler-rt libraries
> > > relative to the resource directory, which you can find out the
> > > location of with clang --print-resource-dir . By default it is
> > > lib/clang/9.0.0 assuming you are building from master. I think that
> > > -DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that
> > > you would either need to take out the CMAKE_INSTALL_PREFIX or change
> > > the location of the resource directory, which I think that you can
> > > alter at build time.
> > >
> >
> > You're right, I've just checked both packages in Alpine and Arch and
> > they indeed move the /usr/lib/linux directory under the clang's one.
> > I've fixed that it worked, now I still have to find the solution
> > regarding crtbeginS.so and crtendS.so.
> >
>
> https://reviews.llvm.org/D28791 should address this. We're still discussing
> some related aspects on https://reviews.llvm.org/D59264, but I'm hoping to
> reach a conclusion and land these changes this week.
>
Nice, thanks. I've applied D28791 in compiler-rt 8.0.0 without problems. Howerver D59264 does not apply correctly on clang 8.0.0. Will there be an intermediate version before 9.0.0? Otherwise I'll try to update the patch for 8.0.0.
Regards,
--
David Demelier
Andrew Kelley via llvm-dev
> Hello,
>
> I'm trying to create a pure LLVM toolchain (that will not depend on GNU
> and produce GNU-free code too) on a musl based distribution.
>
> # compiler-rt
> # libc++
> # libc++abi
> # libunwind
> # clang
David,
You may be interested to know that Zig (https://ziglang.org/) ships with
such a toolchain already. It cross compiles components (musl, libunwind,
etc) for the target lazily from source, and supports these libc targets:
aarch64_be-linux-gnu
aarch64_be-linux-musl
aarch64-linux-gnu
aarch64-linux-musleabi
armeb-linux-gnueabi
armeb-linux-gnueabihf
armeb-linux-musleabi
armeb-linux-musleabihf
arm-linux-gnueabi
arm-linux-gnueabihf
arm-linux-musleabi
arm-linux-musleabihf
i386-linux-gnu
i386-linux-musl
mips64el-linux-gnuabi64
mips64el-linux-gnuabin32
mips64el-linux-musl
mips64-linux-gnuabi64
mips64-linux-gnuabin32
mips64-linux-musl
mipsel-linux-gnu
mipsel-linux-musl
mips-linux-gnu
mips-linux-musl
powerpc64le-linux-gnu
powerpc64le-linux-musl
powerpc64-linux-gnu
powerpc64-linux-musl
powerpc-linux-gnu
powerpc-linux-musl
riscv32-linux-musl
riscv64-linux-gnu
riscv64-linux-musl
s390x-linux-gnu
s390x-linux-musl
sparc-linux-gnu
sparcv9-linux-gnu
x86_64-linux-gnu
x86_64-linux-gnux32
x86_64-linux-musl
Here's an example of using Zig to compile hello-world.c for
aarch64-linux-gnu:
andy@xps ~/m/z/hello> cat hello.c
#include <stdio.h>
int main(int argc, char **argv) {
printf("what up dawg\n");
return 0;
}
andy@xps ~/m/z/hello> zig build-exe --c-source hello.c --library c
-target aarch64v8-linux-gnu
andy@xps ~/m/z/hello> file hello
hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV),
dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for
GNU/Linux 2.0.0, with debug_info, not stripped
And now for x86_64-linux-musl:
andy@xps ~/m/z/hello> zig build-exe --c-source hello.c --library c
-target x86_64-linux-musl
andy@xps ~/m/z/hello> ./hello
what up dawg
andy@xps ~/m/z/hello> ldd ./hello
not a dynamic executable
This works on any OS including Windows and macOS since it has no system
dependencies.
This feature is new and not well tested yet, but if it piques your
interest, you can play with this functionality using pre-built binaries
available on https://ziglang.org/download/ or you can build Zig master
branch from source.
Regards,
Andrew