[llvm-dev] libc++ failed to link against musl

1,039 views
Skip to first unread message

Dmitry Golovin via llvm-dev

unread,
Jun 5, 2017, 7:17:20 AM6/5/17
to Dmitry Golovin via llvm-dev
I'm trying to build LLVM, Clang, LLD, compiler-rt, libc++, libc++abi and libunwind with musl-based toolchain.

The configuration is the following:

LIBCXX_HAS_MUSL_LIBC=ON
LIBCXX_HAS_GCC_S_LIB=OFF
CLANG_DEFAULT_CXX_STDLIB=libc++
CLANG_DEFAULT_LINKER=lld
CLANG_DEFAULT_RTLIB=compiler-rt
LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-musl
LLVM_TARGET_ARCH=x86_64
LLVM_TARGETS_TO_BUILD=X86

When linking libc++.so there are a lot of undefined references to __cxa_allocate_exception, __cxa_begin_catch, __cxa_end_catch, __cxa_free_exception, __cxa_guard_abort, __cxa_guard_acquire, __cxa_guard_release, __cxa_pure_virtual, __cxa_rethrow, __cxa_throw, __gxx_personality_v0, _Unwind_Resume, vtable for __cxxabiv1::__class_type_info, vtable for __cxxabiv1::__si_class_type_info, vtable for __cxxabiv1::__vmi_class_type_info. The full output of the linking command is too long and is attached gzipped. The linking command is the following:

x86_64-linux-musl-g++ -fPIC -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O3 -Wl,-z,defs -fPIC -nodefaultlibs -shared -Wl,-soname,libc++.so.1 -o lib/libc++.so.1.0 projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/algorithm.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/any.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/bind.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/chrono.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/condition_variable.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/debug.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/exception.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/functional.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/future.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/hash.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/ios.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/iostream.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/locale.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/memory.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/mutex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/new.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/optional.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/random.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/regex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/shared_mutex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/stdexcept.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/string.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/strstream.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/system_error.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/thread.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/typeinfo.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/utility.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/valarray.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/variant.cpp.o -lpthread -lc -lm -lrt

Please suggest how to fix these linking errors.

Regards,
Dmitry

libcxxlinking.txt.gz

Jonathan Roelofs via llvm-dev

unread,
Jun 5, 2017, 10:17:27 AM6/5/17
to Dmitry Golovin, Dmitry Golovin via llvm-dev


On 6/5/17 5:17 AM, Dmitry Golovin via llvm-dev wrote:
> I'm trying to build LLVM, Clang, LLD, compiler-rt, libc++, libc++abi and libunwind with musl-based toolchain.
>
> The configuration is the following:
>
> LIBCXX_HAS_MUSL_LIBC=ON
> LIBCXX_HAS_GCC_S_LIB=OFF
> CLANG_DEFAULT_CXX_STDLIB=libc++
> CLANG_DEFAULT_LINKER=lld
> CLANG_DEFAULT_RTLIB=compiler-rt
> LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-musl
> LLVM_TARGET_ARCH=x86_64
> LLVM_TARGETS_TO_BUILD=X86
>
> When linking libc++.so there are a lot of undefined references to __cxa_allocate_exception, __cxa_begin_catch, __cxa_end_catch, __cxa_free_exception, __cxa_guard_abort, __cxa_guard_acquire, __cxa_guard_release, __cxa_pure_virtual, __cxa_rethrow, __cxa_throw, __gxx_personality_v0, _Unwind_Resume, vtable for __cxxabiv1::__class_type_info, vtable for __cxxabiv1::__si_class_type_info, vtable for __cxxabiv1::__vmi_class_type_info. The full output of the linking command is too long and is attached gzipped. The linking command is the following:

Those symbols are from libc++abi, and it isn't on your link line.


Jon

>
> x86_64-linux-musl-g++ -fPIC -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O3 -Wl,-z,defs -fPIC -nodefaultlibs -shared -Wl,-soname,libc++.so.1 -o lib/libc++.so.1.0 projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/algorithm.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/any.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/bind.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/chrono.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/condition_variable.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/debug.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/exception.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/functional.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/future.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/hash.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/ios.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/iostream.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/locale.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/memory.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/mutex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/new.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/optional.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/random.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/regex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/shared_mutex.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/stdexcept.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/string.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/strstream.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/system_error.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/thread.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/typeinfo.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/utility.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/valarray.cpp.o projects/libcxx/lib/CMakeFiles/cxx_objects.dir/__/src/variant.cpp.o -lpthread -lc -lm -lrt
>
> Please suggest how to fix these linking errors.
>
> Regards,
> Dmitry
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

--
Jon Roelofs
jona...@codesourcery.com
CodeSourcery / Mentor Embedded / Siemens
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Dmitry Golovin via llvm-dev

unread,
Jun 5, 2017, 11:15:48 AM6/5/17
to Jonathan Roelofs, Dmitry Golovin via llvm-dev
Hi Jonathan,

Thank you very much. I have solved this problem by configuring with additional options:

LIBCXXABI_LIBCXX_PATH=/path/to/llvm/projects/libcxx
LIBCXXABI_LIBCXX_INCLUDES=/path/to/llvm/projects/libcxx/include
LIBCXX_LIBCXXABI_INCLUDES_INTERNAL=/path/to/llvm/projects/libcxxabi/include

I wonder why those paths were not detected automatically.

Regards,
Dmitry

05.06.2017, 17:17, "Jonathan Roelofs" <jona...@codesourcery.com>:

David Chisnall via llvm-dev

unread,
Jun 6, 2017, 1:40:49 PM6/6/17
to Jonathan Roelofs, Dmitry Golovin via llvm-dev
On 5 Jun 2017, at 15:17, Jonathan Roelofs via llvm-dev <llvm...@lists.llvm.org> wrote:
>
> On 6/5/17 5:17 AM, Dmitry Golovin via llvm-dev wrote:
>> I'm trying to build LLVM, Clang, LLD, compiler-rt, libc++, libc++abi and libunwind with musl-based toolchain.
>>
>> The configuration is the following:
>>
>> LIBCXX_HAS_MUSL_LIBC=ON
>> LIBCXX_HAS_GCC_S_LIB=OFF
>> CLANG_DEFAULT_CXX_STDLIB=libc++
>> CLANG_DEFAULT_LINKER=lld
>> CLANG_DEFAULT_RTLIB=compiler-rt
>> LLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-musl
>> LLVM_TARGET_ARCH=x86_64
>> LLVM_TARGETS_TO_BUILD=X86
>>
>> When linking libc++.so there are a lot of undefined references to __cxa_allocate_exception, __cxa_begin_catch, __cxa_end_catch, __cxa_free_exception, __cxa_guard_abort, __cxa_guard_acquire, __cxa_guard_release, __cxa_pure_virtual, __cxa_rethrow, __cxa_throw, __gxx_personality_v0, _Unwind_Resume, vtable for __cxxabiv1::__class_type_info, vtable for __cxxabiv1::__si_class_type_info, vtable for __cxxabiv1::__vmi_class_type_info. The full output of the linking command is too long and is attached gzipped. The linking command is the following:
>
> Those symbols are from libc++abi, and it isn't on your link line.

It depends on how you want to package libc++ whether you actually want to link to libc++abi. If you’re on a GNUish system, you almost certainly don’t, because you then won’t be able to combine libc++ and libstdc++ in the same binary. You want to add -lstdc++ to the link line (or, ideally, -lsupc++, if your distribution ships a separate libsupc++.so).

On FreeBSD, we don’t link libc++ to any ABI library. Instead, both libc++.so and libstdc++.so are linker scripts that instruct anything that tries to link libc++ to link libc++.so.{version} and libcxxrt.so.{version}.

David

Dmitry Golovin via llvm-dev

unread,
Jun 6, 2017, 4:41:40 PM6/6/17
to David Chisnall, Jonathan Roelofs, Dmitry Golovin via llvm-dev
Hi David,

Neither is the case. The system that I want to build with this toolchain is Linux-based, but not GNUish. I would like to use musl instead of glibc and libc++ instead of libstdc++, only use binutils provided by LLVM. I think that in that case I will link libc++abi and libunwind to libc++ statically, so I will not have to add -lunwind and -lc++abi to my LDFLAGS. Is it okay to do that? There is an experimental LIBCXX_ENABLE_STATIC_ABI_LIBRARY option, I will try to enable it.

The host system that I'm compiling on is Alpine Linux (musl-based distro), if it matters.

Regards,
Dmitry

06.06.2017, 20:40, "David Chisnall" <david.c...@cl.cam.ac.uk>:

Dimitry Andric via llvm-dev

unread,
Jun 6, 2017, 6:00:01 PM6/6/17
to David Chisnall, Dmitry Golovin via llvm-dev, Jonathan Roelofs
Note that this is not 100% true, our static libc++.a contains objects from libcxxrt. But this was unavoidable, at least not if you didn't want to inconvenience users.

-Dimitry

signature.asc

David Chisnall via llvm-dev

unread,
Jun 7, 2017, 5:14:55 AM6/7/17
to Dmitry Golovin, Dmitry Golovin via llvm-dev, Jonathan Roelofs
On 6 Jun 2017, at 21:41, Dmitry Golovin <di...@golovin.in> wrote:
>
> Neither is the case. The system that I want to build with this toolchain is Linux-based, but not GNUish. I would like to use musl instead of glibc and libc++ instead of libstdc++, only use binutils provided by LLVM. I think that in that case I will link libc++abi and libunwind to libc++ statically, so I will not have to add -lunwind and -lc++abi to my LDFLAGS. Is it okay to do that? There is an experimental LIBCXX_ENABLE_STATIC_ABI_LIBRARY option, I will try to enable it.
>
> The host system that I'm compiling on is Alpine Linux (musl-based distro), if it matters.

It probably makes sense for libc++abi or libcxxrt, but it makes less sense for libunwind because C libraries that need to interoperate with C++ will link with this (by default assuming libgcc_s.so provides the relevant symbols). If both do then you’re going to have conflicts, if libgcc_s.so or equivalent doesn’t then you’re going to have link failures in pure-C programs that use C/C++ compatible libraries.

Dmitry Golovin via llvm-dev

unread,
Jun 7, 2017, 5:49:45 AM6/7/17
to David Chisnall, Dmitry Golovin via llvm-dev, Jonathan Roelofs
07.06.2017, 12:14, "David Chisnall" <david.c...@cl.cam.ac.uk>:

> On 6 Jun 2017, at 21:41, Dmitry Golovin <di...@golovin.in> wrote:
>>  Neither is the case. The system that I want to build with this toolchain is Linux-based, but not GNUish. I would like to use musl instead of glibc and libc++ instead of libstdc++, only use binutils provided by LLVM. I think that in that case I will link libc++abi and libunwind to libc++ statically, so I will not have to add -lunwind and -lc++abi to my LDFLAGS. Is it okay to do that? There is an experimental LIBCXX_ENABLE_STATIC_ABI_LIBRARY option, I will try to enable it.
>>
>>  The host system that I'm compiling on is Alpine Linux (musl-based distro), if it matters.
>
> It probably makes sense for libc++abi or libcxxrt, but it makes less sense for libunwind because C libraries that need to interoperate with C++ will link with this (by default assuming libgcc_s.so provides the relevant symbols). If both do then you’re going to have conflicts, if libgcc_s.so or equivalent doesn’t then you’re going to have link failures in pure-C programs that use C/C++ compatible libraries.
>
> David

As far as I can understand, all the functions of libgcc are provided by compiler-rt and libunwind, so it is possible to build a toolchain that will not have any parts of GCC at all. For now my toolchain can only produce statically liked binaries, I have some trouble with shared (https://bugs.llvm.org/show_bug.cgi?id=32425#c8), and it only works for C, the C++ test program segfaults.

Regards,
Dmitry

David Chisnall via llvm-dev

unread,
Jun 7, 2017, 6:32:08 AM6/7/17
to Dmitry Golovin, Dmitry Golovin via llvm-dev, Jonathan Roelofs
On 7 Jun 2017, at 10:49, Dmitry Golovin <di...@golovin.in> wrote:
>
> As far as I can understand, all the functions of libgcc are provided by compiler-rt and libunwind, so it is possible to build a toolchain that will not have any parts of GCC at all.

Correct, though on FreeBSD we still install the libunwind / compiler-rt combination as libgcc_s.so, because then we don’t break existing binaries or compilers.

> For now my toolchain can only produce statically liked binaries, I have some trouble with shared (https://bugs.llvm.org/show_bug.cgi?id=32425#c8), and it only works for C, the C++ test program segfaults.

You might find it informative to look at how the FreeBSD toolchain works.

David

Reply all
Reply to author
Forward
0 new messages