aosp prebuilt toolchain building with upstream-master

[Minseong Kim]

Hi~ 

I am trying to build aosp prebuilt toolchain using the upstream-master branches of clang/llvm/compiler-rt. It seemed going well until I met the following error:

Can anyone help please ?

FAILED: /home/minseong.kim/llvm_ndk/out/stage1-install/linux-x86/clang-dev/bin/clang++ -L/home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/bionic/libc/libc/android_arm64_armv8-a_shared/ /home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/bionic/libc/crtbegin_so/android_arm64_armv8-a/crtbegin_so.o @/home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/bionic/libm/libm/android_arm64_armv8-a_shared/libm.so.rsp /home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/external/compiler-rt/libcompiler_rt-extras/android_arm64_armv8-a_static/libcompiler_rt-extras.a /home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/build/soong/libatomic/android_arm64_armv8-a_static/libatomic.a /home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/build/soong/libgcc/android_arm64_armv8-a_static/libgcc.a -lc /home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/bionic/libc/crtend_so/android_arm64_armv8-a/obj/bionic/libc/arch-common/bionic/crtend_so.o -o /home/minseong.kim/llvm_ndk/out/stage2/soong/.intermediates/bionic/libm/libm/android_arm64_armv8-a_shared/libm.so -target aarch64-linux-android -Bprebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/aarch64-linux-android/bin -Wl,--no-undefined -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--build-id=md5 -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,-maarch64linux -Wl,--hash-style=gnu -Wl,--fix-cortex-a53-843419 -fuse-ld=gold -Wl,--icf=safe -Wl,--no-undefined-version -Wl,--allow-shlib-undefined  -nostdlib -Wl,--gc-sections -shared -Wl,-soname,libm.so -Wl,--version-script,bionic/libm/libm.arm64.map

bionic/libm/upstream-freebsd/lib/msun/src/s_fmal.c:183: error: undefined reference to '__unorddf2'

bionic/libm/upstream-freebsd/lib/msun/src/s_fmal.c:185: error: undefined reference to '__unorddf2'

clang++: error: linker command failed with exit code 1 (use -v to see invocation)

It is the second stage of the build process. Before this command, s_fmal.c is compiled using the newly built clang++ from stage 1. If the clang++ is replaced with unmodified clang++ from llvm repo when compiling the file, the above works fine. When I dump s_fmal.o compiled with new clang++, the code contains __unorddf2 calls in two places unlike code compiled from the unmodified clang++, which has none of __unorddf2.

I guess __unorddf2 from compiler-rt should be included when building libc or libm.

Please shed light on this matter. Greatly appreciated.

Cheers,

Min

[Stephen Hines]

Hi Min,

On Tue, Jun 7, 2016 at 6:20 AM, Minseong Kim <minseo...@linaro.org> wrote:

Hi~ 

I am trying to build aosp prebuilt toolchain using the upstream-master branches of clang/llvm/compiler-rt. It seemed going well until I met the following error:

What do you mean by upstream-master? Are you trying to move these projects further towards TOT LLVM by merging from upstream-master? Normally, these branches build against aosp/dev for the LLVM projects. If you are indeed building against aosp/upstream-master, you are going to hit problems because you won't have updated the Android.mk files as well. If your errors are with aosp/dev, we can help try to fix this, but I really think that you are getting ahead of us. We are quite busy trying to stabilize/fix the sources we have, so we aren't going to look at aosp/upstream-master issues for a little while (if at all). I am currently trying to get an upstream cmake build working instead, but there are severe issues with upstream compiler-rt and Android in general.

Thanks,

Steve

 

--
You received this message because you are subscribed to the Google Groups "android-llvm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-llvm...@googlegroups.com.
To post to this group, send email to androi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Renato Golin]

On 7 June 2016 at 17:28, 'Stephen Hines' via android-llvm

<androi...@googlegroups.com> wrote:
> I am currently trying to get an upstream cmake
> build working instead, but there are severe issues with upstream compiler-rt
> and Android in general.

Hi Steve,

Is that related to the CMake 3.4.3 move? I'm not sure it's relevant,
but in the future, if you report upstream any problems with any
upgrades we try to force, we'll be able to hold them off until you can
fix them.

Regarding RT, do you have bugs open in LLVM? Maybe we could help?
IIRC, the way Android builds RT is very different, so I can imagine
the number of issues you guys are finding. If you create bugs for each
one of them, maybe we can split the work and help you upstream the
base first, so we can continue from upstream master built as upstream
LLVM, and submit the patches to LLVM directly.

Min,

It may be beneficial to help them with Android problems and make sure
AOSP has a solution from which they can pull. As Steve said, before
that works well, not many other things are as important.

cheers,
--renato

[Stephen Hines]

On Tue, Jun 7, 2016 at 9:37 AM, Renato Golin <renato...@linaro.org> wrote:

On 7 June 2016 at 17:28, 'Stephen Hines' via android-llvm
<androi...@googlegroups.com> wrote:
> I am currently trying to get an upstream cmake
> build working instead, but there are severe issues with upstream compiler-rt
> and Android in general.

Hi Steve,

Is that related to the CMake 3.4.3 move? I'm not sure it's relevant,
but in the future, if you report upstream any problems with any
upgrades we try to force, we'll be able to hold them off until you can
fix them.

The updated cmake rules are a bit of a pain, but I was able to work around those problems.

 

Regarding RT, do you have bugs open in LLVM? Maybe we could help?
IIRC, the way Android builds RT is very different, so I can imagine
the number of issues you guys are finding. If you create bugs for each
one of them, maybe we can split the work and help you upstream the
base first, so we can continue from upstream master built as upstream
LLVM, and submit the patches to LLVM directly.

Actually, the biggest problem we face is because of "auto-detection" of target architecture. That part of the RT cmake rules really makes it difficult to cross-compile all/any of the necessary targets that we have. I have some hacky workarounds I am using to experiment with building upstream compiler-rt, but I need to finish making it a bit more robust before finalizing some of our new build scripts. I am surprised that anyone can produce working Android components from the upstream sources (although I guess that all of the upstream users I have seen rely on a prior NDK to build compiler-rt, so they can't even bootstrap properly with their newly built clang). I unfortunately can't rely on the prior NDK to build the latest compiler-rt (from a correctness point-of-view - I can obviously use the older NDK clang to start part of the process if I want to ignore that). I will file some bugs and/or post some further email questions as I get things going with our "upstream" build.

Thanks,

Steve

[Renato Golin]

On 7 June 2016 at 17:49, Stephen Hines <srh...@google.com> wrote:
> Actually, the biggest problem we face is because of "auto-detection" of
> target architecture.

Thought so. That's what bit me when trying to port the 128-bit FP patch:

https://android-review.googlesource.com/#/c/125910/

> That part of the RT cmake rules really makes it
> difficult to cross-compile all/any of the necessary targets that we have. I
> have some hacky workarounds I am using to experiment with building upstream
> compiler-rt, but I need to finish making it a bit more robust before
> finalizing some of our new build scripts.

This is a subject that is being put on hold for a long time on the list.

I think it's about time for us to finally have a serious conversation
about how to actually cross-compile RT. A lot of people do,
downstream, using their own devices, and it doesn't make sense that
none of that is upstream.

Of course, such a patch would only be reasonable upstream if it works
on vanilla LLVM. It is harder, I know, but I recommend that we try to
do that, otherwise we'll just fall into another corner. :)

I also think that there are enough people interested in making that
work to warrant a full scale change in RT.

When you send your proposal, you should copy folks like me, Dumbar,
Samsonov, Saleem, Joerg, Roelofs, Chandler, Chris Bieneman, Chris
Mathews, Kritstof Beyls, Ben Kramer, Daniel Sanders, Adhemerval, Reid,
Eric. We need to include libc++ as well as Clang folks, to make sure
the solution works for everybody.

Many of those folks already had a go in cross-compiling, and even
having multiple RT's per build. You may realise that a lot of what
you're trying to do was already accomplished by them.

It may take a while, but the only long lasting solution is one that
works for most people. Anything else will be just another corner we'll
be trying to get out later.

cheers,
--renato

[Minseong Kim]

Hi~ guys

This issue is temporarily resolved with a workaround by setting allow_undefined_symbols: true in bionic/libm/Android.bp.

>> What do you mean by upstream-master? Are you trying to move these projects further towards TOT LLVM by merging from upstream-master? 

Yes, I patched the android-spedicif diff between aosp/dev and aosp/upstream on aosp/upstream-master and build the prebuilt toolchain. The primary purpose of doing this is to see the effects of the TOT LLVM on Android platform and applications.

Anyways, it can now be built successfully with the upstream-master branches of clang/llvm/compiler-rt using the current android-specific patches from aosp/dev, of course with a few workarounds such as the above.

Thank you very much for your help, Stephen.

>> It may be beneficial to help them with Android problems and make sure

>> AOSP has a solution from which they can pull. As Steve said, before
>> that works well, not many other things are as important.

Renato, 

Sure~ if we have fixes for problems arisen when rebasing llvm TOT in aosp/dev. I am more than happy to share with all.

Cheers,

Min