Re: [PATCH stable 5.4 00/11] ARM 32-bit build with Clang IAS

[Nick Desaulniers]

On Thu, Jul 14, 2022 at 10:05 AM Markus Mayer <markus...@broadcom.com> wrote:
>
> On Thu, 30 Jun 2022 at 16:03, Markus Mayer <markus...@broadcom.com> wrote:
>
> > > > > What we have NOT managed to build are:
> > > > >
> > > > > lib/clang/13.0.1/lib/linux/libclang_rt.builtins-armhf.a
> > > > > lib/clang/13.0.1/lib/linux/libclang_rt.builtins-aarch64.a
> > > > >
> > > > > However, the custom Yocto layer Florian mentioned is somehow able to
> > > > > generate them on an x86 host.
> > > >
> > > > Oh, those builtins, yeah. We build them for AOSP's distribution of
> > > > LLVM, let me see if I can find out where/how. Probably buried in here
> > > > somewhere:
> > > > https://android.googlesource.com/toolchain/llvm_android/+/refs/heads/master/builders.py
> > >
> > > I _think_ these are the CMAKE vars you need to play with:
> > > https://android.googlesource.com/toolchain/llvm_android/+/refs/heads/master/builders.py#510
> >
> > Thanks for digging. I played around somewhat, but haven't found the
> > magical combination.
>
> After a week's vacation I am back at it -- and I have made a discovery
> that is a little puzzling. Looks like Android is "cheating" by
> providing the very files I am looking to build from source as
> prebuilts (i.e. they aren't being built from source by Android).
>
> I followed the instructions at
> https://android.googlesource.com/toolchain/llvm_android/.
>
> And I realized that the files in question are being checked out before
> the build even begins. See, for instance,
>
> https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+/b072928f176af7db1929549c4a30e974c0b7d99d/clang-r450784e/lib64/clang/14.0.7/lib/linux/libclang_rt.builtins-arm-android.a
>
> They exist as binaries in the "prebuilt" repo. There are a bunch more
> low level files
> (https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+/b072928f176af7db1929549c4a30e974c0b7d99d/clang-r450784e/lib64/clang/14.0.7/lib/linux/).
> In this case, I was looking at LLVM 14, since that's what the prebuilt
> repo has.
>
> Do you have any thoughts on who might be building these libraries and
> how? At some point, somebody must be building them.

+ android-llvm
--
Thanks,
~Nick Desaulniers

[Stephen Hines]

On Thu, Jul 14, 2022 at 10:30 AM 'Nick Desaulniers' via android-llvm <androi...@googlegroups.com> wrote:

On Thu, Jul 14, 2022 at 10:05 AM Markus Mayer <markus...@broadcom.com> wrote:
>
> On Thu, 30 Jun 2022 at 16:03, Markus Mayer <markus...@broadcom.com> wrote:
>
> > > > > What we have NOT managed to build are:
> > > > >
> > > > > lib/clang/13.0.1/lib/linux/libclang_rt.builtins-armhf.a
> > > > > lib/clang/13.0.1/lib/linux/libclang_rt.builtins-aarch64.a
> > > > >
> > > > > However, the custom Yocto layer Florian mentioned is somehow able to
> > > > > generate them on an x86 host.
> > > >
> > > > Oh, those builtins, yeah. We build them for AOSP's distribution of
> > > > LLVM, let me see if I can find out where/how. Probably buried in here
> > > > somewhere:
> > > > https://android.googlesource.com/toolchain/llvm_android/+/refs/heads/master/builders.py
> > >
> > > I _think_ these are the CMAKE vars you need to play with:
> > > https://android.googlesource.com/toolchain/llvm_android/+/refs/heads/master/builders.py#510
> >
> > Thanks for digging. I played around somewhat, but haven't found the
> > magical combination.
>
> After a week's vacation I am back at it -- and I have made a discovery
> that is a little puzzling. Looks like Android is "cheating" by
> providing the very files I am looking to build from source as
> prebuilts (i.e. they aren't being built from source by Android).

I think you're not understanding how bootstrapping a compiler works, and that there is a somewhat circular dependency there. We start with a known good host toolchain (which comes from prebuilts/clang/host/linux-x86). This then is used to create a stage1 host compiler, which is later used to build a stage2 compiler. The stage2 compiler is used to build the runtime libraries for each target architecture from source. There is no copying of libraries from prebuilts/ into the out/ directory during this build process. Once we are satisfied with the new compiler, we will tag it and commit it back into prebuilts/clang/host/linux-x86, where it will then become the starting bootstrap compiler for future builds (back to where I started this reply).

 

>
> I followed the instructions at
> https://android.googlesource.com/toolchain/llvm_android/.
>
> And I realized that the files in question are being checked out before
> the build even begins. See, for instance,

Checked out doesn't mean that these are being used in the build. The git repository is present because the host toolchain binaries in it are going to be used as the starting bootstrap compiler used for toolchain/llvm_android/build.py. I think you are misunderstanding what is being built and where those outputs end up. Note that unless you modify toolchain/llvm_android/android_version.py, you will end up producing similarly numbered/named libraries in the out/ directory. If you have furthermore not changed any actual code, it might be identical to what the bootstrap compiler has been checked in as (and probably should be in most cases, although I think we've done additional development in tree already such that binaries might be different already).

Steve

 

>
> https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+/b072928f176af7db1929549c4a30e974c0b7d99d/clang-r450784e/lib64/clang/14.0.7/lib/linux/libclang_rt.builtins-arm-android.a
>
> They exist as binaries in the "prebuilt" repo. There are a bunch more
> low level files
> (https://android.googlesource.com/platform/prebuilts/clang/host/linux-x86/+/b072928f176af7db1929549c4a30e974c0b7d99d/clang-r450784e/lib64/clang/14.0.7/lib/linux/).
> In this case, I was looking at LLVM 14, since that's what the prebuilt
> repo has.
>
> Do you have any thoughts on who might be building these libraries and
> how? At some point, somebody must be building them.

+ android-llvm
--
Thanks,
~Nick Desaulniers

--
You received this message because you are subscribed to the Google Groups "android-llvm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-llvm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-llvm/CAKwvOd%3DjZ1JvhKzb2JXd_dsZomNB%2BqP2Huw%3Dsc48ofBLiQQVVw%40mail.gmail.com.