Building the platform compilers

[Minseong Kim]

Hi~

To build the platform compilers, I followed the instructions here (https://android.googlesource.com/platform/external/clang/+/dev/ToolchainPrebuilts.md).

But the build command returns with errors as the following (clang/llvm/compiler-rt is checkout to dev branch):

Please correct me if I am doing something wrong~

Command$ python external/clang/build.py

INFO:__main__:chdir /home/

INFO:__main__:check_call: make -j24 VERBOSE=1 clang-toolchain-minimal clang-toolchain-full llvm-tools

INFO:__main__:check_call additional env:

{'DISABLE_HOST_PIE': 'true',

 'DISABLE_LLVM_DEVICE_BUILDS': 'true',

 'DISABLE_RELOCATION_PACKER': 'true',

 'FORCE_BUILD_LLVM_COMPONENTS': 'true',

 'FORCE_BUILD_SANITIZER_SHARED_OBJECTS': 'true',

 'OUT_DIR': 'out/stage1',

 'SKIP_LLVM_TESTS': 'true',

 'SOONG_ALLOW_MISSING_DEPENDENCIES': 'true',

 'TARGET_BUILD_VARIANT': 'userdebug',

 'TARGET_PRODUCT': 'aosp_arm'}

============================================

PLATFORM_VERSION_CODENAME=P

PLATFORM_VERSION=P

TARGET_PRODUCT=aosp_arm

TARGET_BUILD_VARIANT=userdebug

TARGET_BUILD_TYPE=release

TARGET_ARCH=arm

TARGET_ARCH_VARIANT=armv7-a

TARGET_CPU_VARIANT=generic

HOST_ARCH=x86_64

HOST_2ND_ARCH=x86

HOST_OS=linux

HOST_OS_EXTRA=Linux-4.4.0-97-generic-x86_64-Ubuntu-14.04.5-LTS

HOST_CROSS_OS=windows

HOST_CROSS_ARCH=x86

HOST_CROSS_2ND_ARCH=x86_64

HOST_BUILD_TYPE=release

BUILD_ID=OC-MR1

OUT_DIR=out/stage1

============================================

ninja: no work to do.

[1/1] out/stage1/soong/.bootstrap/bin/soong_build out/stage1/soong/build.ninja

FAILED: out/stage1/soong/build.ninja

out/stage1/soong/.bootstrap/bin/soong_build -t -l out/stage1/.module_paths/Android.bp.list -b out/stage1/soong -n out/stage1 -d out/stage1/soong/build.ninja.d -o out/stage1/soong/build.ninja Android.bp

error: external/clang/tools/clang-check/Android.bp:3:1: module "clang-check": target.host.compile_multilib: mismatched types *string and string

error: external/clang/tools/clang-format/Android.bp:3:1: module "clang-format": target.host.compile_multilib: mismatched types *string and string

error: external/clang/tools/driver/Android.bp:3:1: module "clang": target.host.compile_multilib: mismatched types *string and string

ninja: build stopped: subcommand failed.

13:58:29 soong bootstrap failed with: exit status 1

make: *** [run_soong_ui] Error 1

Traceback (most recent call last):

  File "external/clang/build.py", line 1078, in <module>

    main()

  File "external/clang/build.py", line 1030, in main

    max_jobs=args.jobs, use_updated_version=False)

  File "external/clang/build.py", line 202, in build

    max_jobs, use_updated_version, profile_task)

  File "external/clang/build.py", line 266, in build_product

    cwd=android_path(), env=env)

  File "external/clang/build.py", line 91, in check_call

    subprocess.check_call(cmd, *args, **kwargs)

  File "/usr/lib/python2.7/subprocess.py", line 540, in check_call

    raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command '['make', '-j24', 'VERBOSE=1', 'clang-toolchain-minimal', 'clang-toolchain-full', 'llvm-tools']' returned non-zero exit status 2

cheers,

-- Min

[Pirama Arumuga Nainar]

Hi Minseong,

The documentation there is outdated.  We've switched to building toolchain using upstream CMake rules, and in that process moved to a different branch.  To build in this branch, you have to do:

$ repo init -u https://android.googlesource.com/platform/manifest -b llvm-toolchain

$ repo sync

$ toolchain/llvm_android/build.py

Thanks,

-Pirama

--
You received this message because you are subscribed to the Google Groups "android-llvm" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-llvm+unsubscribe@googlegroups.com.
To post to this group, send email to androi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Minseong Kim]

Hi Pirama, 

For builds for windows, there are some header finding issues but it might well be my local env problems.

It is working perfectly fine with "$ toolchain/llvm_android/build.py --no-build-windows".

Thanks for the precise help.

Cheers,

-- Min

2017년 12월 22일 금요일 오전 1시 42분 50초 UTC+9, Pirama Arumuga Nainar 님의 말:

To unsubscribe from this group and stop receiving emails from it, send an email to android-llvm...@googlegroups.com.

[Stephen Hines]

Did you follow all the download instructions on https://source.android.com/setup/initializing? For Windows, we know that the build isn't as hermetic as we would like, but if there is something else missing there, please let us know.

Thanks,

Steve

To unsubscribe from this group and stop receiving emails from it, send an email to android-llvm+unsubscribe@googlegroups.com.

[Minseong Kim]

Stephen~

Thank you for the prompt help.

I will redo the instructions here (https://source.android.com/setup/initializing) to make sure that my local build env is intact.

I will share it here if I find something else missing.

Thanks again.

Cheers,

-- Min

2017년 12월 22일 금요일 오후 2시 50분 40초 UTC+9, Stephen Hines 님의 말:

[polish code]

Hi All,

Thank you on providing hints on locating/building current Android's llvm/clang - the docs are really outdated. In fact, this thread is the only source I could find that points at -b llvm-toolchain for current version of the compilers...

Nevertheless, I need a little help with building the compilers. I tried two different approaches:

1. checking out latest version of -b llvm-toolchain, repo sync, toolchain\llvm_android\build.py - it fails during build-time tests at test for some patch (do not remember right now, have it on my work computer, I will update my post tomorrow);
2. checking out by tag: repo forall -c git checkout tags/ndk-r16 -b local_r16; but - not all projects in the branch are tagged with ndk-r16 tag; during that command I get a few of those:
    fatal: Cannot update paths and switch to branch 'local_r16' at the same time.
    Did you intend to checkout 'tags/ndk-r16' which can not be resolved as commit?
also, when trying to build.py the cmake tries to run:
    prebuilts/clang/host/linux-x86/clang-4393122/bin/clang
which for that tag does not exist.

I would be grateful for some tips on obtaining a functioning build for current llvm/clang (e.g. for release 16).

Thanks in advance,
Piotr

[polish code]

Ok, it turns out that clean checkout from head does build correctly. I suppose my changes to build.py to skip unwanted outputs prevented it from building.
But two questions remain:

1. old: how to properly build certain release?
2. new: how to integrate generated toolchain with Android NDK, in such a way that when using ndk-build in an NDK project the buil will use my toolchain? -
   
1. this page gives some info: https://android.googlesource.com/platform/external/clang/+/dev/ToolchainPrebuilts.md#generating-new-prebuilts but this is rather for using with the platform, not the NDK itself

Thank you,

Piotr

[Stephen Hines]

Hi Piotr,

On Mon, Feb 12, 2018 at 10:20 PM, polish code <polis...@gmail.com> wrote:

Ok, it turns out that clean checkout from head does build correctly. I suppose my changes to build.py to skip unwanted outputs prevented it from building.
But two questions remain:

1. old: how to properly build certain release?

We don't have a branch for a particular NDK release. The compiler is its own release, which the NDK packages separately. For building a particular compiler, you can get the precise manifest from prebuilts/clang/host/linux-x86/clang-XXXXXX/manifest_XXXXXX.xml. For instance, prebuilts/clang/host/linux-x86/clang-4579689/manifest_4579689.xml corresponds to our latest released toolchain. In order to figure out which manifest you should download, you can run pass --version to the compiler. Once you have the correct manifest, you can do:

$ cp manifest_XXXXXX.xml .repo/manifests/

$ repo init -b manifest_XXXXXX.xml

$ repo sync

This will sync the exact sources used for the build to your local branch.

 

1. new: how to integrate generated toolchain with Android NDK, in such a way that when using ndk-build in an NDK project the buil will use my toolchain? -
   
1. this page gives some info: https://android.googlesource.com/platform/external/clang/+/dev/ToolchainPrebuilts.md#generating-new-prebuilts but this is rather for using with the platform, not the NDK itself

The aosp/dev documentation is indeed out of date, but we haven't finished putting together the latest documentation for our llvm-toolchain branch yet. As far as using it in an NDK, you can follow the instructions for the public NDK build (https://android.googlesource.com/platform/ndk/+/master/README.md#Building-the-NDK) to create a new NDK once you copy your outputs from the toolchain branch into the NDK prebuilts repository. You can essentially take the outputs from out/install/linux-x86/clang-dev and put them into prebuilts/clang/host/linux-x86/clang-XXXXXX inside the NDK tree. You will have to substitute the XXXXXX with a new number of your choosing (and then update references to that number in the NDK build files too) or you can just put your new toolchain into the same version number that the NDK is already using.

I know that you wanted to work with NDK r16's version, but that Clang is very out of date at this point, so I would strongly recommend using our newest version, which will be in the upcoming NDK r17 beta 2. https://android-review.googlesource.com/q/topic:%2522update-clang-4579689%2522+(status:open) is Dan's switchover CL for that (which also shows you the files that would need to be edited if you created your own version number for Clang).

Thanks,

Steve

Thank you,

Piotr

--
You received this message because you are subscribed to the Google Groups "android-llvm" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-llvm+unsubscribe@googlegroups.com.

[polish code]

Thank you Stephen for clarifications. The manifest_XXXXXX.xml did the trick.
Three issues though:

1. I guess you meant -m instead of -b for repo init (for future readers of this thread)
2. prebuilt clang version 5.0.300080 (clang-4053586) - the one delivered with ndk r16 - does not have its manifest, only repo.config file which looks a bit like the manifest :)
3. I cannot see NDKr17 references/tags in the repo, does it correspond to prebuilt clang-4579689 (6.0.1)?
   

Thank you,

Piotr

[Yi Kong]

On Tuesday, 13 February 2018 16:39:27 UTC+8, polish code wrote:

Thank you Stephen for clarifications. The manifest_XXXXXX.xml did the trick.
Three issues though:

1. I guess you meant -m instead of -b for repo init (for future readers of this thread)
2. prebuilt clang version 5.0.300080 (clang-4053586) - the one delivered with ndk r16 - does not have its manifest, only repo.config file which looks a bit like the manifest :)

The manifest was missed when we first pushed the clang-4053586 prebuilds, you can find it from here:

https://android-review.googlesource.com/c/platform/prebuilts/clang/host/linux-x86/+/577564

[polish code]

Thank you Yi Kong!
Piotr