compilation fail with external toolchain.

[Hu David]

Hi,

I'm using the latest bazel.

Does anyone use external toolchain (non-native) but compile without any error currently?

I get an error even with a simple hello world.

project/

    WORKSPACE

    te2/

        main.c

        BUILD

    tools/

        BUILD

        CROSSTOOL

        CT/  <------- symbolic link to my cross toolchain directory

[err message]

$>  bazel build --crosstool_top=//tools:toolchain --host_crosstool_top=//tools/cpp:toolchain --cpu=armeabi-v7a //test:hello

INFO: Found 1 target...

INFO: From Compiling te2/main.c:

src/main/tools/namespace-sandbox.c:633: execvp(argv[0], argv): No such file or directory

ERROR: /root/blayer/te2/BUILD:1:1: C++ compilation of rule '//te2:hello' failed: x86_64-QNAP-linux-gnu-gcc failed: error executing command tools/CT/x86_64-QNAP-linux-gnu/cross-tools/bin/x86_64-QNAP-linux-gnu-gcc -iquote . -iquote bazel-out/QNAP_x86_64-fastbuild/genfiles -isystem tools/cpp/gcc3 ... (remaining 9 argument(s) skipped).

Target //te2:hello failed to build

Use --verbose_failures to see the command lines of failed build steps.

I am wondering if I should tune my external toolchain setting or this is a bug in bazel. Thanks.

David

[Lukács T. Berki]

Hey,

Well, that depends on what you have in your CROSSTOOL file and in your BUILD file. I'll try to explain how this works, and hopefully I'll be right :)

• The process starts with the value of --crosstool_top. On this, we do redirect chasing, meaning that if it points to a filegroup with one entry in srcs, we follow that pointer recursively.
• Once we arrived at something that's not a filegroup with a single srcs, we read the compiler definition from the CROSSTOOL file in the same directory, or if it's a cc_toolchain_suite rule, it can also be in its proto= attribute.
• Then we choose the toolchain: first, if --cpu is not set, CPU will be the default_target_cpu in the CROSSTOOL file. Thenl, if --compiler is not set, we look up the toolchain identifier for the CPU, then we choose a toolchain from the CROSSTOOL file based on the CPU and the toolchain ID. 
• Then we note the target_cpu field of the chosen toolchain.
• Based on this target, we choose the right cc_toolchain rule: if --crosstool_top found a filegroup, we choose "cc-compiler-<CPU>" from its srcs, and if it found a cc_toolchain_suite, we look it up in its toolchains attribute.
• This cc_toolchain rule contains the set of files needed for compiling, linking, and so on...
• To build the actual command line, the selected toolchain is consulted. For example, the gcc binary will be taken from a tool_path message in it.

I hope this helps. If not, report back with your BUILD / CROSSTOOL files and we'll see if we can figure something out.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/20355b2c-1c44-45f6-a9fe-cb284cca40ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Maximillianstr. 11-15 | 80539 München | Germany | Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle | Registergericht und -nummer: Hamburg, HRB 86891

[Philipp Wollermann]

+1 what Lukacs said, and for the specific error message you are getting ("execvp(argv[0], argv): No such file or directory"), it means that the compiler or one of its required shared libraries was not part of the action inputs, thus it's not available in the sandbox. This means, that in the filegroup of your toolchain, you are probably missing the files making up your compiler.

I can help debugging this, but would need the following data:

- BUILD / CROSSTOOL file

- Output of "bazel build"... with your usual command-line, but please add the additional flag "--sandbox_debug". This will generate lots of output, but we'll see which files are actually present in the sandbox then.

Cheers,

Philipp

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAOu%2B0LUp0NLEt_DST-q4DBqnMK5WKmNNrKzEochayCAMaCrhKA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Google Germany GmbH
Dienerstraße 12
80331 München

Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

[Hu David]

Thanks for the explanation.

src/main/tools/namespace-sandbox.c:633: execvp(argv[0], argv): No such file or directory

The reason I ran into this err message is because the tool_path in my own CROSSTOOL file is not mounted during bazel mounts the sandbox. I fix this by moving my own crosstool to /usr/bin since I don't know how to add additional (my own) source dir to the sandbox mounting.

Then, I face another problem.

ERROR: /root/blayer/david/BUILD:1:1: undeclared inclusion(s) in rule '//david:hello':

this rule is missing dependency declarations for the following files included by 'david/main.c':

  '/usr/bin/CT/arm-QNAP-linux-gnueabi/fs/include/stdc-predef.h'.

Bazel cannot identify my crosstool's system include path correctly.

The header does exist in the path describe above & my crosstool also configured properly with it's own system include path. How can I fix it? I notice there are several attribute (compiler_files/all_files/linker_files) in cc_toolchain. Is it related to my problem? If so, how should I configure it?

Thanks.

David

[Hu David]

I found the answer.

I didn't notice this part of the CROSSTOOL file before.

I set the cxx_builtin_include_directory manually according to my own crosstool setting. Then, everything works fine.

I guess this part could be auto-detected by bazel from the toolchain itself, right?

David

[Lukács T. Berki]

Probably correct, although we haven't investigated yet. It'd probably be something like "gcc -E -x c++ - -v" .

[Onath Claridge]

I'm running into a similar problem - my builds with a custom toolchain fail with

src/main/tools/namespace-sandbox.c:633: execvp(argv[0], argv): No such file or directory

I've gotten my toolchain components to mount in the sandbox. Running with --sandbox_debug gives me a bunch of entries like this:

src/main/tools/namespace-sandbox.c:511: mount: /home/claridge/.cache/bazel/_bazel_claridge/55dc5e1f68c8d183e88ba3315f76c03b/makani/external/third_party/gcc_arm_none_eabi/bin/arm-none-eabi-gcc

But I can't get Bazel to reference the toolchain correctly in its commandlines. My CROSSTOOL is at lib/bazel/CROSSTOOL. If I set the tool_path for gcc to "external/third_party/gcc_arm_none_eabi/bin/arm-none-eabi-gcc", then Bazel interprets that path relative to lib/bazel. If I try to compensate by including "../../" at the beginning of the tool_path, the build dies with

java.lang.RuntimeException: Unrecoverable error while evaluating node 'CONFIGURATION_FRAGMENT:com.google.devtools.build.lib.skyframe.ConfigurationFragmentValue$ConfigurationFragmentKey@8ecc95ca' (requested by nodes 'CONFIGURATION_COLLECTION:com.google.devtools.build.lib.skyframe.ConfigurationCollectionValue$ConfigurationCollectionKey@6d7c1bc6')

at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982)

at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

It looks like I might be able to make this work by moving CROSSTOOL to the top of my repo, but it would be nice to avoid that if possible.

[Austin Schuh]

Brian Silverman got this working for us.  The following is an excerpt from how we do it.  (I believe he upstreamed all the patches.  My goal is that we run a mainline bazel, but that never seems to be reality...)

  tool_path { name: "gcc" path: "debian_amd64_compilers/clang" }

  cxx_builtin_include_directory: "%package(@amd64_compilers_repo//usr/lib/gcc)%"

  cxx_builtin_include_directory: "%package(@amd64_compilers_repo//usr/local/include)%"

  cxx_builtin_include_directory: "%package(@amd64_compilers_repo//usr/include)%"

There is then a shell script in @//tools/cpp/debian_amd64_compilers/ called "clang" with the following contents:

#!/bin/bash --norc

export LD_LIBRARY_PATH="${BAZEL_OUTPUT_ROOT}external/clang_3.7_repo/lib"

exec "${BAZEL_OUTPUT_ROOT}external/clang_3.7_repo/bin/clang" "$@"

Austin

--

You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/c5e65e0a-ab3b-4de7-a839-90b51c09f7e3%40googlegroups.com.

[Onath Claridge]

Interesting, thanks for the suggestion!

[Austin Schuh]

I can't remember if a wiki/standard recipe location ever get set-up or if it was just discussed, but it would probably be good to document best practices for writing CROSSTOOL files for compilers.  It is tricky to get it set up correctly.

Austin

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/6b94e35c-ef27-45a5-b6ae-8b136f1134fa%40googlegroups.com.

[Brian Silverman]

I didn't ever write something on the wiki, but I plan to some time soon...

In case anybody's wondering, the BAZEL_OUTPUT_ROOT variable is for our Go rules which support cgo. It expands to an empty string for normal Bazel-invoked compile commands.

[Onath Claridge]

This approach got me running without much difficulty. Thanks, Austin and Brian!

A wiki entry on CROSSTOOL would be super-helpful. I can usually coax it to do what I need, but it involves a lot of trial and error.