Generate LDFLAGS/CXX/... from the toolchain

[4bu...@gmail.com]

Hello,

I depend on lot of 3rd party libraries which I download and build using `repository_rule` and `ctx.execute(build_script)`, and then I wrap the generated files into `cc_library` targets.

That all works great except when using different toolchains. It seems that some env. variables are picked inside the build script (like `CXX`), but some aren't - like `LDFLAGS` (my problem right now: C++ stdlib when using clang).

It's easy enough to do shell magic and generate environment based on the value of CXX, but that's not the general solution. I would like to generate all these values inside Skylark and pass it to the ctx.execute via `env` parameter. Is there an easy way to generate "make" environment from the current toolchain?

[László Csomor]

The word "toolchain" has a special meaning in Bazel context (see Toolchains [1]) -- do you mean that, or do you mean various compiler toolchains in general?

Bazel executes repository rules as part of the loading phase, before it has any information about the build configuration and thus about toolchains. This means that repository rules can only depend on the local environment -- whatever env. variables you have defined in the environment you ran Bazel in, that's about all you have.

Could you be a bit more specific about what the problem is and why the shell magic workaround is undesired? I'm afraid I don't fully understand the problem.

[1] https://docs.bazel.build/versions/master/toolchains.html

--
László Csomor | Software Engineer | laszlo...@google.com

Google Germany GmbH | Erika-Mann-Str. 33 | 80636 München | Germany
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/05da6d20-d7dc-4de5-acf0-6578cd47a64f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[4bu...@gmail.com]

Yes, I was referring to bazel's toolchain support. Let me try to make it more clear:

Suppose that we have some dependency that's not trivial to write bazel BUILD files for, so we end up compiling it, installing it with the appropriate prefix and wrapping the resulting library and header files inside `cc_library`. For simplicity, I'll use libzmq, although it should be possible to make bazel port (if not one already), but there is software for which that would not be possible. So, as the part of the `repository_rule`, we'll do:

```
export BUILD_SHARED_LIBS=OFF
cmake -DCMAKE_INSTALL_PREFIX=../install -DBUILD_SHARED_LIBS=$BUILD_SHARED_LIBS ..
cmake WITH_PERF_TOOL=OFF -DCMAKE_INSTALL_PREFIX=../install \
-DZMQ_BUILD_TESTS=OFF -DENABLE_CPACK=OFF -DCMAKE_BUILD_TYPE=Release ..
make install
```

to build it and install it. Then it's trivial to wrap the resulting files with `cc_library`:

```
cc_library(
name = "libzmq",
srcs = glob(["install/lib/*.a"]),
hdrs = glob(["install/include/*.h"]),
strip_include_prefix = "install/include",
linkopts = ["-lpthread"],
)
```

And that all works, until the point where we use the different toolchain than what `CXX` was inside the build script (and it seems that that's inherited from the environment where `bazel build` was issued). One example would be compiling libzmq with gcc and trying to use it with clang compiled bazel project - it will fail to link due to libstdc++/libc++ issues.

One way to work around this (and this is what I mean by "shell magic") would be to add this to the script:

```
if [[ $CXX == "clang-5"]]; then
export CXXFLAGS=-stdlib=libc++
export LDFLAGS="-B/usr/bin -fuse-ld=gold -L/usr/local/lib -Wl,-rpath,/usr/local/lib -l:libc++.a -l:libc++abi.a"
fi
``` (I probably have shell syntax error, but you get the idea).

However, that doesn't work if my toolchain suddenly changes (`bazel build --compiler=somecompiler`) and if the new compiler needs the same (or different) flags.

I hope my question is now also clearer - is there a way to query the toolchain configuration (for example `linker_flag` and `cxx_builtin_include_directory`) as part of the repository rule and build the `LDFLAGS` and `CXXFLAGS` and pass them to the `ctx.execute` via environment (there's a parameter there that allows specifying the environment).

Cross compiling is also an obvious problem and I'm not sure how it would fit in there.

[László Csomor]

Yes, that clarifies a lot, thanks!

What's the reason you're using a repository rule instead of a build rule (genrule or custom Skylark build rule)?

In the context of a build rule you'd have access to the active toolchain, in the context of repo rules you don't.

--
László Csomor | Software Engineer | laszlo...@google.com

Google Germany GmbH | Erika-Mann-Str. 33 | 80636 München | Germany
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/f10ffe52-aeb3-4279-9d2e-c2faaf173b2a%40googlegroups.com.

[4bu...@gmail.com]

Yes, I think I'm going to try with `genrule`/Skylark rule approach.
The main reason was the convenience, since with repository_rule one
doesn't need to enumerate the files, but actually just compile/install
and glob the install directory for artifacts.

Pretty much this issue explains what would be perfect for me as well:
https://github.com/bazelbuild/bazel/issues/2792

Thanks!