cross compiles to multiple platforms

[ma...@mtgrasse.com]

I'm investigating the use of bazel with a project I work on, and am struggling a bit with whether or not it has sufficient capabilities for our cross compilation needs.

We are building a CPU simulator and applications that run on top of the simulator. As a part of our build, we need to compile the simulator itself, as well as test applications that will run in the simulation. The simulator should be compiled for the host system that the build is running on. The test applications need to be cross-compiled to the particular target that will be simulated. A particular test application may be built for multiple different target platforms.

I think what I'm looking for is the ability to explicitly specify for each cc_binary and cc_library what my platform target is and have the tooling use the appropriate tool chain for that target. Is this something I can accomplish with bazel?

-Matt

[Brian Silverman]

Bazel can do most of that right now, and I've had good experiences with it. Rather than specifying which platform each rule (cc_binary, cc_library, genrule, etc) is for, you ask Bazel to build a label for a specific platform and it figures out how to do that. This makes sharing libraries between the architectures (for example) very easy, if that's something you want to do.

The only piece that's currently somewhat painful is building targets for multiple target platforms at the same time (with the same bazel command). Support for doing that via dynamic configurations is coming eventually, and there are some alternatives (details at the end).

Bazel is very good about the concept of a "configuration", which includes the target CPU architecture, C/C++ toolchain to use, etc. It currently has "host" and "target" configurations, which seems like it matches most of what you're doing well. As documented in the user manual, there are various command-line flags to configure what these two configurations are. Most attributes (deps, srcs, etc) get the same configuration of the target as the rule itself, but a few (genrule.tools for example) change to the host configuration. For more sophisticated things, custom rules can specify it explicitly for each of their attrs (I think you will want a custom rule for "package the simulator to run these programs" or something similar for example).

You can also specify which platforms a given rule supports (to give better errors if you try to build it for the wrong one, for example) via the environment rule and the restricted_to/compatible_with attrs. This support is pretty recent and still considered experimental, but it's been working well for me. See this thread for details on setting it up. If you set that up, then `bazel test //...` and similar commands just work, and explicitly building a target for the wrong platform gives a reasonable error.

Building code for multiple target platforms within the same command is pretty important for some projects, and Bazel doesn't really support that yet. For some use cases, scripting multiple `bazel build` commands and then packaging up the result outside of bazel works. Another alternative that I'm currently using with OK results is to write custom rules that re-implement all of the cc_binary/cc_library/etc ones for other architectures. The C/C++ rules come with a lot of functionality, some of which can't be re-implemented in a custom rule, and a lot of which is unnecessary. However, which pieces are needed is kind of project-specific, so the rules tend to end up kind of specialized depending on what you want to do. Writing and maintaining those is nontrivial, but it is definitely possible.  I've also written macros to combine creating a target with my custom rule and the builtin one, along with suffixes on the name and the labels of all the deps, which allows writing rules that use both without duplicating everything in the BUILD file.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/f1ff6551-5d5f-4a8a-8f44-0fb93d99eea2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ma...@mtgrasse.com]

Hi Brian,

Your thorough reply is appreciated. These are just the sort of direction I was looking for. I'm off to do some more experimentation.

Thanks!
Matt

> To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

[Indra Gunawan]

Hi Brian,

I would like to chime in.  I hope it is ok.

Do you have an example code of custom rule where I can make it to use host tool toolchain by default.  I have hostool and crosstool defined in bazelrc file.
The object the rule is building is static C libraries, binary tool that generates gen-code/files/headers.  For the later, in skylark of rule that needs to execute the gen-tool, I can specify "host" mode to the tool passed in so it is compiled with host-tool if there is no cache yet.

config_setting is only permissible in BUILD file?

How do I say to cc_binary to use host tool gcc, and use ccxx_include_directories already in hostool's CROSSTOOL if I am passing it to native.cc_binary to build/compile.
Similarly, how can I make sure I can use host tool's gcc for skylark as in ctx.var.get("CC").  

Thank you.
-Indra

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

[Greg Estren]

Hi Indra,

Loosely speaking, if a cc_binary is in the "host" configuration, it will use the host toolchain. That includes the host CROSSTOOL, ctx.var.get("CC"), and so on. So the question is what puts it into that configuration.

If you build bazel build //foo:binary, that automatically builds //foo:binary with the regular (target) toolchain / CROSSTOOL, because that's how all rules explicitly requested at the command line are built.

If you want to build a cc_binary as a dep of another binary, for for example generating C libraries / headers for consumption by another tool, there are straightforward ways to accomplish that.

Is that what you're asking for?

[Greg Estren]

On Saturday, November 4, 2017 at 7:35:20 AM UTC-4, Matthew Grasse wrote:

 

is the ability to explicitly specify for each cc_binary and cc_library what my platform target is and have the tooling use the appropriate tool chain for that target.  Is this something I can accomplish with bazel?

Regarding mixing different target platforms in the same build:

bazel build //foo:binary_for_platform1 //foo:binary_for_platform_2 ...

we're making important progress on this and can make this possible fairly soon. But there's a laundry list of caveats we'd need to explain so you can use it in an informed way (so your builds don't get unnecessarily slow or memory-hoggy).

If anyone else is interested in this please reach out so we can calibrate the demand for this sort of thing.

[jafpe...@gmail.com]

Hi Greg,

We are also very much interested in such functionality!

We would basically like to build libraries and binaries for both host (often Windows) and an embedded target (often ARM Cortex-M using armcc/armclang) at the same time. E.g. building a library for host to unit test, while building the same library for the embedded target to do real testing. Sometimes we also write generator tools for the host that generates sources for the embedded target, which means we have dependencies from the embedded library to the host tool.

Until now we have been experimenting with writing custom rules mimicking the cc_binary/cc_library rules, but that quickly becomes a major task.

Best,
Jacob

[Greg Estren]

Jacob,

Do you have any specific  examples?

I ask because, particularly for host vs. target, Bazel already supports this in a limited way. The catch is which version builds for host and which for target isn't obvious. And it only works under certain parent/dependency relationships that may or may not fit with what you need.

So depending on your precise needs there may (or may not) be options today.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/04b2a140-22c7-497d-bfa4-f4c156d96062%40googlegroups.com.

[Jacob Pedersen]

Hi Greg,

Sure! One example is that we have a host tool written in C++ that is used during the build step of the C++ for the embedded target. Thus, we need to build C++ with multiple toolchains in the same build/run.

Right now it appears like our only option is to implement custom C++ rules for the libraries and binaries for the embedded target?

Best,

Jacob

[Greg Estren]

Right. Today you need to route the host tool through a HOST transition, which means it automatically uses whatever --host_cpu, --host_crosstool_top, etc. are set to (which default to match the machine you run Bazel on).

As Brian mentioned above, the primary two ways to accomplish this now are

1. Route through a genrule.tools (which automatically applies HOST)
2. Write custom rules that apply HOST on attributes of your choice. I suppose this is what you're doing when you talk about writing custom C++ rules.

The genrule approach could look like:

cc_binary(

    name = "toplevel",

    srcs = ["main.cc"],

    data = [

        ":binary", # Applies --cpu, --crosstool_top, etc.

        ":binary_for_host" # Applies --host_cpu, --host_crosstool_top, etc.

    ])

genrule(

    name = "build_binary_for_host",

    srcs = [],

    outs = ["binary_for_host"],

    cmd = "cp $(location :binary) $@",

    tools = [":binary"])

cc_binary(

    name = "binary",

    srcs = ["main.cc"])

$ bazel build //foo:toplevel

$ bazel-bin/foo/toplevel.runfiles/__main__/foo/binary

I'm running on X86!

$ bazel-bin/foo/toplevel.runfiles/__main__/foo/binary_for_host

I'm running on PPC!

John Cater is doing some great work on platforms / toolchains that will eventually make this kind of pattern possible:

cc_binary(

    name = "always_target_macs",

    target_compatible_with = ["@bazel_tools//platforms:mac"])

Then you can mix and match rules however you want and they'll automatically target whatever architecture you've tagged them with.  But we really need community feedback on how quickly to expose this: it comes with efficiency consequences that we need to make sure everyone is onboard with. 

Also, if you want to build the same binary or library in different configurations under different circumstances, this simple tagging approach won't suffice (since it's a universal property of the rule). In cases like that, we'll need some more nuanced syntax. What that looks like again depends on your needs and input.

Note that even though defining a custom cc_ rule that applies host transitions is burdensome, writing a macro that munges together a genrule and native cc_ rule like the above should be more straightforward and, depending on your needs, sufficient.

[m...@8thwall.com]

I'm going to try the genrule.tools trick for 8th Wall, but we're looking to solve a similar problem, as we have packaging rules that build dynamic libraries from multiple architectures into a single distributable file. We'll also be very happy if this gets added to Bazel.

# Simplified example:
custom_zip_rule(
name = "my-package"
srcs = [
"//foo:bar-so" # Build shared library for --cpu=darwin
"//foo:bar-dll" # Build shared library for --cpu=win_x86_64
...
],
)

On Tuesday, February 27, 2018 at 1:57:18 PM UTC-8, Greg Estren wrote:
> Right. Today you need to route the host tool through a HOST transition, which means it automatically uses whatever --host_cpu, --host_crosstool_top, etc. are set to (which default to match the machine you run Bazel on).
>
>
> As Brian mentioned above, the primary two ways to accomplish this now are

> Route through a genrule.tools (which automatically applies HOST)Write custom rules that apply HOST on attributes of your choice. I suppose this is what you're doing when you talk about writing custom C++ rules.

[Greg Estren]

For thread reference:

Skylark Build Configuration will cover these themes and is being prioritized for implementation.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/bfb7a3c9-3cfb-49a1-afe3-539c32c2118b%40googlegroups.com.

[Erik Murphy-Chutorian]

Thanks for sending the doc. This looks great!

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/bfb7a3c9-3cfb-49a1-afe3-539c32c2118b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

8th Wall . m...@8thwall.com . www.8thwall.com