Hermetic builds using Bazel and multiarchitecture workspaces

[edvard.f...@gmail.com]

Hi there,

I just left Google and looked at bazel to see if I could set it up in order to feel myself at home. I'm wondering if someone could point me to resources wrt the following:

1. At Google libc was in the workspace somehow under //third_party, i.e. the system compiler/libraries were not used. Unfortunately, I never looked too closely how it was done. How can I easily use a custom libc(++)/compiler?

2. How should I setup projects if I have libraries/binaries in a workspace that should be usable as dependencies also for Arduino and ARM projects? In other words, I have projects with both amd64 and arduino/arm binaries (robots) and need to specify compilers for each.

For (1) I basically see two options:

1. Include the sources in the workspace, but this seems overkill as it's quite a bit of work to write BUILD files for all non-blaze based dependencies (I assume in this case libc++ would be compiled for each binary). On the plus side, it puts everything under git for easy rollback when updating.

2. Manually put the toolchains and libraries under something like /opt/toolchains/<arch> mounted over NFS to all machines that need to compile things and somehow tell blaze to pick system headers/compilers/tools from here as well as statically link in libraries instead of using the ones provided by the system.

What would be considered best practices and wouldn't require tons of maintenance? I haven't thought about it too deeply yet, so maybe someone has a better idea?

Best,
Edvard

[Brian Silverman]

On Sun, Nov 12, 2017 at 1:24 PM, <edvard.f...@gmail.com> wrote:

Hi there,

I just left Google and looked at bazel to see if I could set it up in order to feel myself at home. I'm wondering if someone could point me to resources wrt the following:

1. At Google libc was in the workspace somehow under //third_party, i.e. the system compiler/libraries were not used. Unfortunately, I never looked too closely how it was done. How can I easily use a custom libc(++)/compiler?

Write a custom CROSSTOOL (or find somebody else's). This is a fair amount of work right now unfortunately... There are several wiki pages about it, and lots of bazel-discuss conversations. Somebody recently mentioned bazelbuild/bazel-toolchains as a place to put examples to start from. tensorflow also has a few to look at as examples. I'm trying to get around to putting some of mine out into separate repositories...

2. How should I setup projects if I have libraries/binaries in a workspace that should be usable as dependencies also for Arduino and ARM projects? In other words, I have projects with both amd64 and arduino/arm binaries (robots) and need to specify compilers for each.

Same solution basically. Once you're doing a custom CROSSTOOL, having toolchain entries both for a cross compiler (or two) and the native amd64 one is easy. You then switch between those with the --cpu= flag when doing builds. Bazel handles compiling tools that run at build time with the amd64 compiler and target binaries with the other one very nicely. One of the toolchains of mine I keep meaning to split out somewhere is for bare metal on a Cortex-M4F.

For (1) I basically see two options:

1. Include the sources in the workspace, but this seems overkill as it's quite a bit of work to write BUILD files for all non-blaze based dependencies (I assume in this case libc++ would be compiled for each binary). On the plus side, it puts everything under git for easy rollback when updating.

2. Manually put the toolchains and libraries under something like /opt/toolchains/<arch> mounted over NFS to all machines that need to compile things and somehow tell blaze to pick system headers/compilers/tools from here as well as statically link in libraries instead of using the ones provided by the system.

What would be considered best practices and wouldn't require tons of maintenance? I haven't thought about it too deeply yet, so maybe someone has a better idea?

I'm a fan of using WORKSPACE new_http_archive entries to download the toolchains (including libc++ usually). That means they're versioned alongside all the code (assuming you keep all the old archives available for download and use new names for new versions), but they're also not checked into the git repository and making it huge. That also makes it easy to include the precompiled standard libraries etc alongside the toolchain, because setting up BUILD files to compile those is indeed a lot of work and usually not too useful. I leave the CROSSTOOL and BUILD files checked in directly because they're small and they change faster than the binaries.

In general, any of the new_* rules will work basically the same and you can decide where you prefer to keep the binaries. new_local_repository with an NFS mount works out pretty much the same, for example.