Bzlmod toolchain and platform configuration

Haixin Tie

unread,

Dec 3, 2023, 12:54:58 AM12/3/23

to bazel-discuss

Hi Bazel Team,

I've read many docs and Git repos related to Bzlmod custom C++ toolchain and platform configurations for ARM processors many times, but I am still very confused. It looks like that I am not the only one, per https://github.com/uber/hermetic_cc_toolchain:

"Configuring toolchains in Bazel is complex, under-documented, and fraught with peril. We, the team behind hermetic_cc_toolchain,are still confused on how this all works, and often wonder why it works at all."

Related repos, docs and tutorials that I have found and studied:

Problem that I am trying to solve:

How to create C++ toolchains for arm-none-eabi that allow bazel build to work on Linux, Windows and MacOS host systems?

The closest solution I could find is hexdae/bazel-arm-none-eabi: ARM none eabi gcc embedded toolchain for Bazel (github.com), but its Bzlmod support is still experimental and doesn't work in my case.

Tthe Bazel official documentation is very good, but I also find the documentation related to custom C++ toolchain setups using the new Bzlmod based system really lacking, Having more detailed and more typical examples would allow people to better understand Bazel and use it for a wider array of applications (e.g. embedded systems, robotics, etc) I'd appreciate your considerations!

Regards,

HaiXin

Haixin Tie

unread,

Dec 3, 2023, 1:33:45 PM12/3/23

to bazel-discuss

Some pain points in particular:

Migrating WORKSPACE to MODULE.bazel sometimes is not as straightforward as Bzlmod Migration Guide | Bazel suggested. For example, use_repo("<mod_extension>", "<external_repo>") is not evaluated immediately (by design, I know), which means that one cannot call register_toolchains("@<external_repo>//toolchain:*") immediately after in in MODULE.bazel, unlike its equivalent in the legacy WORKSPACE file. A non-working example of this: hexdae/bazel-arm-none-eabi: ARM none eabi gcc embedded toolchain for Bazel (github.com). If I missed anything, please advise, as I am still confused on this;
Many code examples and documents related to custom toolchain setup found on Google are largely based on the --crosstool_top toolchain setups. It's not immediately obvious how to convert them to the newer Platform based approach.
There are often many layers of obscure Bazel function calls with dynamic file generations to implement a custom toolchain, making the code quite difficult to follow and understand. It's unclear to me whether all the indirections are absolutely necessary, or because the code creators did not fully understand Bazel and created working codes after many trials & errors.
Toolchain and platform setup is one of the most important foundational pieces of a a build system. It's complicated by nature, but the official documentation and code examples are lacking, making it difficult for users to fully understand and leverage Bazel's power.

As a former Googler who had used Blaze for nearly 10 years, I didn't really appreciate how great Blaze/Bazel was until after leaving Alphabet and started using CMake. Bazel has make the build system end-user's life so much easier, such that I didn't really have to pay much attention to it, and just used it to get my job done. But now as a build system admin trying to setup custom toolchains and platforms, I find it much harder to grasp. I would greatly appreciate it if the Bazel team would come up with a comprehensive code example based on the latest bzlmod and platform designs, to perform the necessary steps below for a custom toolchain setup, e.g.:

How to specify external HTTP or Git repos in MODULE.bazel to download the build tools, similar to bazel-arm-none-eabi/deps.bzl at master · hexdae/bazel-arm-none-eabi (github.com)
How to specify custom BUILD files associated with the external repos downloaded in the previous step to various compiler/linker/etc tools in a toolchain for major host/execution OS'es (e.g. Linux, Windows and MacOS);
How to properly define target platforms associated with the toolchains defined in the previous step;
Simple hello world build target examples to show how the code can be compiled against different target environments on different host/execution environments.

I feel this may be a major pain point for many companies to adopt Bazel as their build system. If the Bazel's custom toolchain and platform setup design can be simplified, and/or its documentation enriched with more comprehensive working code examples, it would make Bazel more accessible to a much wider user base. Thanks for reading my long post. I hope at least some points would make sense to you.

Regards,

HaiXin

Xudong Yang

unread,

Dec 4, 2023, 2:22:46 PM12/4/23

to Haixin Tie, bazel-discuss

> Migrating WORKSPACE to MODULE.bazel sometimes is not as straightforward as Bzlmod Migration Guide | Bazel suggested. For example, use_repo("<mod_extension>", "<external_repo>") is not evaluated immediately (by design, I know), which means that one cannot call register_toolchains("@<external_repo>//toolchain:*") immediately after in in MODULE.bazel, unlike its equivalent in the legacy WORKSPACE file. A non-working example of this: hexdae/bazel-arm-none-eabi: ARM none eabi gcc embedded toolchain for Bazel (github.com). If I missed anything, please advise, as I am still confused on this;

This does work. You _can_ call `register_toolchains("@abc//:def")` right after `use_repo(ext, "abc")`. In fact, you can call it _before_ the `use_repo` clause; the order doesn't matter at all.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/33196d45-9655-45f3-8926-536790b9e4a8n%40googlegroups.com.

Haixin Tie

unread,

Dec 5, 2023, 2:19:02 PM12/5/23

to bazel-discuss

Hi XuDong and the rest of the Bazel team,

Thanks for your reply. Yes, you are right. After adding "git_override() clause I was able to pull in the dependency from the remote @arm_none_eabi repo and register toolchains. I guess the points that I've been struggling with are:

1. How to reuse the toolchains and platforms defined in a remote repo like https://github.com/hexdae/bazel-arm-none-eabi? Some naive questions can be found in the example folder under
https://github.com/thx123/bazel-custom-toolchains.

2. Is https://github.com/hexdae/bazel-arm-none-eabi doing everything the right way per Bazel team's design? Are there more straightforward ways to define toolchains and platforms that are easily reusable outside the module?

Regards,

HaiXin

Haixin Tie

unread,

Dec 5, 2023, 2:24:06 PM12/5/23

to bazel-discuss

Forgot to include direct links to the Question 1 in https://github.com/thx123/bazel-custom-toolchains. Here is one:

https://github.com/thx123/bazel-custom-toolchains/blob/5e08927e95f9b8e8f79e35a457fef7d9b7e8950f/examples/simple/BUILD#L17

Thanks!

HaiXin

Haixin Tie

unread,

Dec 12, 2023, 2:00:18 PM12/12/23

to bazel-discuss

Friendly ping after some waiting for the official answer from the Bazel/Google team...

Just saw the exciting announcement of Bazel 7.0.0. LTS, which includes many C++ toolchain and platform related upgrades. It'd be immensely helpful if the official tutorial documentations (which is great BTW) could include a complete C++ toolchain and platform configuration tutorial similar to the existing Bazel Tutorial: Configure C++ Toolchains and https://github.com/hexdae/bazel-arm-none-eabi. In particular, Bazel officially endorsed ways to configure:

Repositories for C++ compiler and other tools for custom toolchains, like https://github.com/hexdae/bazel-arm-none-eabi/blob/master/deps.bzl;
Definition of platforms based on the types of CPU, OS and the custom toolchains defined above;
Toolchains and platforms reused by other Bazel modules after adding the current module as a dependency.

I'd greatly appreciate Bazel team's consideration to this request.

Regards,

HaiXin (a former Googler)

HaiXin Tie

unread,

Dec 19, 2023, 12:30:15 AM12/19/23

to David Given, bazel-discuss

Dear Bazel team,

FYI, the "see also" section of the "Platform" page for the latest Bazel v7.0.0 release contains mostly outdated documents and non-working code examples that are more than 4 years old: https://bazel.build/versions/7.0.0/concepts/platforms. Please consider revamping the toolchain and platform documentation and example code at your early convenience, as they are critical fundamental pieces of the Bazel build system, and a P1 blocker for me to convince my company to use Bazel. I really love Bazel, and really want to make it available for more people to use it. We just need some good documentation and code examples to understand how to set up toolchains and platforms correctly. Thank you!

Bzlmod toolchain and platform configuration

Haixin Tie

Haixin Tie

Xudong Yang

Haixin Tie

Haixin Tie

Haixin Tie

HaiXin Tie

See also

John Cater

Haixin Tie

Haixin Tie