sample of building boot image with cross compilation

[Ian Smith]

https://github.com/iansmith/bazel-cross-q

I tried to create a small repository to illustrate a bazel problem that I have been unable to fix, despite much reading and trying various approaches.

The scenario that this repository illustrates is attempting to build a bootable disk.  The tools to manipulate disks have to run on linux--which is the host system.  These tools build isos, partitioned disks, bios-friendly boot records, etc.  However, the *content* of the bootable disk needs to compiled on the target system, arm64 with no os.  

I tested this on (bazel version)

Bazelisk version: v1.11.0
Build label: 5.2.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 7 16:02:26 2022 (1654617746)
Build timestamp: 1654617746
Build timestamp as int: 1654617746

This problem seems to be ideal for the current bazel toolchain approach, but I have not been able to get that to work.  You'll see my two toolchains in the repository, t1_arm64 and t2_linux_host. I am SURE that this is my lack of understanding of bazel or toolchains... or maybe something else.  Trying to get it work, I created platforms for the two situations (p1_arm64, p2_linux_host) and tried to use transitions.   This approach also did not work correctly.

I use this line to test this repository:

bazel run -s --toolchain_resolution_debug=@bazel_tools//tools/cpp:toolchain_type  --verbose_failures --sandbox_debug  --platforms=//platforms:p1_arm64 //src/host:make_volume

Note that I'm trying to run the tool on linux.  My expectation was that this would result in building the tool for making disks on host,  building the program that is the content for target, and then using the host-based program to make the disk.  In every case, the target is built with /usr/bin/gcc, which is not in the t1_arm64 toolchain.  The t1_arm64 toolchain has defined gcc to be /usr/bin/aarch64-linux-gnu-gcc-12.

I have two simple programs, src/host/volbuilder.c (14 lines!) and src/target/myprogram.c (6 lines!) to represent the build tool(s) and the content for this example.

There are numerous comments in the bazel code about various changes I have tried making and what effect they had... which can be summarized as "nothing."

Any thoughts, corrections, or new ideas would be appreciated.  If I actually get this to work, I promise to write a tutorial on how it works so others can learn from it.

thanks

ian

[Lars Sønderby Jessen]

Hi,

I'd start by ensuring that the cc toolchain resolution works. So basically that:
bazel build --platforms=//platforms:p1_arm64 //src/target:myprogram (should build myprogram using the 't1_arm' cross-compiler)
bazel build //src/host:volbuilder (should build volbuilder using the host compiler)

I can see that you've commented out 'build --incompatible_enable_cc_toolchain_resolution', I think this flag needs to be enabled otherwise toolchain resolution using platforms won't work (maybe someone else can explain why this feature flag is disabled by default and why the 'platform()' rule is not guarded by the same feature flag?).

You normally do not need to define a toolchain and platform for host, so consider removing the 't2_linux_host' stuff.

I find this example: https://github.com/ltekieli/bazel_cross_compile/tree/master/04_platforms and this tutorial: https://ltekieli.com/cross-compiling-with-bazel/ to be valuable sources of information on how to properly setup cc toolchain resolution using platforms.

When you've managed to get the toolchain resolution up and running you can add 'build --platforms=//platforms:p1_arm64' to .bazelrc and move on and try to get the //src/host:make_volume target to build.

If you want to build everything using 'bazel build ...' or similar you'll also need to add either tag = ["manual"] or a target_compatible_to constaint to 'volbuilder' to prevent it from being build for 'arm64'

Best regards,
Lars Jessen