Cross-platform tests design question

[Filipe Mendes]

Hello, in my project we are doing cross-platform builds, let's consider "platform A" as the host/execution and "platform B" as the target platform.

If I want to cross-build my target, I would have a build toolchain declared that is able to cross-build, which then I could use with the following command:

bazel build --platforms=//platforms:platform_B --extra_toolchains=//toolchains:cross-toolchain //:my_target


So far so good, now let's consider the problematic use case, which are the integration tests. In this case we want to build some targets for the target platform "platform B" while the test itself is built for the host/execution platform "platform A".

I found no issues/documentation on how this should actually be designed, so there's 2 options, either the top-level target sets the platform (then I have to transition some runtime deps to the target platform) or I keep a similar CLI to the build command and transition the test target itself. Let's put that into code (maybe you'll see a better way to implement this):

Option 1:
Command:
    # NOTE: Execution platform here is auto-detected, so don't specify the --platforms flag
    bazel test --extra_toolchains=//toolchains:cross-toolchain --//flags:do_platform_B_transition //:my_integration_test

Code:
# BUILD
    cc_binary(
        name = "my_target",
        ...
    )

    # This is a custom implementation of an alias with platform transition
    alias_and_do_platform_transition(
        name = "my_transitioned_target",
        actual = ":my_target",
        platform = select(
        {
            "//settings:do_platform_B_transition": "//platforms:platform_B",
            # We expect to support more than one target platform
            "//settings:do_platform_C_transition": "//platforms:platform_C",
            },
            no_match_error = ""
        ),
    )

    py_test(
        name = "my_integration_test",
        data = [":my_transitioned_target"],
        ...
    )

Main issues:
    -> Need an extra setting/flag for the transition, per supported target platforms, which implies a more complicated CLI/configuration, which will be different for builds and tests.
    -> Project semantic of target platform is lost in the command, which will confuse regular devs that are not proficient in platform semantics in bazel.


Option 2:
Command:
    bazel test --extra_toolchains=//toolchains:cross-toolchain --platforms=//platforms:platform_B //:my_test

Code:
# NOTE: This custom rule is written as executable instead of test, but the idea stands
# my_integration_test_macro.bzl
    def _run_as_exec_impl(ctx):
        args = ctx.actions.args()
        args.add(ctx.executable.executable)

        link = ctx.actions.declare_file(ctx.attr.name)
        ctx.actions.symlink(
            output = link,
            target_file = ctx.executable.executable,
            is_executable = True,
        )

        return DefaultInfo(
            executable = link,
            runfiles = ctx.runfiles(
                files = ctx.files.data + ctx.files.deps + ctx.files.executable,
            ),
        )

    _run_as_exec = rule(
        implementation = _run_as_exec_impl,
        attrs = {
            # In order for args expansion to work in bazel for an executable rule
            # the attributes must be one of: "srcs", "deps", "data" or "tools".
            # See Bazel's LocationExpander implementation, these attribute names
            # are hardcoded.
            "data": attr.label_list(
                allow_files = True,
                cfg = "target",
            ),
            "deps": attr.label_list(
                allow_files = True,
                cfg = "exec",
            ),
            "executable": attr.label(
                allow_files = True,
                cfg = "exec",
                executable = True,
            ),
        },
        executable = True,
    )

    def run_as_exec(**kwargs):
        """
        This marcro remaps the arguments for clarity:
        deps -> data_as_exec: Runtime deps built with execution platform configuration.
        """
        data_as_exec = kwargs.pop("data_as_exec", None)
        _run_as_exec(
            deps = data_as_exec,
            **kwargs
        )

# BUILD
    cc_binary(
        name = "my_target",
        ...
    )
   
    py_test(
        name = "my_integration_test",
        data = [":my_transitioned_target"],
        ...
    )
   
    run_as_exec(
        name = "my_integration_test_as_exec",
        executable = ":my_integration_test",
        data_target = ":my_target",
    )
   
Main Issues:
    -> Added complexity on implementation of integration tests, the rule above might also have unforseen side effects (unforseen by me, that is).

   
   
So I believe that's the whole picture, if there are any design/product documents that can guide me or if you already foresaw a solution, please comment.

Thanks for the support and BR,
Filipe Mendes.