Toolchain resolution based on specific target

[suny...@gmail.com]

Hello,

Is there a way to force an rule to resolve toolchain differently based on some attributes in the rule? 

My specific need is: I have a bunch of rules that share an single rule_impl function which is  proto_compile_impl from a external package: in this function it resolve and uses a protoc toolchain to do the real thing. However, I want one of my rule resolve to a different protoc version than others. But I found that toolchain resolution constraint require setting “platform” or “target” through bazel command line. Which would affect all my other targets. Plus require changes to the ci/cd pipeline to set that value. 

One way to workaround is to duplicate the rule implementation entirely and make it use a totally different toolchain . But Is there a way to influence toolchain resolution process per target basis? That way I can set all targets that needs a lower version protoc to that , while make the rule impl remain unchanged.

[david...@gmail.com]

I keep running into this issue myself. You can override command line settings via transitions but they're pretty cumbersome to use and the documentation's hard to follow. One of my use cases is producing both debug and release binaries in a single pass, and I end up doing this:

---defs.bzl---

def _compilation_mode_transition(settings, attr):
    return [
        {"//command_line_option:compilation_mode": attr.mode},
    ]

compilation_mode_transition = transition(
    implementation = _compilation_mode_transition,
    inputs = [],
    outputs = [
        "//command_line_option:compilation_mode",
    ],
)

def _nop_impl(ctx):
    return [
        DefaultInfo(files = depset(ctx.files.deps))
    ]

compilation_mode = rule(
        implementation = _nop_impl,
        cfg = compilation_mode_transition,
        attrs = {
            "deps": attr.label_list(),
            "mode": attr.string(),
            "_whitelist_function_transition": attr.label(
                default = "@bazel_tools//tools/whitelists/function_transition_whitelist"),
        },
    )

---BUILD---

cc_program(name="myprogram", ...)

compilation_mode(name="myprogram-debug", mode="dbg", deps=":myprogram")

compilation_mode(name="myprogram-release", mode="opt", deps=":myprogram")

Toolchain options should work the same way. I was unable to find a way to get the actual flag name itself into the BUILD file, so for each and every option you want a transition for you need to create all that boilerplate in your defs.bzl. One downside of using transitions is that the output files end up with obfuscated names so actually accessing the binaries is difficult --- bazel likes you to use bazel itself to create zip files or tgz files, which you then need to pull manually out of the bazel-bin directory.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAJygYd2viA%2BoxMFAkj_NsAyi6CActrHNGXsUa-a_yRfXRyq%3Dog%40mail.gmail.com.

[Yucong Sun]

Cool, thanks for pointing me at right direction, I've managed to do following:

1. add a new build_setting. . --//mytarget:protoc_flavor=XXX

ProtocFlavorProvider = provider(fields = ["type"])
valid_flavors = [
"XXX",
"DEFAULT",
]

def _impl(ctx):
# use `ctx.build_setting_value` to access the raw value
# of this build setting. This value is either derived from
# the default value set on the target or from the setting
# being set somewhere on the command line/in a transition, etc.
flavor = ctx.build_setting_value

# Things you can do in a build setting implementation function
# include more advanced type checking
if flavor not in valid_flavors:
fail(str(ctx.label) + " build setting allowed to take values {" +
", ".join(valid_flavors) + "} but was set to unallowed value " +
flavor)

print("USING protoc flavor: " + flavor + " now.")

# Returns a provider like a normal rule
return ProtocFlavorProvider(type = flavor)

protoc_flavor = rule(
# This line separates a build setting from a regular target, by using
# the `build_setting` atttribute, you mark this rule as a build setting
# including what raw type it is and if it can be used on the command
# line or not (if yes, you must set `flag = True`)
build_setting = config.string(flag = True),
implementation = _impl,
)


2.  adding a config_settings and checks so that 

//mypackage:protoc is always pointing to the right binary base on the above build_setting

native_binary(
name = "protoc",
src = select({
":prebuilt_protoc_darwin_aarch64": "@com_google_protobuf_protoc_osx_aarch_64//file:protoc",
":prebuilt_protoc_darwin_x86_64": "@com_google_protobuf_protoc_osx_x86_64//file:protoc",
":prebuilt_protoc_linux_x86_64": "@com_google_protobuf_protoc_linux_x86_64//file:protoc",
":prebuilt_protoc_darwin_x86_64_ruby": "@com_google_protobuf_protoc_osx_x86_64_ruby//file:protoc",
":prebuilt_protoc_linux_x86_64_ruby": "@com_google_protobuf_protoc_linux_x86_64_ruby//file:protoc",
}),
out = "protoc",
visibility = ["//visibility:public"],
)

config_setting(
name = "prebuilt_protoc_darwin_aarch64",
constraint_values = [
"@bazel_tools//platforms:osx",
"@bazel_tools//platforms:aarch64",
],
flag_values = {
"//third_party/protoc:protoc_flavor": "DEFAULT",
},
values = {"proto_compiler": "//third_party/protoc:protoc"},
)

.... 

3. register this binary as the toolchain

protoc_toolchain(
name = "prebuilt_protoc_toolchain",
protoc = ":protoc",
)

toolchain(
name = "protoc_toolchain",
toolchain = ":prebuilt_protoc_toolchain",
toolchain_type = "@rules_proto_grpc//protobuf:toolchain_type",
)

4.  adding a transition for some targets to automatically change the build settings value

# in transitions.bzl

def _impl(settings, attr):
_ignore = (settings, attr)
return {"//third_party/protoc:protoc_flavor": "RUBY"}

old_protoc_transition = transition(
implementation = _impl,
inputs = [],
outputs = ["//mypkg:protoc_flavor"],
)

# in some build file

my_rule = rule(
attrs = dict(
proto_compile_attrs,
# override protos params in proto_compile_attrs to trigger transition
protos = attr.label_list(
mandatory = True,
providers = [ProtoInfo],
doc = "List of labels that provide the ProtoInfo provider (such as proto_library from rules_proto)",
cfg = ruby_protoc_transition,
),
_allowlist_function_transition = attr.label(
default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
),
),
cfg = ruby_protoc_transition,
toolchains = [str(Label("@rules_proto_grpc//protobuf:toolchain_type"))],
)


Wohoo, now building this rule will automatically flip the build_settings and back, it's perfect.