[Proposal] Setting platforms directly on targets

[John Cater]

There's been a standing request for years to be able to directly set the target platform on a target, instead of at the command line with the "--platforms" flag. I've just created a proposal for a way to do that, so please take a look and let me know what people think about it.

The proposal is at https://github.com/bazelbuild/proposals/blob/main/designs/2022-08-03-platforms-on-targets.md

Thanks!

John Cater

[Brentley Jones]

You mention "Many binaries are single-platform: Android or iOS apps, Windows-specific executables, or tools that use low-level Linux syscalls". How does this proposal plan to handle "single-ish" platform binaries, like fat/universal Apple binaries?

[John Cater]

I'll update the proposal, but my thinking is that Bazel can by default only handle simple cases. More complicated cases, such as Android fat APKs or Apple applications, will need rule-level support for specific logic.

For the Android example, we've recently added the "--android_platforms" flag, and the android_binary rule could be updated to override the common "platform" attribute and use a special "platforms" attribute with a transition to set "--android_platforms" instead.

[Keith Smiley]

On the open question of "Should this be a label_list valued platforms attribute?", we have cases where we want to build targets for some closed set of platforms but more than just 1. For example building a macOS binary as x86_64, and arm64, where we actually don't want a fat binary because they're deployed only to machines with the matching architecture. I think supporting a list here would solve that, although I guess we'd probably also want it to be select-able so you could vary this behavior for development to only build the current platform in dbg, and then both when deploying.

[Keith Smiley]

On the open question of "Should this be a label_list valued platforms attribute?", we have cases where we want to build targets for some closed set of platforms but more than just 1. For example building a macOS binary as x86_64, and arm64, where we actually don't want a fat binary because they're deployed only to machines with the matching architecture. I think supporting a list here would solve that, although I guess we'd probably also want it to be select-able so you could vary this behavior for development to only build the current platform in dbg, and then both when deploying.

--
Keith Smiley

--
You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/9acb4cae-5eca-47e7-a1ad-caf7238f1a5dn%40googlegroups.com.

[Tony Aiuto]

> 1. For example building a macOS binary as x86_64, and arm64, where we actually don't want a fat binary because they're deployed only to machines with the matching architecture. 

How would you distinguish the things you build easily without the platform reflected in the name?  You are going to have to end up looking at OutputGroupInfo keyed by arch to get the paths to the distinct files returned.

To me, the case of multiple top level binaries is best done with a comprehension

[ foo_binary(

    name = "targ_" + arch, ...,

    platform="//buildenv:"+arch,
  )

  for arch in ['x86_64', 'arm64']

]

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CANw2gKMGF2miu045oioAb%3DVUDdUQrhNCgN_-EXJnVO7DOqMdcA%40mail.gmail.com.

[John Cater]

This sort of scenario is why I was considering a multi-valued "platforms" attribute. I think the ideal semantics would be that

a) For top-level builds, there would be an effective split transition, where the target is built for each platform declared in "platforms"

  i) Possibly if "--platforms" is set, and is one of the platforms in "platforms", only that is built? This is slightly confusing.

b) When checking the platform, if the target platform is any platform declared in "platforms", that is acceptable.

I'm worried that the complexity here isn't worth the addition.

What if a binary rule (like macos_binary) had a custom attribute (rule author's choice on the name) and did its own split transition to get the same effect? This is outside of the core Bazel logic (which is good both for simplifying Bazel and for allowing rule-specific semantics), but then the platform checking part of this proposal is unused. Is that a problem? Is there a simple API we can expose to let rule authors opt in to that?

[Keith Smiley]

> How would you distinguish the things you build easily without the platform reflected in the name?  You are going to have to end up looking at OutputGroupInfo keyed by arch to get the paths to the distinct files returned.

If I understand what you're asking, I believe this would be the responsibility of `--use_top_level_targets_for_symlinks`, when you're only building one it would just be the right one in the expected location. A comprehension definitely works, but I think for better developer experience we would immediately follow that up with an alias that did a select on which one of those you wanted based on the current host architecture for debugging.

> I'm worried that the complexity here isn't worth the addition.

I would imagine that --platforms should always be ignored if the value is set on this attribute, and the flag should only apply to targets being built that don't specify their platform, but I can see how it would be nice if multiple were supported to handle it somehow. I agree that gets complex.

> What if a binary rule (like macos_binary) had a custom attribute (rule author's choice on the name) and did its own split transition to get the same effect? This is outside of the core Bazel logic (which is good both for simplifying Bazel and for allowing rule-specific semantics), but then the platform checking part of this proposal is unused. Is that a problem? Is there a simple API we can expose to let rule authors opt in to that?

So actually some of the apple rules do this today, but I think you'd virtually want this everywhere, our case is a cc_binary (we could wrap it in an apple specific rule instead, but that requires some BUILD file duplication).

[Gregg Reynolds]

On Wednesday, August 3, 2022 at 2:27:42 PM UTC-5 John Cater wrote:

There's been a standing request for years to be able to directly set the target platform on a target, instead of at the command line with the "--platforms" flag. I've just created a proposal for a way to do that, so please take a look and let me know what people think about it.

The proposal is at https://github.com/bazelbuild/proposals/blob/main/designs/2022-08-03-platforms-on-targets.md

I think this should be rethunk.  It's not really about platforms, its about toolchain selection. What the proposed attribute really does is modify the constraints on toolchain selection, and that need not be restricted to platform constraints.

A possible alternative would be to simply provide a 'toolchains' attribute to override the rule definition's toolchain attribute.  Users can define a toolchain type for their desired target platform and then just write something like

```

cc_binary( name = "some_binary",

   srcs = [...],

   deps = [...], 

   toolchains = ["//my/custom:toolchain_type"]

)

This would have the advantage of also supporting 'target_settings" (in the toolchain rule). Fixing a constant toolchain (type) for a given build target need not be restricted to "platforms" (i.e. target_compatible with); we might want to stipulate that the build can use any toolchains that satisfies some target_settings rather than some platform description.

Gregg

[Gregg Reynolds]

Maybe a little more explanation is in order.  I think of toolchain selection in terms of dispatch tables.  The top-level dispatches on toolchain type.  If the toolchains in the type target multiple targethosts, then we move to the second-level dispatcher, which is controlled by *_compatible_with and target_settings.

Now if my toolchain type only contains toolchains targeting the same targethost, then I don't need target_compatible_with in the second-level dispatch, since targethost constraints are already implicit in the toolchain type.  So if I can specify my toolchain type for a build target, I don't need to also specify any (target) platform constraints.  Put differently, the value for target_compatible_with (on the toolchain rule) is effectively "any", and that's ok since every available toolchain in fact targets the desired host.

Whether or not such a target should be built at all is a separate issue.  The proposal says:

"When a target is being configured, and the value of the platform attribute is different from the target platform for the configuration, that will be an error and the target will not be built."

I'm not sure that's a good idea. Maybe the user wants it to be built anyway; this should be controlled by target_compatible_with on the build target.

More generally, "platform" is confusing enough already.  The platform rule, contrary to what the docs say, does not define a "platform", it only declares a (named) set of constraints, which may or may not be satisfied for a particular build run, and - crucially - need not be used together, as a unit, in toolchain selection.  The '--platform=foo' CLI argument effectively means "The constraints associated with foo are hereby satisfied", but the documentation on this is far from clear.  All this suggests (to me, at least) that a platform is a first-class thing (in the language), but that's not the case. You cannot write e.g. target_compatible_with=":myplatform", but the proposal is to allow 'platform=":myplatform"'.  I think that's just going to confuse a lot of people.

From the proposal: "When built directly, the platform attribute will be used to set the target platform."  But there is no "target platform" to set, there is only a collection of attributes to stipulate. The critical point being that the *compatible_with attributes only traffic in constraints, not platforms, and they are not required to match up point-by-point with any "platform".

Consider what 'platform=":myplatform"' looks like to a beginner.  It's an assignment, right? Well, no, it's not. It's there purely for side-effects. Ditto for '--platform=foo' - it does not mean that "platform" should have the value "foo".

The proposal says "The platform attribute and the target_compatible_with attribute have similar but distinct functionality. "  I think they're completely different animals, just as 'i=3' and 'i: Int' are different animals.

Then of course there is the simple ambiguity  of using 'platform' when you mean 'target_platform'.

To sum up, I think the problem addressed by the proposal should be viewed not as "how to indicate a desired target platform for a build rule" but  the more fundamental "how to select the desired toolchain", which does not necessarily involve platforms.

gregg

[Guillaume Maudoux]

I think the proposal is great because it matches a design we do use, and that is currently implemented by forcing a transition on the selected targets with a macro.
But we do not use it on binaries, rather on a toplevel bundle rule that packages several binaries together.

I took some time to think about the platforms list, instead of the single value. It seems overkill and ill defined. Does it mean that the binary will always have to be built for all the mentioned platforms ? I imagine that only one supported platform is fine most of the time, for test purposes of a single developer. Conversely, if the list is taken as a subset of allowed platforms, then it is much like target_compatible_with, and loses most of it's interest.

I like the idea that platforms can be used directly, instead of platform constraints. But it also creates yet another way to configure platforms. Soon enough, the difference between `platform` and `target_compatible_with` will make people want to pass platforms into target_compatible_with. I certainly would like to do that sometimes.

I think the biggest issue I have with this proposal is how easy it is to implement with ad-hoc transitions, and the fact that it does not unify very well with existing platform selection as supported by target_compatible_with. But it will be useful in several cases. So if it's relatively easy to implement and keep the bazel code simple, why not :-).

-- Guillaume.

[Guillaume Maudoux]

That being said, a problem we have with our design is that toplevel targets are annotated with (equivalent of) `platform = //foo` but not the intermediate targets. While everything works fine when building the top-level target, intermediate targets are more complex to build because is is not always obvious which platforms they support. They can be build as part of several toplevel targets, and thus for different platforms. But when developers want to build one of these smaller intermediate targets, we have no way to provide a default platform.

In that specific context, a list-valued platform attribute would make sense, so as to annotate intermediate targets with the set of platforms they support. That way the target can be build as a top-level goal, and also as part of an higher-level target. When built as a top-level target, it should probably default to the first one in the list, with an INFO: message explaining how to access the other platforms. That would make a really nice UX for developers.

That being said, this approach leads to lots of duplication: all the intermediate targets need to be annotated with the right set of platform, leading to heavy changes when a platform is added (yes, we have > 30 platforms to encode low-level hardware stuff) or when a new intermediate target is added (the list of platforms it supports must be guessed. It would be easier if that information was inferred from the set of toplevel target it is included in. Ultimately, this can be automated with external tools and enforced in CI.

As you see, my main concern is with the end-user UX, rather than the build-system/tooling team in the company.

-- Guillaume.

[Tony Aiuto]

>In that specific context, a list-valued platform attribute would make sense, so as to annotate intermediate targets with the set of platforms they support. That way the target can be build as a top-level goal, and also as part of an higher-level target. When built as a top-level target, it should probably default to the first one in the list, with an INFO: message explaining how to access the other platforms. That would make a really nice UX for developers.

We should not conflate the two concepts.  platform is strictly to mean "build this target for the specified platform". It is an imperative instruction. target_compatible_with declares that a target only makes sense if a platform fulfills a set of constraints. It is a hint used for skipping targets in //pkg/... expansions. 

What you are describing sounds like you want a top level package to have a list of platforms, then intermediate ones to say what platforms they support (target_compatible_with), but want the first element there to be the default platform for building. That seems a little magic and mixes things. target_compatible_with should more often be about constraints than specific platforms. You might have a constraint like "has_bluetooth", which may be true on some of the platforms you build for, but not all of them.  

But the missing thing in your example is how the top level package knows what to gather if you have target skipping in place.

Let's say you have

pkg_zip(

  name = "my_product",

  srcs = [foo, bar, baz]

  platforms = [linux, macos, windows]

)

cc_binary(name=foo)

cc_binary(name=bar, target_compatible_with=[linux,macos])

cc_binary(name=baz, target_compatible_with=[macos,windows])

I think your intent is that my_product would only have foo and baz in it if you build for windows.

This is too big a change. It would have to mean that the srcs of my_product could gracefully

not fail to build one of the explicitly mentioned things.  For most cases, not being able to build

a target is a failure.  Your case, though, is an interesting one. Do you have a more concrete

example of the kind of fan-out and complexity of what you are doing.

--

You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/28a9e506-4935-4db8-a6e2-7f9909c9790en%40googlegroups.com.

[Guillaume Maudoux]

Well my concern is about the dependencies of targets that have the platform attribute. For them, there is no hint about the platform that should be used.
Even worse, if a target is used as a dependency of several top-level targets built for different platforms, then it is impossible to use the platform parameter there.

You are right, an example is needed.
Expanding on yours, our setup looks like this:

pkg_zip(

  name = "linux_product",

  srcs = [foo, bar]

  platform = "linux"

)

pkg_zip(

  name = "windows_product",

  srcs = [foo, baz]

  platform = "windows"

)

cc_binary(name=foo)

cc_binary(name=bar)

cc_binary(name=baz)

Running `bazel build //linux_product` or `bazel build //windows_product` would work pretty well. Everything is setup by default to the right platform.

But running `bazel build //foo` has a terrible UX. There is no transition happening to either linux or windows. Even worse, if the build defaults to macos platform, then the commands builds foo for a target platform that is not needed, and possibly not supported. This makes it more difficult to build a sub-component than the whole top-level goal, and makes a counter-intuitive UX. Some targets cannot build with the default platform, and there is no way to provide a default that works for everything.

Maybe what we need is a `default_platform` instead of a `platform` parameter. This removes the coercive aspect, where a build fails if you do not use the right platform to build the target. This would give

pkg_zip(

  name = "linux_product",

  srcs = [foo, bar]

  default_platform = "linux"

)

pkg_zip(

  name = "windows_product",

  srcs = [foo, baz]

  default_platform = "windows"

)

cc_binary(name=foo, default_platform = "windows")

cc_binary(name=bar, default_platform = "linux")

cc_binary(name=baz, default_platform = "windows")

and still allow //foo to be compiled in the linux platform, as a dependency of //linux_product.

Conversely, this is where a platforms list makes sense, so that an error can still be raised if the targeted platform is not in the list. And the first one can be used as the default one when used as a top-level build. Or all of them, but that means a lot of platforms in our case.

pkg_zip(

  name = "linux_product",

  srcs = [foo, bar]

  platforms = ["linux"]

)

pkg_zip(

  name = "windows_product",

  srcs = [foo, baz]

  platforms = ["windows"]

)

cc_binary(name=foo, platforms = ["windows", "linux"])

cc_binary(name=bar, platforms = ["linux"])

cc_binary(name=baz, platforms = ["windows"])

At this points, it starts to look like target_compatible_with, because we specify the platforms that the target supports (as a enumeration of platforms instead of a list of required platform features, but still). The main difference being the default_platform semantics that target_compatible_with does not (and cannot) carry.

I hope it makes our potential uses of the feature a bit more clear.

Best regards,

-- Guillaume.

[Gregg Reynolds]

On Wed, Aug 10, 2022 at 6:29 AM Guillaume Maudoux <guillaum...@tweag.io> wrote:

Well my concern is about the dependencies of targets that have the platform attribute. For them, there is no hint about the platform that should be used.
Even worse, if a target is used as a dependency of several top-level targets built for different platforms, then it is impossible to use the platform parameter there.

You are right, an example is needed.
Expanding on yours, our setup looks like this:

pkg_zip(

  name = "linux_product",

  srcs = [foo, bar]

  platform = "linux"

)

pkg_zip(

  name = "windows_product",

  srcs = [foo, baz]

  platform = "windows"

)

cc_binary(name=foo)

cc_binary(name=bar)

cc_binary(name=baz)

Running `bazel build //linux_product` or `bazel build //windows_product` would work pretty well. Everything is setup by default to the right platform.

But running `bazel build //foo` has a terrible UX. There is no transition happening to either linux or windows.

Isn't that an unavoidable side effect of transitions in general? Even without a platform attribute, pkg_zip could have a transition function that would not be invoked if you built the cc_binary targets separately.

Maybe what we need is a `default_platform` instead of a `platform` parameter.

Or a target-local 'transition' attribute whose value is a transition function.

[Tony Aiuto]

Thanks for the example, Guillaume.  What may be missing in the conversation are the implicit requirements the Bazel team have on their minds. We want to provide expressivity, but also want to prune down the number of actions actually executed.  We have target_compatible_with to filter out what won't work. We want platforms to mean 'build for all of these'.  What we don't want is that the default behavior for bazel build of an intermediate target to build for all the platforms they *could* build for.  What we want is that the user (which is either the person typing bazel build, or a target which has to be built for a particular platform) to control the target platform. There could be a default platform, but we would want that as a different attribute from platforms.

But, coming back to an earlier post of mine, I don't think platforms is the right choice most of the time. If you build a target for two different platforms at the same time, then bazel must put them under two distinct output paths, and whatever consumes that must understand it, and package them up in two distinct places.  That is going to be tricky logic.  The easier to use case is to have a single platform, and let you specify N versions of the target, each with a distinct name based on the platform.

--

You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAO40MinCQ0VwppjEqjD%3D0zh4HqfhY8F6A49dFcuYBrT3Zn9hLw%40mail.gmail.com.

[John Cater]

On Monday, August 8, 2022 at 11:18:54 AM UTC-4 d...@mobileink.com wrote:

I think this should be rethunk.  It's not really about platforms, its about toolchain selection. What the proposed attribute really does is modify the constraints on toolchain selection, and that need not be restricted to platform constraints.

While there has also been a long-standing request for the ability to "hard-code" a specific toolchain for a specific portion of the build graph, that's not something I'm trying to specifically address with this proposal (and for reasons I'm happy to get into in another thread, I don't think it's actually a practical request).

This proposal is aimed squarely at reducing the number of boilerplate flags that users need to supply when they invoke Bazel: if you have a target that can only every run on a specific platform, currently you need to always pass the relevant "--platforms" flag when building that (either at the command line, in a blazerc, or by writing a wrapper script that invokes Bazel for you). In these cases, which seem to not be particularly rare after talking to users, specifying the platform directly on the target will reduce overall complexity by making the build more explicit and requiring less out-of-band information on how to invoke Bazel.

[Greg Estren]

FYI both John Cater and I were on (separate) vacations the past few weeks. We're both getting back up to speed.

Thoughts:

• I agree we should be really careful about the relationship between "platform"  and "target_compatible_with". Ideally we could collapse them conceptually, but I'm not sure that's practical. Lacking that we should do everything we possibly can to guide clear understanding of their differences.
• I also suspect fat-style targets are best handled by custom rule logic. It's not immediately obvious exactly how a multi-platform target should build and collate its dependencies, and I think different rules would naturally use different logic. But from an API perspective it would be amazing if we could keep the API standard: is it possible for a custom rule to re-use the "platform" attribute and apply its own logic over it? It'd definitely be confusing if a rule had both a "platform" and "platforms" attribute, with unclear intuition on which applies when.
• How to build non-top-level libraries is an ongoing challenge. I've seen proposals from "make libraries unbuildable" to "auto-set compatible platforms" to "build with default platforms" to "skip". Practically, can we just handle them through the existing target_compatible_with and incompatible target skipping? Was the concern from the last example performance or build failure or both?
• Reminder that the new cquery --output=files can help find build outputs
• This is a step toward flagless builds. Aside from the benefits listed here, flagless builds can guarantee consistency: target A will always build exactly the same way no matter who invokes it from where (user A, user B, CI system, deployment system). I imagine that naturally leads towards other flag settings besides platforms. Is there a story for that? If not supported in this specific proposal, what's the forward looking view of the API that would support that?
• I have to re-emphasize the ever-present efficiency concern. Building multiple targets at once with different platforms can make the build N times larger and slower, quite easily. The list comprehension example above, for example, I could see easily regressing build performance: user A adds two new platforms to that comprehension and user B who runs "bazel build //:all" suddenly sees 2x slower builds. We're trying to work on efficiency improvements and measurement tooling. Does anyone have concerns in the interim?
• I could imagine scenarios where users want to build a sub-target with a different platform. The current proposal would make that an error, right? Are there scenarios where we'd want that to work fluidly? Anything in Apple land?
• Small nit: the "fail if 'platform" doesn't match --platforms" logic: does that apply when --platforms is at its default value? What's the rationale?

Are there any other requirements not captured so far by this thread? Can we ensure every meaningful requirement (i.e. some combination of targets with some combination of platform requirements expected to build correctly by some invocation) are capture in the Background section?

--

You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/2c51d750-c70d-4621-9ab1-28d04ecc0e52n%40googlegroups.com.

[Greg Estren]

+Philipp Schrader, incompatible target skipping expert: this doc and the comments explore a) relationship between declaring the target platform and target_compatible_with, b) what to do when directly building libraries that are deps of platform-specifying binaries (see example code 5 messages up).

[Konstantin]

This whole effort looks like an attempt to achieve what "one to many" transitions already can do, but hopefully simpler at least in some cases. Do I get it right? If so the feature must stay simple otherwise there is no point in replacing (or even adding) one complexity with another.

[Konstantin]

Honestly instead of investing efforts into this new feature I'd rather like to see very much necessary "target_compatible_with" improvements, such as ability to skip incompatible targets from the filegroup (instead of declaring the whole file group incompatible) and/or making IncompatiblePlatformProvider available to Starlark rules to let them make their own decision on being compatible or not depending on their attributes compatibility.

[Philipp Schrader]

I think the restrictions in the proposal make sense in order to keep it well scoped. Some thoughts based on previous responses:

• Making the new attribute a label list creates various challenges that I don't know how to deal with. E.g. to Tony's point earlier, how do you deal with having multiple output paths? And building for all platforms in the list when the target is a top-level target seems unnecessary most of the time.
• The check on --platforms and the platform attribute makes sense to me. Otherwise, it'll be too easy to create more issues like #16099. I.e. in my experience so far, it's suboptimal when rules (or rule sets) automatically transition without user control.
• Despite the previous bullet, I think erroring out on --platforms and platform mismatch is a little too strict. Perhaps the target should become incompatible at that point? We set --platforms internally for every build. That means we can't make use of the platforms attribute. Using it would mean that we will introduce new errors for at least one build variant.
• To add to my previous bullet, if a --platforms/platform mismatch were to instead make the target incompatible, perhaps it may be better to extend target_compatible_with to accept a platform. I feel like it would be relatively easy to extend the existing logic to support matching an exact platform.
• To summarize the previous few bullets, I continue to want to make "bazel build //..." work for all platforms. That is what our work on target_compatible_with is inspired by. If I'm understanding the proposal correctly, using the platform attribute would introduce new errors rather than eliminate them.
• If we were to loosen the restriction and make the platform attribute an unconditional transition, then I feel like it's not expressive enough. At least internally we have a "transition_filegroup" rule that lets users specify things like compilation_mode, features, platform, and a couple other things. But maybe not being expressive enough is the perfect way to introduce the feature and get user feedback on it.
• If the platform transition becomes an unconditional transition (i.e. ignores --platforms) then it probably makes sense to error out if target_compatible_with is also set. I don't see a use case for specifying both a platform and target_compatible_with.
  
• I don't think this proposal should deal with dependencies of targets that specify the platform attribute. I can understand the desire to make this kind of problem easier, but that's a whole separate discussion that involves more than this proposal.
• On the topic of flagless builds: I can understand the desire to make "bazel build :a" mean "build :a the same, no matter where, by whom", but to me "build :a for my current machine so I can actually run it" makes more intuitive sense in general. If the former approach is the goal, that's fine. The attribute doesn't prohibit either approach.
  

Some alternate syntaxes that I can think of, but may be undesirable for various reasons.

Create a way to express an incoming edge transition right in the BUILD file. I think this is basically an extension of the proposed syntax.

cc_binary(
    name = "some_binary",

    transitions = {
        "platforms": ":windows",
        "compilation_mode": "opt",

    },

)

Or create a way to express an outgoing edge transition via an extended, ugly Label syntax,

pkg_tar(

    name = "win_files",
    deps = [

        ":some_binary?platforms=:windows",

        ":other_binary?platforms=:windows",

    ],

)

pkg_tar(

    name = "linux_files",

    deps = [

        ":some_binary?platforms=:linux",

        ":other_binary?platforms=:linux",

    ],

)

Anyway, I think the proposal makes sense and the restrictions are good to start with. That being said, I don't think we would end up using this feature. We possibly may use it in some specific use cases if the mismatch restriction went away.

- Phil Schrader

[Philipp Schrader]

Really sorry for not taking a look earlier!

I hope to have useful feedback by the end of the week.

At a high level, I worry that a feature like this makes problems like #16099 more common. I.e. it'll be easier to trigger transitions to platforms for which you may not even have a toolchain. E.g. if you're not set up to cross-compile for MacOS on Linux, setting platform="//:macos" is a bad idea. And setting target_compatible_with won't help because the transition makes the target compatible.

But again, I hope to actually spend some time thinking about this this week. Apologies for the delay!

Phil

On Tuesday, September 13, 2022 at 12:38:47 PM UTC-7 gre...@google.com wrote:

[Greg Estren]

On Mon, Sep 19, 2022 at 4:26 AM Philipp Schrader <philipp....@gmail.com> wrote:

• On the topic of flagless builds: I can understand the desire to make "bazel build :a" mean "build :a the same, no matter where, by whom", but to me "build :a for my current machine so I can actually run it" makes more intuitive sense in general. If the former approach is the goal, that's fine. The attribute doesn't prohibit either approach.
  

This is the crux of it - the fundamental motivating factor. 

You're making some good points about ensuring this motivation retains elegant integration with existing "platformy" declarations like target_compatible_with.

Actually, what does  "bazel build :a" --> "build :a the same" mean if two users call that from different host machines (and different executor machine access)? There's a clear idea of promoting consistency for building ":a:" no matter who's doing it, without the invoker having to remember an arcane set of flags or even have access to another bazelrc file. But yeah, actually getting the same result also depends on the host machine's setup.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/35274e02-ef79-4209-8fa0-7d0dee212fd1n%40googlegroups.com.

[Gregg Reynolds]

On Wed, Aug 3, 2022 at 2:27 PM 'John Cater' via bazel-dev <baze...@googlegroups.com> wrote:

There's been a standing request for years to be able to directly set the target platform on a target, instead of at the command line with the "--platforms" flag. I've just created a proposal for a way to do that, so please take a look and let me know what people think about it.

The proposal is at https://github.com/bazelbuild/proposals/blob/main/designs/2022-08-03-platforms-on-targets.md

Can't this be handled by the simple expedient of defining a proxy rule whose sole purpose is to transition the toolchain, and whose dependencies are just those targets that must be built with that toolchain?

This idea came to me when working on rules for the js_of_ocaml compiler, which transpiles OCaml bytecode to javascript.  The jsoo_library rule depends on ocaml_module targets, which must produce .cmo (bytecode) files. But ocaml_module may also produce .cmx files (native code).  It's the toolchain that makes the difference.  So jsoo_library runs a transition on its deps attribute that sets the platform such that a vm-producing toolchain is always selected.  The jsoo_library rule itself runs a tool that will usually be built with a native-code emitting toolchain.  So no matter what platform you set for your command-line build command, jsoo_library targets will serve as platform-setting proxies, so to speak, switching the toolchain selected to build their deps.

I used a similar concept a few days ago to get around the restriction that transitions on configurable attributes cannot handle selects.  An ocaml rule may need to select  dependency, but there's a transition involved. So I defined a rule, ocaml_selection_proxy, with a single 'selection' attribute. That's where I put the select function, and the original rule depends on the ocaml_selection_proxy, which just passes on the selected dep.

To set a platform I think it would suffice (I have not tested this) to write a rule with a single attribute, to be set to the target needed the constant platform constraints, and put a transition on that attribute, whose implementation would look something like this (from my jsoo_library rule):

    return {
        "//command_line_option:platforms" : "@ocaml//host/target:vm"
    }

The proxy rule would then just pass on the providers from the dependency.  This kind of rule is pretty trivial, and a few simple examples would probably suffice for anybody  who needs this sort of thing.

Cheers,

Gregg

 

[John Cater]

Apologies for the long silence: first I was out with COVID and then I had a dozen other things land on my desk.

My conclusion from this discussion is that the proposal is flawed: specifically, there is no way for Bazel to provide a simple functionality that is useful for all or most rules/users. I think the way forward will be to come up with a simple Starlark-only rule that accomplishes some of the limited goals of this proposal, in a way that users can opt in as needed and rules can extend in ways that make sense.

I'm going to close this out and hopefully come back early next year with that simpler version of this feature.

Thanks all!

John Cater