platforms, build_settings, transitions and features

[fredrik...@gmail.com]

Dear Bazel community,

I've tried to find a path to describe a multi-platform environment where my build target is a package including executables for multiple operating systems on hardware architectures in multiple compilation modes.

Previous discussions has focused on build_settings or built in command line flags:

Defining configuration in existing rules

https://github.com/gregestren/snippets/tree/master/cc_binary_selectable_copts

How do I define a rule that combines a target built for 2 different CPU architectures into one package ?

The following concepts have been on my mind:

• platform, constraint_setting, constraint_value
• build_setting, label_flag
• config_setting
• features
• transition

Transitions can operate on build_settings, but build_settings are set individually on the command line. Platforms on the other hand are setting constraint_values, which is handy for the user when building just one of my configurations.

1. Problem

For user convenience, I would like to use --platforms=HardwareA with constraint_values, but in transition mode I have to use build_settings. How to combine?

1a. Solution

* Create matching constraint_settings and build_settings.

* Create config_settings, through config_setting_group etc., which checks a constraint_value if the build_setting is set to a dummy default, otherwise checks the build_setting (has been set on the command line or through a transition).

1b. Solution

Use --config=HardwareA where all the build_settings are set in a .bazelrc file, i.e. skip the use of platforms. This does not seem like "the Bazel way" to do it.

2. Problem

Transitions does not support toolchain --features. How should it be possible to package with different --features activated in a one-to-many transition, e.g. opt and dbg? (This specific example is possible today because compilation_mode is still a built in command line flag, but I wonder about the more general concept and our internal --features.)

3. Problem

I cannot get transition to change label_flags. There is no test for it in the Bazel source code either.

4. Suggestion

Let platforms take a build_settings dictionary, not just a constraint_values list.

Does this suggestion mean that constraint_settings and constraint_values can be removed all together? What other purpose do they fulfill?

Also, build_settings are more flexible in supporting string, int, bool and labels, but constraint_settings only support string.

[Fredrik Medley]

5. Problem (transitions and toolchain resolution)

The toolchain rule's exec_compatible_with and target_compatible_with only takes constraint_values whereas transitions do not support changing contraint_settings. That means that we cannot change to another cross compiler over a transition. How is this thought to work? Can config_settings be allowed as well?

Best regards,

Fredrik

[Greg Estren]

+John Cater for platform/toolchain expertise.

Apologies I didn't see this thread earlier, Fredrik.

On Fri, Nov 29, 2019 at 4:27 PM <fredrik...@gmail.com> wrote:

 

1. Problem

For user convenience, I would like to use --platforms=HardwareA with constraint_values, but in transition mode I have to use build_settings. How to combine?

Transitions can set platforms via the "native options" syntax: https://docs.bazel.build/versions/master/skylark/config.html#transitions-on-native-options. See https://github.com/bazelbuild/bazel/blob/d660cf22b7ffae833456a24e4c10368252fbb229/src/test/shell/bazel/platform_mapping_test.sh#L146 for a trivial example (it doesn't actually set the value, but it shows the basic syntax).

2. Problem

Transitions does not support toolchain --features. How should it be possible to package with different --features activated in a one-to-many transition, e.g. opt and dbg? (This specific example is possible today because compilation_mode is still a built in command line flag, but I wonder about the more general concept and our internal --features.)

I'm not an expert on toolchain features. Is it possible to just set --features in the transition using the same syntax as above?

Also note https://docs.google.com/document/d/12n5QNHmFSkuh5yAbdEex64ot4hRgR-moL1zRimU7wHQ includes plans for providing better access to toolchain properties that aren't otherwise exposed as constraint_values or command-line flags. That might not be adequate for what you're asking... it's more about reading toolchain properties vs. setting. But it's a similar area. i'd hope that my suggestion in the last paragraph is adequate.

 

3. Problem

I cannot get transition to change label_flags. There is no test for it in the Bazel source code either.

label_flags have required a bit more tender loving care to fully integrate (e.g. https://github.com/bazelbuild/bazel/issues/10097). I've heard some other reports of quirks coming from them. Can you file a GitHub issue describing the specific issue you're seeing?

 

4. Suggestion

Let platforms take a build_settings dictionary, not just a constraint_values list.

Does this suggestion mean that constraint_settings and constraint_values can be removed all together? What other purpose do they fulfill?

Also, build_settings are more flexible in supporting string, int, bool and labels, but constraint_settings only support string.

We have some principled reasons not to want to mege build settings and platforms. John can address this more thoroughly.

 

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/dac5be84-165f-4abe-b11b-51301c2f245e%40googlegroups.com.

[Greg Estren]

On Fri, Dec 6, 2019 at 5:11 AM Fredrik Medley <fredrik...@gmail.com> wrote:

5. Problem (transitions and toolchain resolution)

The toolchain rule's exec_compatible_with and target_compatible_with only takes constraint_values whereas transitions do not support changing contraint_settings. That means that we cannot change to another cross compiler over a transition. How is this thought to work? Can config_settings be allowed as well?

These documents explore these themes:

https://docs.google.com/document/d/1CgU-GKocMAfsUSI3bbGZ0YRkOWczitIoKs29x3zR914/view

https://docs.google.com/document/d/1cjR8oUwGAtokKbpBJR-jp26l2HxfOrKRJe20VZ7Rbvw/edit#heading=h.kzp6mqja2p3t

https://docs.google.com/document/d/12n5QNHmFSkuh5yAbdEex64ot4hRgR-moL1zRimU7wHQ/edit?pli=1#

In short, we're trying to evolve a coherent language for expressing practical platform / configuration needs using all these basic primitives. The right combinations should ultimately do these kinds of things well, and guidance and support is evolving on figuring out what they are.

 

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/90bc586c-1229-4be2-97ea-a329d591923f%40googlegroups.com.

[Fredrik Medley]

Thank you for the list of design documents. That list would be helpful to put into the online documentation about transitions.

Regarding 3, I've now filed bug reports for sorting out label_flag in transitions:

https://github.com/bazelbuild/bazel/issues/10499 - Broken repository name handling for label_flag

https://github.com/bazelbuild/bazel/issues/10500 - label_flag transition crash after shutdown when using string

/Fredrik

Den onsdag 18 december 2019 kl. 23:31:01 UTC+1 skrev Greg Estren:

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-...@googlegroups.com.

[Fredrik Medley]

From another discussion with Greg I got the following guideline that I will remember:

 

We generally recommend trying to stay pure to the intention of platforms: something belongs in a platform if running the same build on two different machines might produce different results ("exec platform" differences) or running the same built binary on different machines produces different output ("target platform" differences).

We have identified a diamond graph with different transitions on each side, so the common bottom of the graph ends up in two configurations. It turns out that the common piece does not use the build_settings that differs between the paths and should therefore only be built once. Then we have four alternatives:

1. Transition into the common bottom and reset all unused build_settings to a constant (empty string). This requires us to maintain a global list of all build_settings in the whole system, so not ideal, but it is working. (A built in way of accessing the default values of a build_setting/label_flag in Starlark would be handy here, but not necessary.)
2. Set the outputDirectoryName to a specific name, https://github.com/bazelbuild/bazel/issues/7477. The --platform_suffix=/../foo hack does not work because “-ST-<hash>” is appended after platform_suffix. (I might just move platform_suffix last for a quick fix.)
3. Let Bazel look through all dependencies to determine what build_settings and label_flags are actually used transitively. Only those configurations should be hashed into the output directory name. Basically, let each node in the analysis phase propagate used configurations up in the graph.
4. Use your clever output path solution per action.

Alternative 1 and 2 are feasible for us. Would 3 be a short term solution until 4 is in place?

 

Also, will the clever output path handling results in too many file operations on Windows? Otherwise number 3 might be a good enough solution.

Another useful feature would be to apply a transition on an alias rule. I’ll write a feature request on GitHub.

/Fredrik

[Greg Estren]

On Wed, Jan 15, 2020 at 3:54 PM Fredrik Medley <fredrik...@gmail.com> wrote:

From another discussion with Greg I got the following guideline that I will remember:

 

We generally recommend trying to stay pure to the intention of platforms: something belongs in a platform if running the same build on two different machines might produce different results ("exec platform" differences) or running the same built binary on different machines produces different output ("target platform" differences).

We have identified a diamond graph with different transitions on each side, so the common bottom of the graph ends up in two configurations. It turns out that the common piece does not use the build_settings that differs between the paths and should therefore only be built once. Then we have four alternatives:

1. Transition into the common bottom and reset all unused build_settings to a constant (empty string). This requires us to maintain a global list of all build_settings in the whole system, so not ideal, but it is working. (A built in way of accessing the default values of a build_setting/label_flag in Starlark would be handy here, but not necessary.)
2. Set the outputDirectoryName to a specific name, https://github.com/bazelbuild/bazel/issues/7477. The --platform_suffix=/../foo hack does not work because “-ST-<hash>” is appended after platform_suffix. (I might just move platform_suffix last for a quick fix.)
3. Let Bazel look through all dependencies to determine what build_settings and label_flags are actually used transitively. Only those configurations should be hashed into the output directory name. Basically, let each node in the analysis phase propagate used configurations up in the graph.
4. Use your clever output path solution per action.

Alternative 1 and 2 are feasible for us. Would 3 be a short term solution until 4 is in place?

3 is exactly the "trimming" work we're working on: the first proof of concept of said work being at https://github.com/bazelbuild/bazel/issues/6842 (being newly prioritized now that someone's dedicated to it). It's not obvious how shorter-term it is than 4, though. The most generic version of 3 has its own complexities: namely that determining the transitive uses itself requires a new traversal down the graph with its own efficiency costs. If you're willing to consider adding tags to rules (like "this rule doesn't use this flag in its transitive deps"), we could conceptually avoid that cost in preference for the user maintenance cost.

FWIW I added a "secret" new feature to cquery that outputs the transitive fragments used by rules: https://github.com/bazelbuild/bazel/commit/386b25b866bc2d68be6af04ad8ae40b0fd73a6e7 - I'm planning to integrate this into a tool that can measure these sorts of inefficiencies automatically. But you may find it useful for your own measurement or refactoring tools. For example, one could imagine a tool that measures transitive uses, then refactors BUILD fils to auto-apply the manual tags I describe above. It's still a bit clunky. But it should work fairly well in practice until more proper solutions evolve.

 

 

Also, will the clever output path handling results in too many file operations on Windows? Otherwise number 3 might be a good enough solution.

I don't know. It's too early to evaluate these issues.

 

Another useful feature would be to apply a transition on an alias rule. I’ll write a feature request on GitHub.

/Fredrik

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/006b1a5a-3912-4d49-8775-893723cd892e%40googlegroups.com.