Platforms and bazel-out layout

[Maxim Yurchuk]

Hello, 

I'm trying to use platforms in our code, because platforms looks much more better than awful --cpu with strange values such as "darwin".  

But there is problems with bazel-out layout.

Consider following BUILD file:

platform(

    name = "foobar",

    constraint_values = [

        "@platforms//cpu:arm",

        "@platforms//os:android",

    ],

)

cc_library(

    name = "lib",

)

And following command (i'm using windows)

c:\git\bazel-issues\platforms_output_dir2>c:\bazel\bazel-master build --platforms //:foobar   --incompatible_enable_cc_toolchain_resolution --toolchain_resolution_debug --compilation_mode opt ...

INFO: Writing tracer profile to 'C:/users/yurchuk/_bazel_yurchuk/dyw5dx65/command.profile.gz'

INFO: ToolchainResolution: Looking for toolchain of type @bazel_tools//tools/cpp:toolchain_type...

INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-armeabi-v7a...

INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-x64_windows...

INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_64, which does not match value @platforms//cpu:arm from the target platform //:foobar

INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:windows, which does not match value @platforms//os:android from the target platform //:foobar

INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-x64_windows, because of target platform mismatch

INFO: ToolchainResolution:   For toolchain type @bazel_tools//tools/cpp:toolchain_type, possible execution platforms and toolchains: {@local_config_platform//:host -> @local_config_cc//:cc-compiler-armeabi-v7a}

INFO: ToolchainResolution: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/cpp:toolchain_type -> toolchain @local_config_cc//:cc-compiler-armeabi-v7a

INFO: ToolchainResolution: Selected execution platform @local_config_platform//:host,

INFO: Analyzed target //:lib (1 packages loaded, 2 targets configured).

INFO: Found 1 target...

Target //:lib up-to-date (nothing to build)

INFO: Elapsed time: 0.433s, Critical Path: 0.01s

INFO: 0 processes.

INFO: Build completed successfully, 1 total action

Bazel is built from sources, commit c731933e93c19b5e70275e4f4ae2e7da90fa3a32 (Fri Oct 11 15:45:20 2019 -0700)

Since toolchain @local_config_cc//:cc-compiler-armeabi-v7a is built-in toolchain and there are nothing to compile (library is empty) build is succeeded. 

But if someone check bazel-out content you will see:

c:\git\bazel-issues\platforms_output_dir2>dir bazel-out

 Volume in drive C has no label.

 Volume Serial Number is B0A8-33A4

 Directory of c:\git\bazel-issues\platforms_output_dir2\bazel-out

14.10.2019  14:02    <DIR>          .

14.10.2019  14:02    <DIR>          ..

14.10.2019  14:02                57 stable-status.txt

14.10.2019  14:02                27 volatile-status.txt

14.10.2019  14:02    <DIR>          x64_windows-opt

14.10.2019  14:02    <DIR>          _tmp

               2 File(s)             84 bytes

               4 Dir(s)  103 032 303 616 bytes free

x64_windows-opt is confusing, because this is not x64_windows. It is maybe OK for local usage, but not suitable to use on build conveyour (I have no idea how to transform arm+android to x64_windows).

Also it is possible to use bazel-bin directory, but if use --symlink_prefix=/ option bazel-bin will not created. On our build conveyour we use --symlink_prefix=/  to make source tree clean. 

Any thoughts how to obtain name of output directory? Also, maybe it's issue and name of output directory should be more correct?

[Greg Estren]

<This is a good moment to mention Bazel's  "Building With Platforms" guidance doc. It's meant for Bazel 1.0 but because Bazel documentation is  versioned with releases and it wasn't committed until after the 1.0 candidate was already built, it won't appear by default until 1.1 is released.>

Hi Maxim,

Excellent point. The reality is output directory naming is its own delicate challenge that hasn't yet been synced with the new platform APIs. The full story is complicated. But the basic issue is we have to be extremely careful what tokens (e.g. "x86_windows", "opt") go into the output path: Too few and we risk outputs for different platforms writing to the same path and clobbering each other. Too many and we have unwieldy paths that exceed path length limits, are hard to read, etc. 

So this is an API that has to evolve. Ideas are brewing but a generic solution is going to be its own big initiative. One interim workaround is to define platform mappings: Today, all output paths are determined by the value of --cpu ("x64_windows") and --compilation_mode ("opt"). When you set --platforms you're not setting --cpu (because you don't need it), so --cpu is set to its default: "x64_windows". So you could write a platform mapping that says "when --platforms=--platforms //:foobar, set --cpu=<some Android ARM CPU>". This would make the output paths look more accurate.

We could also consider an interim Bazel mode that automatically does this: when computing output paths take the cpu from the current platform instead of --cpu. That still has some nuances to make sure we don't run into correctness problems, but it could presumably offer a better API while the more generic stuff continues to evolve.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/2f16f5da-297e-499b-9d6e-b004735edd2a%40googlegroups.com.

[Maxim Yurchuk]

Hello, Greg!

Thank you for detailed answer. As I'm understood I can specify platform mappings, and output directory will use cpu from mappings. But seems, this is not working as I'm expected. As an example I put following platform_mappings in my workspace root

platforms:

  //:foobar

    --cpu=k8

And I've expected, that (with same bazel invocation from first message)

- since I specified --platform option bazel will not rely on --cpu=k8 in toolchain resolutions 

- since bazel has cpu value it will create output directory with name "k8-opt" (or something like that)

But bazel start trying to use cpu value in toolchain resolution:

c:\git\bazel-issues\platforms_output_dir>c:\bazel\bazel-master build --platforms //:foobar   --incompatible_enable_cc_toolchain_resolution --toolchain_resolution_debug --compilation_mode opt ...

INFO: Writing tracer profile to 'C:/users/yurchuk/_bazel_yurchuk/ubpijymk/command.profile.gz'

INFO: Build option --cpu has changed, discarding analysis cache.

INFO: ToolchainResolution: Looking for toolchain of type @bazel_tools//tools/cpp:toolchain_type...

INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-armeabi-v7a...

INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-x64_windows...

INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_64, which does not match value @platforms//cpu:arm from the target platform //:foobar

INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:windows, which does not match value @platforms//os:android from the target platform //:foobar

INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-x64_windows, because of target platform mismatch

INFO: ToolchainResolution:   For toolchain type @bazel_tools//tools/cpp:toolchain_type, possible execution platforms and toolchains: {@local_config_platform//:host -> @local_config_cc//:cc-compiler-armeabi-v7a}

INFO: ToolchainResolution: Selected execution platform @local_config_platform//:host, type @bazel_tools//tools/cpp:toolchain_type -> toolchain @local_config_cc//:cc-compiler-armeabi-v7a

INFO: ToolchainResolution: Selected execution platform @local_config_platform//:host,

INFO: ToolchainResolution: Selected execution platform @local_config_platform//:host,

ERROR: C:/users/yurchuk/_bazel_yurchuk/ubpijymk/external/local_config_cc/BUILD:47:1: in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'k8'

ERROR: Analysis of target '//:lib' failed; build aborted: Analysis of target '@local_config_cc//:toolchain' failed; build aborted

INFO: Elapsed time: 0.447s

INFO: 0 processes.

FAILED: Build did NOT complete successfully (0 packages loaded, 54 targets configured)

So, no idea how to make predictable names in bazel-out with platform mappings. Or, maybe, I missed something? 

Few thoughts about platform output naming: why not just add field "platform_canonical_name" as platform argument and create directory with that name in bazel-out? There are no problems mentioned by you: 

- bazel built-in directories can be declared in compatible way with existing --cpu stuff

- there is no problem mentioned as "Too few and .... Too many and ...", because it is user's business. User can by self specify not intersecting names for own platforms. Moreover,  I think it's totally unsolvable problem to generate platform directory output name on bazel side, because user can put own constraint values (for example glibc version, compiler version, sanitaizer settings and many others) which are not built-in in bazel, so bazel can't use its in directory naming. 

вторник, 15 октября 2019 г., 21:45:33 UTC+3 пользователь Greg Estren написал:

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-...@googlegroups.com.

[Greg Estren]

CCing Marcel if he has any idea why cc_toolchain_suite is trying to resolve against --cpu when   --incompatible_enable_cc_toolchain_resolution is on.

On Wed, Oct 16, 2019 at 9:45 AM Maxim Yurchuk <maxim....@gmail.com> wrote:

Hello, Greg!

Thank you for detailed answer. As I'm understood I can specify platform mappings, and output directory will use cpu from mappings. But seems, this is not working as I'm expected. As an example I put following platform_mappings in my workspace root

platforms:

  //:foobar

    --cpu=k8

And I've expected, that (with same bazel invocation from first message)

- since I specified --platform option bazel will not rely on --cpu=k8 in toolchain resolutions 

- since bazel has cpu value it will create output directory with name "k8-opt" (or something like that)

Yes on all counts. (recognizing above that //:foobar implies "@platforms//cpu:arm").

 

But bazel start trying to use cpu value in toolchain resolution:

ERROR: C:/users/yurchuk/_bazel_yurchuk/ubpijymk/external/local_config_cc/BUILD:47:1: in cc_toolchain_suite rule @local_config_cc//:toolchain: cc_toolchain_suite '@local_config_cc//:toolchain' does not contain a toolchain for cpu 'k8'

So, no idea how to make predictable names in bazel-out with platform mappings. Or, maybe, I missed something? 

I think your intuition is correct and this is a confusing result. I remember running into this exact issue when I first experimented with custom C++ toolchains.  But I think there's still a workaround. It's involved enough where I wrote it out in a little doc. Please see https://github.com/gregestren/snippets/blob/master/custom_cc_toolchain_with_platforms/using-with-platform-mappings.md.

I'll leave it the  C++ rule experts for input on better principled solutions.

Also, for what it's worth, --platform_suffix is a nice flag you might also find interesting.

 

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/22bf223d-406d-4412-afaa-65a9870013c6%40googlegroups.com.

[Greg Estren]

 

Few thoughts about platform output naming: why not just add field "platform_canonical_name" as platform argument and create directory with that name in bazel-out? There are no problems mentioned by you: 

- bazel built-in directories can be declared in compatible way with existing --cpu stuff

- there is no problem mentioned as "Too few and .... Too many and ...", because it is user's business. User can by self specify not intersecting names for own platforms. Moreover,  I think it's totally unsolvable problem to generate platform directory output name on bazel side, because user can put own constraint values (for example glibc version, compiler version, sanitaizer settings and many others) which are not built-in in bazel, so bazel can't use its in directory naming. 

For the status quo I agree different users have different needs and it may not be Bazel's place to be too opinionated for everyone. One theoretical challenge that may or may not matter by just using "bazel-out/myplatform" is if a user builds with --platforms=//platform1, then builds again with --platforms=//platform2, none of the common build work can be shared even if these platforms aren't very different from each other.

We have deeper plans to change all of this, wildly. See Experimental Content-Based Output Paths for super-future stuff.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/22bf223d-406d-4412-afaa-65a9870013c6%40googlegroups.com.

[Maxim Yurchuk]

Hello Greg!

Thanks for the answer I understood problem with that common work will not shared. 

Another few thoughts about this 

1) why common stuff can't be shared via bazel cache? 

2) also, if common stuff will be shared via cache, another problem suffers: too many IO operations to copy same files to bazel-out/myplatform1 and bazel-out/myplatform2 from cache. 

Maybe Experimental Content-Based Output Paths proposal can be modified in such way:

1) If someone build target foo for myplatform1, it will be placed in bazel-out/myplatform1/../foo. But only foo outputs, not intermediate targets. 

2) Under the hood bazel can use content-based output paths (or any suitable way), and just copy from scary md5-based paths to bazel-out/myplatform1/../foo  

If such plan is acceptable, then bazel can start use bazel-out/myplatform1 without implementation of tricky layouts in bazel-out. When tricky layout will implemented you can just start copying from new layout to user friendly layout  bazel-out/myplatform/... (and only needed targets)

So, no overhead here except of copying requested targets (without intermediate targets). But, seems such copying it's unavoidable, because user must be able find requested artifacts in some predictable place. 

суббота, 19 октября 2019 г., 2:24:19 UTC+3 пользователь Greg Estren написал:

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/22bf223d-406d-4412-afaa-65a9870013c6%40googlegroups.com.

[Maxim Yurchuk]

Also thank you for pointing to "--platform_suffix", but seems it's unusable for me, because its just add suffix to output path, which is still unpredictable for me. 

понедельник, 21 октября 2019 г., 18:46:18 UTC+3 пользователь Maxim Yurchuk написал: