Multi-platform build question

[Jonathan Perry]

I have a repo that internally builds and executes entirely on jdk20.

I also export some parts of that repo for external-to-my-repo consumption; when I do so, I need the exported artifacts to be built with jdk11.  Further, I have a few artifacts which I need to cut out, and replace with maven coordinates when I export.

To do this, I have done the following:

- Defined two java toolchains, one for jdk20, one for jdk11

- Defined a config_setting to indicate if we are building for export, which defaults to False

- Defined the default setup in .bazelrc which sets all of --[tool_]java_[language|runtime]_version=20

- Defined a transition which sets 'building_for_export=true' and 'java_[language|runtime]_version=11' (note it does _not_ set tool_* to 11)

- Used select(...) on the building_for_export to appropriately resolve the artifacts I need to cut&replace

This all works super-smoothly except for one wrinkle: for java_binaries built to be invoked in genrules, I would very much like to have building_for_export revert to the default, so that the resultant binaries have the non-cut artifacts.  This appears to happen correctly for the jdk (i.e. inputs to genrules are built at jdk20 and the genrules themselves are invoked using jdk20, while inputs to the final jar outputs are built at jdk11).  So I _think_ what I'm asking is how I write either a config_setting that only applies when cfg != host, or else a select that is able to discern when cfg == tool?  Or maybe I'm barking up the wrong tree.

Thank you!

[Brian Silverman]

The simple way is to trigger based on the JDK version in use, instead of your hand-set config_setting. I think you can do that with config_setting(values = {"java_language_version": "20"}), although I've never tried it with the Java rules. There's no way to make a custom config_setting do what you want, fundamentally you want to add an additional change to the target->exec transition, but you can't do that directly.

If you look in `src/main/java/com/google/devtools/build/lib/rules/java/JavaOptions.java` in the Bazel source tree, --tool_java_language_version (etc) are defining a transition which changes --java_language_version (etc) for the exec configuration (which it still calls host, which is the old name for a very similar concept).

The other option I see is to define custom platforms for target and exec, with a constraint_setting to tell them apart. Selecting on the constraint_setting is easy, but taking over your platform definitions is fiddly. If you're already writing your own platforms though, then it's easy to add something to them. If you do go this route, I would simplify by removing the separate config_setting and including that in the platform, so you have two different target platforms distinguished by building_for_export, and one exec platform with building_for_export=false.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/b29076df-7453-411c-9090-a2ae3559ec17n%40googlegroups.com.

[Jonathan Perry]

Thank you for this.  I think that taking over my platform definitions might be the right long-term solution, because I anticipate a) having more export options, and b) having more flags to control the various behaviours.  I shall go away and try to figure out how to do that (though if you have pointers on where to begin with the fiddles, that would be much appreciated!).

[Jonathan Perry]

Wow, platforms.  Pretty amazing stuff.

So.  I have defined some platforms, and I'm trying to wire them all up.  I have done the following:

- constraint_setting of java_runtime, with associated constraint_values of 'legacy_java_runtime' and 'modern_java_runtime'

- constraint_setting of export with associated constraint_values of 'building_for_export' and 'not_building_for_export'

- platform of 'host_and_execution' with constraint_values of 'modern_java_runtime' and 'not_building_for_export'

- platform of 'targetting_normal_service_deployment' with constraint_values of 'modern_java_runtime' and 'not-building_for_export'

- platform of 'targetting_legacy_export' with constraint_values of 'legacy_java_runtime' and 'building_for_export'

- java_toolchain 'modern_java_toolchain' at version 20, with an associated zulu jvm @ 20, registered with a toolchain of type @bazel_tools//tools/jdk:toolchain_type

- toolchain of type @bazel_tools//tools/jdk:runtime_toolchain_type, pointing at a zulu jvm @ 20

- java_toolchain 'legacy_java_toolchain' at version 11, with an associated zulu jvm @ 11, registered with a toolchain of type @bazel_tools//tools/jdk:toolchain_type

- registration of all the platforms in WORKSPACE

- addition to .bazelrc of 'common --host_platform=host_and_execution --platforms=host_and_execution

- transition which sets --platforms=targetting_legacy_export

So my questions now are:

- does this setup seem approximately sane?

- is it right to do the transition in platform space?

- what should I be setting in the platforms' exec_compatible_with and target_compatible_with configuation so that I end up with that desired outcome I layed out above, please?  Or should I set something else somewhere else?

Thank you so much for your time.

[Jonathan Perry]

Closing this out - it turns out all I need to do is set target_compatible_with to modern/legacy on each of the compilation toolchains, and then exec_compatible_with to modern on the single jdk_runtime toolchain. Then I do my transition on platforms, and it all works. 

Incredible stuff. Everything is intentional and clean. Thank you for your help. 

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/fecbe7d1-759c-4380-808e-0b9c02c31dbfn%40googlegroups.com.