Adding a constraint to default target platform

[Ivo List]

Hi everyone,

I am working on toolchain resolution for Java rules. One of the obstacles is that target and host platforms are identical which necessarily results in resolving the same toolchains (where java uses two different JDKs). Solution to this is to add a specific constraint to auto-generated target platform. 

More on PR: https://github.com/bazelbuild/proposals/pull/196

Regards,

Ivo List

[John Cater]

The actual proposal is now live at https://github.com/bazelbuild/proposals/blob/master/designs/2020-09-29-new-default-target-constraint.md

Specific comments to follow.

--
You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/9bd2d56e-1eab-4274-837d-72555b9355acn%40googlegroups.com.

[John Cater]

Specific comments:

I think I mentioned to you last week, the special targets @bazel_tools//platforms:target_platform and @bazel_tools//platforms:host_platform are deprecated and scheduled to be removed (support for them only still exists for internal-to-Google reasons, otherwise they'd be gone). See https://github.com/bazelbuild/bazel/pull/9109 and related issues, and https://github.com/bazelbuild/proposals/blob/master/designs/2018-10-22-autoconfigured-host-platform.md for the design proposal.

Currently in Bazel, the default value of the `--host_platform` flag is @local_config_platform//:host, and the value of the `--platforms` flag is whatever was chosen as the host platform. The logic is present in PlatformOptions.computeTargetPlatform() and PlatformOptions.computeHostPlatform() (see https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/analysis/PlatformOptions.java). It's complicated by the support for disabling "--incompatible_auto_configure_host_platform", but that's all legacy for Bazel. Support only still exists for Google-internal reasons.

In your proposal, you state:

```

Rationalia for those two requirements are:

-  Use embedded JDK for compilation in order to have hermetic compiles.
-  Use locally installed JDK in order to execute Java code without Bazel installed.

```

I thought we weren't supporting using the embedded JDK for builds, just to execute Bazel? Why can't we require users of Java rules to install a local JDK (and auto-configure that properly for them, including both toolchain types), or to use a hermetic remote JDK (and configure that properly in the WORKSPACE)? That seems the cleanest approach and should allow us to properly handle local and remote builds.

Fundamentally, Bazel shouldn't be trying to distinguish between "builds for the target platform" and "builds for an execution platform that is identical to the target platform": they should be handled the same way because they are the same builds.

[Ivo List]

On Tue, Sep 29, 2020 at 4:16 PM John Cater <jca...@google.com> wrote:

Specific comments:

I think I mentioned to you last week, the special targets @bazel_tools//platforms:target_platform and @bazel_tools//platforms:host_platform are deprecated and scheduled to be removed (support for them only still exists for internal-to-Google reasons, otherwise they'd be gone). See https://github.com/bazelbuild/bazel/pull/9109 and related issues, and https://github.com/bazelbuild/proposals/blob/master/designs/2018-10-22-autoconfigured-host-platform.md for the design proposal.

Currently in Bazel, the default value of the `--host_platform` flag is @local_config_platform//:host, and the value of the `--platforms` flag is whatever was chosen as the host platform. The logic is present in PlatformOptions.computeTargetPlatform() and PlatformOptions.computeHostPlatform() (see https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/analysis/PlatformOptions.java). It's complicated by the support for disabling "--incompatible_auto_configure_host_platform", but that's all legacy for Bazel. Support only still exists for Google-internal reasons.

Thanks, I'm aware of this. I didn't refer to deprecated labels in the proposal. I have PR https://github.com/bazelbuild/bazel/pull/12192, of course if we go forward with this proposal.

 

In your proposal, you state:

```

Rationalia for those two requirements are:

-  Use embedded JDK for compilation in order to have hermetic compiles.
-  Use locally installed JDK in order to execute Java code without Bazel installed.

```

I thought we weren't supporting using the embedded JDK for builds, just to execute Bazel?

Sorry for the slip of pen. Instead of embedded JDK should be JDK from @remotejdk11.

 

Why can't we require users of Java rules to install a local JDK (and auto-configure that properly for them, including both toolchain types),

or to use a hermetic remote JDK (and configure that properly in the WORKSPACE)? That seems the cleanest approach and should allow us to properly handle local and remote builds.

Basically currently we require both for the reasons explained in the proposal. Requiring only one of them breaks the other requirement.  

First option to depend only on local JDK for compilation and running. The builds are not hermetic. More specifically our Java toolchains add command line parameters to JDK which don't work with all JDKs. Also they depend on JDK's internal compiler. This breaks easily. It could be fixed with auto-configuration, but this potentially requires a lot of effort.

I like the second option better, i.e. to depend on @remotejdk11 only for both compilation and running. It is simple, no special handling for local JDK is needed, users that don't have local JDK are happy (for example https://github.com/bazelbuild/bazel/issues/12012).

I am cautious about it though, because it might break current user's builds. I think I saw a reported issue, where at one release --javabase already pointed to @remotejdk, however I cannot find it again. If I find it, I'll post it. But the main reason why I don't want to go down this path is: "You cannot distribute built files, without specifying a different target platform." 

A combination - use autoconfigured local jdk if it passes all checks for both compilation and running, otherwise @remotejdk. This seems too complicated and without a proper user story to back it up. 

Fundamentally, Bazel shouldn't be trying to distinguish between "builds for the target platform" and "builds for an execution platform that is identical to the target platform": they should be handled the same way because they are the same builds.

I see. Using an extra constraint in the proposal, makes the builds you mention slightly different. For target platform we compile with @remote and execute with @localjdk, and for execution platform we compile and execute with @remotejdk. I don't know if this is the best selection, it is not identical for sure. One more thing - on the execution platform you always get the same JDK to compile and execute. The latter can be a counter argument to my proposal.

Perhaps then using an extra toolchain models this better. Previously this was discarded as too complicated. Rephrasing it in a more simple way: We have a toolchain for JRE, a toolchain for JDK (both currently java_runtime). We use JRE for running and JDK for compiling. The same happens for builds for target and builds for execution. We can handle default selection the way it was done before. And the user can independently specify JDK and JRE to use on any platform.

 

On Tue, Sep 29, 2020 at 9:50 AM John Cater <jca...@google.com> wrote:

The actual proposal is now live at https://github.com/bazelbuild/proposals/blob/master/designs/2020-09-29-new-default-target-constraint.md

Specific comments to follow.

On Tue, Sep 29, 2020 at 7:16 AM 'Ivo List' via bazel-dev <baze...@googlegroups.com> wrote:

Hi everyone,

I am working on toolchain resolution for Java rules. One of the obstacles is that target and host platforms are identical which necessarily results in resolving the same toolchains (where java uses two different JDKs). Solution to this is to add a specific constraint to auto-generated target platform. 

More on PR: https://github.com/bazelbuild/proposals/pull/196

Regards,

Ivo List

--
You received this message because you are subscribed to the Google Groups "bazel-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/9bd2d56e-1eab-4274-837d-72555b9355acn%40googlegroups.com.

--

Ivo List |

Software Engineer |

il...@google.com |

[John Cater]

Sorry for the delays, I've been trying to wrap my head around the issues you've raised here. I am still struggling to understand the root issue. Is it that the Java rules have historically used the locally-installed JDK for local builds, and a remote JDK for remote builds, and we want to maintain that?

The first question I have is, who wants to maintain that behavior? And the second is, can users of Bazel's Java rules be asked to configure this? We currently have the `register_execution_platform` function available in WORKSPACE. Bazel users who utilize both Java rules and remote execution should be able to declare a new execution platform describing their remote execution, and a new hermetic Java toolchain that works on that platform, with the needed constraints to ensure they are selected in tandem. Meanwhile, users of Java rules who do not use remote execution can either use their local JDK for all builds, or configure a hermetic JDK for local builds as well.

On Thu, Oct 1, 2020 at 6:08 AM Ivo List <il...@google.com> wrote:

I'm not so fond about this proposal anymore. 

I found a couple of use-cases that are weird:

• building using java14
  what happens is that I need to define both local and host platform to select JDK14 for compilation (the weird part is that I should only prescribe target platform, but if I do so JDK11 gets used, without compiler for JDK14)
• bootstrapping with only local JDK:
  local JDK is used for compilation (weird part is that I need to use located_on_target_and_host constraint in host platform constraint)

The problem seems to be that we should match host execution JDK with target Java toolchain.

This can be done by adding a third toolchain type - which is complex and can still use a wrong pair of JDK and toolchain.

Another option is to add an attribute to java_toolchain rule, whose value points to the most appropriate java_runtime. 

This would ensure that for compilation for either target or host, a consistent pair of JDK and toolchains are used. It would decouple JVM used to execute on the execution platform from JDK used to compile for the target.

Examples: 

• default toolchain11 points to @remote_jdk11//:jdk
• toolchain14 points to @remote_jdk14//:jdk
• boostrap_toolchain can point to @local_jdk//:jdk

Cross-product doesn't really happen, because for each toolchain you have one (or perhaps two JDKs). 

What do you think?

[Ivo List]

I'm not so fond about this proposal anymore. 

I found a couple of use-cases that are weird:

• building using java14
  what happens is that I need to define both local and host platform to select JDK14 for compilation (the weird part is that I should only prescribe target platform, but if I do so JDK11 gets used, without compiler for JDK14)
• bootstrapping with only local JDK:
  local JDK is used for compilation (weird part is that I need to use located_on_target_and_host constraint in host platform constraint)

The problem seems to be that we should match host execution JDK with target Java toolchain.

This can be done by adding a third toolchain type - which is complex and can still use a wrong pair of JDK and toolchain.

Another option is to add an attribute to java_toolchain rule, whose value points to the most appropriate java_runtime. 

This would ensure that for compilation for either target or host, a consistent pair of JDK and toolchains are used. It would decouple JVM used to execute on the execution platform from JDK used to compile for the target.

Examples: 

• default toolchain11 points to @remote_jdk11//:jdk
• toolchain14 points to @remote_jdk14//:jdk
• boostrap_toolchain can point to @local_jdk//:jdk

Cross-product doesn't really happen, because for each toolchain you have one (or perhaps two JDKs). 

What do you think?

On Tue, Sep 29, 2020 at 6:56 PM Ivo List <il...@google.com> wrote:

[Ivo List]

On Thu, Oct 1, 2020 at 8:33 PM John Cater <jca...@google.com> wrote:

Sorry for the delays, I've been trying to wrap my head around the issues you've raised here. I am still struggling to understand the root issue. Is it that the Java rules have historically used the locally-installed JDK for local builds, and a remote JDK for remote builds, and we want to maintain that?

Pre-historically as far as I know Java rules used only embedded JDK. As I understand this has caused issues to users, when they upgraded Bazel, because of changes done in embedded JDK or upgrading it to a new Java version. This spawned an issue https://github.com/bazelbuild/bazel/issues/6105 and debate https://groups.google.com/d/topic/bazel-sig-jvm/wA9f-B6Tl5w/discussion. The result of this discussion and the work that followed was that now Bazel uses a combination of local and remote JDKs (and the embedded JDK only for its own process).

I'm afraid using only remote jdk (no local) might lead to similar issues as using embedded JDK caused in the past.  But I might be wrong - I don't know much about what was happening with embedded JDK, and remote JDK seems quite stable in the last 6 months. Still if default to a version of remote JDK, we will break some users when upgrading it.

The first question I have is, who wants to maintain that behavior? And the second is, can users of Bazel's Java rules be asked to configure this? We currently have the `register_execution_platform` function available in WORKSPACE. Bazel users who utilize both Java rules and remote execution should be able to declare a new execution platform describing their remote execution, and a new hermetic Java toolchain that works on that platform, with the needed constraints to ensure they are selected in tandem. Meanwhile, users of Java rules who do not use remote execution can either use their local JDK for all builds, or configure a hermetic JDK for local builds as well.

I think the goal here is that Bazel works out of the box for as many users as possible. This implies also minimising the configuration asked from the user and being very careful with the auto-configuration we do. Users using remote execution need to be quite advanced already and I guess configuration is not problematic for them. We must be friendly to more basic users. For them we could default to using local JDK for compilation (with or without fail-back to remote), but then we need to auto-configure which toolchain is used (and this in some cases leads to degrading toolchain functions).

I would like to maintain current behaviour or improve it if it gets me closer to that goal.

[Lukács T. Berki]

Historic behavior aside, my understanding is that the current default is to use a downloaded JDK to run Java compilation (i.e. the host JDK) and a local JVM to run whatever is built. The embedded JDK is only used for running Bazel. This is so that:

1. We know what version of javac we are running (this is important because Error Prone uses non-public APIs and Error Prone integration is very popular) and it always works without having to separately download one before you run Bazel
2. The built Java binaries run on the local machine
3. We can upgrade the JDK used to run Bazel without affecting anyone and cut it down to the minimum

This is in line with Ivo's desire to work out of the box for as many people as possible: Java compilation works (because we download a known-good machinery) and the binaries work because they use a local JVM.

The part I don't understand is why we need to "match host execution JDK with target Java toolchain". Does "target Java toolchain" mean the Java runtime version you want the built code to run on?

--

Lukács T. Berki | Software Engineer | lbe...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Ivo List]

On Fri, Oct 2, 2020 at 2:09 PM Lukács T. Berki <lbe...@google.com> wrote:

Historic behavior aside, my understanding is that the current default is to use a downloaded JDK to run Java compilation (i.e. the host JDK) and a local JVM to run whatever is built. The embedded JDK is only used for running Bazel. This is so that:

1. We know what version of javac we are running (this is important because Error Prone uses non-public APIs and Error Prone integration is very popular) and it always works without having to separately download one before you run Bazel
2. The built Java binaries run on the local machine
3. We can upgrade the JDK used to run Bazel without affecting anyone and cut it down to the minimum

This is in line with Ivo's desire to work out of the box for as many people as possible: Java compilation works (because we download a known-good machinery) and the binaries work because they use a local JVM.

The part I don't understand is why we need to "match host execution JDK with target Java toolchain". Does "target Java toolchain" mean the Java runtime version you want the built code to run on?

Putting it more simply: matching JavaBuilder with JDK. This seems to be the "correct" way to do it, because JavaBuilder as you said uses non-public API.

One use-case that proves this is, currently you can compile for Java14 target while using Java11 for tools on the host only with vanilla_toolchian. You have to set --javabase and --host_javabase to remote JDK14. And this then forces you to use vanilla_toolchain on the host (while you could be using remote JDK11 with proper JavaBuilder). This in a way breaks hermetic compiles.

[Lukács T. Berki]

On Fri, Oct 2, 2020 at 2:43 PM Ivo List <il...@google.com> wrote:

On Fri, Oct 2, 2020 at 2:09 PM Lukács T. Berki <lbe...@google.com> wrote:

Historic behavior aside, my understanding is that the current default is to use a downloaded JDK to run Java compilation (i.e. the host JDK) and a local JVM to run whatever is built. The embedded JDK is only used for running Bazel. This is so that:

1. We know what version of javac we are running (this is important because Error Prone uses non-public APIs and Error Prone integration is very popular) and it always works without having to separately download one before you run Bazel
2. The built Java binaries run on the local machine
3. We can upgrade the JDK used to run Bazel without affecting anyone and cut it down to the minimum

This is in line with Ivo's desire to work out of the box for as many people as possible: Java compilation works (because we download a known-good machinery) and the binaries work because they use a local JVM.

The part I don't understand is why we need to "match host execution JDK with target Java toolchain". Does "target Java toolchain" mean the Java runtime version you want the built code to run on?

Putting it more simply: matching JavaBuilder with JDK. This seems to be the "correct" way to do it, because JavaBuilder as you said uses non-public API.

One use-case that proves this is, currently you can compile for Java14 target while using Java11 for tools on the host only with vanilla_toolchian. You have to set --javabase and --host_javabase to remote JDK14. And this then forces you to use vanilla_toolchain on the host (while you could be using remote JDK11 with proper JavaBuilder). This in a way breaks hermetic compiles.

To recap our VC: if you set --javabase <JVM 14> --host_javabase <JVM 14> you currently get a JavaBuilder that assumes that it runs on JVM 11, which doesn't work. The workaround is to use the --java_toolchain <vanilla builder> so that you get the javac from the JDK 14 and not the one packaged with the default JavaBuilder (which you must use since we don't have one yet for JDK 14). Is this correct?

[Ivo List]

On Fri, Oct 2, 2020 at 3:04 PM Lukács T. Berki <lbe...@google.com> wrote:

On Fri, Oct 2, 2020 at 2:43 PM Ivo List <il...@google.com> wrote:

On Fri, Oct 2, 2020 at 2:09 PM Lukács T. Berki <lbe...@google.com> wrote:

Historic behavior aside, my understanding is that the current default is to use a downloaded JDK to run Java compilation (i.e. the host JDK) and a local JVM to run whatever is built. The embedded JDK is only used for running Bazel. This is so that:

1. We know what version of javac we are running (this is important because Error Prone uses non-public APIs and Error Prone integration is very popular) and it always works without having to separately download one before you run Bazel
2. The built Java binaries run on the local machine
3. We can upgrade the JDK used to run Bazel without affecting anyone and cut it down to the minimum

This is in line with Ivo's desire to work out of the box for as many people as possible: Java compilation works (because we download a known-good machinery) and the binaries work because they use a local JVM.

The part I don't understand is why we need to "match host execution JDK with target Java toolchain". Does "target Java toolchain" mean the Java runtime version you want the built code to run on?

Putting it more simply: matching JavaBuilder with JDK. This seems to be the "correct" way to do it, because JavaBuilder as you said uses non-public API.

One use-case that proves this is, currently you can compile for Java14 target while using Java11 for tools on the host only with vanilla_toolchian. You have to set --javabase and --host_javabase to remote JDK14. And this then forces you to use vanilla_toolchain on the host (while you could be using remote JDK11 with proper JavaBuilder). This in a way breaks hermetic compiles.

To recap our VC: if you set --javabase <JVM 14> --host_javabase <JVM 14> you currently get a JavaBuilder that assumes that it runs on JVM 11, which doesn't work. The workaround is to use the --java_toolchain <vanilla builder> so that you get the javac from the JDK 14 and not the one packaged with the default JavaBuilder (which you must use since we don't have one yet for JDK 14). Is this correct?

Well I double checked and "--javabase <JVM 14> --host_javabase <JVM 14>" with Java11 Javabuilder works, except that you get two warnings about JVM options that are deprecated for _each_ compile action. And hopefully javac patch is complete - i.e. it's not using some internal classes from JVM14.

There is a bit better toolchain for Java14, that uses JavaBuilder, but doesn't patch javac modules see /bazel/tools/jdk/BUILD.java_tools;l=124. 

[Lukács T. Berki]

Hey Ivo,

After a weekend and acquiring a bit (but just a bit) clearer head, here are what I think the fundamental limitations are:

1. java_toolchain rules with an embedded javac version X (i.e. JavaBuilder versions) can use any JVM version >= X (Java bytecode should be forward compatible)
2. java_toolchain rules that embed knowledge about that particular javac version but which don't embed a javac and which rely on the one supplied by JDK (i.e. the vanilla versions) should probably use only that JDK version X because they generally

Is this correct? 

The most naive way to model these is with exec constraints: we'd make an exec constraint that says "I have Java version X" a java_toolchain embedding javac X would say that it runs on that version or above, whereas a java_toolchain relying on the javac supplied by the JDK would say that it needs that particular version. 

This gives rise to two kinds of coupling: 

1. If you want to use javac version X to build your Java code, you put constraints on the Java version that everything else uses on the target platform according to the above rules
2. If you want to target a particular Java language version, you are forced to use a java_toolchain that sets that target_version and thus the constraints arising from (1) if you don't have multiple java_toolchains that have the same javac version but different target_version

(2) seems solvable by making target_version not a property of the java_toolchain rule but a configuration flag or something; then java_toolchain would have a property that says "this is the maximum language level I support". 

If (1) proves to be a problem, we can go with the "attach a java_runtime to a java_toolchain rule" idea.

WDYT? This all doesn't cover any migration we may need to do (I didn't think about it yet), but it looks like a reasonable end state to me.

[Lukács T. Berki]

On Mon, Oct 5, 2020 at 12:56 PM Ivo List <il...@google.com> wrote:

On Mon, Oct 5, 2020 at 8:56 AM Lukács T. Berki <lbe...@google.com> wrote:

Hey Ivo,

After a weekend and acquiring a bit (but just a bit) clearer head, here are what I think the fundamental limitations are:

1. java_toolchain rules with an embedded javac version X (i.e. JavaBuilder versions) can use any JVM version >= X (Java bytecode should be forward compatible)
2. java_toolchain rules that embed knowledge about that particular javac version but which don't embed a javac and which rely on the one supplied by JDK (i.e. the vanilla versions) should probably use only that JDK version X because they generally

Is this correct? 

Correct. 

But this is more of a P2, kind of a problem: support different versions of Java.

Let's first just focus to support javac version X, out of the box. The P1 problem is compiling it with "embedded javac version X" +remote_jdk11 JVM and executing with local_jdk JVM.

The most naive way to model these is with exec constraints: we'd make an exec constraint that says "I have Java version X" a java_toolchain embedding javac X would say that it runs on that version or above, whereas a java_toolchain relying on the javac supplied by the JDK would say that it needs that particular version. 

This gives rise to two kinds of coupling: 

1. If you want to use javac version X to build your Java code, you put constraints on the Java version that everything else uses on the target platform according to the above rules
2. If you want to target a particular Java language version, you are forced to use a java_toolchain that sets that target_version and thus the constraints arising from (1) if you don't have multiple java_toolchains that have the same javac version but different target_version

(2) seems solvable by making target_version not a property of the java_toolchain rule but a configuration flag or something; then java_toolchain would have a property that says "this is the maximum language level I support". 

If (1) proves to be a problem, we can go with the "attach a java_runtime to a java_toolchain rule" idea.

WDYT? This all doesn't cover any migration we may need to do (I didn't think about it yet), but it looks like a reasonable end state to me.

I don't see other ways to solve P1 above, except:

- drop/change the requirement - for example using remote_jdk11 JVM to compile and execute by default and provide another predefined platform that uses local_jdk

Let's not do this.

 

- add a constraint on one of the default platforms

I thought that the "this platform is the same as the one Bazel runs on" constraint is a reasonable one. John, what's the argument against it?

 

- attach a java_runtime to a java_toolchain rule

And this would also mean not registering that java_runtime toolchain in the java_tools repositories, since then you wouldn't be able to guarantee that Bazel doesn't choose that one as the one for the target platform (fwiw, this "additional toolchain accidentally gets selected" issue is a good argument for the "this toolchain is available locally" constraint)

There is at least another option, which is to autodetect a JVM, but not a JDK, i.e. a java_runtime one but not a java_toolchain one. WDYT? I'm not suggesting that we should do this, only that this would be an option; I haven't fully thought through its implications.

[Lukács T. Berki]

On Mon, Oct 5, 2020 at 1:47 PM Lukács T. Berki <lbe...@google.com> wrote:

On Mon, Oct 5, 2020 at 12:56 PM Ivo List <il...@google.com> wrote:

On Mon, Oct 5, 2020 at 8:56 AM Lukács T. Berki <lbe...@google.com> wrote:

Hey Ivo,

After a weekend and acquiring a bit (but just a bit) clearer head, here are what I think the fundamental limitations are:

1. java_toolchain rules with an embedded javac version X (i.e. JavaBuilder versions) can use any JVM version >= X (Java bytecode should be forward compatible)
2. java_toolchain rules that embed knowledge about that particular javac version but which don't embed a javac and which rely on the one supplied by JDK (i.e. the vanilla versions) should probably use only that JDK version X because they generally

Is this correct? 

Correct. 

But this is more of a P2, kind of a problem: support different versions of Java.

Let's first just focus to support javac version X, out of the box. The P1 problem is compiling it with "embedded javac version X" +remote_jdk11 JVM and executing with local_jdk JVM.

The most naive way to model these is with exec constraints: we'd make an exec constraint that says "I have Java version X" a java_toolchain embedding javac X would say that it runs on that version or above, whereas a java_toolchain relying on the javac supplied by the JDK would say that it needs that particular version. 

This gives rise to two kinds of coupling: 

1. If you want to use javac version X to build your Java code, you put constraints on the Java version that everything else uses on the target platform according to the above rules
2. If you want to target a particular Java language version, you are forced to use a java_toolchain that sets that target_version and thus the constraints arising from (1) if you don't have multiple java_toolchains that have the same javac version but different target_version

(2) seems solvable by making target_version not a property of the java_toolchain rule but a configuration flag or something; then java_toolchain would have a property that says "this is the maximum language level I support". 

If (1) proves to be a problem, we can go with the "attach a java_runtime to a java_toolchain rule" idea.

WDYT? This all doesn't cover any migration we may need to do (I didn't think about it yet), but it looks like a reasonable end state to me.

I don't see other ways to solve P1 above, except:

- drop/change the requirement - for example using remote_jdk11 JVM to compile and execute by default and provide another predefined platform that uses local_jdk

Let's not do this.

 

- add a constraint on one of the default platforms

I thought that the "this platform is the same as the one Bazel runs on" constraint is a reasonable one. John, what's the argument against it?

Ivo kindly reminded me that this is right in this thread: "Fundamentally, Bazel shouldn't be trying to distinguish between "builds for the target platform" and "builds for an execution platform that is identical to the target platform": they should be handled the same way because they are the same builds."

But I'm not sure if this is an argument against the "this toolchain is available locally" constraint: the "target platform" is different from the execution platform in that you also have requirements with respect to it outside the build: for an execution platform, all you need is that the actions you execute on it be able to run. For a target platform, you also need that the binaries are runnable outside the control of Bazel. IOW saying "this Java toolchain works if Bazel downloads it, links it into the execroot, etc." is different from saying "this toolchain is there without any extra preparations", which is what the requirement is wrt. the target java_runtime.

[Ulf Adams]

I didn't completely follow the discussion, but this doesn't seem right. Bazel should trust the user if they say that they have a specific target Jvm and it should not prefer that one way or the other over what happens to be installed on the local machine. It's the user's responsibility to ensure consistency, not Bazel's. Note that this is also true for remote execution - as long as Bazel cannot query the remote execution system, it has to trust that the user tells it the right properties. This is where rbe_autoconfig lives - it does query the docker image and then tell Bazel, but note that it is an optional extension of Bazel and not part of Bazel core.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAOu%2B0LWTosWjaW7BaBhyNP5Hupdc%3Ds90CwZfkq-MbD%2Bpb_HrNg%40mail.gmail.com.

[John Cater]

In addition to the philosophical problem with using a constraint to distinguish the target and host platforms, there's an implementation problem: there is no separate target platform. Currently (in Bazel), the default host platform is `@local_config_platform//:host`, and the default target platform is "whatever --host_platform happens to be set to". If we add a new default target platform (so we can distinguish it from the default host platform), we lose that second property, which means that anyone currently using `--host_platform` will see behavior changes (and probably will select the wrong JVM, if I am understanding the current proposals correctly). Due to the issues with execution strategy and execution platform selection, my understanding is that many users of remote execution currently set `--host_platform`, especially if they are starting builds on a Mac laptop but executing them on linux remote workers.

On Mon, Oct 5, 2020 at 8:21 AM Ivo List <il...@google.com> wrote:

I kind of have second thoughts on this solution. The good thing about it, it is simple and easy to implement.

Leaving this aside it could introduce some technical dept and with it issues in the future. One such possible issue is when supporting multiple versions of Java. If we hack around constraints to select the right combination, on every next release of Java or our toolchain, somebody will have to hack again. After those "hacks" are implemented it might become hard to remove this constraint from the default platform.

 

 

- attach a java_runtime to a java_toolchain rule

And this would also mean not registering that java_runtime toolchain in the java_tools repositories, since then you wouldn't be able to guarantee that Bazel doesn't choose that one as the one for the target platform (fwiw, this "additional toolchain accidentally gets selected" issue is a good argument for the "this toolchain is available locally" constraint)

No, we would still register java_runtime toolchain, which would be then used only for execution. Java rules would use java_runtime from java_toolchain for compilation and java_runtime for execution. Nice thing is, that there wouldn't be any host/exec transition in Java rules anymore. In target configuration this would mean local_jdk for execution and remote one for compilation. 

There is one change (just realised about it while writing) - also in host/exec configuration targets would be executed using local jdk (right now remotejdk is used in host/exec for execution). I don't think this is very problematic (android is an example that heavily depends on this). If it is problematic then we're back at adding a constraint to autoconfigured platforms.

About accidental selection of toolchains: toolchains get selected by the order of their appearance, so if local_jdk is on the first place in WORKSPACE suffix, this should be fine. We can also use it in our advantage - i.e. if there is no local jdk, failover to remote jdk.

[Lukács T. Berki]

On Mon, Oct 5, 2020 at 2:27 PM John Cater <jca...@google.com> wrote:

In addition to the philosophical problem with using a constraint to distinguish the target and host platforms, there's an implementation problem: there is no separate target platform. Currently (in Bazel), the default host platform is `@local_config_platform//:host`, and the default target platform is "whatever --host_platform happens to be set to".

Is this by design or did we just end up doing this "accidentally?"

 

If we add a new default target platform (so we can distinguish it from the default host platform), we lose that second property, which means that anyone currently using `--host_platform` will see behavior changes (and probably will select the wrong JVM, if I am understanding the current proposals correctly). Due to the issues with execution strategy and execution platform selection, my understanding is that many users of remote execution currently set `--host_platform`, especially if they are starting builds on a Mac laptop but executing them on linux remote workers.

...and then they build binaries for Linux?

[Lukács T. Berki]

@Ulf: that's a nice principle, but what do you think about the difference between "any JDK fulfilling these constraints that Bazel can provide" and "the JDK that is known to be installed in this environment without Bazel taking extra steps?" If you don't make this difference, there is no guarantee that instead of the locally installed JDK, Bazel will choose one as the target JDK that happens to fulfill the same constraints and then the binary you built will not be executable since the wrapper script references the wrong thing.

On Mon, Oct 5, 2020 at 2:21 PM Ivo List <il...@google.com> wrote:

I kind of have second thoughts on this solution. The good thing about it, it is simple and easy to implement.

Leaving this aside it could introduce some technical dept and with it issues in the future. One such possible issue is when supporting multiple versions of Java. If we hack around constraints to select the right combination, on every next release of Java or our toolchain, somebody will have to hack again. After those "hacks" are implemented it might become hard to remove this constraint from the default platform.

What hacks do you have in mind? I thought that a virtue of this approach was that it's free of hacks. 

 

 

- attach a java_runtime to a java_toolchain rule

And this would also mean not registering that java_runtime toolchain in the java_tools repositories, since then you wouldn't be able to guarantee that Bazel doesn't choose that one as the one for the target platform (fwiw, this "additional toolchain accidentally gets selected" issue is a good argument for the "this toolchain is available locally" constraint)

No, we would still register java_runtime toolchain, which would be then used only for execution. Java rules would use java_runtime from java_toolchain for compilation and java_runtime for execution. Nice thing is, that there wouldn't be any host/exec transition in Java rules anymore. In target configuration this would mean local_jdk for execution and remote one for compilation. 

There is one change (just realised about it while writing) - also in host/exec configuration targets would be executed using local jdk (right now remotejdk is used in host/exec for execution).

I suppose what you are saying here is the same as what John is saying, that the current default is that the host and the target platforms are the same?

 

I don't think this is very problematic (android is an example that heavily depends on this). If it is problematic then we're back at adding a constraint to autoconfigured platforms.

How is Android special? 

 

About accidental selection of toolchains: toolchains get selected by the order of their appearance, so if local_jdk is on the first place in WORKSPACE suffix, this should be fine. We can also use it in our advantage - i.e. if there is no local jdk, failover to remote jdk.

Relying on the order the toolchains are registered in looks like a bit of a hack, doesn' it?

[John Cater]

On Mon, Oct 5, 2020 at 8:30 AM Lukács T. Berki <lbe...@google.com> wrote:

On Mon, Oct 5, 2020 at 2:27 PM John Cater <jca...@google.com> wrote:

In addition to the philosophical problem with using a constraint to distinguish the target and host platforms, there's an implementation problem: there is no separate target platform. Currently (in Bazel), the default host platform is `@local_config_platform//:host`, and the default target platform is "whatever --host_platform happens to be set to".

Is this by design or did we just end up doing this "accidentally?"

This is by design. The original thought was "bazel should default to building for the host platform", so we evolved that to "Bazel should default to building for what the user claims is the the host platform."

 

 

If we add a new default target platform (so we can distinguish it from the default host platform), we lose that second property, which means that anyone currently using `--host_platform` will see behavior changes (and probably will select the wrong JVM, if I am understanding the current proposals correctly). Due to the issues with execution strategy and execution platform selection, my understanding is that many users of remote execution currently set `--host_platform`, especially if they are starting builds on a Mac laptop but executing them on linux remote workers.

...and then they build binaries for Linux?

If they haven't set "--platforms" to a more specific value, yes.

[Ivo List]

On Mon, Oct 5, 2020 at 8:56 AM Lukács T. Berki <lbe...@google.com> wrote:

Hey Ivo,

After a weekend and acquiring a bit (but just a bit) clearer head, here are what I think the fundamental limitations are:

1. java_toolchain rules with an embedded javac version X (i.e. JavaBuilder versions) can use any JVM version >= X (Java bytecode should be forward compatible)
2. java_toolchain rules that embed knowledge about that particular javac version but which don't embed a javac and which rely on the one supplied by JDK (i.e. the vanilla versions) should probably use only that JDK version X because they generally

Is this correct? 

Correct. 

But this is more of a P2, kind of a problem: support different versions of Java.

Let's first just focus to support javac version X, out of the box. The P1 problem is compiling it with "embedded javac version X" +remote_jdk11 JVM and executing with local_jdk JVM.

The most naive way to model these is with exec constraints: we'd make an exec constraint that says "I have Java version X" a java_toolchain embedding javac X would say that it runs on that version or above, whereas a java_toolchain relying on the javac supplied by the JDK would say that it needs that particular version. 

This gives rise to two kinds of coupling: 

1. If you want to use javac version X to build your Java code, you put constraints on the Java version that everything else uses on the target platform according to the above rules
2. If you want to target a particular Java language version, you are forced to use a java_toolchain that sets that target_version and thus the constraints arising from (1) if you don't have multiple java_toolchains that have the same javac version but different target_version

(2) seems solvable by making target_version not a property of the java_toolchain rule but a configuration flag or something; then java_toolchain would have a property that says "this is the maximum language level I support". 

If (1) proves to be a problem, we can go with the "attach a java_runtime to a java_toolchain rule" idea.

WDYT? This all doesn't cover any migration we may need to do (I didn't think about it yet), but it looks like a reasonable end state to me.

I don't see other ways to solve P1 above, except:

- drop/change the requirement - for example using remote_jdk11 JVM to compile and execute by default and provide another predefined platform that uses local_jdk

- add a constraint on one of the default platforms

- attach a java_runtime to a java_toolchain rule

[Ivo List]

I kind of have second thoughts on this solution. The good thing about it, it is simple and easy to implement.

Leaving this aside it could introduce some technical dept and with it issues in the future. One such possible issue is when supporting multiple versions of Java. If we hack around constraints to select the right combination, on every next release of Java or our toolchain, somebody will have to hack again. After those "hacks" are implemented it might become hard to remove this constraint from the default platform.

 

 

- attach a java_runtime to a java_toolchain rule

And this would also mean not registering that java_runtime toolchain in the java_tools repositories, since then you wouldn't be able to guarantee that Bazel doesn't choose that one as the one for the target platform (fwiw, this "additional toolchain accidentally gets selected" issue is a good argument for the "this toolchain is available locally" constraint)

No, we would still register java_runtime toolchain, which would be then used only for execution. Java rules would use java_runtime from java_toolchain for compilation and java_runtime for execution. Nice thing is, that there wouldn't be any host/exec transition in Java rules anymore. In target configuration this would mean local_jdk for execution and remote one for compilation. 

There is one change (just realised about it while writing) - also in host/exec configuration targets would be executed using local jdk (right now remotejdk is used in host/exec for execution). I don't think this is very problematic (android is an example that heavily depends on this). If it is problematic then we're back at adding a constraint to autoconfigured platforms.

About accidental selection of toolchains: toolchains get selected by the order of their appearance, so if local_jdk is on the first place in WORKSPACE suffix, this should be fine. We can also use it in our advantage - i.e. if there is no local jdk, failover to remote jdk.

 

[Ivo List]

On Mon, Oct 5, 2020 at 2:36 PM John Cater <jca...@google.com> wrote:

On Mon, Oct 5, 2020 at 8:30 AM Lukács T. Berki <lbe...@google.com> wrote:

On Mon, Oct 5, 2020 at 2:27 PM John Cater <jca...@google.com> wrote:

In addition to the philosophical problem with using a constraint to distinguish the target and host platforms, there's an implementation problem: there is no separate target platform. Currently (in Bazel), the default host platform is `@local_config_platform//:host`, and the default target platform is "whatever --host_platform happens to be set to".

Is this by design or did we just end up doing this "accidentally?"

This is by design. The original thought was "bazel should default to building for the host platform", so we evolved that to "Bazel should default to building for what the user claims is the the host platform."

 

Well this seems to be easy to change, https://github.com/bazelbuild/bazel/pull/12192, or am i missing something?

 

 

If we add a new default target platform (so we can distinguish it from the default host platform), we lose that second property, which means that anyone currently using `--host_platform` will see behavior changes (and probably will select the wrong JVM, if I am understanding the current proposals correctly). Due to the issues with execution strategy and execution platform selection, my understanding is that many users of remote execution currently set `--host_platform`, especially if they are starting builds on a Mac laptop but executing them on linux remote workers.

...and then they build binaries for Linux?

If they haven't set "--platforms" to a more specific value, yes.

I am adding a constraint to the auto-generated target platform, just so that it doesn't meddle with the host and probably popular option --host_platform. The users that have already written their own specification of host_platform don't need to change anything. Even if they depend on HOST_CONSTRAINTS, they remain the same. They couldn't have dependent on new target platform or on new constraint because it doesn't exist yet.

Possible problem is when users are specifying target platform, and this is mentioned in the proposal.

[Lukács T. Berki]

On Mon, Oct 5, 2020 at 2:36 PM John Cater <jca...@google.com> wrote:

On Mon, Oct 5, 2020 at 8:30 AM Lukács T. Berki <lbe...@google.com> wrote:

On Mon, Oct 5, 2020 at 2:27 PM John Cater <jca...@google.com> wrote:

In addition to the philosophical problem with using a constraint to distinguish the target and host platforms, there's an implementation problem: there is no separate target platform. Currently (in Bazel), the default host platform is `@local_config_platform//:host`, and the default target platform is "whatever --host_platform happens to be set to".

Is this by design or did we just end up doing this "accidentally?"

This is by design. The original thought was "bazel should default to building for the host platform", so we evolved that to "Bazel should default to building for what the user claims is the the host platform."

Okie. If that's the design, then Ivo's argument applies that we can't possibly select different JVMs for executing the code built and running javac to build it. If we don't want to change it, then only the "staple a java_runtime to the java_toolchain" option remains, right?

 

 

If we add a new default target platform (so we can distinguish it from the default host platform), we lose that second property, which means that anyone currently using `--host_platform` will see behavior changes (and probably will select the wrong JVM, if I am understanding the current proposals correctly). Due to the issues with execution strategy and execution platform selection, my understanding is that many users of remote execution currently set `--host_platform`, especially if they are starting builds on a Mac laptop but executing them on linux remote workers.

...and then they build binaries for Linux?

If they haven't set "--platforms" to a more specific value, yes.

[Ivo List]

So one thing that is hacky, if you want to build for Java14 target, you need either to change your host/execution platform constraints (so that Java14 JVM is selected on the host/execution) or you define only one toolchain using selects (which then becomes Java14 when needed)

 

 

 

- attach a java_runtime to a java_toolchain rule

And this would also mean not registering that java_runtime toolchain in the java_tools repositories, since then you wouldn't be able to guarantee that Bazel doesn't choose that one as the one for the target platform (fwiw, this "additional toolchain accidentally gets selected" issue is a good argument for the "this toolchain is available locally" constraint)

No, we would still register java_runtime toolchain, which would be then used only for execution. Java rules would use java_runtime from java_toolchain for compilation and java_runtime for execution. Nice thing is, that there wouldn't be any host/exec transition in Java rules anymore. In target configuration this would mean local_jdk for execution and remote one for compilation. 

There is one change (just realised about it while writing) - also in host/exec configuration targets would be executed using local jdk (right now remotejdk is used in host/exec for execution).

I suppose what you are saying here is the same as what John is saying, that the current default is that the host and the target platforms are the same?

Yes, executing using local jdk in host/exec configuration would be a consequence of having the same host and target.

 

 

I don't think this is very problematic (android is an example that heavily depends on this). If it is problematic then we're back at adding a constraint to autoconfigured platforms.

How is Android special? 

Android builds its own tools, written in Java that it then uses for further building (everything in exec/host configuration). I didn't see any other target's doing this, and even if they are, I don't think it is common.

 

 

About accidental selection of toolchains: toolchains get selected by the order of their appearance, so if local_jdk is on the first place in WORKSPACE suffix, this should be fine. We can also use it in our advantage - i.e. if there is no local jdk, failover to remote jdk.

Relying on the order the toolchains are registered in looks like a bit of a hack, doesn' it?

Perhaps. I'm thinking of the order as an additional criteria in the selection when it becomes irrelevant which tool is selected. Then you probably want the one that is closer to you. I.e. if the user has JDK11 installed, why not use it instead of remote jdk11. It is hacky if the matching toolchains are not really equivalent.

[Lukács T. Berki]

Let's assume that we don't want to split the host/target platform because that's quite a bit of collateral damage for the use case of "I want to compile Java 14".

I think the weird part is that as far as Bazel is concerned, it is right, because from its perspective, giving you any JDK that fulfills the requirements "my OS, my CPU, Java 14", be it a downloaded or a local one, should be fine. Except that it's not, due to the silly wrapper script we have Do I understand correctly that that's the only issue? Then we are using the vanilla Java builder and thus the javac from that very same JDK.

[Ulf Adams]

On Mon, Oct 5, 2020 at 2:35 PM 'Lukács T. Berki' via bazel-dev <baze...@googlegroups.com> wrote:

@Ulf: that's a nice principle, but what do you think about the difference between "any JDK fulfilling these constraints that Bazel can provide" and "the JDK that is known to be installed in this environment without Bazel taking extra steps?" If you don't make this difference, there is no guarantee that instead of the locally installed JDK, Bazel will choose one as the target JDK that happens to fulfill the same constraints and then the binary you built will not be executable since the wrapper script references the wrong thing.

I'm afraid I don't follow your argument. If the two JDKs are equivalent, they should have the same constraints. If not, then not.

The wrapper script is a different question - for java_test, it seems reasonable, but why use the wrapper script for Java binaries outside of Bazel? What is the right way to package a Java binary? I think there are a few options: 1) a binary with an embedded Jdk (there's no rule for that right now - why not?), 2) a .war file for deployment with a servlet container (may use the _deploy.jar), 3) a docker image. What else? The first two options don't do anything with the wrapper script. The third one probably doesn't either, although I guess it's possible to use it that way?

 

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-dev/CAOu%2B0LU9jibAa4dwDx_USE5q9MXVoQiJrq8UmfTpAjMph49gAw%40mail.gmail.com.

[Lukács T. Berki]

On Mon, Oct 5, 2020 at 10:23 PM Ulf Adams <ulf...@gmail.com> wrote:

On Mon, Oct 5, 2020 at 2:35 PM 'Lukács T. Berki' via bazel-dev <baze...@googlegroups.com> wrote:

@Ulf: that's a nice principle, but what do you think about the difference between "any JDK fulfilling these constraints that Bazel can provide" and "the JDK that is known to be installed in this environment without Bazel taking extra steps?" If you don't make this difference, there is no guarantee that instead of the locally installed JDK, Bazel will choose one as the target JDK that happens to fulfill the same constraints and then the binary you built will not be executable since the wrapper script references the wrong thing.

I'm afraid I don't follow your argument. If the two JDKs are equivalent, they should have the same constraints. If not, then not.

They are equivalent within the build. You could argue that the wrapper script is not to be used outside of Bazel (I think so), but it's there, so we must consider it. Although if you are arguing that we should not make decisions based on misfeatures, that does have some weight...