Custom default_java_toolchain for bazel 17/18/19 at the same time

[ittai zeidman]

Hi,

I'm trying to have a custom default_java_toolchain which works for bazel 17/18/19 at the same time due to the need to support different environments.

The problem AFAIU is that the bootclasspath target from bazel has different non-compatible labels. 17/18(@bazel_tools//tools/jdk:platformclasspath8) 19 (@bazel_tools//tools/jdk:platformclasspath).

Am I mistaken or missing something?

[Liam Miller-Cushon]

Can you provide some context about what you need to customize compared to the built-in toolchains, and why you need to support 17/18/19 at the same time?

[ittai zeidman]

I need to globally customize the javacopts (disable errorprone as well as add -deprecation).
Additionally I need to run with Oracle jdk though I’m fairly certain this is related to the java_runtime and not the toolchain (mentioning in case I’m wrong).

I need to support them since we have a few different environments which don’t necessarily update in tandem.
In general even when we are able to converge them, even for one environment I need to be able to rollback easily if I find issues in 19.
I need to support them for a week or two.

Previously (between 15/17) I just copied over the code of DumpClasspath.java and the genrule. Problem is that when we upgraded to later revisions I forgot about it and had strange failures due to missing fixes to this file.

Is there a better alternative?

Thanks,
Ittai

[Liam Miller-Cushon]

On Sat, Nov 3, 2018 at 9:41 PM ittai zeidman <itt...@gmail.com> wrote:

I need to globally customize the javacopts (disable errorprone as well as add -deprecation).

If the only part of the java_toolchain you need to customize is javacopts, it might be simpler to set the top-level --javacopt flag in your .bazelrc.

 

Additionally I need to run with Oracle jdk though I’m fairly certain this is related to the java_runtime and not the toolchain (mentioning in case I’m wrong).

Right, that's the --javabase (and maybe also --host_javabase?). If you're using a custom --host_javabase, what are the blockers for using the default one?

 

I need to support them since we have a few different environments which don’t necessarily update in tandem.
In general even when we are able to converge them, even for one environment I need to be able to rollback easily if I find issues in 19.
I need to support them for a week or two.

I wonder if the 'bazelversion' idea discussed here might help: https://groups.google.com/d/msg/bazel-discuss/wYocSoGTJOg/zQDt-oI-FAAJ

It doesn't solve the problem of needing to roll back if issues are discovered with the new release, but it would make it easier to use the same version consistently and coordinate other changes together with the upgrade.

[ittai zeidman]

Thanks
Re javacopt global flag- I forgot to mention we need to be able to set the source/target versions to 8 which I think Bazel currently supports but I don’t know when Bazel won’t support it.
Wdyt?

[Liam Miller-Cushon]

On Sun, Nov 4, 2018 at 2:19 PM ittai zeidman <itt...@gmail.com> wrote:

Re javacopt global flag- I forgot to mention we need to be able to set the source/target versions to 8 which I think Bazel currently supports

Yep, the supported toolchains currently default to the Java 8 language level:

https://github.com/bazelbuild/bazel/blob/f8be43cebfe5d9ee80e838c883fb305969ece743/tools/jdk/BUILD#L213-L214

https://github.com/bazelbuild/bazel/blob/f8be43cebfe5d9ee80e838c883fb305969ece743/tools/jdk/BUILD#L223-L224 

 

I don’t know when Bazel won’t support it.

If/when the default changes it'll go through the incompatible change process, and it will still be possible to explicitly configure the current language level (probably with top-level flags, not a custom java_toolchain definition). It isn't necessary to explicitly configure -source 8/-target 8 for now.

[itt...@wix.com]

Thanks. I hope to try rolling forward again today or tomorrow.
If I use the command line flag for javacopt is it in addition to the default opts (https://github.com/bazelbuild/bazel/blob/f8be43cebfe5d9ee80e838c883fb305969ece743/tools/jdk/default_java_toolchain.bzl#L47)?

[ittai zeidman]

I just remembered why we need default_java_toolchain:

1. We have many repositories. If each of them needs to specify in the bazelrc file the opts that's duplication which is hard to manage. Instead each repo just encodes the toolchain from a central repo and we can iterate on the contents there.

2. Even if you encode it in the bazelrc file then this will probably not work well if someone wants to specify opts via the flag themselves (our current solution allows aggregation).

We'll be able to converge on 0.19 again on Sunday so I have a solution but I think that such changes should be backwards compatible or with a flag (if it's even possible).

Thank you for your help

[Liam Miller-Cushon]

On Fri, Nov 9, 2018 at 5:44 AM ittai zeidman <itt...@gmail.com> wrote:

I just remembered why we need default_java_toolchain:

1. We have many repositories. If each of them needs to specify in the bazelrc file the opts that's duplication which is hard to manage. Instead each repo just encodes the toolchain from a central repo and we can iterate on the contents there.

I see, thanks. Do you not use `.bazelrc`s at all? 

 

2. Even if you encode it in the bazelrc file then this will probably not work well if someone wants to specify opts via the flag themselves (our current solution allows aggregation).

The manual entry for --javacopt mentations that the flag "can be used ... multiple times with individual arguments".

If --javacopt is set in the .bazelrc and on the command line, the options get concatenated together with the semantics described in the manual.

 

I think that such changes should be backwards compatible or with a flag (if it's even possible).

It's worth thinking about what the approach to incompatible changes should be for things like default_java_toolchain.bzl. For what it's worth, I'm hoping there aren't going to be a lot more changes to the toolchain configuration as we finish adding Java 11 support.

[ittai zeidman]

On Fri, 9 Nov 2018 at 19:35 Liam Miller-Cushon <cus...@google.com> wrote:

On Fri, Nov 9, 2018 at 5:44 AM ittai zeidman <itt...@gmail.com> wrote:

I just remembered why we need default_java_toolchain:

1. We have many repositories. If each of them needs to specify in the bazelrc file the opts that's duplication which is hard to manage. Instead each repo just encodes the toolchain from a central repo and we can iterate on the contents there.

I see, thanks. Do you not use `.bazelrc`s at all? 

 

We do but we try to have very little there

2. Even if you encode it in the bazelrc file then this will probably not work well if someone wants to specify opts via the flag themselves (our current solution allows aggregation).

The manual entry for --javacopt mentations that the flag "can be used ... multiple times with individual arguments".

If --javacopt is set in the .bazelrc and on the command line, the options get concatenated together with the semantics described in the manual.

You’re right, I was mistaken 

🤞🏽

[Liam Miller-Cushon]

On Fri, Nov 9, 2018 at 11:33 AM ittai zeidman <itt...@gmail.com> wrote:

I see, thanks. Do you not use `.bazelrc`s at all?  

We do but we try to have very little there

Because you have to keep all of them in sync, or other reasons?

[ittai zeidman]

yes, keeping them all in sync