Next baseline shift

[Basil Crow]

Historically we have supported old baselines for a long time in the
build toolchain. The current build toolchain supports baselines as far
back as 2.249. This provides a high degree of flexibility for plugin
maintainers and ultimately end users.

Sustaining this level of compatibility across complex transitions like
the move to Java 11 and Jetty 10 comes at significant build toolchain
complexity, either in the form of conditional logic in a single-branch
build toolchain scheme (the current status quo) or in the form of
multiple branches (a hypothetical scenario). For example, the test
harness needs to support Java 8 in order to support 2.249, the need to
support Java 8 means the test harness cannot move to Jetty 10 (which
requires Java 11), and the test harness' continued use of Jetty 9
requires core to retain support both Jetty 9 and 10 (even though only
Jetty 10 is used in production use cases). Similarly, conditional
logic is required to set the Maven compiler release version to 11 for
plugins when newer baselines are in use, and even then that
conditional logic tends to trip up IDEs.

For these reasons, it is desirable to require a baseline of 2.361 or
later (and therefore Java 11) for plugins in the build toolchain
sooner rather than later. The advantage is that this allows us to
simplify and clean up conditional logic that is increasingly becoming
less relevant and a maintenance burden. The disadvantage of requiring
a baseline of 2.361 or later for plugins in late 2022 or early 2023
(as opposed to our usual timeline of e.g. late 2023 or early 2024) is
that it is more of an imposition on plugin maintainers and users of
older LTS lines. I believe that the advantages outweigh the
disadvantages and that we should do this sooner rather than later,
ideally in late 2022.

If the consensus is that we are OK with requiring a baseline of 2.361
or later for plugins in late 2022 (i.e., after the release of 2.375.x
LTS) or early 2023 (i.e., after the release of the next major LTS line
after 2.375), then I explicitly commit to implement the change in the
plugin parent POM (e.g., with regard to Java 8), test harness (e.g.,
with regard to Unicode properties files), HPI plugin (e.g., with
regard to the Maven compiler release version), core (e.g., with regard
to Jetty 9), annotation indexer, symbol annotation library, version
number library, and access modifier library, noting in the release
notes for all eight (8) components that this is a breaking change for
plugin maintainers and that plugin maintainers must use a baseline of
2.361 or later when upgrading to the latest build toolchain.

If you are in favor of requiring a baseline of 2.361 or later for
plugins in late 2022 or early 2023 and with me implementing this
transition in the manner described, please reply with your timeline
preference (either late 2022 or early 2023). If you are in favor of
maintaining a stable branch of the abovementioned eight (8) components
for older LTS lines, please reply stating whether you are willing to
do the implementation and release work (noting that I explicitly
decline to do this implementation and release work). If you are in
favor of any other proposal, please reply stating the alternative
proposal and whether you are willing to do the implementation work.

Thanks,
Basil

[Jesse Glick]

Your proposal sounds great to me, and thanks for volunteering to implement the changes. I think a late-2022 timeline (sometime after 2.375.1 has been released) makes sense—if there is some advantage in waiting longer, I do not get what. The only likely drawback for plugin maintainers who deliberately wish to stay on a pre-2.361.x baseline is that Dependabot (if configured) will offer `plugin-pom` updates that fail CI builds, which is not so bad since the release notes will explain why, and the PR and its replacements can simply be ignored indefinitely. (And of course you will not get access to new stuff in the toolchain, but that seems fair.)

Independently of that, we can consider switching the default JDK for ci.jenkins.io `buildPlugin` from 8 to 11. Even if your plugin supports older baselines and Java 8, running the build on JDK 11 is typically fine, since `-release 8` will ensure the right bytecode and even prevent the infamous `ByteBuffer` problem. The likely issues: you will lose Java 8 test coverage (while gaining 11 test coverage, which is more important!); and occasionally some tool like `javadoc` will be stricter (in a way you ought to fix anyway though you might have preferred to take your time).

[Basil Crow]

On Mon, Nov 28, 2022 at 11:38 AM 'Jesse Glick' via Jenkins Developers
<jenkin...@googlegroups.com> wrote:
>
> Independently of that, we can consider switching the default JDK for ci.jenkins.io `buildPlugin` from 8 to 11.

+1 from me regarding the idea. Regarding the implementation, I am not
sure to whom you are referring when you write "we". Is this something
you are planning to implement, something you are asking me to
implement, or something that you are looking for a third party to
implement?

[Jesse Glick]

On Mon, Nov 28, 2022 at 2:53 PM Basil Crow <m...@basilcrow.com> wrote:

> Independently of that, we can consider switching the default JDK for ci.jenkins.io `buildPlugin` from 8 to 11.

I am not sure to whom you are referring when you write "we". Is this something you are planning to implement, something you are asking me to implement, or something that you are looking for a third party to
implement?

Not really any of those—it is basically a one-line change, if “we” want to do it, so I was soliciting opinions on whether it sounded desirable and safe enough to try (as opposed to leaving the default as 8 indefinitely).

[Basil Crow]

On Mon, Nov 28, 2022 at 1:30 PM 'Jesse Glick' via Jenkins Developers
<jenkin...@googlegroups.com> wrote:
> Not really any of those […] I was soliciting opinions on whether it sounded desirable and safe enough to try

And I was pointing out that it is of more practical value to discuss
designs alongside concrete implementation plans rather than to merely
discuss designs in the abstract. In any case, I think it is desirable
and safe enough to try, and I am not going to try it. I would prefer
to focus this discussion on things that people are willing to commit
to implementing.

[Tim Jacomb]

+1 to any time after 2.375.1

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjr4RgMeyj91qqrmEK3Nfg-LqufRgCP49HAutXVpQLrbaw%40mail.gmail.com.

[Ullrich Hafner]

+1 from me as well (after release of 2.375.1)

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjo4AV%3DuY-f1R%3DjTFTfwe-m8TdxONt4Ly7yhU3JpPTJj6w%40mail.gmail.com.

[Damien Duportal]

+1 for me on the proposal to require a 2.361 baseline. I'm in favor of a late 2022 (after 2.375.1 of course) as the end of year is usually spent on low activity so it will let plugin maintainer some time to discover and fix any issues caused by this change.

> Independently of that, we can consider switching the default JDK for ci.jenkins.io `buildPlugin` from 8 to 11.

I volunteer to make this change in the form of a draft PR to the jenkins-infra/pipeline-library until we are ready to go.

Damien

[James Nord]

> Independently of that, we can consider switching the default JDK for ci.jenkins.io `buildPlugin` from 8 to 11. Even if your plugin supports older baselines and Java 8, running the build on JDK 11 is typically fine, since `-release 8` will ensure the right bytecode and even prevent the infamous `ByteBuffer` problem. 

Alas the bytecode produced is not exactly the same and this is legal. 

This bytecode difference trips up spotbugs (causing several new false positives) that have no option other than to be suppressed, so this change will likely  cause "unrelated" build failures somewhere.

I generally do not like build failures coming from "unrelated" changes, and to may any build change should really be configured and controlled as source (so it can be pre validated).

That said most of my plugins are already up to date so this should be a non issue for me.

On the simificationn of the parent pom I am +1

/James

On Mon, 28 Nov 2022, 19:38 'Jesse Glick' via Jenkins Developers, <jenkin...@googlegroups.com> wrote:

Your proposal sounds great to me, and thanks for volunteering to implement the changes. I think a late-2022 timeline (sometime after 2.375.1 has been released) makes sense—if there is some advantage in waiting longer, I do not get what. The only likely drawback for plugin maintainers who deliberately wish to stay on a pre-2.361.x baseline is that Dependabot (if configured) will offer `plugin-pom` updates that fail CI builds, which is not so bad since the release notes will explain why, and the PR and its replacements can simply be ignored indefinitely. (And of course you will not get access to new stuff in the toolchain, but that seems fair.)

Independently of that, we can consider switching the default JDK for ci.jenkins.io `buildPlugin` from 8 to 11. Even if your plugin supports older baselines and Java 8, running the build on JDK 11 is typically fine, since `-release 8` will ensure the right bytecode and even prevent the infamous `ByteBuffer` problem. The likely issues: you will lose Java 8 test coverage (while gaining 11 test coverage, which is more important!); and occasionally some tool like `javadoc` will be stricter (in a way you ought to fix anyway though you might have preferred to take your time).

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1mdb5zZx7%3D%2B2M%2BgtUrCMXtfCQcJ406WhvQojhx5NTbXw%40mail.gmail.com.

[Jesse Glick]

On Tue, Nov 29, 2022 at 6:19 PM James Nord <james...@gmail.com> wrote:

This bytecode difference trips up spotbugs (causing several new false positives)

Mm, OK, that sounds like a real risk that would preclude changing the default any time soon.

Ideally a plugin repository would pin a version of the library, and Dependabot or updatecli could offer updates. Historically we have not done that. I am not even sure it would be practical, since `buildPlugin.groovy` contains a mixture of actual build instructions that ought to be synchronized with source code changes in general (such as the choice of JDK), as well as interactions with ci.jenkins.io infrastructure that are better synchronized with the site configuration.

[Basil Crow]

On Tue, Nov 29, 2022 at 3:19 PM James Nord <james...@gmail.com> wrote:
>
> This bytecode difference trips up spotbugs (causing several new false positives) that have no option other than to be suppressed, so this change will likely cause "unrelated" build failures somewhere.

https://github.com/spotbugs/spotbugs/issues/756 presumably. That is
fixed in recent versions of SpotBugs (adopted in recent versions of
the plugin parent POM), so it should only be an issue for plugins with
an out-of-date build toolchain.

For what it's worth, JEP-229 CD builds are already running with Java
11 but `-release 8` and this has not caused any major problems. I do
not feel strongly and could go either way.

[James Nord]

There was about 3 or 4 issues all around null checking and try-with-resouces at the time I migrated.  

They may all have been probably are all fixed now, but if you keep up to date with the parent pom you are probably also likely to keep up to date with the Jenkins version, so this would  affect plugins that have less active development more, which means users are probably less familiar with our tooling to begin with and so a more stable build env would be an aid in these circumstances.

Maybe worth trying in a PR to  a few older plugins.  My concern may be invalid, I may have just been unlucky

/James

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjqhkNyfL5NiHs%2BM5UmHQsN12eovcnj_%2Br59mtcm2WxT5w%40mail.gmail.com.

[Jesse Glick]

On Wed, Nov 30, 2022 at 2:41 PM James Nord <james...@gmail.com> wrote:

this would affect plugins that have less active development more, which means users are probably less familiar with our tooling to begin with

Right. Maybe safer to leave the default as is, but formally deprecate running `buildPlugin` without `configurations`, and then we have a tool we could use to propose  `Jenkinsfile` changes if we first fix https://issues.jenkins.io/browse/JENKINS-46795 and push that to ci.jenkins.io.

[Basil Crow]

I released the first tranche of PRs in 4.52. Next week I will take a look at migrating from Jetty 9 to Jetty 10 in the plugin build toolchain.

[Damien Duportal]

Prepared the draft PR for "buildPlugin()": https://github.com/jenkins-infra/pipeline-library/pull/541

[Basil Crow]

On Thu, Dec 1, 2022 at 10:15 AM Basil Crow <m...@basilcrow.com> wrote:
>
> Next week I will take a look at migrating from Jetty 9 to Jetty 10 in the plugin build toolchain.

I released jenkins-test-harness 1912.v0cdf15450b_fb with a Jetty 10 based JenkinsRule. jenkinsci/maven-hpi-plugin#409 covers the same upgrade for mvn hpi:run.

[Damien Duportal]

As explained by Jesse in https://github.com/jenkins-infra/pipeline-library/pull/541#issuecomment-1342884874, changing the default of buildPlugin() could create problems. I'll close the PR as there is no value in changing the default as it.

The archetypes used to generate a new plugin project are already up to date: there is a documentation effort to be done that would be much more valuable (I heard that some documentation contributors are going to propose help) which make sense.

Eventually, once the documentation is up to date, we could add a message when "buildPlugin()" is called without argument, recomending to list explicit configurations.