Changing LTS baseline in buildPlugin.recommendedConfigurations() on ci.jenkins.io

[Oleg Nenashev]

Hi all,

I have a question about changing the Jenkins LTS versions in buildPlugin.recommendedConfigurations() which is a part of the Jenkins Pipeline Library. This is a utility method which provides a list of the recommended configurations for plugin build pipelines (see the code below).

The method was introduced as a part of the Java 11 support effort, and the version was pinned to 2.164.1 which was the first LTS to officially support Java 11.The intent was to change the recommended configuration over time, and it is reflected in the documentation: "It is also possible to use a buildPlugin.recommendedConfigurations() method to get recommended configurations for testing. Note that the recommended configuration may change over time, and hence your CI may break if the new recommended configuration is not compatible".

There are around 70 plugins using the method to retrieve the recommended configurations, and there is a pull request from Jesse Glick to update the version. Although the pull request is totally legit w.r.t to the documentation, there is a risk of breaking master branch builds for some plugins (as designed and documented). There is also a pull request from Baptiste Mathus which pins the version and allows overriding it; this PR changes the purpose of the method a bit.

Although there are great suggestions from Jesse about a longer-term solution (pulling the version from pom.xml and using Dependabot to bump it), I would like to ask plugin maintainers about short-term steps:

• Bump the version, potentially break CI for plugins (Jesse's PR)
• Pin the version, allow overriding it (Baptiste's PR)
• Reject both PRs, keep status quo and old Jenkins core until a plugin-repo-side suggestion is implemented by somebody

Any feedback/and comments will be appreciated.

Thanks in advance, Oleg

P.S: Code:

	static List<Map<String, String>> recommendedConfigurations() {
	def recentLTS = "2.164.1"
	def configurations = [
	[ platform: "linux", jdk: "8", jenkins: null ],
	[ platform: "windows", jdk: "8", jenkins: null ],
	[ platform: "linux", jdk: "8", jenkins: recentLTS, javaLevel: "8" ],
	[ platform: "windows", jdk: "8", jenkins: recentLTS, javaLevel: "8" ],
	[ platform: "linux", jdk: "11", jenkins: recentLTS, javaLevel: "8" ],
	[ platform: "windows", jdk: "11", jenkins: recentLTS, javaLevel: "8" ]
	]
	return configurations
	}

[Jesse Glick]

On Thu, Aug 29, 2019 at 5:42 AM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> Bump the version, potentially break CI for plugins (Jesse's PR)
> Pin the version, allow overriding it (Baptiste's PR)
> Reject both PRs, keep status quo and old Jenkins core until a plugin-repo-side suggestion is implemented by somebody

I would add a fourth option: deprecate `recommendedConfigurations` and
advise plugin authors to use an explicit list of configurations
(updating the latest `jenkinsVersion` on their own schedule) until we
have the time to create a more easily maintained system.

[Gavin]

I'm okay breaking CI for plugins because it means a plugin doesn't work the same on newer versions and that seems bad to me.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr0HVzA10n7KSv%3DqeJN8QhncocQx%2BNSY_yut2VfGVvesjQ%40mail.gmail.com.

[Jesse Glick]

On Thu, Aug 29, 2019 at 10:33 AM Gavin <hal...@gmail.com> wrote:
> it means a plugin doesn't work the same on newer versions

Possibly, but much more commonly it just means some _test_ does not
work the same on newer versions of Jenkins, sometimes for pretty minor
technical reasons. Yes that ought to get fixed, but there is no rush.

The problem with changing the library configuration in “YOLO mode” is
that somebody (especially not the plugin maintainer) files a PR, it
gets a test failure, and they waste half a day trying to figure out
what they did wrong…before figuring out that there is nothing wrong
with the PR at all, `master` builds would be broken the same way if
there were any, and the apparent regression is coming from some change
to another repository no one was even paying attention to. Then
somebody then needs to drop whatever they were doing and get the
plugin’s `master` build fixed.

Using Dependabot would let us avoid this kind of problem, while still
guiding maintainers toward testing against the latest and greatest in
a timely manner. (You would get an automated PR proposing to test
against a new LTS, or more generally proposing to use a new version of
the library; in the normal case that the build passes, you would merge
it; if not, you would track down the issue as soon as you have time to
focus on it; `master` would stay blue, barring infrastructure
problems.) There is not yet any Dependabot support for Jenkins
libraries, however, so we would need to do some groundwork first.

[Oleg Nenashev]

I agree with all points, Jesse. Dependabot + pom.xml definition should be our target goal. I am just not sure we ready to source versions in such way immediately.

One of the major issues is lack of the node step where we can retrieve and process POM.xml

From the resources standpoint, it would make sense to have an ACI-based step which does some spotchecks before running full-size test runs on multiple configurations.

I did some experiments with such preflight checks in https://github.com/jenkins-infra/pipeline-library/pull/21 , but it is not ready so far.

We will also need to inject a mock/test dependency to pom.xml to make the Dependabot operational.

Something we can do, but it also needs to be implemented.

BR, Oleg

[Jesse Glick]

On Thu, Aug 29, 2019 at 11:24 AM Oleg Nenashev <o.v.ne...@gmail.com> wrote:
> Dependabot + pom.xml definition should be our target goal.

Not quite sure what you are proposing here. Some kind of POM profile
to test against newer stuff? I was actually proposing something
completely different (but perhaps complementary): that you would write
something like

@Library('jenkins-infra/pipeline-library@73')
buildPlugin(configurations: buildPlugin.recommendedConfigurations())

which would run on a matrix including, say, 2.176.3 as the high core
version. Dependabot would then suggest incrementing the library
number, which might pull in various changes, including but not limited
to testing against a newer LTS.

> lack of the node step where we can retrieve and process POM.xml

You can

try {
String pom = readTrusted 'pom.xml'
// process…
} catch (x) {
// untrusted PR may have touched POM; see JENKINS-45970
}

outside of a `node` block, if you think you can do something useful with that.