Proposed changes to update center tiers

[Daniel Beck]

Hi everyone,

A few weeks ago I announced an improvement to the the update site generator[1]. Since then I've been busy modernizing the tool further.

One change I'm implementing is flexible tiering. Right now, we determine the last five LTS baselines, and generate weekly and LTS update sites for those, only offering plugins compatible with the latest LTS release in that line. You can see the effect in the directory listing: https://updates.jenkins.io/

This approach has a number of drawbacks:

- LTS users on lower patch level (e.g. 2.222.1) get advertised incompatible plugins requiring a higher patch level (2.222.3)
- Users of 2.222 get offered updates requiring 2.222.1, or 2.222.3.
- In general, users on weekly releases between LTS baselines get offered either incompatible updates (if we serve the next higher tier, which is the current behavior) or we hide compatible plugin updates from them (when serving next lower tier). The latter would be especially problematic for plugins requiring a core newer than the current LTS: It wouldn't be served to anyone.

To address this, I propose to dynamically determine which tiered update sites to generate based on core dependencies declared in plugins. While this will increase the number of tiers (every release ever used as a plugin's core dependency will be included), it'll result in every (recent) version of Jenkins being offered all latest compatible plugin releases.

For details, see my PR here:
https://github.com/jenkins-infra/update-center2/pull/376

As a side effect, there will no longer be the existing, predictable, tiered update sites; any direct access to URLs like /stable-2.222/update-center.json or /2.204/update-center.json may fail. At least install-plugins.sh in the Docker image will need to be fixed to not rely on these internals, and I filed an issue[2] for that. As a solution, URLs like /update-center.json?version=2.204 should be used, and redirects need to be followed (which matches the default behavior in Jenkins).

Please let me know if you expect further problems with this change. I checked some parts of Jenkins infra and most seem to just use /current/ and /stable/, which can still be supported. It's the specific version tiers that will be changed.


Please also note that this isn't a move to add support for older releases. It should be project policy IMO to discourage users running obsolete releases, as that's a bad idea for many reasons (security, useless bug reports, difficulty upgrading, silently missing out on plugin fixes, etc.). This change doesn't add support for older releases, it just basically fixes bugs in the support that already existed. In fact, weekly releases aren't supported for as long in the PR as they are now: The oldest weekly update site would be 2.172, while it is 2.164 today, and could be further restricted easily and even independently of LTS.


1: https://groups.google.com/d/msg/jenkinsci-dev/RKSEfWt1wSY/WfyqPueDAwAJ
2: https://github.com/jenkinsci/docker/issues/954

[Daniel Beck]

> On 17. May 2020, at 14:26, Daniel Beck <m...@beckweb.net> wrote:
>
> To address this, I propose to dynamically determine which tiered update sites to generate based on core dependencies declared in plugins. While this will increase the number of tiers (every release ever used as a plugin's core dependency will be included), it'll result in every (recent) version of Jenkins being offered all latest compatible plugin releases.

Jesse points out that the parenthetical note there,

> every release ever used as a plugin's core dependency will be included

is misleading; there's a cutoff: right now it's 400 days, or just over 13 months, for the max age of the earliest supported core releases. Tiers corresponding to earlier releases will simply not be published. Earlier Jenkins releases will be served the lowest actually generated tier (the "most compatible" one).

The point I was trying to make is that a few plugins (or just one), released frequently, with each release depending on a different version of Jenkins core, could increase the number of generated tiers substantially.

The impact on generation time after recent improvements should be negligible, but might end up affecting disk caches etc. a bit, and the .htaccess file doing the version-based redirects would be getting pretty long too. I doubt it'll come to that, and in the worst case we can start blacklisting plugin releases to reduce the numbers.

[Daniel Beck]

> On 17. May 2020, at 14:26, Daniel Beck <m...@beckweb.net> wrote:
>

> To address this, I propose to dynamically determine which tiered update sites to generate based on core dependencies declared in plugins. While this will increase the number of tiers (every release ever used as a plugin's core dependency will be included), it'll result in every (recent) version of Jenkins being offered all latest compatible plugin releases.

This is now live.

For now we generate both the dynamically determined update site tiers, as well as the fixed tiers we had before, to retain compatibility with the Docker plugins installation script.