Proposal to ensure new plugin hosting requests use Maven instead of Gradle

[Alexander Brandes]

Hey everyone,

the Gradle JPI plugin lacks fundamental support of the jenkins.io infrastructure.

There's no support for automated releases (CD, JEP-229), missing metadata for the plugin page and a few other limitations, which don't make it a great experience using it.

Additionally, worth to note, the components mentioned above aren't new. People are proactively switching away from Gradle to Maven for better support and acceptance of our tooling, see ivy-plugin/49 for example.

For the reasons stated, I would like to propose, that new plugin hosting requests use Maven, to have better toolchain support.

I would be ready to lift this restriction again if the JPI plugin developers provide the same scope of tools like we have for Maven.

Best,

Alexander Brandes, for the hosting team

[Daniel Beck]

On Wed, Dec 7, 2022 at 11:20 AM Alexander Brandes <mc.ca...@gmail.com> wrote:

There's no support for automated releases (CD, JEP-229), missing metadata for the plugin page and a few other limitations, which don't make it a great experience using it.

Last I checked, generated poms were also invalid, which is a problem for any consumers of these artifacts that do more than just download the hpi.

[Basil Crow]

+1

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/bb8f56d3-727b-481a-9132-17bf5eddbbcdn%40googlegroups.com.

[Baptiste Mathus]

+1.

At least such a move will make it very explicit for any new plugin that only Maven really has /production/ support for Jenkins plugin dev.

I think that we should still "allow" new plugins to use Gradle if they wish so. But then they'll be very well aware that they got to be ready for a lot of pain and much meta-work around the tooling itself for developing their plugins.

Which right now might be overlooked by new developers joining the community and thinking that choosing one or another is simply a matter of taste and habits.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjq9xnyvJFoB33n07jORm7Hw6fSBf8FY5ho5FdGMDAiKug%40mail.gmail.com.

[Damien Duportal]

+1

[Adrien Lecharpentier]

+1 as well from me.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/5f43f822-37e4-45e2-85a2-3c60840ffcdfn%40googlegroups.com.

[wfoll...@cloudbees.com]

+1

[Daniel Beck]

On Wed, Dec 7, 2022 at 11:20 AM Alexander Brandes <mc.ca...@gmail.com> wrote:

There's no support for automated releases (CD, JEP-229), missing metadata for the plugin page and a few other limitations, which don't make it a great experience using it.

…

I would be ready to lift this restriction again if the JPI plugin developers provide the same scope of tools like we have for Maven.

Is the maintainer of Gradle JPI Plugin aware of these requirements and/or this discussion? I don't see him in cc.

[Alexander Brandes]

I submitted a dedicated issue on the plugin's repository outlining the TODOs and linking to this discussion.

[sghil...@gmail.com]

Hi all,

I've been maintaining the gradle-jpi-plugin for the last several years. I'm in support of this decision. My most recent OSS plugin used maven as well to take advantage of the excellent tooling the community has put together.

I elaborated a bit more on the mentioned issues in the repository, but wanted to share here as well.

My main goal for maintaining this plugin is compatibility with new Gradle versions for the internal plugins I manage. Most of the evolution over the past several years have been in support of this. For example, there are now more than 400 acceptance tests that are easily run against arbitrary versions of Gradle (including pre-releases) and Java.

I think feature parity with maven-hpi-plugin is achievable and would unlock some productivity gains, but I don't have the bandwidth or the need for this myself. I would be supportive of anyone who wanted to take that on.

> Last I checked, generated poms were also invalid, which is a problem for any consumers of these artifacts that do more than just download the hpi.

I believe this was fixed in 2019 as v0.31.0.

Best,

Steve

[Daniel Beck]

On Fri, Dec 16, 2022 at 7:43 PM sghil...@gmail.com <sghil...@gmail.com> wrote:

> Last I checked, generated poms were also invalid, which is a problem for any consumers of these artifacts that do more than just download the hpi.

I believe this was fixed in 2019 as v0.31.0.

Great, thanks for the correction. Seems "last I checked" was quite a while ago :) 

[Basil Crow]

Here are the most popular plugins still using Gradle for their build:

gradle 232,746
multiple-scms 22,241
allure-jenkins-plugin 17,121
terraform 5,046
checkmarx 4,502
jira-trigger 3,996
blackduck-detect 2,838
npm-yarn-wrapper-steps 2,152
ez-templates 2,145
templating-engine 1,742
synopsys-coverity 1,480

Multiple SCMs is already deprecated. Gradle, Allure, Checkmarx, and
the Synopsys plugins appear to be maintained by corporate teams that
may have their own build system policies. Terraform, Jira Trigger, NPM
and Yarn Wrapper and Steps, EZ Templates, and Templating Engine appear
to be community-maintained.

I wonder if there is a way we could more clearly communicate to the
above parties that we offer only partial support for Gradle and full
support only for Maven.

[Herve Le Meur]

For those using buildPluginWithGradle() from the shared pipeline library, we could add a note about it in the logs.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjoXcdRte%3DXuUwp9XbTcFte_kiT4-CBqXqvgJzzRU98LcQ%40mail.gmail.com.

[Basil Crow]

On Mon, Mar 27, 2023 at 7:27 AM 'Herve Le Meur' via Jenkins Developers <jenkin...@googlegroups.com> wrote:
> For those using buildPluginWithGradle() from the shared pipeline library, we could add a note about it in the logs.

This was done in jenkins-infra/pipeline-library#629.

[Alexis Tual]

Hi,

lately I've been working on improving the Gradle support for building Jenkins plugins, see https://github.com/jenkinsci/gradle-jpi-plugin/issues/213 for more details

The latest JPI plugin release 0.49.0 and recent changes on the pipeline-library (https://github.com/jenkins-infra/pipeline-library/pull/654 and https://github.com/jenkins-infra/pipeline-library/pull/652) enable quality checks and incremental builds on ci.jenkins.io. Where can we document this and would it be ok to re-enable Gradle support ?

One thing I've noticed though is that the cd release with Gradle would not be supported by the dedicated Github action. I wonder if it worth supporting, since I don't see it used by a lot of plugins currently: https://github.com/search?q=org%3Ajenkinsci%20%22maven-cd.yml%40v1%22%20path%3A*.yml&type=code. What do you think ?

Thanks!

CONFIDENTIALITY NOTICE: The contents of this email message, and any attachments, are intended solely for the addressee(s) and may contain confidential, proprietary and/or privileged information legally protected from disclosure. If you are not the intended recipient of this communication, or if you received this communication by mistake, please notify the sender immediately and delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, retransmission, dissemination, copying or storage of this message or its attachments is strictly prohibited.

[Mark Waite]

On Thursday, May 25, 2023 at 6:09:42 AM UTC-6 you wrote:

Hi,

lately I've been working on improving the Gradle support for building Jenkins plugins, see https://github.com/jenkinsci/gradle-jpi-plugin/issues/213 for more details

Thanks for doing that.

 

The latest JPI plugin release 0.49.0 and recent changes on the pipeline-library (https://github.com/jenkins-infra/pipeline-library/pull/654 and https://github.com/jenkins-infra/pipeline-library/pull/652) enable quality checks and incremental builds on ci.jenkins.io. Where can we document this and would it be ok to re-enable Gradle support ?

https://github.com/jenkinsci/gradle-jpi-plugin/issues/213#issue-1499881868 indicates the following reasons that the hosting team started rejecting new hosting requests for plugins built with Gradle:

• No support for JEP-229
• No support for JEP-305
• Lacking end-user documentation on jenkins.io
• No support for publishing static analysis and coverage reports on ci.jenkins.io
• No support for building on ACI (Azure Container Instance) agents
• Suspect to produce invalid pom.xml files causing issues with third parties and the update center in the past

You noted that the second bullet (JEP-305 - incremental builds) is now supported and that the fourth bullet (quality checks) are now supported and that the first bullet (JEP-229 - automated release / continuous delivery) is not supported.

Are you willing to create end user documentation on jenkins.io for those who use Gradle?

I'm not a member of the hosting team, but I think they will want to be assured that Jenkins plugins can be successfully maintained and released using Gradle.

 

One thing I've noticed though is that the cd release with Gradle would not be supported by the dedicated Github action. I wonder if it worth supporting, since I don't see it used by a lot of plugins currently: https://github.com/search?q=org%3Ajenkinsci%20%22maven-cd.yml%40v1%22%20path%3A*.yml&type=code. What do you think ?

Your query is not showing you how widely the automated release process is used.  

https://github.com/search?q=repo%3Ajenkins-infra%2Frepository-permissions-updater+%22cd%3A%22&type=code shows that 340 repositories have enabled automated release. 

I'm not sure what stops Gradle from being supported by automated release, but the number of plugins using automated releases has been steadily increasing for a 

[Alexis Tual]

Thanks for the quick answer!

On Thursday, May 25, 2023 at 3:16:30 PM UTC+2 Mark Waite wrote:

On Thursday, May 25, 2023 at 6:09:42 AM UTC-6 you wrote:

Hi,

lately I've been working on improving the Gradle support for building Jenkins plugins, see https://github.com/jenkinsci/gradle-jpi-plugin/issues/213 for more details

Thanks for doing that.

 

The latest JPI plugin release 0.49.0 and recent changes on the pipeline-library (https://github.com/jenkins-infra/pipeline-library/pull/654 and https://github.com/jenkins-infra/pipeline-library/pull/652) enable quality checks and incremental builds on ci.jenkins.io. Where can we document this and would it be ok to re-enable Gradle support ?

https://github.com/jenkinsci/gradle-jpi-plugin/issues/213#issue-1499881868 indicates the following reasons that the hosting team started rejecting new hosting requests for plugins built with Gradle:

• No support for JEP-229
• No support for JEP-305
• Lacking end-user documentation on jenkins.io
• No support for publishing static analysis and coverage reports on ci.jenkins.io
• No support for building on ACI (Azure Container Instance) agents
• Suspect to produce invalid pom.xml files causing issues with third parties and the update center in the past

You noted that the second bullet (JEP-305 - incremental builds) is now supported and that the fourth bullet (quality checks) are now supported and that the first bullet (JEP-229 - automated release / continuous delivery) is not supported.

Are you willing to create end user documentation on jenkins.io for those who use Gradle?

Sure, I can start a dedicated page with what we have and enrich later (see below)

 

I'm not a member of the hosting team, but I think they will want to be assured that Jenkins plugins can be successfully maintained and released using Gradle.

 

One thing I've noticed though is that the cd release with Gradle would not be supported by the dedicated Github action. I wonder if it worth supporting, since I don't see it used by a lot of plugins currently: https://github.com/search?q=org%3Ajenkinsci%20%22maven-cd.yml%40v1%22%20path%3A*.yml&type=code. What do you think ?

Your query is not showing you how widely the automated release process is used.  

https://github.com/search?q=repo%3Ajenkins-infra%2Frepository-permissions-updater+%22cd%3A%22&type=code shows that 340 repositories have enabled automated release. 

My understanding of https://www.jenkins.io/doc/developer/publishing/releasing-cd/ is that this step will just set the Maven repository credentials as secrets on the relevant Github repository.

The actual continuous release will happen only if using https://raw.githubusercontent.com/jenkinsci/.github/master/workflow-templates/cd.yaml, or am I missing something ? People maybe just have their own workflow using credentials to publish ?

 

I'm not sure what stops Gradle from being supported by automated release, but the number of plugins using automated releases has been steadily increasing for a 

Currently the Github action uses Maven to rebuild and publish https://github.com/jenkins-infra/jenkins-maven-cd-action/blob/master/run.sh#L8.

We could either provide a similar Github action, or simply a template and some docs on how to use it, but first I'd like to make sure it's used.

Getting a step back, I'd be curious to know why it requires rebuilding since it's supposed to be built already on ci.jenkins.io and published to the incrementals repository. The automatic releasing would just consist in promoting from the incrementals repository to the release repository in Artifactory.

 

Thanks!

[Mark Waite]

On Thursday, May 25, 2023 at 9:07:41 AM UTC-6 at...@gradle.com wrote:

Thanks for the quick answer!

On Thursday, May 25, 2023 at 3:16:30 PM UTC+2 Mark Waite wrote:

On Thursday, May 25, 2023 at 6:09:42 AM UTC-6 you wrote:

Hi,

lately I've been working on improving the Gradle support for building Jenkins plugins, see https://github.com/jenkinsci/gradle-jpi-plugin/issues/213 for more details

Thanks for doing that.

 

The latest JPI plugin release 0.49.0 and recent changes on the pipeline-library (https://github.com/jenkins-infra/pipeline-library/pull/654 and https://github.com/jenkins-infra/pipeline-library/pull/652) enable quality checks and incremental builds on ci.jenkins.io. Where can we document this and would it be ok to re-enable Gradle support ?

https://github.com/jenkinsci/gradle-jpi-plugin/issues/213#issue-1499881868 indicates the following reasons that the hosting team started rejecting new hosting requests for plugins built with Gradle:

• No support for JEP-229
• No support for JEP-305
• Lacking end-user documentation on jenkins.io
• No support for publishing static analysis and coverage reports on ci.jenkins.io
• No support for building on ACI (Azure Container Instance) agents
• Suspect to produce invalid pom.xml files causing issues with third parties and the update center in the past

You noted that the second bullet (JEP-305 - incremental builds) is now supported and that the fourth bullet (quality checks) are now supported and that the first bullet (JEP-229 - automated release / continuous delivery) is not supported.

Are you willing to create end user documentation on jenkins.io for those who use Gradle?

Sure, I can start a dedicated page with what we have and enrich later (see below)

 

I'm not a member of the hosting team, but I think they will want to be assured that Jenkins plugins can be successfully maintained and released using Gradle.

 

One thing I've noticed though is that the cd release with Gradle would not be supported by the dedicated Github action. I wonder if it worth supporting, since I don't see it used by a lot of plugins currently: https://github.com/search?q=org%3Ajenkinsci%20%22maven-cd.yml%40v1%22%20path%3A*.yml&type=code. What do you think ?

Your query is not showing you how widely the automated release process is used.  

https://github.com/search?q=repo%3Ajenkins-infra%2Frepository-permissions-updater+%22cd%3A%22&type=code shows that 340 repositories have enabled automated release. 

My understanding of https://www.jenkins.io/doc/developer/publishing/releasing-cd/ is that this step will just set the Maven repository credentials as secrets on the relevant Github repository.

The actual continuous release will happen only if using https://raw.githubusercontent.com/jenkinsci/.github/master/workflow-templates/cd.yaml, or am I missing something ? People maybe just have their own workflow using credentials to publish ?

 

I can't explain why that query does not accurately show the contents of the jenkinsci repositories, but I'm confident that it does not.  I see over 200 references to maven-cd.yml in files that are named either cd.yml or cd.yaml in the .github/workflows directories of repositories in the jenkinsci GitHub organization.

 

I'm not sure what stops Gradle from being supported by automated release, but the number of plugins using automated releases has been steadily increasing for a 

Currently the Github action uses Maven to rebuild and publish https://github.com/jenkins-infra/jenkins-maven-cd-action/blob/master/run.sh#L8.

We could either provide a similar Github action, or simply a template and some docs on how to use it, but first I'd like to make sure it's used.

Getting a step back, I'd be curious to know why it requires rebuilding since it's supposed to be built already on ci.jenkins.io and published to the incrementals repository. The automatic releasing would just consist in promoting from the incrementals repository to the release repository in Artifactory.

The ci.jenkins.io machine is intentionally not considered "trusted".  We don't place credentials on ci.jenkins.io

 

 

Thanks!

[Basil Crow]

In addition to what Mark wrote, I would want to see some of our top
open-source Gradle-based Jenkins plugins (e.g., Gradle, Terraform,
Checkmarx, Jira Trigger, etc.) upgraded to use the latest
gradle-jpi-plugin toolchain, a recent LTS release as the Jenkins core
baseline, and a recent plugin Bill of Materials (BOM) and inspect the
resulting JPI that is produced to ensure it is up to parity with a JPI
produced with the maven-hpi-plugin toolchain (e.g., equivalent
manifest entries, equivalent compiler options, etc). Right now, most
of the abovementioned plugins are using an ancient Jenkins core
baseline and an ancient plugin Bill of Materials (BOM), which does not
speak to the viability of the gradle-jpi-plugin toolchain.

In general the gradle-jpi-plugin toolchain needs to be at feature
parity with the maven-hpi-plugin toolchain before we could accept new
plugin hosting requests. That means, in addition to what Mark
described, that every feature in jenkinsci/plugin-pom and
jenkinsci/maven-hpi-plugin should have an equivalent in
gradle-jpi-plugin, including Java 11/17 support for running unit
tests, integration with the latest versions of JUnit/Mockito/Hamcrest
out-of-the-box, opt-in code formatting using our project-wide Spotless
configuration, equivalents to our Maven Enforcer rules, etc. Since I
am constantly adding new features to plugin-pom/maven-hpi-plugin, this
implies a commitment to proactively follow such development and
sideport the same features to the gradle-jpi-plugin toolchain as they
are introduced.

[Alexis Tual]

Thanks all for your answers, after discussing it internally at Gradle, we won't be able to implement all those new (for us) requirements and maintain them over time without help.