Enabling Continuous Delivery / automated release process for Gradle project

[Jamie Tanna]

Hey folks,

I saw https://groups.google.com/g/jenkinsci-dev/c/4o75QPEslyU and https://www.jenkins.io/doc/developer/publishing/releasing-cd/ and was very interested in setting it up for the Job DSL Plugin, but it's a Gradle project.

Has anyone set up CD for a Gradle project before that they can share?

[Tim Jacomb]

I don't think it will have been done yet because the GitHub actions only support maven.

There's not many projects using gradle so I guess you're the first to look at it.

The action wouldn't need much work to support it,

Probably just a flag defaulted to maven and then a change in this script:

https://github.com/jenkins-infra/jenkins-maven-cd-action/blob/master/run.sh

That's assuming that the gradle plugin supports incrementals which I don't know if it does.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/0419de88-72d1-4562-92d3-3a1723e99d92n%40googlegroups.com.

[Jesse Glick]

On Wed, Nov 17, 2021 at 4:05 AM Tim Jacomb <timja...@gmail.com> wrote:

That's assuming that the gradle plugin supports incrementals which I don't know if it does.

It does not, and the whole concept is rather specific to Maven.

Stepping back, there are basically two sides to JEP-229: a backend patch to the RPU processor whose sole purpose is the provisioning (and rotating) of a GitHub secret in your repo which allows Actions in that repo to deploy to the associated paths in Artifactory, when `cd.enabled = true`; and various components under your control (a Maven extension originally created for JEP-305 Incrementals, conventional Maven metadata edits, a stock workflow with a couple specialized Actions) that cause pushes to trunk to automatically trigger a `mvn deploy` and publish a GitHub Release based on Release Drafter when there have been “interesting” changes.

One thing which should be possible in principle, but which I have not had reason to try, is to use MRP with the backend part. You would create a manually triggered workflow which just runs `mvn -B release:{prepare,perform}`. The `$GITHUB_TOKEN` has write permission, so it can push the two junk commits MRP generates as well as pushing the tag. This could be integrated with Release Drafter or the release/changelog aspect could be manual. You would not get the CD system, but you would at least not need to have personal credentials to Artifactory, and there would be no need to change anything else about plugin sources (would use the traditional manually-maintained version scheme). Basically take your existing local release command and wrap it in a workflow which just binds the Artifactory authentication but otherwise makes no process changes.

I presume the same would apply to Gradle-based plugins, using whatever command it is you use to perform releases from Gradle. This would be an easy step. If you wanted actual CD for a Gradle-based plugin you would need to put some thought into what that would look like, especially in terms of version maintenance and overall experience.

Remember that even without JEP-305 you can trivially deploy a Maven-based project in a CD style. You pick one of the three magic property names that Maven allows in `version`, and override it from a deployment script. Thus configure your POM

<version>${revision}</version>

<properties>

  <!-- fallback so local dev builds do something -->

  <revision>0-SNAPSHOT</revision>

</properties>

Then pick a version during deployment automatically, via human input, whatever, and:

 

version=$(date -u +%Y%m%d%H%M%S)

mvn -s settings-referring-to-MAVEN_TOKEN.xml -B -Drevision=$version -DaltDeploymentRepository=maven.jenkins-ci.org::default::https://repo.jenkins-ci.org/releases/ -Pquick-build clean deploy

gh api -F ref=refs/tags/$version -F sha=$GITHUB_SHA /repos/$GITHUB_REPOSITORY/git/refs

(I am passing `-Pquick-build` here under the assumption that tests are already run by ci.jenkins.io, which JEP-229 systems enforce, and would be redundant here. You can omit it if you do not wish to make that assumption and you know your tests are able to run inside the workflow.)

This works fine for a self-contained project: one whose only consumer is the update center, paying attention solely (I suppose) to the `*.hpi!/META-INF/MANIFEST.MF`. It does not work well if any other project might declare a dependency on it, if the deployed POM is used for any purpose, if it might be run via `plugin-compat-tester`, etc., which is why JEP-305 is so tricky. I really am not sure what considerations apply to Gradle-based projects.

[Jesse Glick]

On Wednesday, November 17, 2021 at 8:29:07 AM UTC-5 Jesse Glick wrote:

One thing which should be possible in principle, but which I have not had reason to try, is to use MRP with the backend part.

Implemented in https://github.com/jenkins-infra/jenkins-maven-cd-action/pull/14. Not as smooth as full JEP-229 but basically works, with the usual MRP downsides. Probably this could be cloned and adapted somehow to support Gradle.

[jn...@cloudbees.com]

there is possibility of something part way between the current CD flow (no junk pushes) and m-r-p.

m-r-p does not have to just take the current version and remove snapshot and increment the last digit, nor does it need to push the commits back (just the tag) (which I believe are your main objections to m-r-p?)  e.g. https://github.com/stephenc/git-timestamp-maven-plugin#release-assistance 

WDYT? 

[Jean-Marc Meessen]

Hello,

It is possible to configure it so that the release process is triggered manually (accumulating several changes) instead of an automatic trigger.

The documentation (or sample code) shows what is the section to "comment out".

/- Jmm

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/a169dcff-2f4c-4353-8549-03f8d0676475n%40googlegroups.com.

[Jesse Glick]

On Tue, Dec 14, 2021 at 9:53 AM 'jn...@cloudbees.com' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

there is possibility of something part way between the current CD flow (no junk pushes) and m-r-p

As mentioned previously it is possible to just `mvn deploy` whatever stuff you like, including say timestamps, and optionally pushing a tag, without pushing MRP-style commits. Fine for things like services, but this will generally confuse any tools which expect to consume your project as a dependency, since the version coming from a local checkout of that tag will not match whatever you deployed. That is why for purposes of Jenkins components including plugins I do not recommend attempting to do anything between full JEP-229, with a deterministic version scheme, and full MRP with the pair of junk commits.

[James Nord]

But my point is the flatten plugin confuses tools and is awkward for many.  Using MRP as I described would not confuse any tool as it would  reqrite a pom as happens today. 

I'm also confused now as you are using MRP which the referenced  comment is about.

[Jesse Glick]

On Tue, Dec 14, 2021 at 6:04 PM James Nord <james...@gmail.com> wrote:

the flatten plugin confuses tools

Which? I think we have ironed out any problems as lots of components have picked up JEP-305. Even seems to work fine on BOMs; the last things I am hesitant to flatten without more careful testing are parent POMs.

Using MRP as I described would not confuse any tool

Depends on the tool. The problem would be that a source checkout at the commit/tag would, if loaded as a project rather than a deployed artifact, have a version different from the artifact. (Also true in JEP-229 components by default, but you can simply pass a fixed and unconditional `-Dset.changelist` and the resulting version is deterministic.)

And as mentioned previously, if you drop that constraint, you can flatten and trivially `deploy` any version string you like (and tag it) without going through all the complexity of MRP. The only reason to deal with MRP is to get the two junk commits that rewrite `<version>` (and `<tag>`) in a tagged SCM commit.

[jn...@cloudbees.com]

>> Using MRP as I described would not confuse any tool

>Depends on the tool. The problem would be that a source checkout at the commit/tag would, if loaded as a project rather than a deployed artifact, have a version different from the artifact. (Also true in JEP-229 components by default, but you can simply pass a fixed and unconditional `-Dset.changelist` and the resulting version is deterministic.)

I beleive we are not understanding each other.  if a tool would be broken by what I am suggesting then it is a broken tool period in that it would have exactly the same issue with a project released today by running "mvn release:prepare release:perform"  the tag contains the fully resolved pom you are still tagging the commit "[maven-release-plugin] prepare release project-a.bcd"  the only difference is that these commits never appear on the mainline branch (master) and are only in the tag.

thus this would actually be in a better state than JEP-229 currently is.

> And as mentioned previously, if you drop that constraint, you can flatten and trivially `deploy` any version string you like (and tag it) without going through all the complexity of MRP. The only reason to deal with MRP is to get the two junk commits that rewrite `<version>` (and `<tag>`) in a tagged SCM commit.

but we do not want to drop that containt - that is the biggest issue we have with JEP229 currently and I am proposiing a solution does not have the "junk commits" on master, nor has the "tagged sources are junk" issue.

happy to jump on a Hangout if you want further explanation.

/James

[Jesse Glick]

On Thu, Dec 16, 2021 at 10:44 AM 'jn...@cloudbees.com' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

you are still tagging the commit "[maven-release-plugin] prepare release project-a.bcd"  the only difference is that these commits never appear on the mainline branch (master)

Yes, this is possible. Generally undesirable for release tags not to be on the trunk branch, though. Compare views and bisect will be all messed up, history views like gitg will not indicate where release points were, etc.

that is the biggest issue we have with JEP229 currently

What is the issue? JEP-229 is working fine so far AFAICT.