[Announce] Gerrit CI workflow to become a brand-new Jenkins plugin

[Luca Milanesio]

Posting to the jenkinsci-users mailing list as well: even if I mainly needed feedback from the plugins developers, feedback from a user-perspective of the new plugin would be *really appreciated* as well :-)

-- See below the original post --

Hi Gerrit and Jenkins Community,

after over two years of iterations and improvements of the Gerrit CI workflow (https://gerrit-ci.gerritforge.com), I have decided that is about time to extract the logic of our workflow and make it available as a brand-new Jenkins plugin!

Why?

I wanted to use the Gerrit CI validation workflow with potentially any project, including Gerrit plugins or anybody else wanting to adopt it.

I could just "copy & paste" our Groovy workflow, however, that does not seem a sensible and long-term approach.

I wanted to have "something more" than a pure triggering mechanism: I wanted to extend the power of Jenkisfile with the Gerrit review workflow verbs.

Why not?

Why should I write yet another Gerrit/Jenkins plugin? Isn't Gerrit Trigger Plugin (https://wiki.jenkins.io/display/JENKINS/Gerrit+Trigger) enough?

We couldn't use it against gerrit-review.googlesource.com because stream events are just not accessible.

There are unresolved issues about:

- Stability: stream-events are based on SSH, which isn't scalable, reliable against downtime, etc.

- Usability: at every JenkinsWorld conference people still come to me asking "how do I setup correct the Gerrit Trigger plugin"?

- Integration: using it inside a JenkinsFile isn't that straightforward and multi-branch projects aren't supported either

What would it be?

I will publish a new plugin named "Gerrit Pipeline Plugin."

The new name indicates:

- Deep integration with Jenkins Pipeline

- Out-of-the-box integration with Gerrit validation workflow in the pipeline

A simple example scripted Jenkinsfile would be:

node {
    checkout scm

    gerrit.withServer("http://gerrit:8080/", "gerrit") {

        try {
            docker.image('gerritforge/play-sbt-8-jdk-alpine').inside {
                stage('Build') {
                    sh 'sbt compile'
                }
                stage('Test') {
                    sh 'sbt 'test'
                }
            }

            gerrit.review("Verified", 1, "It works !")
        } catch (e) {
            gerrit.review("Verified", -1, "Breaks the build ;-(")
            throw e
        }
    }
}

(Where:

- https://gerrit-review.example.com would be the Gerrit URL

- mycredentialsid would be the id of the credentials to access Gerrit)

One key-aspect will be: stateless, configuration-less (apart from the credentials stored in Jenkins keychain)

That means that multiple Jobs, multiple branches of the same Job, can have their own Gerrit integration defined and working out-of-the-box.

No more people asking "how do I configure the Gerrit integration"?

You'll just define a gerrit.withServer() { } and ... it will just work.

When?

I am planning to showcase the first prototype of the new plugin at the Jenkins World Conference in San Francisco inside my "Data-Driven Pipeline workshop."

(https://jenkinsworld20162017.sched.com/event/APTd/data-driven-pipeline-workshop-free)

Shortly afterward, I will publish the plugin on the GerritForge's GitHub account and will start the integration process into the Jenkins CI organization.

Next steps?

A second iteration of the plugin would have a Declarative pipeline equivalent as well, which would require even less code required.

A third iteration of the plugin would support BlueOcean as well, to have a fully UX-integrated experience.

The goal is to have Gerrit Code Review to be a 1st class citizen in the Jenkins ecosystem :-)

So what happens to the Gerrit Trigger Plugin?

The new plugin is not going to replace the current Gerrit Trigger Plugin, but would rather represent an alternative to simpler scenarios when you just require a standard Jenkinsfile Gerrit validation workflow. For all the current users of the Gerrit Trigger Plugin things wouldn't change, unless they need a more Jenkisfile-integrated experience.

Feedback? Like it? Hate it? Have your say :-)

Luca.

[Luca Milanesio]

On 18 Aug 2017, at 14:10, Robert Sandell <rsan...@cloudbees.com> wrote:

Cool,

So is this intended as a Branch API implementation? Something a la GitHub Branch Source Plugin?

Yes, exactly.

That would allow to have separate Jenkinsfile definitions even on different Changes and Patch-Sets.

Which is what my mind goes to when reading things like "Out-of-the-box integration with Gerrit validation workflow in the pipeline".

Yep.

From a quick reading I'm not sure that is what you are planning, it seems like something else?

Nope, that's exactly my goal.

The gerrit-branch-source-plugin is something that has been on my todo list for over two years and I haven't been able to free enough cycles to implement it yet. So getting that ghost off my chest would be nice ;)

Yes, I remember we talked about that during the last Jenkins World Summit in 2016, lots of people wanted that and came to our Booth asking if we did something in that direction.

This year, we'll have some answers for them :-)

Looking forward to get your reviews ;-)

Luca.

/B

2017-08-18 13:44 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com>:

On 18 Aug 2017, at 12:19, Gustaf Lundh <gustaf...@axis.com> wrote:

Cool stuff! Really looking forward to try it out!

> - Stability: stream-events are based on SSH, which isn't scalable, reliable against downtime, etc.

Please note: The Gerrit Trigger plugin does support consuming events from rabbit-mq and also from the Gerrit event-log plugin.

Yes, with RabbitMQ becomes much more reliable and scalable ... but it is *a lot* of extra complexity and infrastructure :-(

Additionally, event-log plugin stores all the events on the Gerrit DB, which is *again* a lot of extra overhead :-((

I think it is also worth nothing that Gerrit Trigger is push based and not pull based. Which makes quite a difference when you have a great amount of projects.

My new plugin is push based as well, based on WebHooks.

But I agree with you, for a very large large scale setup, the Gerrit Trigger Plugin remains the 1st choice and the extra overhead of infrastructure is justified by the size of the installation.

For medium-small scale setups, it is way too complex for the people to setup and manage, based from the feedback I got from my clients :-(

/Gustaf

--
--
To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Robert Sandell

Software Engineer

CloudBees Inc.

[Yotam Shapira]

Sounds great, seems like something we would definitely use. This has been a missing functionality for a long time!

[Cuong Tran]

Hi Luca,

Have you had a chance to publish this to github?

[luca.mi...@gmail.com]

Not yet, I'm just back from the conference. Need to apply some suggestions from Robert and Jesse and then it'll be ready for publishing.

Luca

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2c459ab4-cc59-42b4-a02d-92a1069a56fa%40googlegroups.com.

[shipingc]

This is the plugin I am looking for to add customized gerrit comments from Jenkins pipeline script.

One question: does your new plugin support using env variables in message string? For example, I want to say:   

    
         gerrit.review("Verified", 1, "Build Successful $BUILD_URL")

I tried to implement the same functionality in jenkins pipleline script by directly doing ssh gerrit review call, such as:

       stage('Build') {
sh '''export home_dir="$PWD/lib-dependent"
ssh -p $GERRIT_PORT BuildLinuxDebug@$GERRIT_HOST gerrit review --project "$GERRIT_PROJECT" --message \'"Build Started $BUILD_URL"\' --verified '-1' --code-review '-1' "$GERRIT_PATCHSET_REVISION"

However $BUILD_URL was not expanded in gerrit comment due to the quote issues.

Best regards,

Shiping

On Friday, August 18, 2017 at 2:17:00 AM UTC-7, Luca Milanesio wrote: