Extension points for declarative pipeline in shared library

[Francois Marot]

Hello all,

I have switched all my Java build job to use a common declarative pipeline, so in all my JenkinsFile, I have the very simple following content:

@Library('classicJavaBuild') _

classicJavaBuild()

and the shared pipeline is defined as:

def call(skipSonarParam = false, skipTestsCompileAndExecParam = false, skipDocGenerationParam = false, doMavenReleaseParam = false) {
  
    pipeline {
        agent { node { label 'linuxWithXvfb' } }
        
        parameters {
            booleanParam(defaultValue: skipSonarParam,
                description: 'skip Sonar reporting',
                name: "skipSonar")
            booleanParam(defaultValue: skipTestsCompileAndExecParam,
                description: 'skip tests execution BUT ALSO TESTS COMPILATION',
                name: "skipTestsCompileAndExec")
            }
         
        tools {
            maven 'maven-3'
            jdk 'jdk-8'
        }
        
        stages {
    ......a few stages....

 

 

So far everything is fine. But now I have a new project where I would like to "extend" this pipeline:

1- add a new parameter exclusively in the new project while retaining the common ones
2- add a new 'stage' (or maybe a new step in an existing stage).

Is there any way to add extension points to a pipeline in a shared library so that my Jenkinsfile can still benefit from my common pipeline while defining also it's own additions ?

Regards

[Reinhold Füreder]

Hello Francois,

 

I think this -- still rather small amount of additional flexibility -- already asks for scripted pipelines instead of declarative ones (cf. https://issues.jenkins-ci.org/browse/JENKINS-46547).

 

At least for now, but I guess/assume not just for now… (Although I just stumbled over https://issues.jenkins-ci.org/browse/JENKINS-42224 “Need ability to create reusable chunks of Declarative Pipeline”…)

 

Of course, scripted pipelines within shared pipelines allow for even more fancy and flexible stuff whilst providing the same API for usage in Jenkinsfile, so from the perspective of the users it may not make any difference.

 

Regards,

Reinhold

 

P.S.: Nonetheless I therefore hope that (really) all cool features from declarative pipelines will be available to scripted ones too: hm, admittedly at the moment only https://issues.jenkins-ci.org/browse/JENKINS-47286 “Support skipping stages in scripted pipelines for nice visualization in blue ocean and classic UI stage view” comes to my mind…

[Francois Marot]

Thanks Reinhold.
In the end I could not find a really flexible way to add "extension points".
Nevertheless, I came out with an alternative with default parameters that each job using the common pipeline may override.
My common pipeline named "classicJavaBuild" looks like:

./vars/classicJavaBuild.groovy

def call(Map defaultProvidedParamsMap = [:]) {

    // init default parameters
    Map defaultMap = [skipSonarParam: false,
        skipTestsCompileAndExecParam: false,
        skipDocGenerationParam: false,
        agentParam: 'linuxWithXvfb']    // can be overriden with any label of a Jenkins node
    defaultMap << defaultProvidedParamsMap   // override default values with provided ones, so each job can just provide the overriden ones, while mostly relying on the defaults values
   
    pipeline {
        agent { node { label defaultMap.agentParam } }    // the build will only run on nodes (ie slaves) tagged with 'linuxWithXvfb' showing that those builds are quite standard BUT require Xvfb installed on the host
        environment {
            MAVEN_CMD = "mvn -U --batch-mode -Dmaven.repo.local=${isUnix() ? '$OLEA_JENKINS_HOME' : '%OLEA_JENKINS_HOME%'}/.m2/repository/ -s ${isUnix() ? '$MAVEN_SETTINGS_XML' : '%MAVEN_SETTINGS_XML%'} "
        }
        parameters {
            booleanParam(defaultValue: defaultMap.skipSonarParam,

                description: 'skip Sonar reporting',
                name: "skipSonar")

            booleanParam(defaultValue: defaultMap.skipTestsCompileAndExecParam,

                description: 'skip tests execution BUT ALSO TESTS COMPILATION',
                name: "skipTestsCompileAndExec")

            booleanParam(defaultValue: defaultMap.skipDocGenerationParam,
                description: 'skip DocGeneration',
                name: "skipDocGeneration")
....

So you can see above how I declare some default values for some parameters that each specific job may decide to override.
That more or less solved the problem for me.
As a bonus I should try to pass a groovy closure inside one of those parameter to know if it is possible to pass a specific chunk of code, defined by a job with some specific needs. If I try it I'll tell you !



def call(Map defaultProvidedParamsMap = [:]) {

[Francois Marot]

OK so it works: we can have jobs using a common pipeline AND defining script to be run at specific point in the pipeline (for exemple after as in my exemple below):

@Library('classicJavaBuild') _

def postPipelineRunSomeJob = { argsMap ->
    build job: "someJob/master"
}

classicJavaBuild(postPipelineScript: postPipelineRunSomeJob)

In the common pipeline file, add this in the "defaultMap":

postPipelineScript: {param -> echo "no specific postPipelineScript defined."}

and at the end of the file, before the end of the call() method, add:

    // Run a post-pipeline script that the caller may have passed - otherwise a default 'doing nothing' script is called
    defaultMap.postPipelineScript.call(defaultMap)

Obviously there is no need to define a post-pipeline script, but you can use the same technique to call you job-defined script in any step. So you can have the "extension point feature".

Cheers !

[Reinhold Füreder]

@Francois: Thanks for sharing your approach (that did not come to my mind to be honest).

 

Just for the sake of completeness and kind of praising Jenkins development listening to the needs of us users, please have a look at some of the new features for declarative pipelines just published/announced yesterday: https://jenkins.io/blog/2018/04/09/whats-in-declarative/ might make such (real-world) ideas easier or more straight forward to implement, I think.

 

HTH Reinhold