[JIRA] (JENKINS-50548) Shared Library should be allowed to declare reusable stages

15 views
Skip to first unread message

tobilarscheid@gmail.com (JIRA)

unread,
Apr 3, 2018, 10:50:03 AM4/3/18
to jenkinsc...@googlegroups.com
Tobias Larscheid created an issue
 
Jenkins / Improvement JENKINS-50548
Shared Library should be allowed to declare reusable stages
Issue Type: Improvement Improvement
Assignee: Unassigned
Components: pipeline
Created: 2018-04-03 14:49
Priority: Minor Minor
Reporter: Tobias Larscheid

Currently, we have a lot of reusable steps we use in our Jenkinsfiles like so:

pipeline {
  stages {
    stage('Build') {
      steps {
        reusableBuild()
      }
    }
    stage('Test') {
      steps {
        reusableTest()
      }
    }
  } 
}

I feel it is very repetitive to always replicate the stage definition - the way I look at this, these should be reusable as well. A pipeline would then simply consist of reusable, composable stages:

pipeline {
  stages {
    reusableBuild()
    reusableTest()
    stage 'Something individual' {
      echo 'only for this project'
    }
  } 
}

Maybe this is already possible, but at the moment I have no idea how to define reusable stages in a shared library - any hint would be very much appreciated. I would also be willing to provide a PR, if only I had an idea which code to touch

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

tobilarscheid@gmail.com (JIRA)

unread,
Apr 3, 2018, 11:37:03 AM4/3/18
to jenkinsc...@googlegroups.com
Tobias Larscheid updated an issue
Change By: Tobias Larscheid
Currently, we have a lot of reusable steps we use in our Jenkinsfiles like so:
{code:java}

pipeline {
  stages {
    stage('Build') {
      steps {
        reusableBuild()
      }
    }
    stage('Test') {
      steps {
        reusableTest()
      }
    }
  }
}{code}

I feel it is very repetitive to always replicate the stage definition - the way I look at this, these should be reusable as well. A pipeline would then simply consist of reusable, composable stages:
{code:java}

pipeline {
  stages {
    reusableBuild()
    reusableTest()
    stage 'Something individual' {
      steps {
      echo 'only for this project'
    }
  }
  }

}{code}
Maybe this is already possible, but at the moment I have no idea how to define reusable stages in a shared library - any hint would be very much appreciated. I would also be willing to provide a PR, if only I had an idea which code to touch ;)

andrew.bayer@gmail.com (JIRA)

unread,
Apr 3, 2018, 1:05:03 PM4/3/18
to jenkinsc...@googlegroups.com
Andrew Bayer updated an issue
Change By: Andrew Bayer
Component/s: pipeline-model-definition-plugin
Component/s: pipeline

jeezusjr@gmail.com (JIRA)

unread,
May 3, 2018, 2:28:02 AM5/3/18
to jenkinsc...@googlegroups.com
Jesus Alvarez commented on Improvement JENKINS-50548
 
Re: Shared Library should be allowed to declare reusable stages

I spent all day trying do this with Declarative Pipelines and finally had to give up. It was so easy to do things like this with Scripted Pipelines, but Declarative is the way going forward. It would be nice to have this with the shared pipeline library.

onhate@gmail.com (JIRA)

unread,
May 4, 2018, 11:51:02 AM5/4/18
to jenkinsc...@googlegroups.com

Hello,

As a suggestion you can use the (not well documented) feature that loads scripts on the fly.

this will load the libs from the same repository and branch you are checking out, it will go to folder /vars/ and load all .groovy files.
library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)

<ROOT>/
    - vars/
        - deploy.groovy
        - setup.groovy

The name of the .groovy script will become a step on pipeline, as you can see the steps setup and deploy.
 

#!groovy
script {
    library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
}
pipeline {
    stages {
        stage('Setup & Deploy') {
            steps {
                setup()
                deploy()
            }
        }
    }
}

setup.groovy

#!/usr/bin/env groovy
def call() {
    sh "echo 'YEAH'"
    sh "some/other/script.sh"
}

deploy.groovy

#!/usr/bin/env groovy
def call() {
    def props = readProperties file: 'environment.env'
    env.PROP_1 = props['PROP_1']

    sh "script/that/uses/PROP_1"
}

 

onhate@gmail.com (JIRA)

unread,
May 4, 2018, 11:52:05 AM5/4/18
to jenkinsc...@googlegroups.com
Marcelo Luiz Onhate edited a comment on Improvement JENKINS-50548
Hello,

As a suggestion you can use the (not well documented) feature that loads scripts on the fly.

this will load the libs from the same repository and branch you are checking out, it will go to folder {color:red}/vars/{color} and load all .groovy files.
{ color:red noformat }
library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
{ color noformat }
{noformat}

<ROOT>/
    - vars/
        - deploy.groovy
        - setup.groovy
{noformat}

The name of the .groovy script will become a step on pipeline, as you can see the steps *setup* and *deploy*.
 
{code:java}

#!groovy
script {
    library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
}
pipeline {
    stages {
        stage('Setup & Deploy') {
            steps {
                setup()
                deploy()
            }
        }
    }
}
{code}

{color:red}setup.groovy{color}
{code:java}

#!/usr/bin/env groovy
def call() {
    sh "echo 'YEAH'"
    sh "some/other/script.sh"
}
{code}

{color:red}deploy.groovy{color}
{code:java}

#!/usr/bin/env groovy
def call() {
    def props = readProperties file: 'environment.env'
    env.PROP_1 = props['PROP_1']

    sh "script/that/uses/PROP_1"
}
{code}

 

onhate@gmail.com (JIRA)

unread,
May 4, 2018, 11:52:05 AM5/4/18
to jenkinsc...@googlegroups.com
Marcelo Luiz Onhate edited a comment on Improvement JENKINS-50548
Hello,

As a suggestion you can use the (not well documented) feature that loads scripts on the fly.
{code:java}
#!groovy
script {
    library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
}
pipeline {
    stages {
        stage('Setup & Deploy') {
            steps {
                setup()
                deploy()
            }
        }
    }
}
{code}
this will load the libs from the same repository and branch you are checking out, it will go to folder {color:red}/vars/{color} and load all .groovy files.

{noformat}

library identifier: "setup@$BRANCH_NAME", retriever: legacySCM(scm)
{noformat}

stevengfoster@gmail.com (JIRA)

unread,
Jul 16, 2018, 9:51:01 AM7/16/18
to jenkinsc...@googlegroups.com

Declarative stages have become so flexible now: they can have their own options, environment, post stages, agent etc.

This makes the ability to reuse and share them even more powerful. My use case is running tests against a selection of specialised hardware, accessible by a generic build agent. In conjunction with the lockable resources plugin, the typical stage for this in the middle of a pipeline looks something like:

stage ('Run tests on kit') {
  agent { label 'local-kits' }
  options {
    lock(label: 'kit-type-1', variable: 'KIT_IP', quantity: 1)
  }
  steps {
    runTest('example-test.zip') // existing shared function to download and run previously archived binary on reserved kit
  }
  post {
    always {
      junit 'results.xml'
    }
  }
}

I use this in a ton of places, and help others include it in their pipelines. If I could replace all that (or maybe just the contents of stage { } ) with a call to a library, it would be really handy.
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

andrew.bayer@gmail.com (JIRA)

unread,
Nov 16, 2018, 1:31:06 PM11/16/18
to jenkinsc...@googlegroups.com
Change By: Andrew Bayer
Status: Open Fixed but Unreleased
Resolution: Duplicate
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

tom.ghyselinck@excentis.com (JIRA)

unread,
Nov 19, 2018, 2:02:08 AM11/19/18
to jenkinsc...@googlegroups.com
Tom Ghyselinck commented on Improvement JENKINS-50548
 
Re: Shared Library should be allowed to declare reusable stages

Hi Andrew Bayer,

Is this really a duplicate of JENKINS-49135?

In PR #241, you explicitly say that you won't support the directives stage, stages, steps, etc.
and this is actually what we do want here.

With best regards,
Tom.

tom.ghyselinck@excentis.com (JIRA)

unread,
Nov 19, 2018, 2:04:02 AM11/19/18
to jenkinsc...@googlegroups.com
Tom Ghyselinck edited a comment on Improvement JENKINS-50548
Hi [~abayer],

Is this really a duplicate of [ # JENKINS-49135 |https://issues.jenkins-ci.org/browse/JENKINS-49135 ]?

In [PR #241|https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/241], you explicitly say that you won't support the directives {{stage}}, {{stages}}, {{steps}}, etc.
and this is actually what we _do_ want here.

With best regards,
Tom.

frankgenerated@gmail.com (JIRA)

unread,
Apr 1, 2019, 9:54:09 AM4/1/19
to jenkinsc...@googlegroups.com

Hello,

This fix is marked as "Fixed but Unreleased", can someone add a pointer to the PR implementing this? Or any documentation?

Thanks

  • Frank

colmose@gmail.com (JIRA)

unread,
Apr 12, 2019, 11:28:02 AM4/12/19
to jenkinsc...@googlegroups.com

Would also love to see if there is docs or something for this?

csaba.harmath@gmail.com (JIRA)

unread,
Jun 7, 2019, 3:28:03 PM6/7/19
to jenkinsc...@googlegroups.com

I would like this as well.

Use case:

share build stages across similar projects, to keep things DRY and have a central shared library defining the various build stages per project types.

i.e.: I might have many many microservices all using the same 1) package restore 2) lint 3) build 4) run unit tests  stages with only parameter differences and it would be awesome if app devs can just include a single line with the necessary parameters to call these 4 steps.

 

so instead of

stages {
   stage('restore') {
    steps

{      echo 'package restore'      restore()     }

  }

stage('build') {
  steps

{     echo 'building'     build()    }

  }

stage('test') {
   steps

{      echo 'testing'      test()     }

 }
}

you could do this:

stages

{    sharedMicroserviceStages() }

 

 

 

 

 

csaba.harmath@gmail.com (JIRA)

unread,
Jun 7, 2019, 3:30:03 PM6/7/19
to jenkinsc...@googlegroups.com
CJ Harmath edited a comment on Improvement JENKINS-50548
I would like this as well.

Use case:

share build stages across similar projects, to keep things DRY and have a central shared library defining the various build stages per project types.

i.e.: I might have many many microservices all using the same 1) package restore 2) lint 3) build 4) run unit tests  stages with only parameter differences and it would be awesome if app devs can just include a single line with the necessary parameters to call these 4 steps.

 

so instead of
 
{code:java}
stages {
   stage('restore') {
    steps {
      echo 'package restore'
    
restore()
   
}
  }
stage('build') {
  steps {
     echo 'building'
     build()
    }
  }
stage('test') {
   steps {
      echo 'testing'
     test()
     }
 }
}
{code}
 

 you could do this:

  
{code:java}
 stages {   
   sharedMicroserviceStages()
}
 {code}
 

 

 

 

 

csaba.harmath@gmail.com (JIRA)

unread,
Jun 7, 2019, 3:37:02 PM6/7/19
to jenkinsc...@googlegroups.com

Okay, so I just re-read the docs and as I understand what I should be doing is to just put the entire pipeline into my shared library.

https://jenkins.io/doc/book/pipeline/shared-libraries/#defining-declarative-pipelines

This kind of makes me wonder if we allow defining entire pipeline, why not allow stages as well ?

But I guess I am already happy with this as I only need to maintain one pipeline per app type.

alvizu@gmail.com (JIRA)

unread,
Aug 9, 2019, 5:12:03 PM8/9/19
to jenkinsc...@googlegroups.com

For anyone else who runs into this: Andrew closed this as 'FIxed but unreleased' but with a resolution of 'Duplicate'.

 

The duplicate issue is JENKINS-49135 and track there
Reply all
Reply to author
Forward
0 new messages