Writing integration tests for Global Libraries

[Michael Kobit]

I'm working on writing some global libraries using https://github.com/jenkinsci/workflow-cps-global-lib-plugin. My current process for iterating locally is:

• Start up a local Jenkins instance
• Point the globalLibs directory to my global libraries repository
• Create a few jobs that use my global libraries
• Write some code
• Run jobs
• Check for results in logs

This is inefficient and error prone.

The workflow doesn't seem much better when using the recent @Library support.

My question is, what do I need to do to get some of the Jenkins infrastructure in place to write automated, integration tests for Jenkins pipelines jobs and Global libraries code?

I am using Gradle, and was hoping to see if anybody else has been successful in setting up testing locally so I can validate that I my libraries work in an actual Jenkins pipeline execution with all of the Groovy CPS transformations and other nuances of writing Groovy libraries for Jenkins pipelines.

My current build.gradle looks something like this, but I still haven't gotten it working:

plugins {
  id 'build-dashboard'
  id 'groovy'
}
description = 'Libraries written for use with Jenkins Global Pipeline Libraries'

repositories {
  jcenter()
  maven {
    url 'http://repo.jenkins-ci.org/public'
  }
  mavenLocal()
}

sourceSets {
  main {
    groovy {
      // Jenkins Global Workflow Libraries requires sources to be at 'src'
      srcDirs = ['src']
    }
    java {
      srcDirs = []
    }
    resources {
      srcDirs = []
    }
  }
  test {
    groovy {
      // configure the test source set so that it is not part of the Global Pipeline Libraries
      srcDirs = ['unitTest']
    }
    java {
      srcDirs = []
    }
    resources {
      srcDirs = []
    }
  }
  jenkinsIntegrationTest {
    groovy {
      srcDirs = ['jenkinsIntegrationTest']
    }
    java {
      srcDirs = []
    }
    resources {
      srcDirs = []
    }
    compileClasspath += sourceSets.main.runtimeClasspath
    runtimeClasspath += sourceSets.main.runtimeClasspath
  }
}

configurations {
  jenkinsIntegrationTestCompile.extendsFrom testCompile
  jenkinsIntegrationTestRuntime.extendsFrom testRuntime
}

tasks.create('jenkinsIntegrationTest', Test) {
  group = LifecycleBasePlugin.VERIFICATION_GROUP
  description = 'Runs tests against of actual Jenkins Pipelines'
  testClassesDir = sourceSets.jenkinsIntegrationTest.output.classesDir
  classpath = sourceSets.jenkinsIntegrationTest.runtimeClasspath
}

dependencies {
  compile 'org.codehaus.groovy:groovy-all:2.4.7'

  testCompile 'org.spockframework:spock-core:1.0-groovy-2.4'
  testCompile 'junit:junit:4.12'

  jenkinsIntegrationTestCompile 'org.jenkins-ci.main:jenkins-core:2.17'
  jenkinsIntegrationTestCompile 'org.jenkins-ci.main:jenkins-test-harness:2.17'
  jenkinsIntegrationTestCompile 'org.jenkins-ci.main:jenkins-war:2.17:war-for-test@jar'
  jenkinsIntegrationTestCompile 'org.jenkins-ci.plugins:git:3.0.0:tests'
  jenkinsIntegrationTestCompile 'org.jenkins-ci.plugins.workflow:workflow-cps-global-lib:2.4'
  jenkinsIntegrationTestCompile 'org.jenkins-ci.plugins.workflow:workflow-support:1.15:tests'
  jenkinsIntegrationTestCompile 'org.jenkins-ci.plugins.workflow:workflow-job:2.6'
}

// Make sure only the Groovy dependency is available.
// Other dependencies must be used with @Grab in the defined classes due to how Jenkins Global Libraries work
project.afterEvaluate {
  final compile = it.configurations.compile.dependencies
  if (compile.size() != 1 || compile.first().name != 'groovy-all') {
    throw new GradleException('groovy-all must be the only dependency for the compile configuration')
  }
}

And an example spec (not executing yet because of missing dependencies) 

import jenkins.plugins.git.GitSampleRepoRule
import org.junit.Rule
import org.jvnet.hudson.test.RestartableJenkinsRule
import spock.lang.Specification

class ExampleIntegrationSpec extends Specification {

  @Rule
  RestartableJenkinsRule rr = new RestartableJenkinsRule()

  @Rule
  GitSampleRepoRule sampleRepo = new GitSampleRepoRule()

  def "my test"() {
    expect:
    true
  }
}

This is just based on some of the other test code I've seen in the actual Jenkins plugins.

Anybody have any luck or pointers for getting Global Library tests? What are the actual dependencies I need to declare to get this up and running?

[Michael Lasevich]

There is no reason why you cannot automate the local running of the Jenkins tests. See how plugin development works - they have setup to start a local Jenkins instance with plugin installed, and it should be relatively simple to automate that to run test jobs and check for output. 

That said, you are WAY overthinking this. A much simpler solution is to store your library in a proper SCM and use the @Library's version support to allow a job to specify the branch of the repo. Create a pre-production branch and set up a validation job to trigger on commit of code to preprod to test the pre-production version of the library. Validation job can execute various tests against the pre-prod branch (which, if you desire, can include executing of other jobs) and if everything passes, either auto-merge the commit tested to production, or signal you that it is safe to merge that commit. No additional infrastructure is required, use the same Jenkins server you are already using with this library.

The tricky part would be figuring out how to write proper validation tests.

-M

[Michael Kobit]

Using the support for @Library seems like a much simpler way to go about testing it, and was thought about as the option. We were hoping to have turnaround time locally as well as having more codified tests. We have seen the Groovy CPS plugin bite us in the ass a few times, so we wanted to get some testing infrastructure in place and were hoping we could it as close to the writer as possible.

It will definitely be easier to set up a few simple jobs or something for us to run tests against the global libraries.

Thanks for the response.