Jenkins job dsl stash api - FATAL: Expecting Ant GLOB pattern, but saw

[Francisca Munhoz]

Hi guys,

I'm new to job dsl plugin but I can see it is very useful. I found this page http://www.rightmove.co.uk/dev/blog/scaling-jenkins-job-dsl-stash-api/ that has a pretty cool example in how to call stash and iterates between projects and repos. So I've decided to give it a go, but I believe something is missing from my settings or I just don't know how to setup. I'm using a Jenkins docker image from https://hub.docker.com/_/jenkins/ jenkins 2.0

home jenkins: /var/jenkins_home/

jobs: /var/jenkins_home/jobs

/var/jenkins_home/jobs:

                                      jenkins-job-dsl.groovy 

                                      seed                    

                                      seed.groovy

This is the error message:

Building in workspace /var/jenkins_home/workspace/seed

FATAL: Expecting Ant GLOB pattern, but saw '/var/jenkins_home/jobs/jenkins-job-dsl.groovy'. See http://ant.apache.org/manual/Types/fileset.html for syntax

java.io.IOException: Expecting Ant GLOB pattern, but saw '/var/jenkins_home/jobs/jenkins-job-dsl.groovy'. See http://ant.apache.org/manual/Types/fileset.html for syntax

at hudson.FilePath.glob(FilePath.java:1738)

at hudson.FilePath.access$1700(FilePath.java:190)

at hudson.FilePath$32.invoke(FilePath.java:1719)

at hudson.FilePath$32.invoke(FilePath.java:1716)

at hudson.FilePath.act(FilePath.java:990)

at hudson.FilePath.act(FilePath.java:968)

at hudson.FilePath.list(FilePath.java:1716)

at hudson.FilePath.list(FilePath.java:1701)

at hudson.FilePath.list(FilePath.java:1687)

at hudson.FilePath$list$2.call(Unknown Source)

at javaposse.jobdsl.plugin.ScriptRequestGenerator$_getScriptRequests_closure2.doCall(ScriptRequestGenerator.groovy:44)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)

at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)

at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)

at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1021)

at groovy.lang.Closure.call(Closure.java:426)

at groovy.lang.Closure.call(Closure.java:442)

at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)

at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1890)

at org.codehaus.groovy.runtime.dgm$159.invoke(Unknown Source)

at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)

at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)

at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)

at javaposse.jobdsl.plugin.ScriptRequestGenerator.getScriptRequests(ScriptRequestGenerator.groovy:43)

at javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:210)

at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)

at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)

at hudson.model.Build$BuildExecution.build(Build.java:205)

at hudson.model.Build$BuildExecution.doRun(Build.java:162)

at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)

at hudson.model.Run.execute(Run.java:1738)

at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)

at hudson.model.ResourceController.execute(ResourceController.java:98)

at hudson.model.Executor.run(Executor.java:410)

Finished: FAILURE

Here it is my seed configure page:

[Daniel Spilker]

You can't use absolute paths, the path must be relative to the seed job's workspace. You need to put the scripts in /var/jenkins_home/workspace/seed/jobs/jenkins-job-dsl.groovy and then use jobs/jenkins-job-dsl.groovy to locate the scripts in the workspace.

Daniel

--
You received this message because you are subscribed to the Google Groups "job-dsl-plugin" group.
To unsubscribe from this group and stop receiving emails from it, send an email to job-dsl-plugi...@googlegroups.com.
To post to this group, send email to job-dsl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/job-dsl-plugin/fa335390-e91f-4d26-b43e-32dc4103b3b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Francisca Munhoz]

Thanks Daniel for your help!

I have another question, I have setup in credentials my user and password to access stash and also another credential with the private key, on the groovy script I'm passing my credential id (stash-user) for the stash user and password, however I'm getting an authentication error. Do I need to extend it and pass my credentials in any other way?

Error message:

Building in workspace /var/jenkins_home/workspace/seed
Processing DSL script seed.groovy
INFO - Searching for repos in project TEST
FATAL: Server returned HTTP response code: 401 for URL: https://stash.xxxxx.au/rest/api/1.0/projects/TEST/repos
java.io.IOException: Server returned HTTP response code: 401 for URL: https://stash.xxxxx.au/rest/api/1.0/projects/TEST/repos

Here are the groovy scripts:

seed.groovy:

import groovy.json.JsonSlurper

def jenkinsGitCredential = 'stash-user'
def jenkinsJobDslFileName = 'jenkins_job_dsl.groovy'
def stashProjectKey = 'TEST'

if (stashProjectKey) {
    println "INFO - Searching for repos in project $stashProjectKey"
    def reposApi = new URL("https://stash.xxxxx.au/rest/api/1.0/projects/${stashProjectKey}/repos")
    def repos = new JsonSlurper().parse(reposApi.newReader())

    if (repos) {
        def repoList = repos.get("values")

        println "INFO - Found ${repoList.size} repos for project ${stashProjectKey}."

        repoList.each { repo ->
            processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName)
        }
    }
}


private void processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName) {
    def repoSlug = repo.get("slug")

    if (repoSlug) {

        println "INFO - Searching for branches in repo $repoSlug"

        def branchApi = new URL("https://stash.xxxxx.au/rest/api/1.0/projects/${stashProjectKey}/repos/${repoSlug}/branches")
        def branches = new JsonSlurper().parse(branchApi.newReader())

        branches.get("values").each { branch ->
            processBranch(jenkinsGitCredential, branch, stashProjectKey, repoSlug, jenkinsJobDslFileName)
        }
    }
}

private void processBranch(jenkinsGitCredential, branch, stashProjectKey, repoSlug, jenkinsJobDslFileName) {
    def branchName = branch.get("displayId")
    def branchSimpleName = branchName.replace("/", "-")
    def jobDslUrl = new URL("https://stash.xxxxx.au/rest/api/1.0/projects/${stashProjectKey}/repos/${repoSlug}/browse/${jenkinsJobDslFileName}?raw&at=${branchName}")

    println "INFO - Found branch ${repoSlug} - ${branchName}"
    println "INFO - Looking for job dsl ${jobDslUrl}"

    def urlConnection = (HttpURLConnection) jobDslUrl.openConnection()
    def status = urlConnection.getResponseCode()
    def repoAndBranchName = "${repoSlug} - ${branchName}"

    if (status == 200) {
        println "INFO - Found job dsl for " + repoAndBranchName

        def dslJson = new JsonSlurper().parse(jobDslUrl.newReader())

        if (dslJson.get("lines") != null) {
            try {
                println "INFO - Executing job dsl for $repoAndBranchName"

                Binding binding = getJobDslBinding(
                        stashProjectKey,
                        repoSlug,
                        branchSimpleName,
                        branchName,
                        jenkinsGitCredential)

                executeJobDsl(binding, dslJson)
            } catch (Exception e) {
                println "ERROR - Couldn't run job dsl for ${repoAndBranchName}: ${e.getMessage()} \n ${e.printStackTrace()}"
            }
        }

    } else {
        println "WARN - No jenkins job dsl found for $repoAndBranchName"
    }
}

private void executeJobDsl(Binding binding, dslJson) {
    def jobDsl = [];

    dslJson.get("lines").each {
        jobDsl.add(it.text)
    }

    def jobDslString = jobDsl.join("\n")

    GroovyShell shell = new GroovyShell(binding);
    Object value = shell.parse(jobDslString);

    job value.run()
}

private Binding getJobDslBinding(stashProjectKey, repoSlug, branchSimpleName, branchName, jenkinsGitCredential) {
    Binding binding = new Binding();

    binding.setVariable('stashProjectKey', stashProjectKey)
    binding.setVariable('projectName', repoSlug)
    binding.setVariable('branchSimpleName', branchSimpleName)
    binding.setVariable('branchName', branchName)
    binding.setVariable('jenkinsGitCredential', jenkinsGitCredential)

    binding
}

----------------------------------

jenkins_job_dsl.groovy :

---------------------------------

Closure job = {

    // Job Name

    name "${stashProjectKey}-${projectName}-${branchSimpleName}"

    println "INFO - Searching for repos in project $stashProjectKey"

    // Where should jenkins run the job

    label ('master')

    // Where should Jenkins get the source code from

    scm {

        git {

            remote {

                url ("https://stash.xxxxx.au/scm/${stashProjectKey}/${projectName}.git")

                branch (branchName)

                credentials (jenkinsGitCredential)

            }

        }

    }

    // How often should the job run

    triggers {

        scm ('H/10 * * * *')

    }

    // Gradle build steps to execute

    steps {

        def gradleTask = '<GRADLE BUILD STEPS>'

        gradle (gradleTask, null, true) {

            def makeExecutable = it / 'makeExecutable'

            makeExecutable.setValue (true)

        }

    }

    // Additional Report Settings

    publishers {

        jacocoCodeCoverage  {

            minimumLineCoverage '60'

            maximumLineCoverage '90'

            execPattern '**/build/**/*.exec'

        }

        archiveJunit ("**/build/test-results/*.xml", true, true)

    }

}

return job

---------------------------------

[Francisca Munhoz]

I have also tried passing the user and password on the api url but it did not work 

def reposApi = new URL("https://francisca:password@stash.xxxxx.au/rest/api/1.0/projects/${stashProjectKey}/repos")

I'm new to Groovy and job DSL, so I do apologise for my basic questions and I do appreciate your help!

Error:

Processing DSL script seed.groovy

INFO - Searching for repos in project TEST

FATAL: Server returned HTTP response code: 401 for URL: https://francisca:pass...@stash.xxxxx.au/rest/api/1.0/projects/TEST/repos

java.io.IOException: Server returned HTTP response code: 401 for URL: https://francisca:pass...@stash.xxxxx.au/rest/api/1.0/projects/TEST/repos

    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1840)

    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)

    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)

    at org.codehaus.groovy.runtime.ResourceGroovyMethods.configuredInputStream(ResourceGroovyMethods.java:2021)

    at org.codehaus.groovy.runtime.ResourceGroovyMethods.newReader(ResourceGroovyMethods.java:2070)

    at org.codehaus.groovy.runtime.dgm$983.invoke(Unknown Source)

    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)

    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)

    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)

    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)

    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)

    at seed.run(seed.groovy:10)

    at seed$run.call(Unknown Source)

    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)

    at seed$run.call(Unknown Source)

​

--
You received this message because you are subscribed to a topic in the Google Groups "job-dsl-plugin" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/job-dsl-plugin/1ybmBMm96-A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to job-dsl-plugi...@googlegroups.com.

To post to this group, send email to job-dsl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/job-dsl-plugin/b2100522-7d97-4d2e-bdd3-c669e1f8afaf%40googlegroups.com.

[Azul Inho]

Try this:



import groovy.json.JsonSlurper
import java.util.regex.*

jenkinsGitCredential = 'username:password'
jenkinsJobDslFileName = 'jenkins-job-dsl.groovy'
stashUrl = 'https://bitbucket.stash.com'
matchBranchesRegex = '.*'
matchProjectsRegex = 'mySTASH_PROJECT'
matchReposRegex = '.*'


def projectsApi = new URL("${stashUrl}/rest/api/1.0/projects?limit=1000")
def projects = new JsonSlurper().parse(projectsApi.newReader())

projects.get("values").each {
    def stashProjectKey = it.get("key")

    if (stashProjectKey) {
        println "Found project: ${stashProjectKey}"
        if ("${stashProjectKey}" ==~ "${matchProjectsRegex}") {

                println "INFO - Searching for repos in project $stashProjectKey"

                def reposApi = new URL("${stashUrl}/rest/api/1.0/projects/${stashProjectKey}/repos?limit=1000")

                def repos = new JsonSlurper().parse(reposApi.newReader())

                if (repos) {
                    def repoList = repos.get("values")

                    println "INFO - Found ${repoList.size} repos for project ${stashProjectKey}."

                    repoList.each { repo ->

                        if (repo.get("slug") ==~ "${matchReposRegex}") {
                            processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName, stashUrl)
                        }
                    }
                }
        }
    }
}

private void processRepo(repo, stashProjectKey, jenkinsGitCredential, jenkinsJobDslFileName, stashUrl) {

    def repoSlug = repo.get("slug")

    if (repoSlug) {

        println "INFO - Searching for branches in repo $repoSlug"

        def branchApi = new URL("${stashUrl}/rest/api/1.0/projects/${stashProjectKey}/repos/${repoSlug}/branches?limit=10000")

        def branches = new JsonSlurper().parse(branchApi.newReader())

        branches.get("values").each { branch ->

            if (branch.get("displayId") ==~ "${matchBranchesRegex}") {
                processBranch(jenkinsGitCredential, branch, stashProjectKey, repoSlug, jenkinsJobDslFileName, stashUrl)
            }
        }
    }
}

private void processBranch(jenkinsGitCredential, branch, stashProjectKey, repoSlug, jenkinsJobDslFileName, stashUrl) {

    def branchName = branch.get("displayId")
    def branchSimpleName = branchName.replace("/", "-")

    def jobDslUrl = new URL("${stashUrl}/rest/api/1.0/projects/${stashProjectKey}/repos/${repoSlug}/browse/${jenkinsJobDslFileName}?raw&at=${branchName}")

[Francisca Munhoz]

Thanks a lot!

--
You received this message because you are subscribed to a topic in the Google Groups "job-dsl-plugin" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/job-dsl-plugin/1ybmBMm96-A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to job-dsl-plugi...@googlegroups.com.
To post to this group, send email to job-dsl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/job-dsl-plugin/b3b92530-bcce-4c2d-aca3-2fd1c5b22259%40googlegroups.com.