BUILD FLOW parallel closure simple example

[pjl8...@gmail.com]

I need to run a number of parameterized jobs in a parallel fashion, and decided to try a simple example I recall seeing.

///////////////////// Build Flow TestHarness /////////////////////////////////

// construct and collect closures for LATER execution

buildClosures = []

for (int i=1; i<50; i++) {

    def curClosure = {

       build("TestParameterized", Parameter: i)

    }

    buildClosures.add(curClosure)

}

// execute the closures in buildClosures in parallel

parallel(buildClosures)

/////////////////////////////////////////////////////////////////////////////

TestParameterized contains the simple groovy script below:

///////////////////////////////// SCRIPT ///////////////////////////////////////

import hudson.model.*

def parameter = build.buildVariableResolver.resolve("Parameter")    

println "parameter:"+ parameter  

def long recursiveFib(long n) {

    if (n == 0) {

        return 0

    } else if (n == 1) {

        return 1

    }

    return recursiveFib(n - 1) + recursiveFib(n - 2)

}

def result = recursiveFib(33)  

println "recursiveFib:"+result

/////////////////////////////////////////////////////////////////////////////////

When I run this the console output in build flow testharness task is

parallel {
    Schedule job TestParameterized
    Schedule job TestParameterized
    Schedule job TestParameterized
    .

    .
    Build TestParameterized #12 started
    Build TestParameterized #12 started
    Build TestParameterized #12 started
    .

    .
    TestParameterized #12 completed 
    TestParameterized #12 completed 
    TestParameterized #12 completed 
    .

    .
    Build TestParameterized #13 started
    Build TestParameterized #13 started
    Build TestParameterized #13 started
    .

    .
    TestParameterized #13 completed 
    TestParameterized #13 completed 
    TestParameterized #13 completed 
    .

    .
}
Finished: SUCCESS

Which shows just two TestParameterized task being created (multiple times),

and reviewing the console for these two tasks #12, #13 the parameter passed in is always 50.

What I was expecting to see was 50 TestParameterized jobs with increasing parameter value 1 to 50.

What am I missing?

Thanks

  

[Marc MacIntyre]

I assume that you only end up defining one curClosure variable, since it never goes out of scope, and subsequent changes to it update the one version you've added to your buildClosures list.

Maybe you want buildClosures.add(curClosure.clone()) 

[pjl8...@gmail.com]

Thanks for the suggestion.. .but same result

[Stuart Rowe]

As Marc pointed out, the variable "i" isn't private to each closure because it's defined outside of the scope.

One solution would be to copy the value of i to a variable defined within the scope of the closure:

for (int i=1; i<50; i++) {

    def curClosure = {

        def j = i

        build("TestParameterized", Parameter: j)

    }

    buildClosures.add(curClosure)

}

Because each of your schedule jobs for TestParamaterized had the same parameters, the Jenkins scheduler actually merged the tasks to reduce redundancies.

[Peter Lenson]

Stuart I still get the same result using your suggestion...below is exactly what I did and my results:

I created a build flow job called TestJobDSL containing the following script in the "Define build flow using flow DSL" section of Build Triggers:

// construct and collect closures for LATER execution

buildClosures = []

for (int i=1; i<50; i++) {

    def curClosure = {

        def j = i

        build("TestParameterized", Parameter: j)

    }

    buildClosures.add(curClosure)

}

// execute the closures in buildClosures in parallel

parallel(buildClosures)

TestParameterized was created as  free-style job with one parameter.

In the build section, I have a "Windows Execute Batch  Command" s containing the following:

echo %Parameter%

When I run TestJobDSL this is what I see in the console:

Started by user XXXXXX
[EnvInject] - Loading node environment variables.
Building on master in workspace C:\Users\XXXXXX\.jenkins\jobs\TestJobDSL\workspace
parallel {
    Schedule job TestParameterized
    Schedule job TestParameterized

... (truncated for brevity)

    Schedule job TestParameterized
    Schedule job TestParameterized

 
    Build TestParameterized #6 started
    Schedule job TestParameterized
    Schedule job TestParameterized
    Build TestParameterized #6 started
    Schedule job TestParameterized
    Build TestParameterized #6 started
    Schedule job TestParameterized
    Build TestParameterized #6 started
    Build TestParameterized #6 started
    Build TestParameterized #6 started
    Build TestParameterized #6 started
    Schedule job TestParameterized
    Build TestParameterized #6 started
    Schedule job TestParameterized
    ... (truncated for brevity)  
    Schedule job TestParameterized

 ... truncated for brevity

    Schedule job TestParameterized
    Build TestParameterized #6 started
    Build TestParameterized #6 started
    Schedule job TestParameterized
    Schedule job TestParameterized
    Build TestParameterized #6 started
    Build TestParameterized #6 started
    Schedule job TestParameterized
    Schedule job TestParameterized

... (truncated for brevity)

    Schedule job TestParameterized
    Schedule job TestParameterized
    Schedule job TestParameterized

    Build TestParameterized #6 started
    Build TestParameterized #6 started
    Build TestParameterized #6 started
    TestParameterized #6 completed 
    TestParameterized #6 completed 
    TestParameterized #6 completed 
    TestParameterized #6 completed 
    TestParameterized #6 completed 

... (truncated for brevity)
    TestParameterized #6 completed 
    TestParameterized #6 completed 
    TestParameterized #6 completed 
    TestParameterized #6 completed 
    Build TestParameterized #7 started
    Build TestParameterized #7 started

... (truncated for brevity)

    Build TestParameterized #7 started
    Build TestParameterized #7 started
    TestParameterized #7 completed 

... (truncated for brevity)
    TestParameterized #7 completed 
}
Finished: SUCCESS

TestParameterized contains two runs 6,7 each containing the following console output:

Started by build flow TestJobDSL#114
Started by build flow TestJobDSL#114
... (truncated for brevity)
Started by build flow TestJobDSL#114
Started by build flow TestJobDSL#114
[EnvInject] - Loading node environment variables.
Building remotely on Slave1 in workspace c:\Slave1RemoteFS\workspace\TestParameterized
[TestParameterized] $ cmd /c call C:\Users\XXXXXX\AppData\Local\Temp\hudson2441697324175984667.bat

c:\Slave1RemoteFS\workspace\TestParameterized>echo 50 
50

c:\Slave1RemoteFS\workspace\TestParameterized>exit 0 
Finished: SUCCESS

As you can see only the terminal loop value "50" is used. 

What do you get when you run the examples above?

Thanks

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/2swVIxjzdZc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Stuart Rowe]

My mistake, the "def j = i" should be outside of the closure:

// construct and collect closures for LATER execution

buildClosures = []

for (int i = 1; i<50; i++) {

   def j = i 

   def curClosure = {

        build("TestParameterized", "Parameter" : j)

    }

    buildClosures.add(curClosure)

}

// execute the closures in buildClosures in parallel

parallel(buildClosures)

I  ran this locally and got the expected results:

arallel {
    Schedule job TestParameterized

    Schedule job TestParameterized
    Schedule job TestParameterized

    (truncated for brevity...)

    Schedule job TestParameterized
    Schedule job TestParameterized

    Build TestParameterized #107 started
    TestParameterized #107 completed 
    Build TestParameterized #108 started
    TestParameterized #108 completed 
    Build TestParameterized #109 started
    (truncated for brevity...)
    TestParameterized #154 completed 
    Build TestParameterized #155 started
    TestParameterized #155 completed 
}
Finished: SUCCESS

[pjl8...@gmail.com]

THANKS! 

Works like a charm!