[JIRA] (JENKINS-55426) Using "for in" loop for generating tasks for "parallel" execution, causes all tasks to have last value from iterated collection

9 views
Skip to first unread message

alpanshin@gmail.com (JIRA)

unread,
Jan 7, 2019, 9:56:02 AM1/7/19
to jenkinsc...@googlegroups.com
Alexandr Panshin created an issue
 
Jenkins / Bug JENKINS-55426
Using "for in" loop for generating tasks for "parallel" execution, causes all tasks to have last value from iterated collection
Issue Type: Bug Bug
Assignee: vjuranek
Attachments: screen1.jpg
Components: groovy-plugin
Created: 2019-01-07 14:55
Environment: Jenkins 2.150.1, Pipeline-Groovy Plugin 2.57
Labels: groovy parallel
Priority: Minor Minor
Reporter: Alexandr Panshin

When I try to generate tasks for parallel execution using the following code:

import jenkins.model.Jenkins

pipeline {
    agent none
    stages {
        stage('Preparing machines') {
            steps {
                script {
                    def machines = ['agent-windows0', 'agent-windows1', 'agent-redhat0', 'agent-redhat1']
                    def machinePreparations = [:]

                    for (machine in machines) {
                        def labelParameters = []
                        labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [machine], nodeEligibility: [$class: 'AllNodeEligibility']])
                        labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])

                        machinePreparations[machine] = {
                            stage(machine + ' preparation') {
                                build job: '../Preparation/' + machine, parameters: labelParameters

                                def node = Jenkins.get().nodes.find({it.name == machine})
                                node.setLabelString(node.getLabelString() + 'successful')
                                println(String.format("Job with machine %s successfully done!", machine))
                            }
                        }
                    }

                    parallel machinePreparations
                }
            }
        }
    }
}

I have all the tasks receiving last value from collection ("agent-redhat1") - see screen attached.
This can be partially fixed by saving "machine" variable into another intermediate variable and using this intermediate variable instead:

import jenkins.model.Jenkins

pipeline {
    agent none
    stages {
        stage('Preparing machines') {
            steps {
                script {
                    def machines = ['agent-windows0', 'agent-windows1', 'agent-redhat0', 'agent-redhat1']
                    def machinePreparations = [:]

                    for (machine in machines) {
                        def agentName = machine
                        def labelParameters = []
                        labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [agentName], nodeEligibility: [$class: 'AllNodeEligibility']])
                        labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])

                        machinePreparations[agentName] = {
                            stage(agentName + ' preparation') {
                                build job: '../Preparation/' + agentName, parameters: labelParameters

                                def node = Jenkins.get().nodes.find({it.name == agentName})
                                node.setLabelString(node.getLabelString() + 'successful')
                                println(String.format("Job with machine %s successfully done!", agentName))
                            }
                        }
                    }

                    parallel machinePreparations
                }
            }
        }
    }
}

However, this also not fixing problem entirely, see JENKINS-55425
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

alpanshin@gmail.com (JIRA)

unread,
Jan 7, 2019, 9:57:02 AM1/7/19
to jenkinsc...@googlegroups.com
Alexandr Panshin updated an issue
Change By: Alexandr Panshin
When I try to generate tasks for parallel execution using the following code:
{code:java}

import jenkins.model.Jenkins

pipeline {
    agent none
    stages {
        stage('Preparing machines') {
            steps {
                script {
                    def machines = ['agent-windows0', 'agent-windows1', 'agent-redhat0', 'agent-redhat1']
                    def machinePreparations = [:]

                    for (machine in machines) {
                     def labelParameters = []
                     labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [machine], nodeEligibility: [$class: 'AllNodeEligibility']])
                     labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])

                     machinePreparations[machine] = {
                     stage(machine + ' preparation') {
                     build job: '../Preparation/' + machine, parameters: labelParameters

                     def node = Jenkins.get().nodes.find({it.name == machine})
                     node.setLabelString(node.getLabelString() + 'successful')
                     println(String.format("Job with machine %s successfully done!", machine))
                     }
                     }
                    }

                    parallel machinePreparations
                }
            }
        }
    }
}
{code}

I have all the tasks receiving last value from collection ("agent-redhat1") - see screen attached.
This can be partially fixed by saving "machine" variable into another intermediate variable and using this intermediate variable instead:

{code:java}

import jenkins.model.Jenkins

pipeline {
    agent none
    stages {
        stage('Preparing machines') {
            steps {
                script {
                    def machines = ['agent-windows0', 'agent-windows1', 'agent-redhat0', 'agent-redhat1']
                    def machinePreparations = [:]

                    for (machine in machines) {
                     def agentName = machine
                     def labelParameters = []
                     labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [agentName], nodeEligibility: [$class: 'AllNodeEligibility']])
                     labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])

                     machinePreparations[agentName] = {
                     stage(agentName + ' preparation') {
                     build job: '../Preparation/' + agentName, parameters: labelParameters

                     def node = Jenkins.get().nodes.find({it.name == agentName})
                     node.setLabelString(node.getLabelString() + 'successful')
                     println(String.format("Job with machine %s successfully done!", agentName))
                     }
                     }
                    }

                    parallel machinePreparations
                }
            }
        }
    }
}
{code}

However, this also not fixing problem entirely, see [ # JENKINS-55425 |https://issues.jenkins-ci.org/browse/JENKINS-55425 ]

stuartr@ea.com (JIRA)

unread,
Jan 22, 2020, 7:34:02 PM1/22/20
to jenkinsc...@googlegroups.com
Stuart Rowe commented on Bug JENKINS-55426
 
Re: Using "for in" loop for generating tasks for "parallel" execution, causes all tasks to have last value from iterated collection

Avoid using "for-in" loops. Instead try:

machines.each { machine ->
    def agentName = machine
    def labelParameters = []
    labelParameters.add([$class: 'NodeParameterValue', name: 'node_name', labels: [agentName], nodeEligibility: [$class: 'AllNodeEligibility']])
    labelParameters.add([$class: 'StringParameterValue', name: 'ARBITRARY_PARAMETER', value: 'ARBITRARY_VALUE'])

    machinePreparations[agentName] = {
        stage(agentName + ' preparation') {
            build job: '../Preparation/' + agentName, parameters: labelParameters

            def node = Jenkins.get().nodes.find({it.name == agentName})
            node.setLabelString(node.getLabelString() + 'successful')
            println(String.format("Job with machine %s successfully done!", agentName))
        }
    }
}

 

 

 
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

zolen@zolen.org (JIRA)

unread,
Apr 20, 2020, 7:02:04 AM4/20/20
to jenkinsc...@googlegroups.com
Sergei S updated an issue
 
Change By: Sergei S
Attachment: image-2020-04-20-14-01-42-072.png
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

zolen@zolen.org (JIRA)

unread,
Apr 20, 2020, 7:03:03 AM4/20/20
to jenkinsc...@googlegroups.com
Sergei S commented on Bug JENKINS-55426
 
Re: Using "for in" loop for generating tasks for "parallel" execution, causes all tasks to have last value from iterated collection

I catched similar bug with ".each" loop inside parallel stages.

 

def deploymentStages = [:]
configGlobal = ["stage-a":["childMap":["key1":"val-1-stage-A", "key2":"val-2-stage-A", "childList":["child-list-value-stage-A"]]], 
                "stage-b":["childMap":["key1":"val-1-stage-B", "key2":"val-2-stage-B", "childList":["child-list-value-stage-B"]]]]

    pipeline {
        agent any

        stages {
            stage('Parent') {
                steps {
                    script {
                        configGlobal.each { item -> 
                            deploymentStages["${item.key}"] = {
                                stage("${item.key}") {
                                    config = item.value.childMap
                                    newList = [config.key1, config.key2]
                                    echo "before loop"
                                    echo newList.toString()
                                    config.childList.each                { value ->      
                                        newList.add(value)
                                    }
                                    echo "after loop"
                                    echo newList.toString()
                                }
                            }
                        }
                        parallel deploymentStages
                    }
                }
            }
        }
        
    }

 Output:

zolen@zolen.org (JIRA)

unread,
Apr 20, 2020, 7:10:03 AM4/20/20
to jenkinsc...@googlegroups.com
Sergei S edited a comment on Bug JENKINS-55426
I catched similar bug with ".each" loop inside parallel stages.
Jenkins 2.222, pipeline groovy plugin 2.80  
{code:java}

def deploymentStages = [:]
configGlobal = ["stage-a":["childMap":["key1":"val-1-stage-A", "key2":"val-2-stage-A", "childList":["child-list-value-stage-A"]]],
                "stage-b":["childMap":["key1":"val-1-stage-B", "key2":"val-2-stage-B", "childList":["child-list-value-stage-B"]]]]

    pipeline {
        agent any

        stages {
            stage('Parent') {
                steps {
                    script {
                     configGlobal.each { item ->
                     deploymentStages["${item.key}"] = {
                     stage("${item.key}") {
                     config = item.value.childMap
                     newList = [config.key1, config.key2]
                     echo "before loop"
                     echo newList.toString()
                     config.childList.each                { value ->      
                     newList.add(value)
                     }
                     echo "after loop"
                     echo newList.toString()
                     }
                     }
                     }
                     parallel deploymentStages
                    }
                }
            }
        }
        
    }
{code}
 Output:
!image-2020-04-20-14-01-42-072.png!

zolen@zolen.org (JIRA)

unread,
Apr 20, 2020, 7:22:02 AM4/20/20
to jenkinsc...@googlegroups.com
Sergei S edited a comment on Bug JENKINS-55426
I catched similar bug with ".each" loop inside parallel stages.

Jenkins 2.222 (tested on 2.232 too) , pipeline groovy plugin 2.80 
Reply all
Reply to author
Forward
0 new messages