[JIRA] (JENKINS-39464) Pipeline with parallel and no stages does not get visualised correctly

3 views
Skip to first unread message

jdumay@cloudbees.com (JIRA)

unread,
Dec 15, 2016, 1:54:02 AM12/15/16
to jenkinsc...@googlegroups.com
James Dumay updated an issue
 
Jenkins / Bug JENKINS-39464
Pipeline with parallel and no stages does not get visualised correctly
Change By: James Dumay
Summary: pipeline Pipeline with parallel but not and no stages shows up only first branch does not get visualised correctly
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

jdumay@cloudbees.com (JIRA)

unread,
Dec 15, 2016, 2:04:01 AM12/15/16
to jenkinsc...@googlegroups.com
James Dumay commented on Bug JENKINS-39464
 
Re: Pipeline with parallel and no stages does not get visualised correctly

Sam Van Oort Vivek Pandey is there some way we could have this show up as if it was wrapped in a stage? not sure if we should do this in the frontend code, Blue Oceam or bismuth level.

jdumay@cloudbees.com (JIRA)

unread,
Dec 15, 2016, 2:05:01 AM12/15/16
to jenkinsc...@googlegroups.com
James Dumay edited a comment on Bug JENKINS-39464
[~svanoort] [~vivek] is there some way we could have this show up as if it was wrapped in a stage? not sure if we should do this in the frontend code, Blue Oceam Ocean or bismuth level.

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:25:01 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon updated an issue
 
Change By: jlpinardon
Attachment: BrokenPipelineDisplay.png

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:26:02 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon updated an issue
Change By: jlpinardon
Attachment: AwaitedDisplay.png

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:27:03 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon updated an issue
Change By: jlpinardon
Attachment: stagesNestedinParallel.xml

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:30:04 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon updated an issue
Change By: jlpinardon
Attachment: Legacy with duration error.png

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:30:05 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon commented on Bug JENKINS-39464
 
Re: Pipeline with parallel and no stages does not get visualised correctly

Dear all,

I made a trial
stagesNestedinParallel.xml
that is equivalent to the example given here. And, for sure, I felt into the same bug.
Here is what it is displayed :

I suggest that such a parallel pipeline (configure, make and test sequential stages run within several parallel "branches") could be displayed as :

Note that the "legacy" visualization is even unable to correctly display the duration of the stages :

jdumay@cloudbees.com (JIRA)

unread,
Dec 19, 2016, 6:33:01 AM12/19/16
to jenkinsc...@googlegroups.com

jlpinardon could you share the Jenkinsfile/Script please?

jdumay@cloudbees.com (JIRA)

unread,
Dec 19, 2016, 6:34:02 AM12/19/16
to jenkinsc...@googlegroups.com
James Dumay edited a comment on Bug JENKINS-39464
[~jlpinardon] could you share the Jenkinsfile/Script please? You might also want try the workaround provided in the ticket description.

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:41:01 AM12/19/16
to jenkinsc...@googlegroups.com

Here it is :

env.sleepTime=10
node('MTS_RH6_X64') {
    env.initialName='Level1'
    // Initializing level 1
    stage('Start') {
        env.stageName=env.initialName
        sh('''
                #!/bn/bash
                echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                sleep $(( 1 + $RANDOM % ${sleepTime} )) 
        ''')
    }

    parallel 'MRS_LDB':{

        node('ENODEB_MRS_LDB') {
            env.stageName='configure'
            stage ('configure') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')
            }
            env.stageName='make'
            stage ('make') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')        }
            env.stageName='test'
            stage ('test') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')
            }
        }
       
    }, 
    'MTS_LRH5':{
        node('MTS_LRH5') {
            env.stageName='configure'
            stage ('configure') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')
            }
            env.stageName='make'
            stage ('make') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')        }
            env.stageName='test'
            stage ('test') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')
            }
        }
    },
    'LB': {
        node('MTS_RH6_X64') {
            env.stageName='configure'
            stage ('configure') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')
            }
            env.stageName='make'
            stage ('make') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')        }
            env.stageName='test'
            stage ('test') {
            sh('''
                    #!/bn/bash
                    echo "Starting ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
                    sleep $(( 1 + $RANDOM % ${sleepTime} )) 
            ''')
            }
        }
    }
    env.stageName=env.initialName+'_Deploy'
    stage('Deploy') {
    sh('''
        #!/bin/bash
        sleep $(( 1 + $RANDOM % ${sleepTime} ))
        echo "This is the end for ${JOB_NAME}.${stageName} at $(date +'%s') on ${NODE_NAME}"
    ''')          
    }       
}

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:45:03 AM12/19/16
to jenkinsc...@googlegroups.com

With the workaround, the display is "correct". I mean it is exact. But I think (as far as it is feasible) that the sequential stuff within could be display. At least for the first level.
Nevertheless, the display during the build run is not correct as shown below :

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:45:03 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon updated an issue
Change By: jlpinardon
Attachment: Display During run.png

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:47:02 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon edited a comment on Bug JENKINS-39464
With the workaround, the display is indeed "correct". I mean it is exact. But I think (as far as it is feasible) that the sequential stuff within could be display displayed as in the my proposed schema . At least for the first level.
Nevertheless, the display during the build run is not correct as shown below :
!Display During run.png|thumbnail!

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 6:48:01 AM12/19/16
to jenkinsc...@googlegroups.com
jlpinardon edited a comment on Bug JENKINS-39464
With the workaround, the display is indeed "correct". I mean it is exact. But I think (as far as it is feasible) that the sequential stuff within could be displayed as in the my proposed schema. At least for the first level.


Nevertheless, the display during the build run is not correct as shown below :
!Display During run.png|thumbnail!


Parallel blocks are shown as running (which is correct) but stages within are all displayed sequentially after, and lokke like achieved (which is for sure totally wrong).

jdumay@cloudbees.com (JIRA)

unread,
Dec 19, 2016, 6:59:02 AM12/19/16
to jenkinsc...@googlegroups.com

@jlpinardon absolutely - we can't render those {{stage}}s within the parallels (see JENKINS-38442).

The reason why it doesn't show correctly after you change the structure of the Pipeline while executing is that we rely on the previous run to "preview" what the flow of the Pipeline will be. Unfortunately as Pipeline is a scripting language we can't read ahead the structure without executing it. The good news is that the new declarative syntax for Pipeline doesn't have this problem - we can read the Jenkinsfile like a config file

jdumay@cloudbees.com (JIRA)

unread,
Dec 19, 2016, 7:00:06 AM12/19/16
to jenkinsc...@googlegroups.com
James Dumay edited a comment on Bug JENKINS-39464
@jlpinardon absolutely - we can't render those {{stage}} '
s within the parallels (see JENKINS-38442).

The reason why it doesn't show correctly after you change the structure of the Pipeline while executing is that we rely on the previous run to "preview" what the flow of the Pipeline will be. Unfortunately as Pipeline is a scripting language we can't read ahead the structure without executing it. The good news is that the [new declarative syntax for Pipeline|https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/getting%20started] doesn't have this problem - we can read the Jenkinsfile like a config file :)

jdumay@cloudbees.com (JIRA)

unread,
Dec 19, 2016, 7:00:10 AM12/19/16
to jenkinsc...@googlegroups.com
James Dumay edited a comment on Bug JENKINS-39464
@jlpinardon absolutely - we can't render those {{stage}}'s within the parallels (see JENKINS-38442).

The reason why it doesn't show correctly after you change the structure of the Pipeline while executing is that we rely on the previous run to "preview" what the flow of the Pipeline will be. Unfortunately as Pipeline is a scripting language we can't read ahead the structure without executing it. The good news is that the [new declarative syntax for Pipeline|https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/getting%20started] doesn't have this problem - we can read the Jenkinsfile like a config file :)


If you run it a second time you shouldn't see the problem.

jdumay@cloudbees.com (JIRA)

unread,
Dec 19, 2016, 7:02:02 AM12/19/16
to jenkinsc...@googlegroups.com
James Dumay edited a comment on Bug JENKINS-39464
@ [~ jlpinardon ] absolutely - we can't render those {{stage}}'s within the parallels (see JENKINS-38442).


The reason why it doesn't show correctly after you change the structure of the Pipeline while executing is that we rely on the previous run to "preview" what the flow of the Pipeline will be. Unfortunately as Pipeline is a scripting language we can't read ahead the structure without executing it. The good news is that the [new declarative syntax for Pipeline|https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/getting%20started] doesn't have this problem - we can read the Jenkinsfile like a config file :)

If you run it a second time you shouldn't see the problem.

jlpinardon@java.net (JIRA)

unread,
Dec 19, 2016, 7:06:03 AM12/19/16
to jenkinsc...@googlegroups.com

Thanks for the good news I will try this as soon as possible.
But concerning the display while running,the bad news is that it is still broken after several run.

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 2:10:02 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay updated an issue
 
Change By: James Dumay
Sprint: kraken post-release

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:22:02 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay updated an issue
*Problem*
If you have a parallel declaration, but no stages, in classic stage view nothing shows up (you just have to look at log). In blue ocean, you get a single node (stage) which shows only the first branch.
Expected behavior: should show 3 parallel branches (ideally)

*Example*
{noformat}
parallel 'branch1': {
        node('node1') {
            stage('Setup') {
                checkout([details removed])
            }
            stage('Unit and Integration Tests') {
                bat '"my command to execute tests"'
            }
        }
}, 'branch2': {
        node('node2') {
            stage('Setup') {
                checkout([details removed])
            }
            stage('Unit and Integration Tests') {
                bat '"my command to execute tests"'
            }
        }
}
{noformat}

*Workaround*
Simply wrap the parallels in a {{stage}} and it will work correctly.

{code}
stage('Build and Test') {
  parallel 'branch1': {
          node('node1') {
              stage('Setup') {
                  checkout([details removed])
              }
              stage('Unit and Integration Tests') {
                  bat '"my command to execute tests"'
              }
          }
  }, 'branch2': {
          node('node2') {
              stage('Setup') {
                  checkout([details removed])
              }
              stage('Unit and Integration Tests') {
                  bat '"my command to execute tests"'
              }
          }
  }
}
{code}

It will then be visualised like:
!snap.png!

*Other examples*
{code}
node {
stage('Checkout') {
  checkout scm
}
}

parallel linux: {
node('Linux') {
  stage('Build') {
   echo 'Build linux'
  }
  stage('Tests') {
   echo 'Tests linux'
  }
  stage('Static analysis') {
   echo 'Static analysis linux'
  }
}
},
windows: {
node('Windows') {
  stage('Build') {
   echo 'Build windows'
  }
  stage('Tests') {
   echo 'Tests windows'
  }
  stage('Static analysis') {
   echo 'Static analysis windows'
  }
}
}
{code}

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:25:04 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay updated an issue
Change By: James Dumay
Attachment: image.png

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:25:05 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay updated an issue

_Parallel defined between stages but not nested in its own stage_
{code}
node {
stage('Checkout') {
  checkout scm
}
}

parallel linux: {
node('Linux') {
  stage('Build') {
   echo 'Build linux'
  }
  stage('Tests') {
   echo 'Tests linux'
  }
  stage('Static analysis') {
   echo 'Static analysis linux'
  }
}
},
windows: {
node('Windows') {
  stage('Build') {
   echo 'Build windows'
  }
  stage('Tests') {
   echo 'Tests windows'
  }
  stage('Static analysis') {
   echo 'Static analysis windows'
  }
}
}
{code}


Result looks like this:
!image.png|thumbnail!

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:26:05 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay updated an issue
Change By: James Dumay
Priority: Minor Major

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:26:05 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay updated an issue
Change By: James Dumay
Sprint: post-release tethys

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:27:03 AM1/6/17
to jenkinsc...@googlegroups.com

Sam Van Oort Vivek Pandey would it be feasible to synthesise a stage around a parallel when it is not within a stage itself? We're starting to see more and more questions about why the visualization doesn't work (obviously not an issue with Declarative)

jdumay@cloudbees.com (JIRA)

unread,
Jan 6, 2017, 6:28:01 AM1/6/17
to jenkinsc...@googlegroups.com
James Dumay edited a comment on Bug JENKINS-39464
[~svanoort] [~vivek] would it be feasible to synthesise a stage around a parallel when it is not within a stage itself? We're starting to see more and more questions about why the visualization doesn't work (obviously not an issue with Declarative) . I would like to get an idea of how feasible this is and a rough estimate.

vivek.pandey@gmail.com (JIRA)

unread,
Jan 6, 2017, 1:06:03 PM1/6/17
to jenkinsc...@googlegroups.com

James Dumay I will start with what I think is work involved or issues we need to solve and then make determination:

  • This virtual stage would collate all orphan (with no parent stage) consecutive top level parallels (with no stages in between)
  • What would be the display name or label of each virtual stage?
  • Current implementation is event based parsing API (bismuth), it sends events for all nodes. For stage it sends events that help compute stages, status, timing info etc.
  • * Computation of stage would require copying/collating status from underlying parallel
  • * Computation of timing will involve capture timing from underlying parallel blocks. This might be incorrect unless we capture all out-of-stage nodes (steps as well) and then compute timing info (duration, start and end)
  • Sythentic stage id generation. node id is generated during execution, its simple integer. I am hoping CpsFlowExecution.iota() might just do it but maybe there are edge cases where there are chances of id conflict - like pipeline is midway execution so id of synthetic stage could be reused later? Or we generate UUID for these nodes ourselves?

Sam Van Oort are there other complications it might cause or better approach to handle it?

jdumay@cloudbees.com (JIRA)

unread,
Jan 7, 2017, 1:29:01 AM1/7/17
to jenkinsc...@googlegroups.com

Vivek Pandey if we have to synthesise a stage I am happy with it being called "Parallel". Would it be possible to attach some data to the synthesised node so that we can show a warning in the UI about how it has been transformed? I'd like to educate users more about the usage of Stage.

jdumay@cloudbees.com (JIRA)

unread,
Jan 7, 2017, 3:41:01 AM1/7/17
to jenkinsc...@googlegroups.com
James Dumay assigned an issue to Vivek Pandey
 
Change By: James Dumay
Assignee: Vivek Pandey

vivek.pandey@gmail.com (JIRA)

unread,
Jan 18, 2017, 2:50:01 PM1/18/17
to jenkinsc...@googlegroups.com
Vivek Pandey started work on Bug JENKINS-39464
 
Change By: Vivek Pandey
Status: Reopened In Progress

vivek.pandey@gmail.com (JIRA)

unread,
Jan 19, 2017, 6:54:01 PM1/19/17
to jenkinsc...@googlegroups.com
Change By: Vivek Pandey
Status: In Progress Review

vivek.pandey@gmail.com (JIRA)

unread,
Jan 19, 2017, 8:46:02 PM1/19/17
to jenkinsc...@googlegroups.com
Change By: Vivek Pandey
Status: In Review Resolved
Resolution: Fixed

jdumay@cloudbees.com (JIRA)

unread,
Jan 19, 2017, 9:03:03 PM1/19/17
to jenkinsc...@googlegroups.com
 
Re: Pipeline with parallel and no stages does not get visualised correctly

This fix will be released in 1.0.0-b19 next week.

trane9991@gmail.com (JIRA)

unread,
Oct 8, 2018, 8:15:03 AM10/8/18
to jenkinsc...@googlegroups.com

Vivek Pandey
I'm facing the same problem. Any progress here?

See my complete issue described here: JENKINS-39464
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

rishirt.us@gmail.com (JIRA)

unread,
May 17, 2019, 12:00:09 PM5/17/19
to jenkinsc...@googlegroups.com

Why does the resolution to this issue say "Fixed" but the issue itself is still "In Porgress"?
Reply all
Reply to author
Forward
0 new messages