[JIRA] (JENKINS-60479) Building a matrix with exclusions in a declarative pipeline using a pipeline library finds no branches to run

49 views
Skip to first unread message

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 13, 2019, 2:35:03 PM12/13/19
to jenkinsc...@googlegroups.com
Matthew Brunton created an issue
 
Jenkins / Bug JENKINS-60479
Building a matrix with exclusions in a declarative pipeline using a pipeline library finds no branches to run
Issue Type: Bug Bug
Assignee: Andrew Bayer
Components: p4-plugin, pipeline-model-definition-plugin
Created: 2019-12-13 19:34
Environment: Docker image jenkins/jenkins:2.208 in a docker network with a Perforce instance (https://github.com/p4paul/helix-docker) using p4-plugin 1.10.7
Priority: Major Major
Reporter: Matthew Brunton

I have run into an issue when attempting to build a matrix in a declarative pipeline using a pipeline library through p4.

If the excludes section is used, the pipeline is run successfully, but it reports that there are no branches to run. Without the excludes section, all targets in the matrix are run.

 

To reproduce, I created a Perforce instance with Jenkins in a docker network and installed the p4-plugin 1.10.7. I then created a multibranch pipeline with a source of //testDepot/... from perforce.

I created two streams and a library stream.

 

In the first stream, I created a Jenkinsfile:

@Library('test-pipeline') _
lib()

Where the test-pipeline is a pipeline library at //testDepot/libstream/... with the pipeline code in //testDepot/libstream/vars/lib.groovy

def call()
{
  pipeline
  {
    agent any
    stages
    {
      stage('Build')
      {
        matrix
        {
          axes
          {
            axis
            {
              name 'PLATFORM'
              values 'Linux', 'Windows', 'OSX'
            }
            axis
            {
              name 'CONFIGURATION'
              values 'Debug', 'DebugMT', 'DebugNoPCH', 'Release', 'ReleaseMT', 'Retail', 'RetailMT'
            }
          }
          excludes
          {
            exclude
            {
              axis
              {
                name 'PLATFORM'
                notValues 'Windows'
              }
              axis
              {
                name 'CONFIGURATION'
                values 'DebugMT', 'DebugNoPCH', 'ReleaseMT', 'RetailMT'
              }
            }
          }
          stages
          {
            stage('Build')
            {
              steps
              {
                echo "Building ${PLATFORM} ${CONFIGURATION}"
              }
            }
            stage('Test')
            {
              steps
              {
                echo "Testing ${PLATFORM} ${CONFIGURATION}"
              }
            }
          }
        }
      }
    }
  }
}

When run, this pipeline is successful, but nothing runs.

[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] parallel
No branches to run
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

If the excludes section is commented out, all targets in the matrix are run.

 

In the second stream, I simply put the pipeline from the groovy into a Jenkinsfile. It runs correctly both with and without the excludes section.

 

Our declarative pipeline is rather large and complex, and we have run into the "Method code too large" issue. We were hoping that we could get around it by building with a matrix in the declarative pipeline, but have run into this issue instead.

 

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

kwirth@perforce.com (JIRA)

unread,
Dec 16, 2019, 5:04:02 AM12/16/19
to jenkinsc...@googlegroups.com

kwirth@perforce.com (JIRA)

unread,
Dec 17, 2019, 12:46:03 PM12/17/19
to jenkinsc...@googlegroups.com
Karl Wirth commented on Bug JENKINS-60479
 
Re: Building a matrix with exclusions in a declarative pipeline using a pipeline library finds no branches to run

Hi Andrew Bayer,

Thanks for reporting this. Before we start, from a pure Perforce streams point of view I'd recommend against having a library stream in the same stream path as code.

The code expects that streams are divergent forms of the same basic code. For example a stream may contain:

 

//Project1Stream/Rel1/...
//Project1Stream/Rel2/...
//Project1Stream/Main/...

//Project1Stream/Dev/...

 

which could import

//LibraryStream/main/...

Alternatively if you have a deeper stream depth set you could have the two stream trees in the same depot:

//StreamDepot/Project1/Rel1/...
//StreamDepot/Project1/Rel2/...
//StreamDepot/Project1/Main/...
//StreamDepot/Project1/Dev/...

//StreamDepot/Library/Main/...
//StreamDepot/Library/Dev/...

If you don't have this structure then there will always be outstanding changes that need to be branched between the library stream and it's parents.

Now back to the problem. I tried setting this up and keep getting compilation errors:

 

 

 

 

 

kwirth@perforce.com (JIRA)

unread,
Dec 17, 2019, 12:49:03 PM12/17/19
to jenkinsc...@googlegroups.com
Karl Wirth edited a comment on Bug JENKINS-60479
Hi [~abayer],


Thanks for reporting this. Before we start, from a pure Perforce streams point of view I'd recommend against having a library stream in the same stream path as code.

The code expects that streams are divergent forms of the same basic code. For example a stream may contain:


 
{code:java}
//Project1Stream/Rel1/...
//Project1Stream/Rel2/...
//Project1Stream/Main/...

//Project1Stream/Dev/...
{code}
 


which could import
{code:java}
//LibraryStream/main/...{code}

Alternatively if you have a deeper stream depth set you could have the two stream trees in the same depot:
{code:java}

//StreamDepot/Project1/Rel1/...
//StreamDepot/Project1/Rel2/...
//StreamDepot/Project1/Main/...
//StreamDepot/Project1/Dev/...

//StreamDepot/Library/Main/...
//StreamDepot/Library/Dev/...
{code}

If you don't have this structure then there will always be outstanding changes that need to be branched between the library stream and it's parents.

Now back to the problem. I tried setting this up and keep getting compilation errors:

 
{code:java}
  org.jenkinsci.plugins.workflow.cps.CpsCompilationErrorsException: startup failed:
/var/lib/jenkins/jobs/JENKINS-60479_Exclusions_MultiBranch/branches/main/builds/2/libs/test-pipeline/vars/lib.groovy: 8: Unknown stage section "matrix". Starting with version 0.5, steps in a stage must be in a ‘steps’ block. @ line 8, column 7.
           stage('Build')
{code}
I am on 2.176.1 so this may be a version problem. Just in case can you please send me the screenshots of the job definition to make sure I'm doing this correctly.

 

 

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 17, 2019, 1:38:05 PM12/17/19
to jenkinsc...@googlegroups.com

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 17, 2019, 1:41:02 PM12/17/19
to jenkinsc...@googlegroups.com

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 17, 2019, 1:42:02 PM12/17/19
to jenkinsc...@googlegroups.com

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 17, 2019, 1:44:03 PM12/17/19
to jenkinsc...@googlegroups.com

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 17, 2019, 1:56:03 PM12/17/19
to jenkinsc...@googlegroups.com

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 17, 2019, 1:58:03 PM12/17/19
to jenkinsc...@googlegroups.com
Matthew Brunton commented on Bug JENKINS-60479
 
Re: Building a matrix with exclusions in a declarative pipeline using a pipeline library finds no branches to run

Thanks for responding. I'll supply screenshots of the setup I'm using to reproduce. I'm also using the Pipeline: Declarative plugin 1.5, which was recently released to add matrix support to declarative pipelines.

kwirth@perforce.com (JIRA)

unread,
Dec 19, 2019, 6:03:03 AM12/19/19
to jenkinsc...@googlegroups.com

kwirth@perforce.com (JIRA)

unread,
Dec 19, 2019, 6:47:03 AM12/19/19
to jenkinsc...@googlegroups.com

kwirth@perforce.com (JIRA)

unread,
Dec 19, 2019, 6:47:03 AM12/19/19
to jenkinsc...@googlegroups.com
 
Re: Building a matrix with exclusions in a declarative pipeline using a pipeline library finds no branches to run

Hi Matthew Brunton - The problem was that I needed the latest version of the declarative pipeline plugin.

I'm easily able to reproduce the behavior and have attached the complete console outputs for with and without excludes.

I'll check where the message 'No branches to run' comes from and discuss with the developers to see if that cab tell us if the is a p4-plugin or something else problem.

PerforceTestingConsoleOutput_WithExcludes.txt

PerforceTestingConsoleOutput_WithOUT_Excludes.txt

kwirth@perforce.com (JIRA)

unread,
Dec 19, 2019, 6:47:04 AM12/19/19
to jenkinsc...@googlegroups.com

mbrunton@eidosmontreal.com (JIRA)

unread,
Dec 19, 2019, 9:11:02 AM12/19/19
to jenkinsc...@googlegroups.com
Matthew Brunton commented on Bug JENKINS-60479
 
Re: Building a matrix with exclusions in a declarative pipeline using a pipeline library finds no branches to run

Thanks for looking into this. I wasn't sure which plugin was causing the issue, it's possible that it's either the p4-plugin or something overlooked when pipeline-model-definition-plugin was updated.

kwirth@perforce.com (JIRA)

unread,
Dec 19, 2019, 10:05:04 AM12/19/19
to jenkinsc...@googlegroups.com

Hi Matthew Brunton - Here's the line of code:

https://github.com/jenkinsci/workflow-cps-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/cps/steps/ParallelStepExecution.java#L42

I have worked through this with the developers and we don't think this is a problem with our code in p4-plugin.

There is a slight chance that maybe we need to be inheriting a new  version of one of our dependencies but as it works when in a 'Jenkinsfile' or without the 'excludes' there's more chance it's problem in a library higher up such as the general SCM module.

If this does turn out to be a bug on our side please let us know.

kwirth@perforce.com (JIRA)

unread,
Dec 19, 2019, 10:49:02 AM12/19/19
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages