[JIRA] [workflow-plugin] (JENKINS-28447) Workflow plugin does not resolve variables in SCM plugin

51 views
Skip to first unread message

nicknewland@outlook.com (JIRA)

unread,
May 17, 2015, 11:26:03 PM5/17/15
to jenkinsc...@googlegroups.com
Nicholas Newland created an issue
 
Jenkins / Bug JENKINS-28447
Workflow plugin does not resolve variables in SCM plugin
Issue Type: Bug Bug
Assignee: Jesse Glick
Components: workflow-plugin
Created: 18/May/15 3:25 AM
Environment: Jenins 1.613, Workflow plugin 1.6
Priority: Major Major
Reporter: Nicholas Newland

When selecting the "Groovy CPS DSL from SCM" option for a worflow job, the SCM plugins do not appear to resolve build parameters or environment variables. I am using the git plugin and when I use it from other jobs I can specify a build parameter, like "BuildBranch", and use that when specifying what branch should be built:

Branches to build: */$

{BuildBranch}

This does not work when I use the Groovy CPS DSL from SCM.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

jglick@cloudbees.com (JIRA)

unread,
May 27, 2015, 2:44:01 PM5/27/15
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
CpsScmFlowDefinition does not resolve variables
Change By: Jesse Glick
Summary: Workflow plugin CpsScmFlowDefinition  does not resolve variables  in SCM plugin

jglick@cloudbees.com (JIRA)

unread,
May 27, 2015, 2:45:01 PM5/27/15
to jenkinsc...@googlegroups.com
Jesse Glick commented on Bug JENKINS-28447
 
Re: CpsScmFlowDefinition does not resolve variables

Not quite sure I understand how to reproduce here, but bear in mind that CpsScmFlowDefinition is merely a convenience. For more control you can use CpsFlowDefinition (inline scipt) and just specify something like

node {
  checkout(…) // anything supported by the SCM
  load 'flow.groovy' // run everything else
}

nicknewland@outlook.com (JIRA)

unread,
May 27, 2015, 4:39:01 PM5/27/15
to jenkinsc...@googlegroups.com

That's what I did. I simply used the CpsFlowDefinition. Maybe my expectations where wrong, but this is what I was expecting to be able to do.

Freestyle project:
1. Define new job
2. Check "This build is parameterized"
3. Create string parameter with name "BuildBranch" and default value "master"
4. Under Source code management select "Git"
5. in "Branches to build" enter "*/$

{BuildBranch}"
6. when the build runs with the default value it checks out the master branch of my git repo and runs the build
7. If a developer runs the build and enters "feature/awesome" for the BuildBranch parameter, the feature/awesome branch is checked out from git and the build runs.

When I use the CpsScmFlowDefinition, I can do all of the above and select Git as the SCM and fill in "*/${BuildBranch}

in the "Branches to build" text box, but git always complains that there is no branch named "*/$

{BuildBranch}

". It doesn't seem to be resolving the build parameter.

nicknewland@outlook.com (JIRA)

unread,
May 27, 2015, 4:54:01 PM5/27/15
to jenkinsc...@googlegroups.com
Nicholas Newland edited a comment on Bug JENKINS-28447

nicknewland@outlook.com (JIRA)

unread,
May 27, 2015, 4:55:02 PM5/27/15
to jenkinsc...@googlegroups.com
Nicholas Newland edited a comment on Bug JENKINS-28447
That's what I did.  I simply used the CpsFlowDefinition.  Maybe my expectations where wrong, but this is what I was expecting to be able to do.

Freestyle project:
1. Define new job
2. Check "This build is parameterized"
3. Create string parameter with name "BuildBranch" and default value "master"
4. Under Source code management select "Git"
5. in "Branches to build" enter "*/$BuildBranch"

6. when the build runs with the default value it checks out the master branch of my git repo and runs the build
7. If a developer runs the build and enters "feature/awesome" for the BuildBranch parameter, the feature/awesome branch is checked out from git and the build runs.

When I use the CpsScmFlowDefinition, I can do all of the above and select Git as the SCM and fill in "*/$
{ BuildBranch }  in the "Branches to build" text box, but git always complains that there is no branch named "*/$ { BuildBranch } ".  It doesn't seem to be resolving the build parameter.

nicknewland@outlook.com (JIRA)

unread,
May 27, 2015, 4:55:03 PM5/27/15
to jenkinsc...@googlegroups.com
Nicholas Newland edited a comment on Bug JENKINS-28447
That's what I did.  I simply used the CpsFlowDefinition.  Maybe my expectations where wrong, but this is what I was expecting to be able to do.

Freestyle project:
1. Define new job
2. Check "This build is parameterized"
3. Create string parameter with name "BuildBranch" and default value "master"
4. Under Source code management select "Git"
5. in "Branches to build" enter "*/$BuildBranch"
6. when the build runs with the default value it checks out the master branch of my git repo and runs the build
7. If a developer runs the build and enters "feature/awesome" for the BuildBranch parameter, the feature/awesome branch is checked out from git and the build runs.

When I use the CpsScmFlowDefinition, I can do all of the above and select Git as the SCM and fill in "*/$BuildBranch in the "Branches to build" text box, but git always complains that there is no branch named "*/$BuildBranch".  It doesn't seem to be resolving the build parameter.

nicknewland@outlook.com (JIRA)

unread,
May 27, 2015, 4:56:01 PM5/27/15
to jenkinsc...@googlegroups.com

mitya.kononchuk@mendix.com (JIRA)

unread,
May 28, 2015, 7:05:01 AM5/28/15
to jenkinsc...@googlegroups.com

The workaround @Jesse Glick proposes has one disadvantage: you have to allocate an extra node for the build, while using CpsScmFlowDefinition you don't.

mitya.kononchuk@mendix.com (JIRA)

unread,
May 28, 2015, 7:07:02 AM5/28/15
to jenkinsc...@googlegroups.com
Mitya Kononchuk edited a comment on Bug JENKINS-28447

jglick@cloudbees.com (JIRA)

unread,
Aug 12, 2015, 8:20:01 AM8/12/15
to jenkinsc...@googlegroups.com

Mitya Kononchuk not if flow.groovy runs inside the same node it was given.

nirvine@bns-g.com (JIRA)

unread,
Oct 20, 2015, 6:10:02 PM10/20/15
to jenkinsc...@googlegroups.com

This is affecting me as well. The Help documentation for the field advertises it as a feature:

$

Unknown macro: {ENV_VARIABLE}

It is also possible to use environment variables. In this case the variables are evaluated and the result is used as described above.
E.g. $

Unknown macro: {TREEISH}

, refs/tags/$

Unknown macro: {TAGNAME}

,...

So one of these is a bug.

nirvine@bns-g.com (JIRA)

unread,
Oct 20, 2015, 6:11:03 PM10/20/15
to jenkinsc...@googlegroups.com
Nick Irvine edited a comment on Bug JENKINS-28447
This is affecting me as well. The Help documentation for the field advertises it as a feature:

{quote}
$ \ {ENV_VARIABLE}

It is also possible to use environment variables. In this case the variables are evaluated and the result is used as described above.
E.g. $ \ {TREEISH}, refs/tags/$ \ {TAGNAME},...
{quote}


So one of these is a bug.

jglick@cloudbees.com (JIRA)

unread,
Oct 20, 2015, 6:23:02 PM10/20/15
to jenkinsc...@googlegroups.com

Of course the inline help was written for the plugin when only freestyle projects existed.

nirvine@bns-g.com (JIRA)

unread,
Oct 20, 2015, 6:35:02 PM10/20/15
to jenkinsc...@googlegroups.com

Jesse Glick: I'm just saying, since this bug is several months old, it would appease me at least if the docs were updated to say something like "(Not available in Workflow Step; see JENKINS-28447)". Seems like a reasonable stop-gap solution, coupled with your suggestion above.

In fact, I'm willing to contribute a patch if you think that's worthwhile. And provided it's as simple a fix as I imagine – I'm not familiar with the codebase whatsoever.

jglick@cloudbees.com (JIRA)

unread,
Oct 20, 2015, 6:53:02 PM10/20/15
to jenkinsc...@googlegroups.com

Well this would apply to any SCM plugin that has any field which gets variables substituted, I expect. Have not investigated in detail.

andrew.bayer@gmail.com (JIRA)

unread,
Jan 6, 2016, 5:43:03 PM1/6/16
to jenkinsc...@googlegroups.com
Andrew Bayer updated an issue
 

Bumping this to critical - integration with things like the Gerrit trigger plugin with scripts from SCM will be problematic without this.
Change By: Andrew Bayer
Priority: Major Critical

jglick@cloudbees.com (JIRA)

unread,
Mar 22, 2016, 6:33:03 PM3/22/16
to jenkinsc...@googlegroups.com
 
Re: CpsScmFlowDefinition does not resolve variables

SCM implementations seem to call Run.getEnvironment(TaskListener) so it is possible an implementation of JENKINS-30910 would automatically fix this, but certainly deserves its own integration tests.

lvader@gmail.com (JIRA)

unread,
May 20, 2016, 5:45:03 AM5/20/16
to jenkinsc...@googlegroups.com

bump. without being able to use variables in pipeline from scm is practically making pipeline type of job useless, as can't be integrated e.g. with Github pull requests processing.

pwolf@cloudbees.com (JIRA)

unread,
May 20, 2016, 1:58:03 PM5/20/16
to jenkinsc...@googlegroups.com
Patrick Wolf updated an issue
 
Change By: Patrick Wolf
Labels: followup

docwhat@gerf.org (JIRA)

unread,
Jun 20, 2016, 2:10:02 PM6/20/16
to jenkinsc...@googlegroups.com
Christian Höltje commented on Bug JENKINS-28447
 
Re: CpsScmFlowDefinition does not resolve variables

So this breaks pipeline jobs using Gerrit. The Gerrit variables are not available in the "Pipeline script from SCM" section. The are not replaced, but left as $FOOBAR; example: git -c core.askpass=true fetch --tags --progress ssh://jen...@gerrit.example.com:29418/noodles/jenkinsfile.git $GERRIT_REFSPEC

It breaks other cases too. For example, in the same job above, I've set a build parameter $GERRIT_REFSPEC and it doesn't work either.
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

docwhat@gerf.org (JIRA)

unread,
Jun 20, 2016, 2:12:07 PM6/20/16
to jenkinsc...@googlegroups.com
Christian Höltje updated an issue
 
Change By: Christian Höltje
Environment:
Jenins 1.613, Workflow plugin 1.6

Also: Jenkins 1.651.1, workflow-job 2.3

kmbulebu@gmail.com (JIRA)

unread,
Jun 22, 2016, 3:38:02 PM6/22/16
to jenkinsc...@googlegroups.com
Kevin Bulebush commented on Bug JENKINS-28447
 
Re: CpsScmFlowDefinition does not resolve variables

The suggested workaround will not work if your script uses 'checkout scm'.

{{java.lang.IllegalStateException: inappropriate context
at org.jenkinsci.plugins.workflow.multibranch.SCMVar.getValue(SCMVar.java:75)}}

'checkout scm' is only allowed for CpsScmFlowDefinition type jobs, not the CpsFlowDefinition type job used by the workaround.

In my case, I fully worked out a Pipeline script to build the master branch, then duped the job and modified it to trigger on patchset-created events in Gerrit. I suppose I'll go back to the original script and modify it to build its own scm object. I'll have to keep in mind that it could be invoked by either a ref-updated or patchset-created event, changing the parameters to the git scm class. It's a bit of a rabbit hole.

gareth@garethwestern.com (JIRA)

unread,
Aug 4, 2016, 6:27:03 AM8/4/16
to jenkinsc...@googlegroups.com

Is this a duplicate of JENKINS-34876?

gareth@garethwestern.com (JIRA)

unread,
Aug 4, 2016, 6:27:09 AM8/4/16
to jenkinsc...@googlegroups.com
Gareth Western edited a comment on Bug JENKINS-28447
Is this a duplicate of JENKINS-34876 and JENKINS-34540 ?

jglick@cloudbees.com (JIRA)

unread,
Aug 16, 2016, 5:22:04 PM8/16/16
to jenkinsc...@googlegroups.com

Have some evidence that JENKINS-30910 would fix this implicitly.

jglick@cloudbees.com (JIRA)

unread,
Aug 29, 2016, 2:03:22 PM8/29/16
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Change By: Jesse Glick
Component/s: workflow-cps-plugin
Component/s: pipeline

jglick@cloudbees.com (JIRA)

unread,
Sep 16, 2016, 7:44:07 PM9/16/16
to jenkinsc...@googlegroups.com
Jesse Glick started work on Bug JENKINS-28447
 
Change By: Jesse Glick
Status: Open In Progress

jglick@cloudbees.com (JIRA)

unread,
Sep 16, 2016, 7:45:07 PM9/16/16
to jenkinsc...@googlegroups.com
 
Re: CpsScmFlowDefinition does not resolve variables

I think I have finally reproduced this in a test, the build ending with something like

…
 > git rev-parse origin/${VERSION}^{commit} # timeout=10
 > git rev-parse ${VERSION}^{commit} # timeout=10
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
Finished: FAILURE

jglick@cloudbees.com (JIRA)

unread,
Sep 16, 2016, 10:04:19 PM9/16/16
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Progress Review

jwillemsen@remedy.nl (JIRA)

unread,
Sep 20, 2016, 11:09:04 AM9/20/16
to jenkinsc...@googlegroups.com
Johnny Willemsen commented on Bug JENKINS-28447
 
Re: CpsScmFlowDefinition does not resolve variables

Would be useful for our use case when the Script Path could use the job name, that way we can have a generic setup where we just have to copy jobs and rename them, the script path will use the correct file for that job from scm

scm_issue_link@java.net (JIRA)

unread,
Sep 21, 2016, 2:17:19 PM9/21/16
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/test/java/org/jenkinsci/plugins/workflow/cps/CpsScmFlowDefinitionTest.java
http://jenkins-ci.org/commit/workflow-cps-plugin/86c7ea988f371a8b50d2a2ed41ad2fcb4cd6de25
Log:
[FIXED JENKINS-28447] CpsScmFlowDefinition does not resolve variables.
Actual fix is implied by JENKINS-30910; this is just the integration test.

jglick@cloudbees.com (JIRA)

unread,
Sep 21, 2016, 2:20:29 PM9/21/16
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Review Resolved
Resolution: Fixed

jglick@cloudbees.com (JIRA)

unread,
Sep 21, 2016, 2:20:29 PM9/21/16
to jenkinsc...@googlegroups.com
Jesse Glick commented on Bug JENKINS-28447
 
Re: CpsScmFlowDefinition does not resolve variables

Johnny Willemsen this issue is about variable expansion in the SCM configuration. Your case is something else, best filed as a separate RFE. (Or really you may just want to be using the Job DSL plugin.)

jenkinsci@daumiller.eu (JIRA)

unread,
Nov 10, 2016, 10:37:07 AM11/10/16
to jenkinsc...@googlegroups.com

Could it be that changing this introduced JENKINS-38734?
I have these versions of pipeline related Plugins:
Pipeline 2.4
Pipeline Graph Analysis Plugin 1.2
Pipeline: API 2.6
Pipeline: Basic Steps 2.3
Pipeline: Build Step 2.3
Pipeline: Groovy 2.23
Pipeline: Input Step 2.5
Pipeline: Job 2.8
Pipeline: Milestone Step 1.1
Pipeline: Multibranch 2.9.2
Pipeline: Nodes and Processes 2.5
Pipeline: REST API Plugin 2.2
Pipeline: SCM Step 2.2
Pipeline: Shared Groovy Libraries 2.4
Pipeline: Stage Step 2.2
Pipeline: Stage View Plugin 2.2
Pipeline: Step API 2.5
Pipeline: Supporting APIs 2.10

jenkinsci@daumiller.eu (JIRA)

unread,
Nov 10, 2016, 11:12:04 AM11/10/16
to jenkinsc...@googlegroups.com
Dorian Daumiller updated an issue
 
Change By: Dorian Daumiller
Comment:
Could it be that changing this introduced JENKINS-38734?
I have these versions of pipeline related Plugins:
Pipeline 2.4    
Pipeline Graph Analysis Plugin 1.2    
Pipeline: API 2.6    
Pipeline: Basic Steps 2.3    
Pipeline: Build Step 2.3    
Pipeline: Groovy 2.23    
Pipeline: Input Step 2.5    
Pipeline: Job 2.8    
Pipeline: Milestone Step 1.1    
Pipeline: Multibranch 2.9.2    
Pipeline: Nodes and Processes 2.5    
Pipeline: REST API Plugin 2.2    
Pipeline: SCM Step 2.2    
Pipeline: Shared Groovy Libraries 2.4    
Pipeline: Stage Step 2.2    
Pipeline: Stage View Plugin 2.2    
Pipeline: Step API 2.5    
Pipeline: Supporting APIs 2.10

matthew.mallard@jetisre.com (JIRA)

unread,
Jul 17, 2019, 6:03:03 PM7/17/19
to jenkinsc...@googlegroups.com
Matthew Mallard commented on Bug JENKINS-28447
 
Re: CpsScmFlowDefinition does not resolve variables

Has this "separate feature" been added yet to the "lightweight checkout" functionality? In my case, I have a pipeline job that is parameterized. One of the parameters is the branch name which I intend to use later on in the job when selecting the definition "Pipeline script from SCM". When selecting that you have the option to select your Git repository and Branches to build, but it doesn't recognize the parameter when "Lightweight checkout" is selected.
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
Reply all
Reply to author
Forward
0 new messages