[JIRA] (JENKINS-57417) configFileProvider / withCredentials not working in declarative pipeline

33 views
Skip to first unread message

crashvb@gmail.com (JIRA)

unread,
May 11, 2019, 11:12:02 AM5/11/19
to jenkinsc...@googlegroups.com
Richard Davis created an issue
 
Jenkins / Bug JENKINS-57417
configFileProvider / withCredentials not working in declarative pipeline
Issue Type: Bug Bug
Assignee: Dominik Bartholdi
Components: config-file-provider-plugin, credentials-binding-plugin, token-macro-plugin
Created: 2019-05-11 15:11
Environment: Jenkins 2.174
Config File Provider Plugin 3.6
Credentials Bind Plugin 1.18
Priority: Major Major
Reporter: Richard Davis

From what I read which googling the issue, I am under the impression that:

1) The credentials bind plugin does not assign environment variables, but rather groovy variables; and even if it did, environment variables do not propagate "up" to the parent context.

2) The "glue" that is available using scripted pipelines does not work in declarative pipelines; specifically the ability to define local variables and assign environment variables within a stage, and access them via the underlying token macro plugin that backs the configFileProvider.

  • Use of groovy variables, ${PIP_USERNAME}, resulted in an "Unrecoginzed macro" error.
  • Use of ${env.PIP_USERNAME} did not substitute the variable at all.
  • Use of ${ENV, var = "PIP_USERNAME"} substitutes the variable with null string.

 

Custom Config File - 05f48227-0980-4313-ab24-f007d78090cf
[global]
index = https://${ENV, var="PIP_USERNAME"}:${ENV, var="PIP_PASSWORD"}@nexus/repository/pypi-all/pypi
index-url = https://${ENV, var="PIP_USERNAME"}:${ENV, var="PIP_PASSWORD"}@nexus/repository/pypi-all/simple
cert = /etc/pki/ca-trust/source/anchors/ca.crt

[list]
format=columns

 

Pipeline
pipeline {
    agent any
    
    environment {
        PIP_FILE = '05f48227-0980-4313-ab24-f007d78090cf'
    }
    stages {
        stage('Build') {
            steps {
                withCredentials([usernamePassword(credentialsId: '0353637f-ef0b-46e5-b95a-6322b1e073d7', passwordVariable: 'PIP_PASSWORD', usernameVariable: 'PIP_USERNAME')]) {
                    configFileProvider([configFile(fileId: "${env.PIP_FILE}", replaceTokens: true, variable: 'PIP_TARGET')]) {
                        sh "cat ${PIP_TARGET}"
                    }
                }
            }
        }
    }
}
Output
...
Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] withCredentials
Masking only exact matches of $PIP_USERNAME or $PIP_PASSWORD
[Pipeline] {
[Pipeline] wrap
provisioning config files...
copy managed file [Python pip.conf for Debugging] to file:/home/jenkins/workspace/OPIN/jenkins-issue@tmp/config8203247759059609056tmp
[Pipeline] {
[Pipeline] sh
+ cat /home/jenkins/workspace/OPIN/jenkins-issue@tmp/config8203247759059609056tmp
[global]
index = https://:@nexus/repository/pypi-all/pypi
index-url = https://:@nexus/repository/pypi-all/simple
cert = /etc/pki/ca-trust/source/anchors/ca.crt

[list]
[Pipeline] }
Deleting 1 temporary files
...

Is this a bug, or am I missing something?

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

domi@fortysix.ch (JIRA)

unread,
May 13, 2019, 11:35:02 AM5/13/19
to jenkinsc...@googlegroups.com
Dominik Bartholdi commented on Bug JENKINS-57417
 
Re: configFileProvider / withCredentials not working in declarative pipeline

Why do you handle the ids different for withCredentials and configFileProvider? fileId is not substituted by the configFileProvider, therefore you have to pass the id as it is. But after all you can use groovy if you like, this should work:

var pipFile = '05f48227-0980-4313-ab24-f007d78090cf'
pipeline {
    agent any
    
    environment {
        PIP_FILE = '05f48227-0980-4313-ab24-f007d78090cf'
    }
    stages {
        stage('Build') {
            steps {
                withCredentials([usernamePassword(credentialsId: '0353637f-ef0b-46e5-b95a-6322b1e073d7', passwordVariable: 'PIP_PASSWORD', usernameVariable: 'PIP_USERNAME'
)]) {
                    configFileProvider([configFile(fileId: pipFile, replaceTokens: true, variable: 'PIP_TARGET')]) {
                        sh "cat ${PIP_TARGET}"
                    }
                }
            }
        }
    }
}
Add Comment Add Comment
 

domi@fortysix.ch (JIRA)

unread,
May 13, 2019, 11:36:03 AM5/13/19
to jenkinsc...@googlegroups.com
Dominik Bartholdi edited a comment on Bug JENKINS-57417
Why do you handle the ids different for {{withCredentials}} and {{configFileProvider}}? {{fileId}} is not substituted by the configFileProvider, therefore you have to pass the id as it is. But after all you can use groovy if you like, this should work:
{code:java}

var pipFile = '05f48227-0980-4313-ab24-f007d78090cf'
pipeline {
    agent any
    
    environment {
        PIP_FILE = '05f48227-0980-4313-ab24-f007d78090cf'
    }
    stages {
        stage('Build') {
            steps {
                withCredentials([usernamePassword(credentialsId: '0353637f-ef0b-46e5-b95a-6322b1e073d7', passwordVariable: 'PIP_PASSWORD', usernameVariable: 'PIP_USERNAME')]) {
                    configFileProvider([configFile(fileId: pipFile, replaceTokens: true, variable: 'PIP_TARGET')]) {
                     sh "cat ${PIP_TARGET}"
                    }
                }
            }
        }
    }
}{code}

domi@fortysix.ch (JIRA)

unread,
May 13, 2019, 11:36:03 AM5/13/19
to jenkinsc...@googlegroups.com
Dominik Bartholdi resolved as Not A Defect
 
Change By: Dominik Bartholdi
Status: Open Resolved
Resolution: Not A Defect

crashvb@gmail.com (JIRA)

unread,
May 13, 2019, 12:15:03 PM5/13/19
to jenkinsc...@googlegroups.com
Richard Davis commented on Bug JENKINS-57417
 
Re: configFileProvider / withCredentials not working in declarative pipeline

Thank you for the reply. This code was isolated from a larger pipeline to reproduce the specific issue being reported.

I ran the same job again with the fileId inline (no local variable), and got the same outcome:

configFileProvider([configFile(fileId: '05f48227-0980-4313-ab24-f007d78090cf', replaceTokens: true, variable: 'PIP_TARGET')]) { ... }

I don't believe that the issue is related to resolution of fileId, as it the plugin is able to replicate the file (with parameters omitted):

...
copy managed file [Python pip.conf for Debugging] to file:/home/jenkins/workspace/OPIN/jenkins-issue@tmp
/config8203247759059609056tmp
...

Since the last post, I have also add a

sh "echo ${PIP_USERNAME} ${PIP_PASSWORD}"

within the withCredentials block, and saw that they were successfully masked:

Confirmation of ENV assignment
...
[Pipeline] withCredentials
Masking only exact matches of $PIP_USERNAME or $PIP_PASSWORD
[Pipeline] {
[Pipeline] sh
+ echo **** ****
**** ****
[Pipeline] wrap
...

I remember seeing an old PR against the config file provider, where it wasn't initially replacing macros within pipelines, only scripts; but i'm not familiar enough with the code base to remember in which file it was located. From looking at the source of the token macro, it appears to accept [environment as a parameter](https://github.com/jenkinsci/token-macro-plugin/blob/master/src/main/java/org/jenkinsci/plugins/tokenmacro/impl/EnvironmentVariableMacro.java#L45). Could this be an issue with how it's being invoked?

crashvb@gmail.com (JIRA)

unread,
May 13, 2019, 1:38:01 PM5/13/19
to jenkinsc...@googlegroups.com

crashvb@gmail.com (JIRA)

unread,
May 15, 2019, 10:22:03 PM5/15/19
to jenkinsc...@googlegroups.com
Richard Davis updated an issue
Change By: Richard Davis
Component/s: credentials-binding-plugin
Component/s: token-macro-plugin

crashvb@gmail.com (JIRA)

unread,
May 15, 2019, 10:27:02 PM5/15/19
to jenkinsc...@googlegroups.com
 
Re: configFileProvider / withCredentials not working in declarative pipeline

I've dug a little deeper into the source repository. it looks like the Util.replaceMacro method is being invoked correctly with the provided environment; however, it appears that the environment is not begin propagated from higher context, and is instead being explicitly provided as null.

I don't know enough about the way jenkins pipelines are implemented to assess if this omission is intentional, due to an alternative data channel and / or design limitation, or if it is a bug ...

Is use of the configFileProvider within declarative pipelines supported, or should this be re-characterized as an feature enhancement?

(I've also removed the 'credentials-binding-plugin' and 'token-macro-plugin' components, as they have been verified in parallel and don't appear to be the root cause.)

crashvb@gmail.com (JIRA)

unread,
May 15, 2019, 10:29:03 PM5/15/19
to jenkinsc...@googlegroups.com
Richard Davis edited a comment on Bug JENKINS-57417
I've dug a little deeper into the source repository. it looks like the [Util.replaceMacro|https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/Util.java#L146] method is being invoked correctly with the [provided environment|https://github.com/jenkinsci/config-file-provider-plugin/blob/master/src/main/java/org/jenkinsci/lib/configprovider/model/ConfigFileManager.java#L120]; however, it appears that the environment is not begin being propagated from higher context, and is instead being explicitly [provided as null|https://github.com/jenkinsci/config-file-provider-plugin/blob/master/src/main/java/org/jenkinsci/plugins/configfiles/buildwrapper/ManagedFileUtil.java#L57].


I don't know enough about the way jenkins pipelines are implemented to assess if this omission is intentional, due to an alternative data channel and / or design limitation, or if it is a bug ...

Is use of the configFileProvider within declarative pipelines supported, or should this be re-characterized as an feature enhancement?

(I've also removed the 'credentials-binding-plugin' and 'token-macro-plugin' components, as they have been verified in parallel and don't appear to be the root cause.)

crashvb@gmail.com (JIRA)

unread,
May 15, 2019, 10:39:03 PM5/15/19
to jenkinsc...@googlegroups.com

crashvb@gmail.com (JIRA)

unread,
May 16, 2019, 11:56:02 PM5/16/19
to jenkinsc...@googlegroups.com

domi@fortysix.ch (JIRA)

unread,
Jun 5, 2019, 3:33:03 AM6/5/19
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages