Using Config File Provider Plugin with placeholders and Token Macro Plugin

[Roger Lehmann]

Hi,

I'm having trouble with replacing tokens inside my config file.

The config file token replacement only respects environment variable values defined on the job, but not inside the withEnv or withCredential statement blocks.

And as far as I know (and used to), Jenkins or rather the statements provide the credentials as env vars.

Could anyone please help me on how to do it properly?

Maybe I'm just using the wrong macro in my config file?

Thank you!

Example code:

node('master'){
    stage('test w/o magic') {
        configFileProvider([configFile(fileId: 'c76d4a37-1233-4f8c-a2bb-e2f9cf4e05b4', replaceTokens: true, targetLocation: 'test.txt', variable: 'testfile')]) {
            sh 'cat $testfile'
            echo "var1: $env.var1"
            echo "var2: $var2"
        }
    }
    stage('testwith centials provider') {
        withCredentials([string(credentialsId: 'aws_vault_password', variable: 'var1'),
        sshUserPrivateKey(credentialsId: 'jenkins-robot-ssh-key',
        keyFileVariable: 'var2',
        passphraseVariable: 'var3',
        usernameVariable: 'var4'),
        usernameColonPassword(credentialsId: 'jenkins_robot_for_gitlab', variable: 'var5')]) {
            configFileProvider([configFile(fileId: 'c76d4a37-1233-4f8c-a2bb-e2f9cf4e05b4', replaceTokens: true, targetLocation: 'test.txt', variable: 'testfile')]) {
                sh 'cat $testfile'
                echo "var1: $var1"
                echo "var2: $var2"
                echo "var5: $var5"
            }
        }
    }
    
    stage('test with env provider') {
        withEnv(['var1=bla', 'var2=blubb', 'undefined=isdefined']) {
            configFileProvider([configFile(fileId: 'c76d4a37-1233-4f8c-a2bb-e2f9cf4e05b4', replaceTokens: true, targetLocation: 'test.txt', variable: 'testfile')]) {
                sh 'cat $testfile'
                echo "var1: $env.var1"
                echo "var2: $var2"
            }
        }
    }
}

ENV vars declared at job level:

var1=test
var2=foobar

Config File:

Test for Token replacement on ${ENV, var="JOB_NAME"}
key1: ${ENV, var="var1"}
key2: ${ENV, var="var2"}
key5: ${ENV, var="var5"}
undefined: ${ENV, var="undefined"}

Output:

[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/IT/credentials-replacement
[Pipeline] {
[Pipeline] stage
[Pipeline] { (test w/o magic)
[Pipeline] wrap
provisioning config files...
copy managed file [Secret-File] to file:/var/lib/jenkins/workspace/IT/credentials-replacement/test.txt
[Pipeline] {
[Pipeline] sh
[credentials-replacement] Running shell script
+ cat /var/lib/jenkins/workspace/IT/credentials-replacement/test.txt
Test for Token replacement on IT/credentials-replacement key1: test
key2: foobar
key5: 
undefined: 
[Pipeline] echo
var1: test
[Pipeline] echo
var2: foobar
[Pipeline] }
[Pipeline] // wrap
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (testwith centials provider)
[Pipeline] withCredentials
[Pipeline] {
[Pipeline] configFileProvider
provisioning config files...
copy managed file [Secret-File] to file:/var/lib/jenkins/workspace/IT/credentials-replacement/test.txt
[Pipeline] {
[Pipeline] sh
[credentials-replacement] Running shell script
+ cat /var/lib/jenkins/workspace/IT/credentials-replacement/test.txt
Test for Token replacement on IT/credentials-replacement key1: test
key2: foobar
key5: 
undefined: 
[Pipeline] echo
var1: ****
[Pipeline] echo
var2: ****
[Pipeline] echo
var5: ****
[Pipeline] }
[Pipeline] // configFileProvider
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (test with env provider)
[Pipeline] withEnv
[Pipeline] {
[Pipeline] configFileProvider
provisioning config files...
copy managed file [Secret-File] to file:/var/lib/jenkins/workspace/IT/credentials-replacement/test.txt
[Pipeline] {
[Pipeline] sh
[credentials-replacement] Running shell script
+ cat /var/lib/jenkins/workspace/IT/credentials-replacement/test.txt
Test for Token replacement on IT/credentials-replacement key1: test
key2: foobar
key5: 
undefined: 
[Pipeline] echo
var1: bla
[Pipeline] echo
var2: blubb
[Pipeline] }
[Pipeline] // configFileProvider
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Expected:

different outputs for "key1:", "key2:" and "key5:" than the values provided on the job level.

Best,

Roger

[Slide]

It looks like the environment variables are not added to the job, which is where token macro can get at them, but are added to the local context instead. I'll have to look into this more to determine how this can be done. 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/bcd50ff1-1e1e-427d-94c5-8861897cdd64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Roger Lehmann]

That would be really nice of you!

Is there a more "standardized" way of providing config files with Jenkins?

I saw I could store the whole file as a secret with the Credentials plugin, but I can't edit it then. Only reupload without an easy way knowing what's in there.

And my use case needs to be able to view and edit a config file with secrets but keeping the secrets secret.

[Liejun Tao]

Hi,

I hit the same problem. Has this been fixed by any chance?

[Slide]

Not yet, work had been hitting hard recently. I'll try and get to it soon. Can you file an issue?

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/35f716f9-022b-4f67-a56f-d54dcaeba491%40googlegroups.com.