variable within a variable

228 views
Skip to first unread message

cw

unread,
Oct 22, 2020, 12:14:06 PM10/22/20
to Jenkins Users
I'm trying to set up a string parameter which will reference a variable which will be created within the pipeline.  For instance, when building a parameterized build, the user enters a file path in this format: /srv/jboss/server/${MAL}/log/

Where MAL is assigned via the pipeline and then I try to ls -ltr the fully qualified path after MAL has been expanded.  However I'm not getting it to expand; wondering what I'm doing wrong.

node {
MALarray = params.multi.split(',')

MALarray.each { MAL ->
         
if (MAL in ['care''] )  {
            servers = ['xx.xx.xx.xx','xx.xx.xx.xx']
        }

def soc_remote = [:]
soc_remote.name = "${MAL} SOC"
soc_remote.allowAnyHosts = true

withCredentials([usernamePassword(credentialsId: 'Credentials', usernameVariable: 'USER_ID', passwordVariable: 'USER_PASSWORD')]) {
soc_remote.user = USER_ID
soc_remote.password = USER_PASSWORD
servers.each { server ->
            soc_remote.host = server
stage("Run script on ${server}") {
    try {
    echo("MAL is ${MAL}")
    sshCommand remote: soc_remote, command: "ls -ltr ${filepath}"
    } catch (err) {
                          echo "something failed"
                          echo "${err}"
                          }
}
}
        }
}
}

Console output:
11:59:34 MAL is care 
11:59:34 [Pipeline] sshCommand 
11:59:34 Executing command on care SOC[xx.xx.xx.xx]: ls -ltr /srv/jboss/server/${MAL}/log/ sudo: false 
11:59:35 ls: cannot access /srv/jboss/server//log/: No such file or directory 
11:59:35 Failed command care SOC#1002 with status 2: ls -ltr /srv/jboss/server/${MAL}/log/  

jeremy mordkoff

unread,
Oct 23, 2020, 12:13:07 PM10/23/20
to Jenkins Users
I think you need to do an "eval" or the equivalent to filepath in order to expand the inner variable. I believe the ssh command is executing "ls /srv/jboss/server/${MAL}/log/" but MAL on the far end is empty. So you need to complete the expansion before calling ssh. 

cw

unread,
Oct 23, 2020, 2:28:57 PM10/23/20
to Jenkins Users
Thank you, it seems that you're right, it's not completing the expansion, but in the research I did, I wasn't able to make anything else work (one suggestion had to do with double quotes instead of single quotes, but wasn't able to get that to work either).  I don't know if it's related to it being a parameter that was specified at build this is not allowing it to expand?

I looked at the link you sent but didn't really understand "using a closure" as the answer states.  I tried to imitate it using this code below but it still didn't expand:

def fp = { filepath -> "${filepath}" }
println fp(filepath)

jeremy mordkoff

unread,
Oct 23, 2020, 3:14:08 PM10/23/20
to Jenkins Users

and I found this snippet worked just fine after I approved the signature in jenkins 

import groovy.text.SimpleTemplateEngine

node("docker") {
    stage("eval") {
        def pathname = "/srv/\${host}/foo/bar"

        def templateMessage = new SimpleTemplateEngine().createTemplate(pathname)

        def result = { host -> templateMessage.make(host: host)}
        def HOST = "myhost"

        println(result(HOST))
    }

cw

unread,
Oct 23, 2020, 4:31:55 PM10/23/20
to Jenkins Users
That's great, thank you very much.  I was able to take your code, substitute MAL for HOST, for instance, and test it in a groovy web console, and it returned the correct value and interpolated the variable even though the variable wasn't defined until after the pathname was defined with the variable in it.  Exactly what I was looking for.  However...

I took the same code and pasted it into my test jenkins pipeline and now I'm getting an error on a subsequent command which I looked up, but I think it may be over my head...  Note that I'm getting the correct result with my substituted variable, but I'm just printing to the console and I'm not using it yet in the sshcommand, and yet sshcommand appears to be failing where before it wasn't.

6:24:05 Executing command on care SOC[xx.xx.xx.xx]: ls -ltr /srv/jboss/server/ sudo: false 
16:24:05 [Pipeline] echo 
16:24:05 something failed 
16:24:05 [Pipeline] echo 
16:24:05 java.io.NotSerializableException: groovy.text.SimpleTemplateEngine$SimpleTemplate  

code block:

import java.text.SimpleDateFormat
import groovy.text.SimpleTemplateEngine

node {
    def dateFormat = new SimpleDateFormat("yyyyMMddHHmm")

MALarray = params.multi.split(',')

MALarray.each { MAL ->

         
if (MAL in ['care', 'conn'] )  {
            echo 'care or conn'
            servers = ['xx.xx.xx.xx']
        }
def soc_remote = [:]
soc_remote.name = "${MAL} SOC"
soc_remote.allowAnyHosts = true

withCredentials([usernamePassword(credentialsId: 'Credentials', usernameVariable: 'USER_ID', passwordVariable: 'USER_PASSWORD')]) {
soc_remote.user = USER_ID
soc_remote.password = USER_PASSWORD
def templateMessage = new SimpleTemplateEngine().createTemplate(filepath)
def result = { mal -> templateMessage.make(mal: mal)}

println(result(MAL))    // this is working correctly

servers.each { server ->
            soc_remote.host = server
    try {
    echo("MAL is ${MAL}")
    sshCommand remote: soc_remote, command: "ls -ltr /srv/jboss/server/"
    } catch (err) {
                          echo "something failed"
                          echo "${err}"    // this is where I'm getting the error on the sshcommand
                          }
}
        }
}
}

jeremy mordkoff

unread,
Oct 26, 2020, 9:41:08 AM10/26/20
to Jenkins Users
Are you running this in the main thread of the enkinsFile itself or in a function? I suspect that you are in a function and you may need to use the @NONCPS decorator 

cw

unread,
Oct 26, 2020, 10:04:59 AM10/26/20
to Jenkins Users
The code block I pasted is exactly the pipeline steps I am running so I don't think I'm running it in a function?  

cw

unread,
Oct 26, 2020, 11:51:19 AM10/26/20
to Jenkins Users
Also I'm confused why I'm getting a  NotSerializableException related to SimpleTemplateEngine$SimpleTemplate when I'm in a try/except block which is running an sshCommand which is not trying to call the SimpleTemplateEngine.

jeremy mordkoff

unread,
Oct 26, 2020, 1:51:28 PM10/26/20
to Jenkins Users
I think it is failing on this line: 
    echo("MAL is ${MAL}")



cw

unread,
Oct 26, 2020, 2:41:09 PM10/26/20
to Jenkins Users
I wondered about that also, but I commented it out and still got the error.  Then I moved the defs outside the withCredentials block and now I get a similar but different error.  It appears to be erroring at the withCredentials line.  I haven't even tried calling these defs yet but am getting the error when going into withCredentials:

14:34:20 in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals 
14:34:20 in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@307ca576 
14:34:20 in field com.cloudbees.groovy.cps.impl.CpsClosureDef.capture 
14:34:20 in object com.cloudbees.groovy.cps.impl.CpsClosureDef@1b7eff10 
14:34:20 in field com.cloudbees.groovy.cps.impl.CpsClosure.def 
14:34:20 in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@2932682d 
14:34:20 in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.closures 
14:34:20 in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@43cd433c 
14:34:20 in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@43cd433c 
14:34:20 Caused: java.io.NotSerializableException: groovy.text.SimpleTemplateEngine$SimpleTemplate  

jeremy mordkoff

unread,
Oct 26, 2020, 4:02:13 PM10/26/20
to Jenkins Users
okay, take this with a grain of salt because this is purely a guess

The ssh call is trying to serialize all of the local variables, but it can't serialize the simpleTemplate. So move all the code that touches simpletemplate into a function that's wrapped with the @NONCPS decorator. 

something like  this (I am writing this off the top of my head ... no verification)

@NONCPS
def make_path(MAL) { 

        def pathname = "/srv/\${host}/foo/bar"
        def templateMessage = new SimpleTemplateEngine().createTemplate(pathname)
        def template = { host -> templateMessage.make(host: host)}
        def result = template(MAL)
       return result
}

   

cw

unread,
Oct 26, 2020, 4:40:39 PM10/26/20
to Jenkins Users
thank you for the suggestion and the explanation.  I'm definitely learning here.  slowly, but hopefully learning.

I'm also investigating just writing a quick python program that does the work for me and calling that to get the result I need.

Reply all
Reply to author
Forward
0 new messages