[JIRA] (JENKINS-46124) EnvStep convert map to list

22 views
Skip to first unread message

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 6:24:03 AM7/5/18
to jenkinsc...@googlegroups.com
Tom Ghyselinck updated an issue
 
Jenkins / Improvement JENKINS-46124
EnvStep convert map to list
Change By: Tom Ghyselinck
Priority: Minor Major
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 6:39:02 AM7/5/18
to jenkinsc...@googlegroups.com
Tom Ghyselinck commented on Improvement JENKINS-46124
 
Re: EnvStep convert map to list

Hi,

This would be a great improvement for use with declarative pipeline.

We now have the case where we only perform a checkout in the stages where applicable.

It could help to have some syntax like:

pipeline {
    stages {
        stage('Build') {
            steps {
                withEnv(checkout(scm)) {
                    echo 'printenv'
                    echo 'Perform the build'
                }
            }
        }
    }
}

On the other hand, we still have the case that checkout and withEnv are in separate steps.

We would something like the following (not that this example won't work because of the variable assignment!):

pipeline {
    stages {
        stage('Build') {
            steps {
                def scmVars = checkout(scm)

                echo 'Preparing the build'

                withEnv(scmVars) {
                    echo 'printenv'
                    echo 'Perform the build'
                }
            }
        }
    }
}

Our current workaround is to set the scmVars as a global variable and then "translate" it into a list for use with withEnv:

import groovy.transform.Field

@Field XRA31_SCM_VARS = null

/**
 * Sets the XRA31_SCM_VARS variables to the output of the `checkout` command
 */
def xra31_prepare_source(steps, scm) {
    XRA31_SCM_VARS = steps.checkout changelog: false, poll: false, scm: scm
}

def flatten_map(Map m) {
    List l = []
    m.each { k,v ->
        l.add([k, v].join('='))
    }
    return l
}

pipeline {
    stages {
        stage('Build') {
            steps {
                xra31_prepare_source(this, scm)

                echo 'Preparing the build'

                withEnv(flatten_map(XRA31_SCM_VARS)) {
                    echo 'printenv'
                    echo 'Perform the build'
                }
            }
        }
    }
}

It would be great to see a solution for the latter case since we use/need it extensively.

With best regards,
Tom.

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 6:41:01 AM7/5/18
to jenkinsc...@googlegroups.com
Tom Ghyselinck updated an issue
Change By: Tom Ghyselinck
Component/s: workflow-scm-step-plugin

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 6:42:02 AM7/5/18
to jenkinsc...@googlegroups.com
 
Re: EnvStep convert map to list

Another solution which would help us is to add an option to the checkout command to inject the environment variables directly instead of only returning them.

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 6:47:01 AM7/5/18
to jenkinsc...@googlegroups.com

Jesse Glick,

I know you work a lot on this stuff.
Is this something you find useful?

Thank you in advance!

With best regards,
Tom.

josephp90@gmail.com (JIRA)

unread,
Jul 5, 2018, 10:20:03 AM7/5/18
to jenkinsc...@googlegroups.com

Hey Tom Ghyselinck when using declarative pipeline you don't have to use checkout scm because the job is configured to use scm and the declarative pipeline will just checkout from that

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 10:30:01 AM7/5/18
to jenkinsc...@googlegroups.com
Tom Ghyselinck edited a comment on Improvement JENKINS-46124
Hi,

This would be a great improvement for use with declarative pipeline.

We now have the case where we only perform a {{checkout}} in the stages where applicable.

It could help to have some syntax like:
{noformat}
pipeline {
    options {
        skipDefaultCheckout true
    }

    stages {
        stage('Build') {
            steps {
                withEnv(checkout(scm)) {
                    echo 'printenv'
                    echo 'Perform the build'
                }
            }
        }
    }
}
{noformat}

On the other hand, we still have the case that {{checkout}} and {{withEnv}} are in separate steps.

We would something like the following (_*not that this example won't work because of the variable assignment!*_):
{noformat}
pipeline {
     options {
         skipDefaultCheckout true
     }

    stages {
        stage('Build') {
            steps {
                def scmVars = checkout(scm)

                echo 'Preparing the build'

                withEnv(scmVars) {
                    echo 'printenv'
                    echo 'Perform the build'
                }
            }
        }
    }
}
{noformat}

Our current workaround is to set the {{scmVars}} as a global variable and then "_translate_" it into a list for use with {{withEnv}}:
{noformat}

import groovy.transform.Field

@Field XRA31_SCM_VARS = null

/**
* Sets the XRA31_SCM_VARS variables to the output of the `checkout` command
*/
def xra31_prepare_source(steps, scm) {
    XRA31_SCM_VARS = steps.checkout changelog: false, poll: false, scm: scm
}

def flatten_map(Map m) {
    List l = []
    m.each { k,v ->
        l.add([k, v].join('='))
    }
    return l
}

pipeline {
     options {
         skipDefaultCheckout true
     }

    stages {
        stage('Build') {
            steps {
                xra31_prepare_source(this, scm)

                echo 'Preparing the build'

                withEnv(flatten_map(XRA31_SCM_VARS)) {
                    echo 'printenv'
                    echo 'Perform the build'
                }
            }
        }
    }
}
{noformat}

It would be great to see a solution for the latter case since we use/need it extensively.

With best regards,
Tom.

tom.ghyselinck@excentis.com (JIRA)

unread,
Jul 5, 2018, 10:31:01 AM7/5/18
to jenkinsc...@googlegroups.com

Hi Joseph Petersen,

It depends

I use skipDefaultCheckout true in my pipeline options.
I updated my comment accordingly.

Thank you for the suggestion anyway!

With best regards,
Tom.

josephp90@gmail.com (JIRA)

unread,
Jul 5, 2018, 12:14:02 PM7/5/18
to jenkinsc...@googlegroups.com

The envs is passed from the default checkout

jglick@cloudbees.com (JIRA)

unread,
Jul 20, 2018, 3:18:02 PM7/20/18
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Change By: Jesse Glick
Component/s: workflow-scm-step-plugin

jglick@cloudbees.com (JIRA)

unread,
Jul 20, 2018, 3:23:02 PM7/20/18
to jenkinsc...@googlegroups.com
Jesse Glick commented on Improvement JENKINS-46124
 
Re: EnvStep convert map to list

Databinding in Jenkins, including for Step, does not generally handle Map. Unfortunately it does not generally handle List<String> either, necessitating a StepDescriptor.newInstance override (JENKINS-27901). So I suppose this step could accept Map<String, String> as a convenience.

Anyway for purposes of the stated request, I would consider it a defect in pipeline-model-definition-plugin that it even allows you pass such a function call as an argument. That is a Scripted thing which should not exist in Declarative, just as the def keyword is forbidden. What you actually want is a declarative option to perform a standard checkout in a given stage. So file that separately please, unless it suffices to move skipDefaultCheckout to the other stages rather than being a top-level option, in which case you do not need anything at all.

jerrywiltse@gmail.com (JIRA)

unread,
Apr 23, 2019, 10:10:02 AM4/23/19
to jenkinsc...@googlegroups.com

Did anyone ever file a separate issue to create an overload of withEnv which takes a Map<String, String>?  Indeed, the overload to pass a Map would be convenient for us as well.  I assumed the reason for the current signature was that Groovy's underlying string helper function "execute()" takes a String[] called envp for environment variables, so withEnv was designed in such a way that the env vars could just be passed through without a type conversion.  Otherwise, I can't imagine why it was designed this way. 

Of note, it seems the GDSL file currently says it takes a Map, which appears to be simply incorrect. 

 

method(name: 'withEnv', type: 'Object', params: [overrides:Map, body:'Closure'], doc: 'Set environment variables')

 

 

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

josephp90@gmail.com (JIRA)

unread,
Jan 25, 2020, 6:14:05 PM1/25/20
to jenkinsc...@googlegroups.com

This is not as easy as it would seem.

Adding a second @DataBoundConstructor since the method is final seems unsupported and throws error

Changing the code to have two @DataBoundSetter changes the step to ignore takesImplicitBlockArgument suddenly it wants named arguments.

 

javax.annotation.processing.FilerException: Attempt to reopen a file for path C:\git\code\workflow.basic.steps.plugin\target\classes\org\jenkinsci\plugins\workflow\steps\EnvStep.stapler
[ERROR]         at jdk.compiler/com.sun.tools.javac.processing.JavacFiler.checkFileReopening(JavacFiler.java:742)
[ERROR]         at jdk.compiler/com.sun.tools.javac.processing.JavacFiler.createResource(JavacFiler.java:534)
[ERROR]         at org.kohsuke.stapler.jsr269.AbstractProcessorImpl.createResource(AbstractProcessorImpl.java:81)
[ERROR]         at org.kohsuke.stapler.jsr269.AbstractProcessorImpl.writePropertyFile(AbstractProcessorImpl.java:67)
[ERROR]         at org.kohsuke.stapler.jsr269.ConstructorProcessor.write(ConstructorProcessor.java:98)
[ERROR]         at org.kohsuke.stapler.jsr269.ConstructorProcessor.access$000(ConstructorProcessor.java:28)
[ERROR]         at org.kohsuke.stapler.jsr269.ConstructorProcessor$1.visitExecutable(ConstructorProcessor.java:36)
[ERROR]         at org.kohsuke.stapler.jsr269.ConstructorProcessor$1.visitExecutable(ConstructorProcessor.java:32)
[ERROR]         at jdk.compiler/com.sun.tools.javac.code.Symbol$MethodSymbol.accept(Symbol.java:1964)
[ERROR]         at java.c...@11.0.6/javax.lang.model.util.ElementScanner6.scan(ElementScanner6.java:153)
[ERROR]         at java.c...@11.0.6/javax.lang.model.util.ElementScanner6.scan(ElementScanner6.java:140)
[ERROR]         at java.c...@11.0.6/javax.lang.model.util.ElementScanner6.visitType(ElementScanner6.java:189)
[ERROR]         at jdk.compiler/com.sun.tools.javac.code.Symbol$ClassSymbol.accept(Symbol.java:1447)
[ERROR]         at java.c...@11.0.6/javax.lang.model.util.ElementScanner6.scan(ElementScanner6.java:153)
[ERROR]         at org.kohsuke.stapler.jsr269.ConstructorProcessor.process(ConstructorProcessor.java:57)
[ERROR]         at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:980)
[ERROR]         at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:896)
[ERROR]         at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1222)
[ERROR]         at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1334)
[ERROR]         at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1258)
[ERROR]         at jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:936)
[ERROR]         at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
[ERROR]         at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:147)
[ERROR]         at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
[ERROR]         at jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
[ERROR]         at org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126)
[ERROR]         at org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174)
[ERROR]         at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1134)
[ERROR]         at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:187)
[ERROR]         at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
[ERROR]         at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR]         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
[ERROR]         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
[ERROR]         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
[ERROR]         at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
[ERROR]         at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
[ERROR]         at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)

 

 

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

jglick@cloudbees.com (JIRA)

unread,
Jan 27, 2020, 9:30:03 AM1/27/20
to jenkinsc...@googlegroups.com

I would suggest leaving the databinding alone and implementing CustomDescribableModel.

josephp90@gmail.com (JIRA)

unread,
Jan 27, 2020, 9:35:02 AM1/27/20
to jenkinsc...@googlegroups.com

Searching on GitHub does not give many results, perhaps you could point to a good implementation:

https://github.com/search?q=org%3Ajenkinsci+CustomDescribableModel&type=Code

josephp90@gmail.com (JIRA)

unread,
Jan 27, 2020, 9:40:03 AM1/27/20
to jenkinsc...@googlegroups.com

Reading on the CustomDescribableModel I would have to implement a customInstantiate

josephp90@gmail.com (JIRA)

unread,
Jan 27, 2020, 4:07:02 PM1/27/20
to jenkinsc...@googlegroups.com
Joseph Petersen assigned an issue to Devin Nusbaum
 
Change By: Joseph Petersen
Assignee: Devin Nusbaum

josephp90@gmail.com (JIRA)

unread,
Jan 27, 2020, 4:09:04 PM1/27/20
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Jan 27, 2020, 4:38:04 PM1/27/20
to jenkinsc...@googlegroups.com
Jesse Glick assigned an issue to Joseph Petersen
 
Change By: Jesse Glick
Assignee: Devin Nusbaum Joseph Petersen

jglick@cloudbees.com (JIRA)

unread,
Jan 27, 2020, 4:38:05 PM1/27/20
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Jan 27, 2020, 4:38:05 PM1/27/20
to jenkinsc...@googlegroups.com
Jesse Glick started work on Improvement JENKINS-46124
 
Change By: Jesse Glick
Status: Open In Progress

josephp90@gmail.com (JIRA)

unread,
Mar 7, 2020, 1:07:24 AM3/7/20
to jenkinsc...@googlegroups.com
Joseph Petersen assigned an issue to Joseph Petersen
Change By: Joseph Petersen
Assignee: Joseph Petersen (old)
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages