[JIRA] (JENKINS-43077) withEnv does not set some environment variables

249 views
Skip to first unread message

dkowis@shlrm.org (JIRA)

unread,
Mar 23, 2017, 3:10:01 PM3/23/17
to jenkinsc...@googlegroups.com
David Kowis created an issue
 
Jenkins / Bug JENKINS-43077
withEnv does not set some environment variables
Issue Type: Bug Bug
Assignee: Unassigned
Components: pipeline
Created: 2017/Mar/23 7:09 PM
Environment: Jenkins 2.32.3
BlueOcean 1.0.0-rc1
Priority: Minor Minor
Reporter: David Kowis

I don't believe this is a duplicate of https://issues.jenkins-ci.org/browse/JENKINS-28317 but I haven't been able to figure out if I have that version of the plugin installed on my jenkins, because I'm not sure what plugin that applies to.

 

I had a `withEnv` section that looked like:
{{
def flowContext(cl) {
withEnv([
'npm_config_cache=npm-cache',
'npm_config_registry=http://npm.artifactory.mycompany.com/artifactory/api/npm/npm',
'http_proxy=http://www-proxy.mycompany.com:8080',
'https_proxy=http://www-proxy.homedepot.com:8080',
'HTTP_PROXY=http://www-proxy.mycompany.com:8080',
'HTTPS_PROXY=http://www-proxy.mycompany.com:8080',
'PROXY_HOST=www-proxy.mycompany.com',
'PROXY_PORT=8080',
'no_proxy=.mycompany.com'
])

{ credentialsContext(cl) }

}

def credentialsContext(cl) {
withCredentials([
usernamePassword(
credentialsId: 'service-account-user-pass',
passwordVariable: 'SVC_PASS',
usernameVariable: 'SVC_USER'
),
usernamePassword(
credentialsId: 'github-auth-token',
passwordVariable: 'GITHUB_TOKEN',
usernameVariable: 'GITHUB_USERNAME'
),
usernamePassword(
credentialsId: 'service-account-maven-artifactory-token',
passwordVariable: 'ARTIFACTORY_USER',
usernameVariable: 'ARTIFACTORY_TOKEN'
),
usernamePassword(
credentialsId: 'service-account-maven-artifactory-password',
passwordVariable: 'ARTIFACTORY_USER',
usernameVariable: 'ARTIFACTORY_PASSWORD'
),
string(credentialsId: 'slack-webhook', variable: 'SLACK_WEBHOOK_URL'),
string(credentialsId: 'tracker-token', variable: 'TRACKER_TOKEN')

])

{ cl() }

}

}}

The credentials context is another function that exports a few credentials into the environment.
All of the credentials would be there, but not all of the environment variables.

This would be executed within a docker container like so:

{{
stage('pip-ci-node Build') {
docker.image(ciNode).inside() {
//Do the node installs in parallel, because the npm install is the slowest part, and we can use lots of cpu
parallel(
"Root Npm Install/Build": {
flowContext

{ sh('env') sh('npm install') sh('gulp') }

},
"webapp/static Install/Build": {
flowContext

{ echo "Due to bug: https://issues.jenkins-ci.org/browse/JENKINS-33510" sh('cd src/main/webapp/static && npm install') sh('cd src/main/webapp/static && grunt prod') }

},
failFast: true
)
}
}
}}

 The sh('env') command within the flowContext would never show the no_proxy environment variable. I eventually gave up on trying to get it set in the Jenkinsfile, and instead put it in the root container image.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

dkowis@shlrm.org (JIRA)

unread,
Mar 23, 2017, 3:16:01 PM3/23/17
to jenkinsc...@googlegroups.com
David Kowis updated an issue
Change By: David Kowis
I don't believe this is a duplicate of https://issues.jenkins-ci.org/browse/JENKINS-28317 but I haven't been able to figure out if I have that version of the plugin installed on my jenkins, because I'm not sure what plugin that applies to.

 

I had a `withEnv` section that looked like:


{ { code:groovy}
def flowContext(cl) {
    withEnv([
        'npm_config_cache=npm-cache',
        'npm_config_registry=http://npm.artifactory.mycompany.com/artifactory/api/npm/npm',
        'http_proxy=http://
www- proxy.mycompany.com:8080',
        'https_proxy=http://
www- proxy. homedepot mycompany .com:8080',
        'HTTP_PROXY=http://
www- proxy.mycompany.com:8080',
        'HTTPS_PROXY=http://
www- proxy.mycompany.com:8080',
        'PROXY_HOST=
www- proxy.mycompany.com',

{code } }

The credentials context is another function that exports a few credentials into the environment.
All of the credentials would be there, but not all of the environment variables.

This would be executed within a docker container like so:

{ { code:groovy}
    stage('
pip-ci-node Node Build') {

        docker.image(ciNode).inside() {
            //Do the node installs in parallel, because the npm install is the slowest part, and we can use lots of cpu
            parallel(
                "Root Npm Install/Build": {
                    flowContext {
                     sh('env')
                     sh('npm install')
                     sh('gulp')
                    }
                },
                "webapp/static Install/Build": {
                    flowContext {
                     echo "Due to bug: https://issues.jenkins-ci.org/browse/JENKINS-33510"
                     sh('cd src/main/webapp/static && npm install')
                     sh('cd src/main/webapp/static && grunt prod')
                    }
                },
                failFast: true
            )
        }
    }

{code
} }

 
The {{sh('env')}} command within the flowContext would never show the {{no_proxy}} environment variable. I eventually gave up on trying to get it set in the Jenkinsfile, and instead put it in the root container image.

dkowis@shlrm.org (JIRA)

unread,
Mar 23, 2017, 3:16:02 PM3/23/17
to jenkinsc...@googlegroups.com
David Kowis updated an issue
I don't believe this is a duplicate of https://issues.jenkins-ci.org/browse/JENKINS-28317 but I haven't been able to figure out if I have that version of the plugin installed on my jenkins, because I'm not sure what plugin that applies to.

 

I had a `withEnv` section that looked like:


{code: groovy none }

def flowContext(cl) {
    withEnv([
        'npm_config_cache=npm-cache',
        'npm_config_registry=http://npm.artifactory.mycompany.com/artifactory/api/npm/npm',
        'http_proxy=http://proxy.mycompany.com:8080',
        'https_proxy=http://proxy.mycompany.com:8080',
        'HTTP_PROXY=http://proxy.mycompany.com:8080',
        'HTTPS_PROXY=http://proxy.mycompany.com:8080',
        'PROXY_HOST=proxy.mycompany.com',
{code: groovy none }
    stage('Node Build') {

        docker.image(ciNode).inside() {
            //Do the node installs in parallel, because the npm install is the slowest part, and we can use lots of cpu
            parallel(
                "Root Npm Install/Build": {
                    flowContext {
                     sh('env')
                     sh('npm install')
                     sh('gulp')
                    }
                },
                "webapp/static Install/Build": {
                    flowContext {
                     echo "Due to bug: https://issues.jenkins-ci.org/browse/JENKINS-33510"
                     sh('cd src/main/webapp/static && npm install')
                     sh('cd src/main/webapp/static && grunt prod')
                    }
                },
                failFast: true
            )
        }
    }

{code}


The {{sh('env')}} command within the flowContext would never show the {{no_proxy}} environment variable. I eventually gave up on trying to get it set in the Jenkinsfile, and instead put it in the root container image.

andrew.bayer@gmail.com (JIRA)

unread,
Mar 23, 2017, 9:29:02 PM3/23/17
to jenkinsc...@googlegroups.com
Andrew Bayer updated an issue
Change By: Andrew Bayer
Component/s: workflow-basic-steps-plugin
Component/s: pipeline

Aaron.Marasco@BIA-Boeing.com (JIRA)

unread,
Feb 9, 2018, 9:24:02 AM2/9/18
to jenkinsc...@googlegroups.com
Aaron Marasco commented on Bug JENKINS-43077
 
Re: withEnv does not set some environment variables

If it helps, I'm even hitting this now in a non-Docker environment.

One call tree is: try => stage => parallel closure => node => timestamps => lock.

alexvolanis@gmail.com (JIRA)

unread,
Jan 1, 2020, 6:58:03 PM1/1/20
to jenkinsc...@googlegroups.com

I believe I identified the faulting code in the docker-workflow-plugin but I fail too see why this apparently intentional environment reduction is performed.

Here: https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/WithContainerStep.java#L134

and here: https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/WithContainerStep.java#L289

If I am reading this correctly docker containers executing pipeline will have none of the slave environment showing in the container environment.

I can understand PATH removal, it makes sense. But what if I am actively trying to override any environment that happens to be on the slave host with a direct withEnv call?

Why am I prevented from actually making this choice?

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

alexvolanis@gmail.com (JIRA)

unread,
Jan 1, 2020, 7:02:03 PM1/1/20
to jenkinsc...@googlegroups.com
Alexander Volanis edited a comment on Bug JENKINS-43077
I believe I identified the faulting code in the docker-workflow-plugin but I fail too see why this apparently intentional environment reduction is performed.

Here: [https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/WithContainerStep.java#L134]

and here: [https://github.com/jenkinsci/docker-workflow-plugin/blob/master/src/main/java/org/jenkinsci/plugins/docker/workflow/WithContainerStep.java#L289]

If I am reading this correctly docker containers executing pipeline will have none of the slave environment showing in the container environment.

I can understand PATH removal, it makes sense. But what if I am actively trying to override any environment that happens to be on the slave host with a direct withEnv call?

Why am I prevented from actually making this choice?


Would this be better logically to remove only if it has the same value not just the same key?

 

jglick@cloudbees.com (JIRA)

unread,
Jan 2, 2020, 9:57:03 AM1/2/20
to jenkinsc...@googlegroups.com

what if I am actively trying to override any environment that happens to be on the slave host with a direct withEnv call?

withEnv inside withDockerContainer should work for that case I guess.

Would this be better logically to remove only if it has the same value not just the same key?

Possibly.

In general, if you have any sort of subtle issues with withDockerContainer, my recommendation is simply not to use it. More transparent and reliable to just use sh 'docker run …' commands where you can precisely control the behavior according to Docker documentation.

Reply all
Reply to author
Forward
0 new messages