[JIRA] [core] (JENKINS-27739) Changes to slave environment variables are ignored by master

31 views
Skip to first unread message

milo@cyberlifelabs.com (JIRA)

unread,
Apr 3, 2015, 2:25:25 PM4/3/15
to jenkinsc...@googlegroups.com
Issue Type: Bug Bug
Assignee: Unassigned
Components: core
Created: 03/Apr/15 6:23 PM
Description:

A slave's environment variables seem to be cached upon first connect and then never considered again. This prevents legitimate changes to a slave's environment from having any effect on builds. The only workaround I can find is to delete a slave and recreate it, which in a complex environment is unacceptable.

This may be a consequence of JENKINS-26755.
Environment: Jenkins 1.606
Project: Jenkins
Labels: envinronment-variables slaves
Priority: Major Major
Reporter: Milo Hyson
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

daniel@beckweb.net (JIRA)

unread,
Apr 3, 2015, 4:24:27 PM4/3/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-27739

Issue is missing installed plugins and their versions. Issue is missing specific steps to (easily) reproduce the issue.

milo@cyberlifelabs.com (JIRA)

unread,
Apr 3, 2015, 4:56:27 PM4/3/15
to jenkinsc...@googlegroups.com
Milo Hyson commented on Bug JENKINS-27739

Installed Plugins:

Ant Plugin (ant) v1.2
buildgraph-view (buildgraph-view) v1.1.1
CloudBees Build Flow plugin (build-flow-plugin) v0.16
conditional-buildstep (conditional-buildstep) v1.3.3
Copy Artifact Plugin (copyartifact) v1.32.1
Credentials Binding Plugin (credentials-binding) v1.3
Credentials Plugin (credentials) v1.21
Email Extension Plugin (email-ext) v2.39
embeddable-build-status (embeddable-build-status) v1.5
Environment Injector Plugin (envinject) v1.90
External Monitor Job Type Plugin (external-monitor-job) v1.4
GitHub API Plugin (github-api) v1.59
GitHub plugin (github) v1.10
GitHub Pull Request Builder (ghprb) v1.16-5
Green Balls (greenballs) v1.14
Javadoc Plugin (javadoc) v1.3
Jenkins Active Directory plugin (active-directory) v1.39
Jenkins build timeout plugin (build-timeout) v1.14
Jenkins Cobertura Plugin (cobertura) v1.9.6
Jenkins CVS Plug-in (cvs) v2.12
Jenkins description setter plugin (description-setter) v1.9
Jenkins Exclusion Plug-in (Exclusion) v0.10
Jenkins GIT client plugin (git-client) v1.11.1
Jenkins GIT plugin (git) v2.3
Jenkins Gradle plugin (gradle) v1.24
Jenkins Gravatar plugin (gravatar) v2.1
Jenkins Job Configuration History Plugin (jobConfigHistory) v2.10
Jenkins jQuery plugin (jquery) v1.7.2-1
Jenkins Mailer Plugin (mailer) v1.12
Jenkins Multijob plugin (jenkins-multijob-plugin) v1.15
Jenkins Parameterized Trigger plugin (parameterized-trigger) v2.25
Jenkins Priority Sorter Plugin (PrioritySorter) v2.9
Jenkins promoted builds plugin (promoted-builds) v2.19
Jenkins SSH Slaves plugin (ssh-slaves) v1.9
Jenkins Subversion Plug-in (subversion) v2.4.5
Jenkins TextFinder plugin (text-finder) v1.10
Jenkins Throttle Concurrent Builds Plug-in (throttle-concurrents) v1.8.4
Jenkins Translation Assistance plugin (translation) v1.12
Job Import Plugin (job-import-plugin) v1.2
Join plugin (join) v1.15
JUnit Plugin (junit) v1.2
LDAP Plugin (ldap) v1.11
MapDB API Plugin (mapdb-api) v1.0.6.0
Matrix Authorization Strategy Plugin (matrix-auth) v1.2
Matrix Project Plugin (matrix-project) v1.4
Maven Integration plugin (maven-plugin) v2.7.1
Monitoring (monitoring) v1.53.1
Next Build Number Plugin (next-build-number) v1.1
Node and Label parameter plugin (nodelabelparameter) v1.5.1
NodeJS Plugin (nodejs) v0.2.1
OWASP Markup Formatter Plugin (antisamy-markup-formatter) v1.3
PAM Authentication plugin (pam-auth) v1.2
Performance plugin (performance) v1.11
Plain Credentials Plugin (plain-credentials) v1.1
Plot plugin (plot) v1.8
Publish Over SSH (publish-over-ssh) v1.12
Run Condition Plugin (run-condition) v1.0
SCM API Plugin (scm-api) v0.2
Script Security Plugin (script-security) v1.13
Semantic Versioning Plugin (semantic-versioning-plugin) v1.7
Simple Theme Plugin (simple-theme-plugin) v0.3
SSH Agent Plugin (ssh-agent) v1.5
SSH Credentials Plugin (ssh-credentials) v1.10
Token Macro Plugin (token-macro) v1.10
Windows Slaves Plugin (windows-slaves) v1.0
Workflow: Step API (workflow-step-api) v1.2
xUnit plugin (xunit) v1.92
youtrack-plugin (youtrack-plugin) v0.6.3

Reproduction:

1) Add a slave node to a master connecting via JNLP.
2) Start slave.jar on the slave machine.
3) On the master, view the new node's system information and note the PATH environment variable.
4) Kill slave.jar.
5) Change the PATH on the slave machine.
6) Restart slave.jar.
7) Look again at the node's system information on the master and note the PATH variable has not changed.

daniel@beckweb.net (JIRA)

unread,
Apr 3, 2015, 5:21:26 PM4/3/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-27739

Weird. For me the only affected variable seems to be PATH.

daniel@beckweb.net (JIRA)

unread,
Apr 3, 2015, 5:23:26 PM4/3/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-27739

Could you confirm the above on your environment?

Is this a regression (i.e. did it ever work for you)? If so, what is the newest version know to not be affected?

milo@cyberlifelabs.com (JIRA)

unread,
Apr 3, 2015, 8:38:25 PM4/3/15
to jenkinsc...@googlegroups.com
Milo Hyson commented on Bug JENKINS-27739

I just used PATH as an example. In my actual case I had three custom variables set on the slave machine. When I removed them and restarted slave.jar, those variables remained. I made other changes to test my suspicions (e.g. altering PATH) and they too remained unaltered.

It's not a regression to my knowledge. We've never needed to change slave variables before.

daniel@beckweb.net (JIRA)

unread,
Apr 3, 2015, 8:45:26 PM4/3/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-27739

My basic 1.607 without any custom plugins had different behavior: Only PATH didn't change (unfortunately I didn't confirm it was set correctly in the shell, so may have been user error), the new variables I defined were correctly updated.

Could you test whether the same issue occurs in the following configurations:

  • Jenkins 1.599 with Env-Inject 1.90
  • Jenkins 1.599 with Env-Inject not installed
  • Jenkins 1.606 with Env-Inject not installed

There was a specific change in 1.600 related to slave caching, but it should clear slave environment after restarting.

As there are, and were, numerous issues with env-inject being weird, knowing whether it affects things either depending on or independent of Jenkins version would be very helpful.

milo@cyberlifelabs.com (JIRA)

unread,
Apr 6, 2015, 3:32:44 PM4/6/15
to jenkinsc...@googlegroups.com
Milo Hyson commented on Bug JENKINS-27739

Looks like the problem might have occurred around v1.606. For each of the tests I conducted the following procedure:

1) Started with the following environment variables on the slave machine
PATH = /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
SOME_ENVVAR = value
2) Launched java -jar jenkins.war on the master machine
3) Added a new node to the master
4) Started slave.jar on the slave
5) Ensured PATH and SOME_ENVVAR showed up on master
6) Killed slave.jar
7) Appended :/nonexistent to PATH
8) Removed SOME_ENVVAR
9) Ensured variables changed on slave machine
10) Restarted slave.jar
11) Refreshed node's system-information page on master

The following were the results:

v1.599 without EnvInject
PATH updated on master
SOME_ENVVAR disappeared from master

v1.599 with EnvInject
PATH updated on master
SOME_ENVVAR disappeared from master

v1.606 with EnvInject
PATH did not update on master
SOME_ENVVAR remained on master

milo@cyberlifelabs.com (JIRA)

unread,
Apr 6, 2015, 3:32:43 PM4/6/15
to jenkinsc...@googlegroups.com
 
Milo Hyson edited a comment on Bug JENKINS-27739

Looks like the problem might have occurred around v1.606. For each of the tests I conducted the following procedure:

1) Started with the following environment variables on the slave machine
PATH = /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
SOME_ENVVAR = value
2) Launched java -jar jenkins.war on the master machine
3) Added a new node to the master
4) Started slave.jar on the slave
5) Ensured PATH and SOME_ENVVAR showed up on master
6) Killed slave.jar
7) Appended :/nonexistent to PATH
8) Removed SOME_ENVVAR
9) Ensured variables changed on slave machine
10) Restarted slave.jar
11) Refreshed node's system-information page on master

The following were the results:

v1.599 without EnvInject
PATH updated on master
SOME_ENVVAR disappeared from master

v1.599 with EnvInject
PATH updated on master
SOME_ENVVAR disappeared from master

v1.606 with EnvInject
PATH did not update on master
SOME_ENVVAR remained on master
This message is automatically generated by JIRA.

milo@cyberlifelabs.com (JIRA)

unread,
Apr 6, 2015, 3:34:42 PM4/6/15
to jenkinsc...@googlegroups.com
 
Milo Hyson edited a comment on Bug JENKINS-27739

Looks like the problem might have occurred around v1.606. For each of the tests I conducted the following procedure:

  1. Started with the following environment variables on the slave machine
    • PATH = /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
    • SOME_ENVVAR = value
  1. Launched java -jar jenkins.war on the master machine
  1. Added a new node to the master
  1. Started slave.jar on the slave
  1. Ensured PATH and SOME_ENVVAR showed up on master
  1. Killed slave.jar
  2. Appended :/nonexistent to PATH
  3. Removed SOME_ENVVAR
  1. Ensured variables changed on slave machine
  1. Restarted slave.jar
  1. Refreshed node's system-information page on master

The following were the results:

  • v1.599 without EnvInject
    • PATH updated on master
    • SOME_ENVVAR disappeared from master
  • v1.599 with EnvInject
    • PATH updated on master
    • SOME_ENVVAR disappeared from master
  • v1.606 with EnvInject
    • PATH did not update on master
    • SOME_ENVVAR remained on master

milo@cyberlifelabs.com (JIRA)

unread,
Apr 6, 2015, 3:34:42 PM4/6/15
to jenkinsc...@googlegroups.com
 
Milo Hyson edited a comment on Bug JENKINS-27739

Looks like the problem might have occurred around v1.606. For each of the tests I conducted the following procedure:

  1. Started with the following environment variables on the slave machine
    PATH = /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
    SOME_ENVVAR = value
  2. Launched java -jar jenkins.war on the master machine
  3. Added a new node to the master
  4. Started slave.jar on the slave
  5. Ensured PATH and SOME_ENVVAR showed up on master
  6. Killed slave.jar
  7. Appended :/nonexistent to PATH
  8. Removed SOME_ENVVAR
  9. Ensured variables changed on slave machine
  10. Restarted slave.jar
  11. Refreshed node's system-information page on master

The following were the results:

v1.599 without EnvInject
PATH updated on master
SOME_ENVVAR disappeared from master

v1.599 with EnvInject
PATH updated on master
SOME_ENVVAR disappeared from master

v1.606 with EnvInject
PATH did not update on master
SOME_ENVVAR remained on master
This message is automatically generated by JIRA.

funeeldy@yahoo.com (JIRA)

unread,
Apr 15, 2015, 9:29:05 AM4/15/15
to jenkinsc...@googlegroups.com

we are seeing the same behavior, although I did not try to delete the slave and recreate it. We keep having to restart the jenkins server. I also tried to uninstall the envinject plugin, but we have a multijob plugin that depends on it. When I restarted jenkins server, none of my jobs would load due to the missing envinject plugin, so I had to reinstall it.

achittur@mercenarytechnology.com (JIRA)

unread,
May 1, 2015, 9:38:40 PM5/1/15
to jenkinsc...@googlegroups.com
A C commented on Bug JENKINS-27739

Verified the same behavior on Jenkins 1.611 with EnvInject.

Changing a slave node's PATH has no effect on what Jenkins sets the PATH to during a build even after a slave service restart. Restarting the slave operating system still has no effect, the PATH is not refreshed. Restarting the master Jenkins server finally did allow PATH to get refreshed.

The server is definitely caching an old PATH and then re-assigning it (via EnvInject?) to the slave node process. This is very bad.

Since changing environment variables necessitates at least a slave process restart, it would seem the proper fix is to refresh the master server's cache of a node's environment during slave-service startup.

daniel@beckweb.net (JIRA)

unread,
May 2, 2015, 3:51:41 AM5/2/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-27739

Could I get confirmation from someone that this issue only occurs when envinject is installed? Milo, could you try "v1.606 without EnvInject" as well?

milo@cyberlifelabs.com (JIRA)

unread,
May 4, 2015, 1:18:01 PM5/4/15
to jenkinsc...@googlegroups.com
Milo Hyson commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

Just tried the above procedure on v1.606 without EnvInject. Problem still occurred.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

milo@cyberlifelabs.com (JIRA)

unread,
May 4, 2015, 4:44:01 PM5/4/15
to jenkinsc...@googlegroups.com

I've only just started looking at the source, so I may be off base here. But it seems to me that hudson.model.Computer is taking a snapshot upon first request and then never querying the slave again:

public EnvVars getEnvironment() throws IOException, InterruptedException {
        EnvVars cachedEnvironment = this.cachedEnvironment;
        if (cachedEnvironment != null) {
            return new EnvVars(cachedEnvironment);
        }

        cachedEnvironment = EnvVars.getRemote(getChannel());
        this.cachedEnvironment = cachedEnvironment;
        return new EnvVars(cachedEnvironment);
    }

Looking at the context for the introduction of this code I understand there was a desire to minimize network calls, but I would think there are times when the cost is worth it. At the start of a build, for instance, or when the system-information is requested. These are relative one-offs that would not unduly suffer from a second or two of delay.

W.BelgraverThissen@SpiritIT.com (JIRA)

unread,
May 5, 2015, 10:49:01 AM5/5/15
to jenkinsc...@googlegroups.com

I just ran into this issue as well after updating one of our tools to a new version that required an update of an environment variable to find it.

W.BelgraverThissen@SpiritIT.com (JIRA)

unread,
May 5, 2015, 10:51:02 AM5/5/15
to jenkinsc...@googlegroups.com
I just ran into this issue as well after updating one of our tools to a new version that required an update of an environment variable to find it.  For now as a workaround I have overruled the variable in the agent settings and that seems to work.

david.rubio@seebyte.com (JIRA)

unread,
May 13, 2015, 5:03:01 AM5/13/15
to jenkinsc...@googlegroups.com

We have the same problem with Jenkins 1.613 (and older versions too) with EnvInject 1.90. This is a pretty bad issue for us because we keep changing the variables of one of our slaves when installing new libraries. Hopefully it'll be fixed soon.

releng@bullhorn.com (JIRA)

unread,
Jun 3, 2015, 12:19:01 PM6/3/15
to jenkinsc...@googlegroups.com

We have envinject 1.91.2 and jenkins 1.606 here (CentOS master, windows 7 slave) and I see the same behaviors. The path the node has is clearly onle from an earlier configuration, and doesn't include some new items from the system path. I have changed the configuration to NOT unset the system environment variables at the global and node/slave level, to no avail. I have not tried recreating the slave (I will try that now).

Actually, I just tried to reset the node configuration settings, and I see under "Prepare jobs environment" that the "Unset System Environment Variables" setting is checked. Same for the global "Prepare jobs environment" -> "Unset System Environment Variables" check box.

releng@bullhorn.com (JIRA)

unread,
Jun 3, 2015, 1:43:02 PM6/3/15
to jenkinsc...@googlegroups.com

Recreating the slave picked up the new PATH settings.

achittur@mercenarytechnology.com (JIRA)

unread,
Jun 3, 2015, 1:43:02 PM6/3/15
to jenkinsc...@googlegroups.com
A C commented on Bug JENKINS-27739

This issue is independent of EnvInject - I consistently reproduce this on Jenkins 1.6.09+ without EnvInject.

This is a very annoying bug indeed.

dbeck@cloudbees.com (JIRA)

unread,
Jun 3, 2015, 3:34:03 PM6/3/15
to jenkinsc...@googlegroups.com
Daniel Beck assigned an issue to Daniel Beck
 
Jenkins / Bug JENKINS-27739
Change By: Daniel Beck
Assignee: Daniel Beck

dbeck@cloudbees.com (JIRA)

unread,
Jun 3, 2015, 3:35:03 PM6/3/15
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: envinronment-variables  lts-candidate  slaves

giladbaru@gmail.com (JIRA)

unread,
Jun 4, 2015, 8:17:01 AM6/4/15
to jenkinsc...@googlegroups.com
Gilad Baruchian commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

I have the same issue.
Adding PATH=<my required path>;%PATH% to a batch build step fixed it.
Also uninstalling the slave using :
installUtil /u c:\jenkins\jenkins-slave.exe
sc delete jenkinsslave-c__jenkins
and then recreating the slave fixed it.

This makes it very hard to manage changes to PATH in the machine, I hope it will be resolved soon.

dbeck@cloudbees.com (JIRA)

unread,
Jun 5, 2015, 7:47:03 AM6/5/15
to jenkinsc...@googlegroups.com
Daniel Beck started work on Bug JENKINS-27739
 
Change By: Daniel Beck
Status: Open In Progress

scm_issue_link@java.net (JIRA)

unread,
Jun 6, 2015, 12:36:07 PM6/6/15
to jenkinsc...@googlegroups.com
SCM/JIRA link daemon resolved as Fixed
 
Change By: SCM/JIRA link daemon
Status: In Progress Resolved
Resolution: Fixed

scm_issue_link@java.net (JIRA)

unread,
Jun 6, 2015, 12:36:07 PM6/6/15
to jenkinsc...@googlegroups.com
SCM/JIRA link daemon commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

Code changed in jenkins
User: Daniel Beck
Path:
core/src/main/java/hudson/model/Computer.java
http://jenkins-ci.org/commit/jenkins/c569036fca43f286ebbb80498f3f0937766e44c5
Log:
[FIX JENKINS-27739] Clear cached env vars when node goes online

scm_issue_link@java.net (JIRA)

unread,
Jun 6, 2015, 12:36:09 PM6/6/15
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:
core/src/main/java/hudson/model/Computer.java

http://jenkins-ci.org/commit/jenkins/c6d4204af1db78a6b1ba8cdd5f40f61756a1d009
Log:

JENKINS-27739 Ensure cache clearing listener runs first

scm_issue_link@java.net (JIRA)

unread,
Jun 6, 2015, 12:36:09 PM6/6/15
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:
core/src/main/java/hudson/model/Computer.java

http://jenkins-ci.org/commit/jenkins/31208c3dc2682fe04f5008dcd794fb39e41d6b13
Log:
Merge pull request #1728 from daniel-beck/

JENKINS-27739

[FIX JENKINS-27739] Clear cached env vars when node goes online

dbeck@cloudbees.com (JIRA)

unread,
Jun 6, 2015, 12:38:03 PM6/6/15
to jenkinsc...@googlegroups.com

Merged towards 1.617.

dogfood@java.net (JIRA)

unread,
Jun 8, 2015, 1:59:07 AM6/8/15
to jenkinsc...@googlegroups.com
dogfood commented on Bug JENKINS-27739

Integrated in jenkins_main_trunk #4168

Result = UNSTABLE

ogondza@gmail.com (JIRA)

unread,
Jun 9, 2015, 3:36:03 AM6/9/15
to jenkinsc...@googlegroups.com
Oliver Gondža updated an issue
 
Change By: Oliver Gondža
Labels: 1.609.2-fixed envinronment-variables  lts-candidate  slaves

scm_issue_link@java.net (JIRA)

unread,
Jun 9, 2015, 4:02:10 AM6/9/15
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:
core/src/main/java/hudson/model/Computer.java


[FIX JENKINS-27739] Clear cached env vars when node goes online

(cherry picked from commit c569036fca43f286ebbb80498f3f0937766e44c5)

scm_issue_link@java.net (JIRA)

unread,
Jun 9, 2015, 4:02:12 AM6/9/15
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:
core/src/main/java/hudson/model/Computer.java

JENKINS-27739 Ensure cache clearing listener runs first

(cherry picked from commit c6d4204af1db78a6b1ba8cdd5f40f61756a1d009)

jonathan.winch@realvnc.com (JIRA)

unread,
Jul 3, 2015, 6:40:02 AM7/3/15
to jenkinsc...@googlegroups.com
Jonathan Winch updated an issue
 
Change By: Jonathan Winch
Labels: 1.609.2-fixed envinronment-variables  lts-candidate  slaves

dbeck@cloudbees.com (JIRA)

unread,
Jul 3, 2015, 6:49:06 AM7/3/15
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 1.609.2-fixed  envinronment  environment -variables slaves

dbeck@cloudbees.com (JIRA)

unread,
Jul 3, 2015, 6:49:08 AM7/3/15
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 1.609.2-fixed envinronment-variables  lts-candidate  slaves

dbeck@cloudbees.com (JIRA)

unread,
Jul 3, 2015, 6:51:01 AM7/3/15
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

Jonathan Winch 1.609.2-fixed means it has already been picked up for LTS inclusion.

moshe.zvi@gmail.com (JIRA)

unread,
Jul 21, 2015, 9:10:02 AM7/21/15
to jenkinsc...@googlegroups.com
Moshe Zvi reopened an issue
 

Still happening on 1.619. Restarting the master indeed fixed it, but that's a bad resolution.
It would be better for this to remain in status Opened for others who encounter the same issue.
Change By: Moshe Zvi
Resolution: Fixed
Status: Resolved Reopened

funeeldy@yahoo.com (JIRA)

unread,
Jul 21, 2015, 9:29:02 AM7/21/15
to jenkinsc...@googlegroups.com

dbeck@cloudbees.com (JIRA)

unread,
Jul 21, 2015, 9:41:04 AM7/21/15
to jenkinsc...@googlegroups.com

Moshe Zvi How is the slave connected to Jenkins? JNLP, SSH slave, …? Does the issue occur with the Env-Inject plugin not installed?

oded@geek.co.il (JIRA)

unread,
Aug 12, 2015, 12:05:04 PM8/12/15
to jenkinsc...@googlegroups.com

I have the same problem with EC2 on demand Windows slaves. The slaves are installed and launched through WinRM ( I believe as a standard Java process).

giladbaru@gmail.com (JIRA)

unread,
Aug 12, 2015, 12:16:01 PM8/12/15
to jenkinsc...@googlegroups.com

Update : it works if I delete and create the slaves again but that is a real pain.
currently i just add an inject variable stage and i add path=<what i want>;%path%

kleinen@gmail.com (JIRA)

unread,
Aug 27, 2015, 4:58:04 PM8/27/15
to jenkinsc...@googlegroups.com

My team has also been experiencing this issue. Our server is running from RedHat Ent. Linux 6.5/Tomcat7 and we are seeing this on nodes that are both WinXP or Win7. We are not seeing this on nodes that are linux-based.

One important item we saw and have confirmed is that new environment variables get added/update/remove just fine, but environment variables at the time the node was first connected seem to be locked in and won't update/remove. Our current workaround is to delete the node in Jenkins and recreate it.

dbeck@cloudbees.com (JIRA)

unread,
Aug 27, 2015, 5:41:03 PM8/27/15
to jenkinsc...@googlegroups.com
Daniel Beck resolved as Fixed
 

Issue needs to occur without envinject installed to be a proven core issue (as was the problem that was fixed here a while back).

Please include much more information and don't just reopen without providing any. This guideline may help. And please make sure that your issue is identical to what was originally reported.
Change By: Daniel Beck
Status: Reopened Resolved
Resolution: Fixed

dogfood@java.net (JIRA)

unread,
Sep 24, 2015, 2:27:32 PM9/24/15
to jenkinsc...@googlegroups.com
dogfood commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

Integrated in jenkins_main_trunk #4292
[FIX JENKINS-27739] Clear cached env vars when node goes online (Revision aaf0afc54d4e320e2d75a4af6a263745d9de9928)

JENKINS-27739 Ensure cache clearing listener runs first (Revision ba69511c4775bb72feada0b2c0d3fd299f5179a7)

Result = UNSTABLE
ogondza : aaf0afc54d4e320e2d75a4af6a263745d9de9928
Files :

  • core/src/main/java/hudson/model/Computer.java

ogondza : ba69511c4775bb72feada0b2c0d3fd299f5179a7
Files :

  • core/src/main/java/hudson/model/Computer.java

jonathan.winch@realvnc.com (JIRA)

unread,
Sep 25, 2015, 11:46:02 AM9/25/15
to jenkinsc...@googlegroups.com
Jonathan Winch updated an issue
Change By: Jonathan Winch
Attachment: support.zip

jonathan.winch@realvnc.com (JIRA)

unread,
Sep 25, 2015, 11:47:04 AM9/25/15
to jenkinsc...@googlegroups.com
Jonathan Winch commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

I have a simple job that reports the slave’s environment variable, I have run this on our production server and can confirm that the environment variable never changes unless the slave is deleted and then re added.

This is being seen on LTS 1.609.1 – we know this was working in LTS 1.565.1, when we upgraded directly to 1.609.1 this issue started occurring.

Using the test system I have confirmed that the issue is still reproducible in the same manner as on the live system.
On the test system I have upgraded the EnvInject plugin from 1.91.3 to the latest version 1.92.1 and that did not fix it.
On the test system I upgraded to the latest LTS version 1.609.3, that did not fix it.
On the test system I removed the EnvInject plugin and that did not fix it.
On the test system I have rolled back to 1.596.3 by replacing the Jenkins.war file, at this point the environment variable is still not being picked up correctly.
On the test system I have upgraded to the latest LTS release 1.609.3 with the EnvInject plugin removed and that did not fix it.
On the test system I have upgraded to the latest release 1.630 with the EnjInject plugin removed and that did not fix it.

The master is an Ubuntu 12.04 64 bit vm, Jenkins has been installed from the Ubuntu package via apt, the slave used in this test is a Windows 7 x64 VM, I am configuring the slave and master + starting the builds via the latest version of Firefox.

I have installed the Support Core Plugin and the support zip from this is attached.

jonathan.winch@realvnc.com (JIRA)

unread,
Sep 25, 2015, 11:47:08 AM9/25/15
to jenkinsc...@googlegroups.com
Jonathan Winch updated an issue
Change By: Jonathan Winch
Comment:

jonathan.winch@realvnc.com (JIRA)

unread,
Sep 25, 2015, 11:47:16 AM9/25/15
to jenkinsc...@googlegroups.com
Jonathan Winch reopened an issue
 

I have a simple job that reports the slave’s environment variable, I have run this on our production server and can confirm that the environment variable never changes unless the slave is deleted and then re added.

This is being seen on LTS 1.609.1 – we know this was working in LTS 1.565.1, when we upgraded directly to 1.609.1 this issue started occurring.

Using the test system I have confirmed that the issue is still reproducible in the same manner as on the live system.
On the test system I have upgraded the EnvInject plugin from 1.91.3 to the latest version 1.92.1 and that did not fix it.
On the test system I upgraded to the latest LTS version 1.609.3, that did not fix it.
On the test system I removed the EnvInject plugin and that did not fix it.
On the test system I have rolled back to 1.596.3 by replacing the Jenkins.war file, at this point the environment variable is still not being picked up correctly.
On the test system I have upgraded to the latest LTS release 1.609.3 with the EnvInject plugin removed and that did not fix it.
On the test system I have upgraded to the latest release 1.630 with the EnjInject plugin removed and that did not fix it.

The master is an Ubuntu 12.04 64 bit vm, Jenkins has been installed from the Ubuntu package via apt, the slave used in this test is a Windows 7 x64 VM, I am configuring the slave and master + starting the builds via the latest version of Firefox.

I have installed the Support Core Plugin and the support zip from this is attached.
Change By: Jonathan Winch
Resolution: Fixed
Status: Resolved Reopened

dbeck@cloudbees.com (JIRA)

unread,
Sep 25, 2015, 8:48:02 PM9/25/15
to jenkinsc...@googlegroups.com
Daniel Beck resolved as Fixed
 

This fix was merged into 1.609.2 and 1.617. That's why this issue is labeled '1.609.2-fixed'. Experiments using 1.609.1 are irrelevant.

(Not sure why the LTS changelog didn't pick this up, but Kohsuke uses some weird generator for that, so I cannot even modify it to report the truth afterwards at the moment – this is being tracked as INFRA-264.)
Change By: Daniel Beck
Status: Reopened Resolved
Resolution: Fixed

michaelcox@hotmail.co.uk (JIRA)

unread,
Oct 1, 2015, 11:17:02 AM10/1/15
to jenkinsc...@googlegroups.com
Michael Cox commented on Bug JENKINS-27739
 
Re: Changes to slave environment variables are ignored by master

With Jenkins ver. 1.631 and 1.617, I see this issue.

Slave has been deleted / recreated a few times but it makes no difference.
I have envinject plugin version 1.92.1.

I have tried unticking the 'Unset environment variables' on the node setup page - makes no difference.
Killed master and restarted.
Disconnected slave and restarted.

This was working a couple of weeks ago and there's been no changes, to my recollection.

If I ssh in to the box and do echo $myvar, the value appears set, when when the job run, the value is unset (running as the same user)

michaelcox@hotmail.co.uk (JIRA)

unread,
Oct 1, 2015, 12:03:03 PM10/1/15
to jenkinsc...@googlegroups.com
Michael Cox edited a comment on Bug JENKINS-27739
With Jenkins ver. 1.631 and 1.617, I see this issue.

Slave has been deleted / recreated a few times but it makes no difference.
I have envinject plugin version 1.92.1.

I have tried unticking the 'Unset environment variables' on the node setup page - makes no difference.
Killed master and restarted.
Disconnected slave and restarted.

This was working a couple of weeks ago and there's been no changes, to my recollection.

If I ssh in to the box and do echo $myvar, the value appears set, when when the job run, the value is unset (running as the same user)


EDIT - slave is a mac, with the value set in ~/.bash_profile

dbeck@cloudbees.com (JIRA)

unread,
Oct 1, 2015, 6:31:04 PM10/1/15
to jenkinsc...@googlegroups.com

If this issue only occurs when envinject is installed, it's not this bug, and likely not a bug in core at all. Please look for a bug filed against envinject.

zionyx@gmail.com (JIRA)

unread,
Nov 30, 2015, 5:12:01 AM11/30/15
to jenkinsc...@googlegroups.com
KY Lee commented on Bug JENKINS-27739

This issue still persists on my 1.606 instance with EnvInject uninstalled. Had both server and slaves rebooted without avail.
I am more inclined to think that this issue is with Jenkins' core than with EnvInject.

kobihk@gmail.com (JIRA)

unread,
Oct 9, 2016, 11:10:08 AM10/9/16
to jenkinsc...@googlegroups.com

I updated environment variable but didn't see the updated value just after disconnect and reconnect the agent.

We use Jenkins 1.656

This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

nehaljw.kkd1@gmail.com (JIRA)

unread,
Nov 2, 2016, 4:14:03 AM11/2/16
to jenkinsc...@googlegroups.com

Please note that bringing the slave online/offline doesn't help. One has to disconnect the slave and relaunch the slave agent. This worked for me for Jenkins version: 1.651.2

paris_bp2s_ci_project@bnpparibas.com (JIRA)

unread,
Jan 23, 2017, 9:24:03 AM1/23/17
to jenkinsc...@googlegroups.com

Hello,

Same as Nehal, I have the problem with Jenkins 1.651.1.

paris_bp2s_ci_project@bnpparibas.com (JIRA)

unread,
Jan 23, 2017, 9:24:10 AM1/23/17
to jenkinsc...@googlegroups.com
Paris BP2S CI Project updated an issue
 
Change By: Paris BP2S CI Project
Comment:
Hello,

Same as Nehal, I have the problem with Jenkins 1.651.1.

paris_bp2s_ci_project@bnpparibas.com (JIRA)

unread,
Jan 23, 2017, 9:25:13 AM1/23/17
to jenkinsc...@googlegroups.com

Hello,

Same as Nehal, I have the problem with Jenkins 1.651.1.
Add Comment Add Comment
 

nedeljko@markoviconline.de (JIRA)

unread,
Nov 12, 2019, 12:04:08 PM11/12/19
to jenkinsc...@googlegroups.com

There still this behavior, within Jenkins version Jenkins ver. 2.190.1

I tried node deleting / creating, Jenkins restarting... Nothing helped    

 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

nedeljko@markoviconline.de (JIRA)

unread,
Nov 12, 2019, 12:14:05 PM11/12/19
to jenkinsc...@googlegroups.com
Nedeljko Markovic reopened an issue
 

this issue still exist in the Jenkins version Jenkins ver. 2.190.1
Change By: Nedeljko Markovic
Resolution: Fixed
Status: Resolved Reopened

dbeck@cloudbees.com (JIRA)

unread,
Nov 13, 2019, 4:01:05 AM11/13/19
to jenkinsc...@googlegroups.com
Daniel Beck assigned an issue to Unassigned
Change By: Daniel Beck
Assignee: Daniel Beck
Reply all
Reply to author
Forward
0 new messages