[JIRA] (JENKINS-56066) EC2 nodes fail to start after jenkins restart

amineko+jenkins@protonmail.com (JIRA)

unread,

Feb 9, 2019, 7:07:14 AM2/9/19

to jenkinsc...@googlegroups.com

Maja Bratseth created an issue

Jenkins /

JENKINS-56066

EC2 nodes fail to start after jenkins restart

Issue Type:	Bug
Assignee:	FABRIZIO MANFREDI
Components:	ec2-plugin
Created:	2019-02-09 12:06
Environment:	Jenkins ver. 2.150.2 org.jenkins-ci.main:jenkins-war:2.150.2 org.jenkins-ci:crypto-util:1.1 commons-httpclient:commons-httpclient:3.1-jenkins-1 aopalliance:aopalliance:1.0 com.google.inject:guice:4.0 org.jenkins-ci.modules:slave-installer:1.6 org.springframework:spring-dao:1.2.9 org.codehaus.groovy:groovy-all:2.4.12 org.kohsuke.stapler:stapler-groovy:1.256 org.jenkins-ci:constant-pool-scanner:1.2 org.connectbot.jbcrypt:jbcrypt:1.0.0 org.jenkins-ci.modules:ssh-cli-auth:1.4 org.ow2.asm:asm-commons:5.0.3 org.jenkins-ci:symbol-annotation:1.1 org.jenkins-ci.modules:windows-slave-installer:1.9.2 commons-digester:commons-digester:2.1 org.kohsuke:libpam4j:1.11 commons-io:commons-io:2.4 com.github.jnr:jnr-posix:3.0.45 org.kohsuke:trilead-putty-extension:1.2 org.jenkins-ci.modules:instance-identity:2.2 commons-codec:commons-codec:1.9 org.kohsuke:asm6:6.2 net.sf.kxml:kxml2:2.3.0 org.kohsuke:libzfs:0.8 org.jenkins-ci.ui:jquery-detached:1.2 org.kohsuke.stapler:json-lib:2.4-jenkins-2 org.jvnet.robust-http-client:robust-http-client:1.2 org.ow2.asm:asm:5.0.3 com.github.jnr:jnr-ffi:2.1.8 com.github.jnr:jnr-constants:0.9.9 org.kohsuke.stapler:stapler-adjunct-timeline:1.5 org.slf4j:log4j-over-slf4j:1.7.25 org.jenkins-ci:version-number:1.4 org.jvnet.hudson:commons-jelly-tags-define:1.0.1-hudson-20071021 org.jenkins-ci.main:jenkins-core:2.150.2 org.kohsuke.stapler:stapler:1.256 org.jenkins-ci:jmdns:3.4.0-jenkins-3 commons-lang:commons-lang:2.6 org.springframework:spring-jdbc:1.2.9 org.codehaus.woodstox:wstx-asl:3.2.9 org.springframework:spring-core:2.5.6.SEC03 org.springframework:spring-aop:2.5.6.SEC03 org.samba.jcifs:jcifs:1.3.17-kohsuke-1 org.apache.ant:ant:1.9.2 com.sun.solaris:embedded_su4j:1.1 com.github.jnr:jffi:1.2.17 javax.inject:javax.inject:1 org.jenkins-ci.modules:upstart-slave-installer:1.1 org.apache.commons:commons-compress:1.10 commons-beanutils:commons-beanutils:1.8.3 org.jvnet.localizer:localizer:1.24 org.fusesource.jansi:jansi:1.11 org.jenkins-ci.modules:sshd:2.4 org.springframework:spring-beans:2.5.6.SEC03 net.java.sezpoz:sezpoz:1.13 javax.xml.stream:stax-api:1.0-2 org.jenkins-ci:winstone:5.1 org.slf4j:slf4j-jdk14:1.7.25 org.jvnet.hudson:activation:1.1.1-hudson-1 commons-jelly:commons-jelly-tags-fmt:1.0 net.i2p.crypto:eddsa:0.2.0 jfree:jfreechart:1.0.9 org.kohsuke.stapler:stapler-jelly:1.256 oro:oro:2.0.8 org.jenkins-ci:commons-jexl:1.1-jenkins-20111212 org.jenkins-ci.plugins.icon-shim:icon-set:1.0.5 stax:stax-api:1.0.1 com.google.code.findbugs:jsr305:2.0.1 org.kohsuke:windows-package-checker:1.2 org.acegisecurity:acegi-security:1.0.7 commons-fileupload:commons-fileupload:1.3.1-jenkins-2 org.jenkins-ci.modules:launchd-slave-installer:1.2 org.jenkins-ci:annotation-indexer:1.12 jline:jline:2.12 org.jenkins-ci:task-reactor:1.5 org.kohsuke.stapler:stapler-adjunct-zeroclipboard:1.3.5-1 org.kohsuke.stapler:stapler-adjunct-codemirror:1.3 org.ow2.asm:asm-util:5.0.3 org.jenkins-ci:bytecode-compatibility-transformer:2.0-beta-2 org.apache.sshd:sshd-core:1.7.0 org.slf4j:jcl-over-slf4j:1.7.25 org.kohsuke:akuma:1.10 javax.mail:mail:1.4.4 org.hamcrest:hamcrest-core:1.3 jfree:jcommon:1.0.12 org.springframework:spring-context-support:2.5.6.SEC03 com.google.guava:guava:11.0.1 org.jvnet.hudson:jtidy:4aug2000r7-dev-hudson-1 org.jenkins-ci:commons-jelly:1.1-jenkins-20120928 org.jenkins-ci.ui:handlebars:1.1.1 org.springframework:spring-context:2.5.6.SEC03 org.jenkins-ci.ui:jquery-detached:1.2.1 org.ow2.asm:asm-analysis:5.0.3 io.github.stephenc.crypto:self-signed-cert-generator:1.0.0 javax.servlet.jsp.jstl:javax.servlet.jsp.jstl-api:1.2.1 commons-discovery:commons-discovery:0.4 com.github.jnr:jffi:1.2.16 org.jenkins-ci.dom4j:dom4j:1.6.1-jenkins-4 org.jenkins-ci:memory-monitor:1.9 org.jenkins-ci.modules:systemd-slave-installer:1.1 org.jvnet.hudson:xstream:1.4.7-jenkins-1 org.jvnet:tiger-types:2.2 com.sun.xml.txw2:txw2:20110809 org.springframework:spring-web:2.5.6.SEC03 org.kohsuke.stapler:stapler-jrebel:1.256 org.kohsuke.jinterop:j-interop:2.0.6-kohsuke-1 org.jvnet.winp:winp:1.27 org.jenkins-ci.main:cli:2.150.2 org.jruby.ext.posix:jna-posix:1.0.3-jenkins-1 org.apache.ant:ant-launcher:1.9.2 net.java.dev.jna:jna:4.5.2 javax.annotation:javax.annotation-api:1.2 org.kohsuke.jinterop:j-interopdeps:2.0.6-kohsuke-1 org.jenkins-ci.main:remoting:3.27 com.infradna.tool:bridge-method-annotation:1.13 org.ow2.asm:asm-tree:5.0.3 org.kohsuke:asm5:5.0.1 io.jenkins.stapler:jenkins-stapler-support:1.0 args4j:args4j:2.0.31 antlr:antlr:2.7.6 relaxngDatatype:relaxngDatatype:20020414 com.jcraft:jzlib:1.1.3-kohsuke-1 org.jenkins-ci.ui:bootstrap:1.3.2 org.kohsuke:access-modifier-annotation:1.14 commons-collections:commons-collections:3.2.2 junit:junit:4.12 org.jenkins-ci:trilead-ssh2:build-217-jenkins-11 org.slf4j:slf4j-api:1.7.25 net.sf.ezmorph:ezmorph:1.0.6 org.springframework:spring-webmvc:2.5.6.SEC03 com.github.jnr:jnr-x86asm:1.0.2 xpp3:xpp3:1.1.4c jaxen:jaxen:1.1-beta-11 commons-jelly:commons-jelly-tags-xml:1.1
Priority:	Major
Reporter:	Maja Bratseth

After restarting jenkins, EC2 nodes fail to start with the following stacktrace:

ERROR: Unexpected error in launching an agent. This is probably a bug in Jenkins
java.lang.NullPointerException
at java.lang.ProcessEnvironment.validateValue(ProcessEnvironment.java:120)
at java.lang.ProcessEnvironment.access$400(ProcessEnvironment.java:61)
at java.lang.ProcessEnvironment$Value.valueOf(ProcessEnvironment.java:203)
at java.lang.ProcessEnvironment$StringEnvironment.put(ProcessEnvironment.java:243)
at java.lang.ProcessEnvironment$StringEnvironment.put(ProcessEnvironment.java:221)
at hudson.slaves.CommandLauncher.launch(CommandLauncher.java:130)
at hudson.plugins.ec2.ssh.EC2UnixLauncher.launchScript(EC2UnixLauncher.java:209)
at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:48)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

If we go to /configure and just save the current configuration, the nodes will successfully start after that. Since we have many nodes, this workaround is quite impractical.

Same problem but different stacktrace: https://issues.jenkins-ci.org/browse/JENKINS-55473