[JIRA] (JENKINS-56759) Unexpected executor death prevents builds

19 views
Skip to first unread message

brian.hewson@cd-adapco.com (JIRA)

unread,
Mar 26, 2019, 8:40:02 AM3/26/19
to jenkinsc...@googlegroups.com
brian hewson created an issue
 
Jenkins / Bug JENKINS-56759
Unexpected executor death prevents builds
Issue Type: Bug Bug
Assignee: Unassigned
Components: core
Created: 2019-03-26 12:39
Environment: Jenkins 2.138.3 master running on Ubutntu 16.04, java openjdk 1.8.0_191, running in a tomcat container, installed via package manager
330 connected slaves
the problem appeared on Windows 10 (1803) slaves running cygwin 2.5.2 for sshd access and oracle java 1.8.0_181
Priority: Minor Minor
Reporter: brian hewson

Problem as seen by user, asking for a new build of <job> has no effect (retry, rebuild, and new build)

 

Problem as seen from the logs:

Log entry 
WARNING: A new build could not be created in job build_win64
java.lang.IllegalStateException: JENKINS-23152: /var/lib/jenkins/jobs/build_win64/builds/865 already existed; will not overwrite with build_win64 #865
	at hudson.model.RunMap.put(RunMap.java:189)
	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:185)
	at hudson.model.AbstractProject.newBuild(AbstractProject.java:967)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1143)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:137)
	at hudson.model.Executor$1.call(Executor.java:365)
	at hudson.model.Executor$1.call(Executor.java:347)
	at hudson.model.Queue._withLock(Queue.java:1439)
	at hudson.model.Queue.withLock(Queue.java:1300)
	at hudson.model.Executor.run(Executor.java:347)

Mar 26, 2019 7:23:14 AM hudson.model.Executor run
SEVERE: Executor #0 for W1803-BUILD56: Unexpected executor death
java.lang.Error: java.lang.IllegalStateException: JENKINS-23152: /var/lib/jenkins/jobs/build_win64/builds/865 already existed; will not overwrite with build_win64 #865
	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:193)
	at hudson.model.AbstractProject.newBuild(AbstractProject.java:967)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1143)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:137)
	at hudson.model.Executor$1.call(Executor.java:365)
	at hudson.model.Executor$1.call(Executor.java:347)
	at hudson.model.Queue._withLock(Queue.java:1439)
	at hudson.model.Queue.withLock(Queue.java:1300)
	at hudson.model.Executor.run(Executor.java:347)
Caused by: java.lang.IllegalStateException: JENKINS-23152: /var/lib/jenkins/jobs/build_win64/builds/865 already existed; will not overwrite with build_win64 #865
	at hudson.model.RunMap.put(RunMap.java:189)
	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:185)
	... 8 more

 I tried building multiple times and can see multiple entries in the log, the build number continues to increment and the job continues to fail because of the problem on the build machine. When I took that build machine offline the job failed for the same reason on another build node 

later in the log 
Mar 26, 2019 7:26:58 AM jenkins.model.lazy.LazyBuildMixIn newBuild
WARNING: A new build could not be created in job build_win64
java.lang.IllegalStateException: JENKINS-23152: /var/lib/jenkins/jobs/build_win64/builds/867 already existed; will not overwrite with build_win64 #867
	at hudson.model.RunMap.put(RunMap.java:189)
	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:185)
	at hudson.model.AbstractProject.newBuild(AbstractProject.java:967)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1143)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:137)
	at hudson.model.Executor$1.call(Executor.java:365)
	at hudson.model.Executor$1.call(Executor.java:347)
	at hudson.model.Queue._withLock(Queue.java:1439)
	at hudson.model.Queue.withLock(Queue.java:1300)
	at hudson.model.Executor.run(Executor.java:347)

Mar 26, 2019 7:26:58 AM hudson.model.Executor run
SEVERE: Executor #0 for W1803-BUILD54: Unexpected executor death
java.lang.Error: java.lang.IllegalStateException: JENKINS-23152: /var/lib/jenkins/jobs/build_win64/builds/867 already existed; will not overwrite with build_win64 #867
	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:193)
	at hudson.model.AbstractProject.newBuild(AbstractProject.java:967)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1143)
	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:137)
	at hudson.model.Executor$1.call(Executor.java:365)
	at hudson.model.Executor$1.call(Executor.java:347)
	at hudson.model.Queue._withLock(Queue.java:1439)
	at hudson.model.Queue.withLock(Queue.java:1300)
	at hudson.model.Executor.run(Executor.java:347)
Caused by: java.lang.IllegalStateException: JENKINS-23152: /var/lib/jenkins/jobs/build_win64/builds/867 already existed; will not overwrite with build_win64 #867
	at hudson.model.RunMap.put(RunMap.java:189)
	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:185)
	... 8 more

Trying again resulted in a self-repaired executor and the job running successfully. 

 

This problem was fixed in 2.13 (JENKINS-27530)

 

This system has been running 2.138.3 for several months, I haven't seen this problem since the system upgrade, although users could have seen the build failure and not reported it, or retried with success.

 

Path to reproduce: Unclear.

Workaround: Take 'Dead Executor' slave offline and disconnect, then reconnect and try the job again. 
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 5:59:02 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino commented on Bug JENKINS-56759
 
Re: Unexpected executor death prevents builds

I experienced this very error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in this issue.  

 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

I am experiencing the errors on Windows slave machines using a Debian Master.

The windows service user permissions appear correct using the service properties menu.

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:00:01 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this very error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in this  [ issue |https://issues . jenkins-ci.org/browse/JENKINS-29902].   


 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

I am experiencing the errors on Windows slave machines using a Debian Master.

The windows service user permissions appear correct using the service properties menu.

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:10:04 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in this issue.  


 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

I am experiencing the errors on Windows slave machines using a Debian Master Jenkins Ver: 2 . 176.1

Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

 The windows jenkins service user permissions appear correct using via the service properties menu.

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:10:04 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in this the   JENKINS-29902 issue.  


 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

Jenkins Ver: 2.176.1


Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

The windows jenkins service user permissions appear correct via the service properties menu.

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:12:01 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in the JENKINS-29902 issue.  


 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

Jenkins Ver: 2.176.1

Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

The same configuration is used successfully using another Debian 9 server with the same slave units.

The windows jenkins service user permissions appear correct via the service properties menu.

 

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:13:02 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in the JENKINS-29902 issue.  

 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

Jenkins Ver: 2.176.1

Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

The same configuration is used successfully using another a diffferent Debian 9 server with the same slave units unit machines .


The windows jenkins service user permissions appear correct via the service properties menu.

 

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:15:02 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino updated an issue
 
Change By: James Rubino
Priority: Minor Blocker

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:17:02 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino updated an issue
Change By: James Rubino
Priority: Blocker Major

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:20:02 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in the JENKINS-29902 issue.  

 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 
JobClass: Freestyle
Jenkins Ver: 2.176.1

Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

The same configuration is used successfully using a diffferent Debian 9 server with the same slave unit machines.


The windows jenkins service user permissions appear correct via the service properties menu.

 

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:33:02 PM7/24/19
to jenkinsc...@googlegroups.com

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 24, 2019, 6:34:02 PM7/24/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in the JENKINS-29902 issue.  

 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 

JobClass: Freestyle

Jenkins Ver: 2.176.1

Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

The same configuration is used successfully using a diffferent different Debian 9 server with the same slave unit machines.


The windows jenkins service user permissions appear correct via the service properties menu.

 

james.rubino+jenkinsjira@gmail.com (JIRA)

unread,
Jul 25, 2019, 10:15:02 AM7/25/19
to jenkinsc...@googlegroups.com
James Rubino edited a comment on Bug JENKINS-56759
I experienced this error during a server change-over today.

I see that this error has been gone over at least once before and a bug report filed and resolved in the JENKINS-29902 issue.  

 

I have tried to take slaves off and back online and this did not resolve the error.

I have tried changing job build numbers and this did not resolve the error.

 
JobClass jobClass : Freestyle


Jenkins Ver: 2.176.1

Master Node OS:  Debian 9

Windows Slave OS: Windows 7 Pro service pack 1

The same configuration is used successfully using a different Debian 9 server with the same slave unit machines.


The windows jenkins service user permissions appear correct via the service properties menu.

 

ronny.borchert@muehlbauer.de (JIRA)

unread,
Mar 12, 2020, 8:23:02 AM3/12/20
to jenkinsc...@googlegroups.com

I got this too today. We are using several axis. We are using no slave. We are on Windows 2012 R2. How could I support?
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

ronny.borchert@muehlbauer.de (JIRA)

unread,
Mar 12, 2020, 8:32:02 AM3/12/20
to jenkinsc...@googlegroups.com
Ronny Borchert edited a comment on Bug JENKINS-56759
I got this too today. We are using several axis. We are using no slave. We are on Windows 2012 R2 and jenkins version is 2 . 225, all plugins are up to date. How could I support?

ronny.borchert@muehlbauer.de (JIRA)

unread,
Mar 12, 2020, 8:54:03 AM3/12/20
to jenkinsc...@googlegroups.com
Ronny Borchert edited a comment on Bug JENKINS-56759
I We got this too today. We are using several axis. We are using no slave. We are on Windows 2012 R2 and jenkins version is 2.225 , all plugins are up but we was copying a project to date wrong location . How could I support?
Reply all
Reply to author
Forward
0 new messages