[JIRA] [core] (JENKINS-28915) Thread Death and build number is already in use

[michael.damberger@hella.com (JIRA)]

Michael Damberger created an issue

Jenkins / JENKINS-28915 Thread Death and build number is already in use Issue Type: Bug Assignee: Unassigned Components: core Created: 16/Jun/15 8:17 AM Environment: Windows Server 2008 R2 Standard java version "1.8.0_45" Java(TM) SE Runtime Environment (build 1.8.0_45-b15) Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode) jenkins 1.614 in apache-tomcat-7.0.62 Priority: Major Reporter: Michael Damberger Jun 16, 2015 5:29:37 AM SEVERE hudson.model.Executor run Unexpected executor death java.lang.IllegalStateException: cannot create a build with number 6 since that (or higher) is already in use among [967] at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361) at hudson.model.RunMap.put(RunMap.java:192) at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178) at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144) at hudson.model.Executor$1.call(Executor.java:328) at hudson.model.Executor$1.call(Executor.java:310) at hudson.model.Queue._withLock(Queue.java:1251) at hudson.model.Queue.withLock(Queue.java:1189) at hudson.model.Executor.run(Executor.java:310) Add Comment

This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)

[dbeck@cloudbees.com (JIRA)]

Daniel Beck commented on JENKINS-28915

Re: Thread Death and build number is already in use How did that happen? Please provide more information.

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger edited a comment on JENKINS-28915

Re: Thread Death and build number is already in use

jenkins runs jobs over night with 4 build processors and one of them dies almost nightly since 1.614 was installed at the time of death all processors were busy. another job  hat  had  a failure at this time. is it possible to determine what build it was that was trying to run, causing the IllegalStateException? because this problem is occurring almost every night, so I could easily provide more information, if some log level can be modified somewhere (?)

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger commented on JENKINS-28915

Re: Thread Death and build number is already in use

jenkins runs jobs over night with 4 build processors and one of them dies almost nightly since 1.614 was installed

at the time of death all processors were busy. another job hat a failure at this time.

is it possible to determine what build it was that was trying to run, causing the IllegalStateException?

because this problem is occurring almost every night, so I could easily provide more information, if some log level can be modified somewhere

Add Comment

[dbeck@cloudbees.com (JIRA)]

Daniel Beck commented on JENKINS-28915

Re: Thread Death and build number is already in use

It's a project with a single build with the build number 967. Does that help?

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger commented on JENKINS-28915

Re: Thread Death and build number is already in use

tonight: java.lang.IllegalStateException: cannot create a build with number 7 since that (or higher) is already in use among [967] if the build number is 967, why does it try to create build number 7? should it not try to create build number 968?

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger commented on JENKINS-28915

Re: Thread Death and build number is already in use

I could identify the project . trying to run it causes the above exception I manually have set nextbuildnumber to 968, now this job works again. thanks for your information!

Add Comment

[dbeck@cloudbees.com (JIRA)]

Daniel Beck commented on JENKINS-28915

Re: Thread Death and build number is already in use

Well, yes, and that's why you filed a bug. Without more information we cannot investigate this further however. Are you using Gerrit Trigger plugin?

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger commented on JENKINS-28915

Re: Thread Death and build number is already in use

no, I'm not using gerrit trigger plugin

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger commented on JENKINS-28915

Re: Thread Death and build number is already in use

the thread death problem should disappear now, I assume. in case another wrong nextbuildnumber occurs, I will post it here. let me know what further information you need.

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger commented on JENKINS-28915

Re: Thread Death and build number is already in use

ok, I think I found the initial problem. something went wrong when it was tried to delete this job: SEVERE: Error while serving http://***/job/***jobname***/doDelete java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor694.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:46) (...ommitting some lines - let me know if you need the full stack tracE) Caused by: java.nio.file.FileSystemException: D:\.jenkins\jobs\PROD_IBS Pool v2_ReportGeneration\builds\967\log: The process cannot access the file because it is being used by another process. at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269) at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) at java.nio.file.Files.delete(Files.java:1126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at hudson.Util.deleteFile(Util.java:247) at hudson.Util.deleteRecursive(Util.java:310) at hudson.Util.deleteContentsRecursive(Util.java:212) at hudson.Util.deleteRecursive(Util.java:301) at hudson.Util.deleteContentsRecursive(Util.java:212) at hudson.Util.deleteRecursive(Util.java:301) at hudson.Util.deleteContentsRecursive(Util.java:212) at hudson.Util.deleteRecursive(Util.java:301) at hudson.model.AbstractItem.performDelete(AbstractItem.java:594) at hudson.model.Job.performDelete(Job.java:275) at hudson.model.AbstractProject.performDelete(AbstractProject.java:372) at hudson.model.AbstractItem.delete(AbstractItem.java:583) at hudson.model.Job.delete(Job.java:665) at hudson.model.AbstractItem.doDoDelete(AbstractItem.java:544) ... 80 more

Add Comment

[michael.damberger@hella.com (JIRA)]

Michael Damberger edited a comment on JENKINS-28915

Re: Thread Death and build number is already in use

ok, I think I found the initial problem. something went wrong when it was tried to delete this job: SEVERE: Error while serving http://***/job/***jobname***/doDelete java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor694.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298) at org.kohsuke.stapler.interceptor.RequirePOST$Processor.invoke(RequirePOST.java:46) (...ommitting some lines - let me know if you need the full stack tracE)

Caused by: java.nio.file.FileSystemException:  D:  *** \ .jenkins\jobs\PROD_IBS Pool v2_ReportGeneration\ builds\967\log: The process cannot access the file because it is being used by another process.

[sdeschenes@log10solutions.com (JIRA)]

Sylvain Deschenes commented on JENKINS-28915

Re: Thread Death and build number is already in use

I've got this issue too. Step to reproduce it: with job DSL, I've re-queue the job. Let it requeue a lot of times. Deactivate the job, Delete it, Re-create the job with same name. Launch it The error is there.... Those are the plugin that I have installed: Git Plugin Job Dsl Gradle Plugin Workspace Cleanup Plugin SSH Agent Plugin Groovy Credentials Binding Plugin Build Blocker Plugin Environment Injector Plugin using Jenkins ver. 1.642.2

Add Comment

[tib@hms.se (JIRA)]

Timmy Brolin commented on JENKINS-28915

Re: Thread Death and build number is already in use

I got the same issue. Log output: ****************** Unexpected executor death java.lang.IllegalStateException: cannot create a build with number 18 since that (or higher) is already in use among [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 195]

at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361) at hudson.model.RunMap.put(RunMap.java:192) at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)

at org.jenkinsci.plugins.workflow.job.WorkflowJob.createExecutable(WorkflowJob.java:230) at hudson.model.Executor$1.call(Executor.java:362) at hudson.model.Executor$1.call(Executor.java:344) at hudson.model.Queue._withLock(Queue.java:1403) at hudson.model.Queue.withLock(Queue.java:1268) at hudson.model.Executor.run(Executor.java:344) ****************** Running Jenkins 1.625.2 If I create a copy of the troublesome job, then the copy will run just fine.

Add Comment

[tib@hms.se (JIRA)]

Timmy Brolin edited a comment on JENKINS-28915

Re: Thread Death and build number is already in use

I got the same issue. Log output: ****************** Unexpected executor death java.lang.IllegalStateException: cannot create a build with number 18 since that (or higher) is already in use among [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 195] at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361) at hudson.model.RunMap.put(RunMap.java:192) at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178) at org.jenkinsci.plugins.workflow.job.WorkflowJob.createExecutable(WorkflowJob.java:230) at hudson.model.Executor$1.call(Executor.java:362) at hudson.model.Executor$1.call(Executor.java:344) at hudson.model.Queue._withLock(Queue.java:1403) at hudson.model.Queue.withLock(Queue.java:1268) at hudson.model.Executor.run(Executor.java:344) ******************

The latest build of this job visible in its build history is #12. So the number 195 is clearly an error somewhere.

Running Jenkins 1.625.2 If I create a copy of the troublesome job, then the copy will run just fine.

Add Comment