| Hi, A number of months ago we ran into an issue where Jenkins would no longer dispatch jobs from the queue. The Jenkins log file contained many messages of the form: SEVERE hudson.triggers.SafeTimerTask#run: Timer task hudson.model.Queue$MaintainTask@6ee3a4d7 failed
java.lang.NullPointerException Based on the advice in https://support.cloudbees.com/hc/en-us/articles/360034998632-Jenkins-stops-processing-builds-in-the-build-queue-after-an-error-appears-in-the-logs , looking back further in the logs revealed a series of errors of the form: 2019-11-16 18:55:13.548+0000 [id=36] SEVERE hudson.triggers.SafeTimerTask#run: Timer task hudson.model.Queue$MaintainTask@6ee3a4d7 failed
java.lang.NullPointerException
at hudson.plugins.buildblocker.BlockingJobsMonitor.checkNodeForQueueEntries(BlockingJobsMonitor.java:108)
at hudson.plugins.buildblocker.BuildBlockerQueueTaskDispatcher.checkAccordingToProperties(BuildBlockerQueueTaskDispatcher.java:171)
at hudson.plugins.buildblocker.BuildBlockerQueueTaskDispatcher.checkForBlock(BuildBlockerQueueTaskDispatcher.java:127)
at hudson.plugins.buildblocker.BuildBlockerQueueTaskDispatcher.canTake(BuildBlockerQueueTaskDispatcher.java:110)
at hudson.model.Queue$JobOffer.getCauseOfBlockage(Queue.java:279)
at hudson.model.Queue.maintain(Queue.java:1614)
at hudson.model.Queue$MaintainTask.doRun(Queue.java:2885)
at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834) Note that the stack trace was not appearing in the newer log message, but the older messages pointed to the 'Build Blocker Plugin'. We tried a reboot but the problem remained after the restart. In addition, we noticed one additional, unique error message that would appear before the other, repeating error message: 2019-11-18 19:03:45.423+0000 [id=1915] WARNING j.util.AtmostOneTaskExecutor$1#call
java.lang.NullPointerException
at hudson.plugins.buildblocker.BlockingJobsMonitor.checkNodeForQueueEntries(BlockingJobsMonitor.java:108)
at hudson.plugins.buildblocker.BuildBlockerQueueTaskDispatcher.checkAccordingToProperties(BuildBlockerQueueTaskDispatcher.java:171)
at hudson.plugins.buildblocker.BuildBlockerQueueTaskDispatcher.checkForBlock(BuildBlockerQueueTaskDispatcher.java:127)
at hudson.plugins.buildblocker.BuildBlockerQueueTaskDispatcher.canTake(BuildBlockerQueueTaskDispatcher.java:110)
at hudson.model.Queue$JobOffer.getCauseOfBlockage(Queue.java:279)
at hudson.model.Queue.maintain(Queue.java:1614)
at hudson.model.Queue$1.call(Queue.java:326)
at hudson.model.Queue$1.call(Queue.java:323)
at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:108)
at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:98)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:112)
at java.base/java.lang.Thread.run(Thread.java:834) This unique message appeared before the original error messages (and appeared both before the initial problem and after the reboot) as well. We were able to "workaround" this problem by removing the 'Build Blocker Plugin', at which point builds were able to resume. We have left it disabled for now. We would like to re-enable the 'Build Blocker Plugin', but would also look to root cause and fix this issue so it does not hang the Jenkins server again. This issue has a similar stacktrace to JENKINS-38729, but the symptoms appear different. The Jenkins server was at version '2.190.2' and the 'Build Blocker Plugin' was at version '1.7.3'. Let me know if you need any more information. Thanks,
Wade |
