[JIRA] (JENKINS-59194) Jenkins Queue freeze on AbstractCloudSlave.terminate

2 views

Skip to first unread message

brainsam@yandex.ru (JIRA)

unread,

Sep 2, 2019, 3:43:03 PM9/2/19

to jenkinsc...@googlegroups.com

Alexander Moiseenko created an issue

Jenkins /

JENKINS-59194

Jenkins Queue freeze on AbstractCloudSlave.terminate

Issue Type:	Bug
Assignee:	Unassigned
Attachments:	stacktrace.txt
Components:	core, durable-task-plugin, kubernetes-plugin
Created:	2019-09-02 19:42
Environment:	core 2.176.2 kubernetes 2.18.1 durable task plugin 1.30
Priority:	Major
Reporter:	Alexander Moiseenko

Jenkins master freeze on hudson.model.Queue.withLock.

At certain moment all jobs have been stucked in waiting state, locked by java.util.concurrent.locks.ReentrantLock without any progress for 20 minutes until jenkins was restarted:

 
                                                                "Computer.threadPoolForRemoting [#26557] / waiting for JNLP4-connect connection from 10.116.113.11/10.116.113.11:35228 id=20559800" #9908709 daemon prio=5 os_prio=0 tid=0x00007f30602ca000 nid=0x4609 in Object.wait() [0x00007f3120f8b000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at hudson.remoting.Request.call(Request.java:177)
        - locked <0x00000004ea5957f0> (a hudson.remoting.UserRequest)
        at hudson.remoting.Channel.call(Channel.java:956)
        at org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave._terminate(KubernetesSlave.java:263)
        at hudson.slaves.AbstractCloudSlave.terminate(AbstractCloudSlave.java:67)
        at org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1$1.run(OnceRetentionStrategy.java:128)
        at hudson.model.Queue._withLock(Queue.java:1379)
        at hudson.model.Queue.withLock(Queue.java:1256)
        at org.jenkinsci.plugins.durabletask.executors.OnceRetentionStrategy$1.run(OnceRetentionStrategy.java:123)
        at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
        at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - <0x00000002c0779d00> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
        - <0x00000004ea595920> (a java.util.concurrent.ThreadPoolExecutor$Worker)
 
                                                            