[JIRA] (JENKINS-59423) Timed out waiting for agent pod ready after concurrent 27 agent pods running

12 views
Skip to first unread message

primebluer@gmail.com (JIRA)

unread,
Sep 18, 2019, 4:34:02 AM9/18/19
to jenkinsc...@googlegroups.com
kwan su created an issue
 
Jenkins / Bug JENKINS-59423
Timed out waiting for agent pod ready after concurrent 27 agent pods running
Issue Type: Bug Bug
Assignee: Unassigned
Components: kubernetes-plugin
Created: 2019-09-18 08:33
Priority: Minor Minor
Reporter: kwan su

I have a jenkins job run build conccurently, but the agent connction timout happened after 27 conccurent builds created. There are only 27 agent pods online, the other agent are just timeout and offline.

My Concurrency Limit setting in Kubernetes plugin and the pod template are empty, which means no limit for the concurrent agent pods.

 

2019-09-18 08:15:32.507+0000 [id=8704] WARNING o.c.j.p.k.KubernetesLauncher#launch: Error in provisioning; agent=Kubern
etesSlave name: test3-183-6j55c-xmhx3-g56pb, template=PodTemplate

{, name='Test3_183-6j55c-xmhx3', namespace='jenkins', l abel='Test3_183-6j55c', nodeUsageMode=EXCLUSIVE, annotations=[org.csanchez.jenkins.plugins.kubernetes.PodAnnotation@aab9 c821]}

io.fabric8.kubernetes.client.KubernetesClientTimeoutException: Timed out waiting for [100000] milliseconds for [Pod] with name:[test3-183-6j55c-xmhx3-g56pb] in namespace [jenkins].
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.await(AllContainersRunningPodWatcher.j
ava:130)
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:131)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:297)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-09-18 08:15:32.508+0000 [id=8704] INFO o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance fo
r agent test3-183-6j55c-xmhx3-g56pb
2019-09-18 08:15:32.526+0000 [id=77] INFO o.c.j.p.k.pod.retention.Reaper#eventReceived: jenkins/test3-183-6j55c-xm
hx3-g56pb was just deleted, so removing corresponding Jenkins agent
2019-09-18 08:15:32.527+0000 [id=8704] INFO o.c.j.p.k.KubernetesSlave#deleteSlavePod: Terminated Kubernetes instance
for agent jenkins/test3-183-6j55c-xmhx3-g56pb
Terminated Kubernetes instance for agent jenkins/test3-183-6j55c-xmhx3-g56pb
2019-09-18 08:15:32.527+0000 [id=8704] INFO o.c.j.p.k.KubernetesSlave#_terminate: Disconnected computer test3-183-6j
55c-xmhx3-g56pb
Disconnected computer test3-183-6j55c-xmhx3-g56pb

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

primebluer@gmail.com (JIRA)

unread,
Sep 18, 2019, 4:35:04 AM9/18/19
to jenkinsc...@googlegroups.com
kwan su updated an issue
Change By: kwan su
I have a jenkins job run build builds conccurently, but the agent connction timout happened after 27 conccurent builds created. There are only 27 agent pods online, the other agent are just timeout and offline.

primebluer@gmail.com (JIRA)

unread,
Sep 18, 2019, 4:36:08 AM9/18/19
to jenkinsc...@googlegroups.com
kwan su updated an issue
I have a jenkins job run builds conccurently, but the agent connction timout happened after 27 conccurent builds created. There are only 27 agent pods online, the other agent are just timeout and offline.


My Concurrency Limit setting in Kubernetes plugin and the pod template are empty, which means no limit for the concurrent agent pods.

 

2019-09-18 08:15:32.507+0000 [id=8704] WARNING o.c.j.p.k.KubernetesLauncher#launch: Error in provisioning; agent=Kubern
etesSlave name: test3-183-6j55c-xmhx3-g56pb, template=PodTemplate

{, name='Test3_183-6j55c-xmhx3', namespace='jenkins', l abel='Test3_183-6j55c', nodeUsageMode=EXCLUSIVE, annotations=[org.csanchez.jenkins.plugins.kubernetes.PodAnnotation@aab9 c821]}

{color:#FF0000} io.fabric8.kubernetes.client.KubernetesClientTimeoutException: Timed out waiting for [100000] milliseconds for [Pod] with name:[test3-183-6j55c-xmhx3-g56pb] in namespace [jenkins]. {color}
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.await(AllContainersRunningPodWatcher.j
ava:130)
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:131)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:297)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-09-18 08:15:32.508+0000 [id=8704] INFO o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance fo
r agent test3-183-6j55c-xmhx3-g56pb
2019-09-18 08:15:32.526+0000 [id=77] INFO o.c.j.p.k.pod.retention.Reaper#eventReceived: jenkins/test3-183-6j55c-xm
hx3-g56pb was just deleted, so removing corresponding Jenkins agent
2019-09-18 08:15:32.527+0000 [id=8704] INFO o.c.j.p.k.KubernetesSlave#deleteSlavePod: Terminated Kubernetes instance
for agent jenkins/test3-183-6j55c-xmhx3-g56pb
Terminated Kubernetes instance for agent jenkins/test3-183-6j55c-xmhx3-g56pb
2019-09-18 08:15:32.527+0000 [id=8704] INFO o.c.j.p.k.KubernetesSlave#_terminate: Disconnected computer test3-183-6j
55c-xmhx3-g56pb
Disconnected computer test3-183-6j55c-xmhx3-g56pb

primebluer@gmail.com (JIRA)

unread,
Sep 18, 2019, 4:37:02 AM9/18/19
to jenkinsc...@googlegroups.com
kwan su updated an issue
Change By: kwan su
Attachment: image-2019-09-18-16-36-36-881.png

primebluer@gmail.com (JIRA)

unread,
Sep 18, 2019, 4:50:02 AM9/18/19
to jenkinsc...@googlegroups.com
kwan su closed an issue as Fixed
Change By: kwan su
Status: Open Closed
Resolution: Fixed
Reply all
Reply to author
Forward
0 new messages