[JIRA] (JENKINS-58301) Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

[autarch@outlook.com (JIRA)]

autarch princeps created an issue

Jenkins / JENKINS-58301 Kubernetes Plugin Repeated Socket Ping Timeout Exceptions Issue Type: Bug Assignee: Carlos Sanchez Components: kubernetes-plugin Created: 2019-07-02 14:11 Environment: Jenkins version: 2.182 Kubernetes Plugin: 1.6.2 Kubernetes: v1.13.7-eks-c57ff8 Labels: plugin kubernetes-plugin kubernetes timeout Priority: Major Reporter: autarch princeps I often get issues like this: java.net.SocketTimeoutException: sent ping but didn't receive pong within 1000ms (after 330 successful ping/pongs) One single issue breaks the entire task and makes it hard to even cancel the task. Should this not be retried rather than break execution? Our Jenkins will run longer running tasks as well. Any single task breaking stop in the middle is a real issue, and I don't see why one network issue after 330 successful ones (in this case) is such a big issue. I set the timeout (both Connection and Read) in the settings of the Kubernetes Plugin as well, but it still only has a timeout of one second. Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[deiwin.sarjas@gmail.com (JIRA)]

Deiwin Sarjas commented on JENKINS-58301

Re: Kubernetes Plugin Repeated Socket Ping Timeout Exceptions We've also been seeing quite a bit of this recently on EKS with Jenkins 2.164.2, Kubernetes plugin 1.15.2.

Add Comment

[autarch@outlook.com (JIRA)]

autarch princeps updated an issue

Jenkins / JENKINS-58301

Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

Change By: autarch princeps Environment: Jenkins version: 2.182 Kubernetes Plugin: 1. 6 16 .2 Kubernetes: v1.13.7-eks-c57ff8

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick updated an issue

Jenkins / JENKINS-58301 Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

Change By: Jesse Glick Labels: kubernetes kubernetes-plugin plugin timeout

Add Comment

[jglick@cloudbees.com (JIRA)]

Jesse Glick assigned an issue to Unassigned

Jenkins / JENKINS-58301 Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

Change By: Jesse Glick Assignee: Carlos Sanchez

Add Comment

[jtcheng@alauda.io (JIRA)]

cheng jingtao commented on JENKINS-58301

Re: Kubernetes Plugin Repeated Socket Ping Timeout Exceptions +1

Add Comment

[grechtyrone1989@gmail.com (JIRA)]

Tyrone Grech commented on JENKINS-58301

Re: Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

We are also encountering this issue fairly often in our CI system running: On premises Kubernetes cluster on version 1.14.1 Jenkins version 2.186 Kubernetes Plugin version 1.17.2

Add Comment

[deiwin.sarjas@gmail.com (JIRA)]

Deiwin Sarjas commented on JENKINS-58301

Re: Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

We configured -Dkubernetes.websocket.ping.interval=30000 for Jenkins based on this comment on another issue. I'll report back if it helps or not.

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[juha.tiensyrja@arm.com (JIRA)]

Juha Tiensyrjä commented on JENKINS-58301

Re: Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

That option helped for us. But the reason why the pings started to fail was actually the JVM garbage collector, which caused the master to hang for more than 1 second. We switched from the default to G1GC to reduce time the master is blocked, and this helped with other timeouts too.

Add Comment

[Lee.McClintock@fisglobal.com (JIRA)]

Lee McClintock updated an issue

Jenkins / JENKINS-58301

Kubernetes Plugin Repeated Socket Ping Timeout Exceptions

Change By: Lee McClintock

I often get issues like this: java.net.SocketTimeoutException: sent ping but didn't receive pong within 1000ms (after 330 successful ping/pongs) One single issue breaks the entire task and makes it hard to even cancel the task. Should this not be retried rather than break execution? Our Jenkins will run longer running tasks as well. Any single task breaking stop in the middle is a real issue, and I don't see why one network issue after 330 successful ones (in this case) is such a big issue.

I set the timeout (both Connection and Read) in the settings of the Kubernetes Plugin as well, but it still only has a timeout of one second. x

Add Comment

[aburdajewicz@cloudbees.com (JIRA)]

Allan BURDAJEWICZ commented on JENKINS-58301

Re: Kubernetes Plugin Repeated Socket Ping Timeout Exceptions I believe that this issue is resolved since the release of version 1.19.3 that uses kubernetes-client 4.6.0 in which the default ping interval is 30 seconds: https://github.com/fabric8io/kubernetes-client/commit/2b1799497f46de81c841ea43808472d3239e7209#diff-7a4b549d7e10b88fbe20ebe680f6b25b https://github.com/jenkinsci/kubernetes-plugin/commit/464320a012fa0fd47b92f3af3d0403afd22c41a5#diff-600376dffeb79835ede4a0b285078036 https://github.com/jenkinsci/kubernetes-client-api-plugin/blob/kubernetes-client-api-4.6.0-1/pom.xml#L20 maybe Vincent Latombe can confirm ?

Add Comment