[JIRA] (JENKINS-58437) Pods entering error state due to ConnectionRefusalException: pod is already connected to this master

10 views
Skip to first unread message

grzegorz.zieba.ext@gigaset.com (JIRA)

unread,
Jul 11, 2019, 3:59:13 AM7/11/19
to jenkinsc...@googlegroups.com
Grzegorz Zieba created an issue
 
Jenkins / Bug JENKINS-58437
Pods entering error state due to ConnectionRefusalException: pod is already connected to this master
Issue Type: Bug Bug
Assignee: Carlos Sanchez
Components: kubernetes-plugin
Created: 2019-07-11 07:58
Priority: Blocker Blocker
Reporter: Grzegorz Zieba

Pod jnlp log:

 

$ kubectl -n jenkins logs pod-zebra-feature-build-89-br6hq-rc0n5 jnlp                                                                                                       [9:40:51]
Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior
Jul 11, 2019 7:32:10 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: pod-zebra-feature-build-89-br6hq-rc0n5
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Jul 11, 2019 7:32:11 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.29
Jul 11, 2019 7:32:11 AM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://jenkins.dev.gigaset-devnet.pl/]
Jul 11, 2019 7:32:11 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Jul 11, 2019 7:32:11 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
  Agent address: jenkins.dev.gigaset-devnet.pl
  Agent port:    50000
  Identity:      15:ff:8e:1d:7c:f3:fe:e0:dc:f3:66:88:a5:84:42:36
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.dev.gigaset-devnet.pl:50000
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 15:ff:8e:1d:7c:f3:fe:e0:dc:f3:66:88:a5:84:42:36
Jul 11, 2019 7:32:13 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connected
Jul 11, 2019 7:32:17 AM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNING: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.envinject.EnvInjectComputerListener$2; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Jul 11, 2019 7:35:19 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Terminated
Jul 11, 2019 7:35:29 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Performing onReconnect operation.
Jul 11, 2019 7:35:29 AM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1 onReconnect
INFO: Restarting agent via jenkins.slaves.restarter.UnixSlaveRestarter@16d81b49
Jul 11, 2019 7:35:30 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: pod-zebra-feature-build-89-br6hq-rc0n5
Jul 11, 2019 7:35:30 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Jul 11, 2019 7:35:30 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.29
Jul 11, 2019 7:35:30 AM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Jul 11, 2019 7:35:30 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://jenkins.dev.gigaset-devnet.pl/]
Jul 11, 2019 7:35:31 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Jul 11, 2019 7:35:31 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
  Agent address: jenkins.dev.gigaset-devnet.pl
  Agent port:    50000
  Identity:      15:ff:8e:1d:7c:f3:fe:e0:dc:f3:66:88:a5:84:42:36
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.dev.gigaset-devnet.pl:50000
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 15:ff:8e:1d:7c:f3:fe:e0:dc:f3:66:88:a5:84:42:36
Jul 11, 2019 7:35:31 AM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins.dev.gigaset-devnet.pl/172.29.7.98:50000] Local headers refused by remote: pod-zebra-feature-build-89-br6hq-rc0n5 is already connected to this master. Rejecting this connection.
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: pod-zebra-feature-build-89-br6hq-rc0n5 is already connected to this master. Rejecting this connection.
	at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
	at hudson.remoting.Engine.innerRun(Engine.java:614)
	at hudson.remoting.Engine.run(Engine.java:474)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: pod-zebra-feature-build-89-br6hq-rc0n5 is already connected to this master. Rejecting this connection.
	at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
	at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
	at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
	at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
	at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
	at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
	at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
	at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
	at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: java.nio.channels.ClosedChannelException
		... 7 moreJul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.dev.gigaset-devnet.pl:50000
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP3-connect is not enabled, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
	at hudson.remoting.Engine.onConnectionRejected(Engine.java:682)
	at hudson.remoting.Engine.innerRun(Engine.java:639)
	at hudson.remoting.Engine.run(Engine.java:474)

Jenkins maser log:

 

java.lang.IllegalStateException: Pod has terminated containers: jenkins/pod-zebra-feature-build-89-br6hq-rc0n5 (jnlp)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:166)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
	at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.await(AllContainersRunningPodWatcher.java:127)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:128)
	at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

 

 

Jenkins - 2.164.3

Kubernetes plugin - 1.16.7

Kubernetes cluster - v1.14.3

 
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

grzegorz.zieba.ext@gigaset.com (JIRA)

unread,
Jul 11, 2019, 4:00:04 AM7/11/19
to jenkinsc...@googlegroups.com
Grzegorz Zieba updated an issue
Change By: Grzegorz Zieba
Pod jnlp log:

 
{noformat}

$ kubectl -n jenkins logs pod-zebra-feature-build-89-br6hq-rc0n5 jnlp                     [9:40:51]
Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior
Jul 11, 2019 7:32:10 AM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: pod-zebra-feature-build-89-br6hq-rc0n5
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Jul 11, 2019 7:32:11 AM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.29
Jul 11, 2019 7:32:11 AM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://jenkins.dev.
gigaset- devnet.pl/]

Jul 11, 2019 7:32:11 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Jul 11, 2019 7:32:11 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
  Agent address: jenkins.dev.
gigaset- devnet.pl

  Agent port:    50000
  Identity:      15:ff:8e:1d:7c:f3:fe:e0:dc:f3:66:88:a5:84:42:36
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Jul 11, 2019 7:32:11 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.dev. gigaset- devnet.pl:50000
gigaset- devnet.pl/]
gigaset- devnet.pl
INFO: Connecting to jenkins.dev. gigaset- devnet.pl:50000

Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 15:ff:8e:1d:7c:f3:fe:e0:dc:f3:66:88:a5:84:42:36
Jul 11, 2019 7:35:31 AM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins.dev. gigaset- devnet.pl/172.29.7.98:50000] Local headers refused by remote: pod-zebra-feature-build-89-br6hq-rc0n5 is already connected to this master. Rejecting this connection.

Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: pod-zebra-feature-build-89-br6hq-rc0n5 is already connected to this master. Rejecting this connection.
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:614)
at hudson.remoting.Engine.run(Engine.java:474)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: pod-zebra-feature-build-89-br6hq-rc0n5 is already connected to this master. Rejecting this connection.
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.nio.channels.ClosedChannelException
  ... 7 moreJul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins.dev. gigaset- devnet.pl:50000

Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP3-connect is not enabled, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Jul 11, 2019 7:35:31 AM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:682)
at hudson.remoting.Engine.innerRun(Engine.java:639)
at hudson.remoting.Engine.run(Engine.java:474){noformat}
Jenkins maser log:

 
{noformat}

java.lang.IllegalStateException: Pod has terminated containers: jenkins/pod-zebra-feature-build-89-br6hq-rc0n5 (jnlp)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:166)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.periodicAwait(AllContainersRunningPodWatcher.java:187)
at org.csanchez.jenkins.plugins.kubernetes.AllContainersRunningPodWatcher.await(AllContainersRunningPodWatcher.java:127)
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:128)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){noformat}

 

 

Jenkins - 2.164.3

Kubernetes plugin - 1.16.7

Kubernetes cluster - v1.14.3

 

grzegorz.zieba.ext@gigaset.com (JIRA)

unread,
Jul 16, 2019, 5:39:02 AM7/16/19
to jenkinsc...@googlegroups.com

grzegorz.zieba.ext@gigaset.com (JIRA)

unread,
Jul 16, 2019, 7:01:02 AM7/16/19
to jenkinsc...@googlegroups.com
Grzegorz Zieba closed an issue as Not A Defect
 
Change By: Grzegorz Zieba
Status: Open Closed
Resolution: Not A Defect
Reply all
Reply to author
Forward
0 new messages