| I noticed about a month ago that I had many un-deleted pods in Kubernetes. The jobs didn't fail according to Jenkins, but the pods exited non-zero in kubernetes. I'm not sure what version this started with, but it's happening on the latest.
kubectl --namespace=jenkins get po -a
NAME READY STATUS RESTARTS AGE
jenkins-75fd5fdbb-jkrmf 1/1 Running 0 53d
jenkins-slave-jxk94-729sl 0/1 Error 0 3h
jenkins-slave-jxk94-bq6kp 0/1 Error 0 25m
jenkins-slave-jxk94-jrdgb 0/1 Error 0 35m
jenkins-slave-jxk94-w0tdh 0/1 Error 0 2m
This bug isn't critical since it's not failing my builds, but something is going wrong. This is the log from the agent:
Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior
Warning: SECRET is defined twice in command-line arguments and the environment variable
Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable
Sep 05, 2018 3:57:44 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: jenkins-slave-jxk94-6c6sd
Sep 05, 2018 3:57:44 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Sep 05, 2018 3:57:44 PM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.25
Sep 05, 2018 3:57:44 PM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Sep 05, 2018 3:57:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://jenkins-discovery/]
Sep 05, 2018 3:57:45 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Sep 05, 2018 3:57:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: jenkins-discovery
Agent port: 50000
Identity: 96:81:a1:68:84:20:aa:12:1f:2b:97:b0:c5:2f:de:25
Sep 05, 2018 3:57:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Sep 05, 2018 3:57:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-discovery:50000
Sep 05, 2018 3:57:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Sep 05, 2018 3:57:45 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 96:81:a1:68:84:20:aa:12:1f:2b:97:b0:c5:2f:de:25
Sep 05, 2018 3:57:46 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connected
Sep 05, 2018 3:57:48 PM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNING: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.envinject.EnvInjectComputerListener$2; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Sep 05, 2018 3:57:49 PM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNING: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.gitclient.Git$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Sep 05, 2018 3:57:51 PM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARNING: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Sep 05, 2018 4:07:36 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Terminated
^^ This terminates because I abort the build. The log then continues:
Sep 05, 2018 4:07:46 PM jenkins.slaves.restarter.JnlpSlaveRestarterInstaller$FindEffectiveRestarters$1 onReconnect
INFO: Restarting agent via jenkins.slaves.restarter.UnixSlaveRestarter@581415ae
Sep 05, 2018 4:07:49 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: jenkins-slave-jxk94-6c6sd
Sep 05, 2018 4:07:49 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Sep 05, 2018 4:07:49 PM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.25
Sep 05, 2018 4:07:49 PM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Sep 05, 2018 4:07:49 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://jenkins-discovery/]
Sep 05, 2018 4:07:49 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: jenkins-discovery
Agent port: 50000
Identity: 96:81:a1:68:84:20:aa:12:1f:2b:97:b0:c5:2f:de:25
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-discovery:50000
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 96:81:a1:68:84:20:aa:12:1f:2b:97:b0:c5:2f:de:25
Sep 05, 2018 4:07:50 PM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection to jenkins-discovery/10.55.243.214:50000] Local headers refused by remote: Unknown client name: jenkins-slave-jxk94-6c6sd
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-slave-jxk94-6c6sd
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)
at hudson.remoting.Engine.innerRun(Engine.java:614)
at hudson.remoting.Engine.run(Engine.java:474)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkins-slave-jxk94-6c6sd
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:832)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.nio.channels.ClosedChannelException
... 7 moreSep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to jenkins-discovery:50000
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP3-connect is not enabled, skipping
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP2-connect not supported, skipping
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Sep 05, 2018 4:07:50 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:675)
at hudson.remoting.Engine.innerRun(Engine.java:639)
at hudson.remoting.Engine.run(Engine.java:474)
It attempts to reconnect but is met with "Unknown client name". Is there some reason it attempts to reconnect and then errors when it cannot? I can launch a 4 job parallel pipeline and if I abort it I end up with 5 Errored pods. One is the pipeline job itself, with four of the sub-jobs. This can also happen when builds succeed. Is this normal behavior for the agent to keep trying to connect after being terminated? |
