[JIRA] (JENKINS-58896) Dynamic kubernetes agents not registering to master

12 views
Skip to first unread message

matt.cengic@sherwin.com (JIRA)

unread,
Aug 12, 2019, 10:48:03 AM8/12/19
to jenkinsc...@googlegroups.com
matt c created an issue
 
Jenkins / Bug JENKINS-58896
Dynamic kubernetes agents not registering to master
Issue Type: Bug Bug
Assignee: Unassigned
Components: kubernetes-plugin
Created: 2019-08-12 14:47
Environment: Jenkins: 2.176.1
org.csanchez.jenkins.plugins:kubernetes:1.18.1
Priority: Minor Minor
Reporter: matt c

I'm receiving the following error when I dynamically deploy Jenkins slave pods to my Kubernetes cluster. I've tried jenkins.slaves.DefaultJnlpSlaveReceiver.disableStrictVerification=true to no effect

I see the pods being created in Kubernetes, they just seem to fail to register to the master for some reason.

 

Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behaviorjava -Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless -tunnel linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000 -url https://linuxjnks.rancher.sherwin.com/ -workDir /home/jenkins/ e7a8c4b7cb92f3058003fa78d23f5ce12f8f975849f20cccaf0075499d927148 jenkinsslav-rpcncAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main createEngineINFO: Setting up agent: jenkinsslav-rpcncAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener <init>INFO: Jenkins agent is running in headless mode.Aug 12, 2019 1:54:31 PM hudson.remoting.Engine startEngineINFO: Using Remoting version: 3.29Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDirINFO: Using /home/jenkins/remoting as a remoting work directoryBoth error and output logs will be printed to /home/jenkins/remotingAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Locating server among https://linuxjnks.rancher.sherwin.com/Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting server accepts the following protocols: [JNLP-connect, Ping]Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolveINFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability checkAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Agent discovery successfulAgent address: linux-jenkins-discovery.linux-jenkins.svc.cluster.localAgent port: 50000Identity: a8:7f:31:f3:34:10:14:cd:42:f2:21:24:82:47:d7:9cAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: HandshakingAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Connecting to linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Server reports protocol JNLP4-connect not supported, skippingAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Server reports protocol JNLP4-plaintext not supported, skippingAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Protocol JNLP3-connect is not enabled, skippingAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Server reports protocol JNLP2-connect not supported, skippingAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Trying protocol: JNLP-connectAug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener statusINFO: Protocol JNLP-connect encountered an unexpected exceptionjava.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake:at java.util.concurrent.FutureTask.report(FutureTask.java:122)at java.util.concurrent.FutureTask.get(FutureTask.java:192)at hudson.remoting.Engine.innerRun(Engine.java:614)at hudson.remoting.Engine.run(Engine.java:474)Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake:at org.jenkinsci.remoting.engine.JnlpProtocol1Handler.sendHandshake(JnlpProtocol1Handler.java:123)at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:162)at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:158)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)at java.lang.Thread.run(Thread.java:748)Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener errorSEVERE: The server rejected the connection: None of the protocols were acceptedjava.lang.Exception: The server rejected the connection: None of the protocols were acceptedat hudson.remoting.Engine.onConnectionRejected(Engine.java:682)at hudson.remoting.Engine.innerRun(Engine.java:639)at hudson.remoting.Engine.run(Engine.java:474)
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

matt.cengic@sherwin.com (JIRA)

unread,
Aug 12, 2019, 10:50:02 AM8/12/19
to jenkinsc...@googlegroups.com
matt c updated an issue
Change By: matt c
I'm receiving the following error when I dynamically deploy Jenkins slave pods to my Kubernetes cluster. I've tried jenkins.slaves.DefaultJnlpSlaveReceiver.disableStrictVerification=true to no effect

I see the pods being created in Kubernetes, they just seem to fail to register to the master for some reason.

 

{{Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior}}{{java -Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless -tunnel linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000 -url [https:// <redacted>/|https:// linuxjnks.rancher.sherwin.com/] -workDir /home/jenkins/ e7a8c4b7cb92f3058003fa78d23f5ce12f8f975849f20cccaf0075499d927148 jenkinsslav-rpcnc}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main createEngine}}{{INFO: Setting up agent: jenkinsslav-rpcnc}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener <init>}}{{INFO: Jenkins agent is running in headless mode.}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.Engine startEngine}}{{INFO: Using Remoting version: 3.29}}{{Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir}}{{INFO: Using /home/jenkins/remoting as a remoting work directory}}{{Both error and output logs will be printed to /home/jenkins/remoting}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Locating server among [https:// <redacted>/|https:// linuxjnks.rancher.sherwin.com/]}}{{Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve}}{{INFO: Remoting server accepts the following protocols: [JNLP-connect, Ping]}}{{Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve}}{{INFO: Remoting TCP connection tunneling is enabled. Skipping the TCP Agent Listener Port availability check}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Agent discovery successful}}{{Agent address: linux-jenkins-discovery.linux-jenkins.svc.cluster.local}}{{Agent port: 50000}}{{Identity: a8:7f:31:f3:34:10:14:cd:42:f2:21:24:82:47:d7:9c}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Handshaking}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Connecting to linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Server reports protocol JNLP4-connect not supported, skipping}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Server reports protocol JNLP4-plaintext not supported, skipping}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Protocol JNLP3-connect is not enabled, skipping}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Server reports protocol JNLP2-connect not supported, skipping}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Trying protocol: JNLP-connect}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status}}{{INFO: Protocol JNLP-connect encountered an unexpected exception}}{{java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake:}}{{at java.util.concurrent.FutureTask.report(FutureTask.java:122)}}{{at java.util.concurrent.FutureTask.get(FutureTask.java:192)}}{{at hudson.remoting.Engine.innerRun(Engine.java:614)}}{{at hudson.remoting.Engine.run(Engine.java:474)}}{{Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Server didn't accept the handshake:}}{{at org.jenkinsci.remoting.engine.JnlpProtocol1Handler.sendHandshake(JnlpProtocol1Handler.java:123)}}{{at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:162)}}{{at org.jenkinsci.remoting.engine.LegacyJnlpProtocolHandler$2.call(LegacyJnlpProtocolHandler.java:158)}}{{at java.util.concurrent.FutureTask.run(FutureTask.java:266)}}{{at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}{{at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}{{at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)}}{{at java.lang.Thread.run(Thread.java:748)}}{{Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener error}}{{SEVERE: The server rejected the connection: None of the protocols were accepted}}{{java.lang.Exception: The server rejected the connection: None of the protocols were accepted}}{{at hudson.remoting.Engine.onConnectionRejected(Engine.java:682)}}{{at hudson.remoting.Engine.innerRun(Engine.java:639)}}{{at hudson.remoting.Engine.run(Engine.java:474)}}

matt.cengic@sherwin.com (JIRA)

unread,
Aug 12, 2019, 10:54:02 AM8/12/19
to jenkinsc...@googlegroups.com
matt c updated an issue
I'm receiving the following error when I dynamically deploy Jenkins slave pods to my Kubernetes cluster. I've tried jenkins.slaves.DefaultJnlpSlaveReceiver.disableStrictVerification=true to no effect

I see the pods being created in Kubernetes, they just seem to fail to register to the master for some reason.

 

{{ Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior }}{{
java -Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless -tunnel linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000 -url [ https:// <redacted>/|https:// linuxjnks.rancher. sherwin somewhere .com/ ] -workDir /home/jenkins/ e7a8c4b7cb92f3058003fa78d23f5ce12f8f975849f20cccaf0075499d927148 jenkinsslav-rpcnc }}{{
Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main createEngine }}{{
INFO: Setting up agent: jenkinsslav-rpcnc }}{{
Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener <init> }}{{
INFO: Jenkins agent is running in headless mode. }}{{
Aug 12, 2019 1:54:31 PM hudson.remoting.Engine startEngine }}{{
INFO: Using Remoting version: 3.29 }}{{
Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir }}{{
INFO: Using /home/jenkins/remoting as a remoting work directory }}{{
Both error and output logs will be printed to /home/jenkins/remoting }}{{
Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status }}{{
INFO: Locating server among [https:// <redacted>/|https:// linuxjnks.rancher. sherwin somewhere .com/] }}{{

matt.cengic@sherwin.com (JIRA)

unread,
Aug 12, 2019, 11:40:02 AM8/12/19
to jenkinsc...@googlegroups.com
matt c updated an issue
I'm receiving the following error when I dynamically deploy Jenkins slave pods to my Kubernetes cluster. I've tried jenkins.slaves.DefaultJnlpSlaveReceiver.disableStrictVerification=true to no effect

I see the pods being created in Kubernetes, they just seem to fail to register to the master for some reason.
With agent protocol 4 enabled on the master:

INFO: [JNLP4-connect connection to linux-jenkins-discovery.linux-jenkins.svc.cluster.local/10.43.209.92:50000] Local headers refused by remote: Unknown client name: jenkinsslav-rh6nj
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Protocol JNLP4-connect encountered an unexpected exception
java.util.concurrent.ExecutionException: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkinsslav-rh6nj
at org.jenkinsci.remoting.util.SettableFuture.get(SettableFuture.java:223)

at hudson.remoting.Engine.innerRun(Engine.java:614)
at hudson.remoting.Engine.run(Engine.java:474)
Caused by: org.jenkinsci.remoting.protocol.impl.ConnectionRefusalException: Unknown client name: jenkinsslav-rh6nj
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.newAbortCause(ConnectionHeadersFilterLayer.java:378)
at org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer.onRecvClosed(ConnectionHeadersFilterLayer.java:433)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:172)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:154)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer.access$1500(BIONetworkLayer.java:48)
at org.jenkinsci.remoting.protocol.impl.BIONetworkLayer$Reader.run(BIONetworkLayer.java:247)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.nio.channels.ClosedChannelException
... 7 more
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener status

INFO: Connecting to linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener status

INFO: Server reports protocol JNLP4-plaintext not supported, skipping
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener status

INFO: Protocol JNLP3-connect is not enabled, skipping
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener status

INFO: Server reports protocol JNLP2-connect not supported, skipping
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Server reports protocol JNLP-connect not supported, skipping
Aug 12, 2019 3:36:36 PM hudson.remoting.jnlp.Main$CuiListener error

SEVERE: The server rejected the connection: None of the protocols were accepted
java.lang.Exception: The server rejected the connection: None of the protocols were accepted
at hudson.remoting.Engine.onConnectionRejected(Engine.java:682)
at hudson.remoting.Engine.innerRun(Engine.java:639)
at hudson.remoting.Engine.run(Engine.java:474)

With only protocol 1 enabled:
 Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior
java -Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless -tunnel linux-jenkins-discovery.linux-jenkins.svc.cluster.local:50000 -url [ https://linuxjnks.rancher.somewhere.com/ ] -workDir /home/jenkins/ e7a8c4b7cb92f3058003fa78d23f5ce12f8f975849f20cccaf0075499d927148 jenkinsslav-rpcnc

Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: jenkinsslav-rpcnc
Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Aug 12, 2019 1:54:31 PM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 3.29
Aug 12, 2019 1:54:31 PM org.jenkinsci.remoting.engine.WorkDirManager initializeWorkDir
INFO: Using /home/jenkins/remoting as a remoting work directory
Both error and output logs will be printed to /home/jenkins/remoting
Aug 12, 2019 1:54:31 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [https://linuxjnks.rancher.somewhere.com/]

matt.cengic@sherwin.com (JIRA)

unread,
Aug 12, 2019, 12:13:02 PM8/12/19
to jenkinsc...@googlegroups.com
matt c commented on Bug JENKINS-58896
 
Re: Dynamic kubernetes agents not registering to master

Logs from Jenkins master:

 

Aug 12, 2019 12:08:39 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Kubernetes agents: 1
Aug 12, 2019 12:08:39 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template for label null: Kubernetes Pod Template
Aug 12, 2019 12:08:39 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Aug 12, 2019 12:08:39 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning Kubernetes Pod Template from kubernetes with 1 executors. Remaining excess workload: 0
Aug 12, 2019 12:08:49 PM hudson.slaves.NodeProvisioner$2 run
INFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)
Aug 12, 2019 12:08:49 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Created Pod: linux-jenkins/jenkinsslav-xh8w3
Aug 12, 2019 12:08:49 PM okhttp3.internal.platform.Platform log
INFO: ALPN callback dropped: HTTP/2 is disabled. Is alpn-boot on the boot class path?
Aug 12, 2019 12:08:49 PM io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1 onFailure
WARNING: Exec Failure: HTTP 403, Status: 403 - null
java.net.ProtocolException: Expected HTTP 101 response but was '403 Forbidden'
at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229)
at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)


at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)
Aug 12, 2019 12:08:49 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
{{WARNING: Error in provisioning; agent=KubernetesSlave name: jenkinsslav-xh8w3, template=PodTemplate{inheritFrom='', name='jenkinsslav', namespace='linux-jenkins', instanceCap=2, label='', nodeSelector='', nodeUsageMode=NORMAL, workspaceVolume=EmptyDirWorkspaceVolume [memory=false], containers=[ContainerTemplate

{name='jnlp', image='docker.cpartdc01.somewhere.com/jenkins/jnlp-slave', alwaysPullImage=true, workingDir='/home/jenkins/', command='', args='', ttyEnabled=true, resourceRequestCpu='', resourceRequestMemory='', resourceLimitCpu='', resourceLimitMemory='', livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe@1b0070bd}

]}}}
io.fabric8.kubernetes.client.KubernetesClientException
at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onFailure(WatchConnectionManager.java:198)
at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:571)
at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:198)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)


at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)
Aug 12, 2019 12:08:49 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent jenkinsslav-xh8w3
Aug 12, 2019 12:08:49 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
WARNING: Slave pod jenkinsslav-xh8w3 was not deleted due to retention policy Always.
Aug 12, 2019 12:08:49 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer jenkinsslav-xh8w3
Disconnected computer jenkinsslav-xh8w3
Aug 12, 2019 12:08:53 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted JNLP4-connect connection #43 from /10.42.5.121:43990
Aug 12, 2019 12:08:53 PM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection from 10.42.5.121/10.42.5.121:43990] Refusing headers from remote: Unknown client name: jenkinsslav-xh8w3
Aug 12, 2019 12:08:53 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
WARNING: Connection #44 failed
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:254)
Aug 12, 2019 12:08:59 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Kubernetes agents: 1
Aug 12, 2019 12:08:59 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template for label null: Kubernetes Pod Template
Aug 12, 2019 12:08:59 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply
INFO: Started provisioning Kubernetes Pod Template from kubernetes with 1 executors. Remaining excess workload: 0
Aug 12, 2019 12:09:09 PM hudson.slaves.NodeProvisioner$2 run
INFO: Kubernetes Pod Template provisioning successfully completed. We have now 2 computer(s)
Aug 12, 2019 12:09:09 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
INFO: Created Pod: linux-jenkins/jenkinsslav-4lxj1
Aug 12, 2019 12:09:09 PM io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1 onFailure
WARNING: Exec Failure: HTTP 403, Status: 403 - null
java.net.ProtocolException: Expected HTTP 101 response but was '403 Forbidden'
at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229)
at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:196)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)


at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)
Aug 12, 2019 12:09:09 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher launch
{{WARNING: Error in provisioning; agent=KubernetesSlave name: jenkinsslav-4lxj1, template=PodTemplate{inheritFrom='', name='jenkinsslav', namespace='linux-jenkins', instanceCap=2, label='', nodeSelector='', nodeUsageMode=NORMAL, workspaceVolume=EmptyDirWorkspaceVolume [memory=false], containers=[ContainerTemplate

{name='jnlp', image='/jenkins/jnlp-slave', alwaysPullImage=true, workingDir='/home/jenkins/', command='', args='', ttyEnabled=true, resourceRequestCpu='', resourceRequestMemory='', resourceLimitCpu='', resourceLimitMemory='', livenessProbe=org.csanchez.jenkins.plugins.kubernetes.ContainerLivenessProbe@1b0070bd}

]}}}
io.fabric8.kubernetes.client.KubernetesClientException
at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onFailure(WatchConnectionManager.java:198)
at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:571)
at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:198)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:206)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)


at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)
Aug 12, 2019 12:09:09 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Terminating Kubernetes instance for agent jenkinsslav-4lxj1
Aug 12, 2019 12:09:09 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
WARNING: Slave pod jenkinsslav-4lxj1 was not deleted due to retention policy Always.
Aug 12, 2019 12:09:09 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate
INFO: Disconnected computer jenkinsslav-4lxj1
Disconnected computer jenkinsslav-4lxj1
Aug 12, 2019 12:09:11 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
INFO: Accepted JNLP4-connect connection #45 from /10.42.5.122:39998
Aug 12, 2019 12:09:12 PM org.jenkinsci.remoting.protocol.impl.ConnectionHeadersFilterLayer onRecv
INFO: [JNLP4-connect connection from 10.42.5.122/10.42.5.122:39998] Refusing headers from remote: Unknown client name: jenkinsslav-4lxj1
Aug 12, 2019 12:09:12 PM hudson.TcpSlaveAgentListener$ConnectionHandler run
WARNING: Connection #46 failed
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:197)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:254)

matt.cengic@sherwin.com (JIRA)

unread,
Aug 15, 2019, 2:45:02 PM8/15/19
to jenkinsc...@googlegroups.com
matt c commented on Bug JENKINS-58896

I believe this is the source of my issue:

 

https://github.com/rancher/rancher/issues/21740

 

Rancher 2.2.5 and 2.2.6 have cross-site scripting exploit denials that are affecting the plugin. We upgraded to 2.2.7 and the plugin works again.

matt.cengic@sherwin.com (JIRA)

unread,
Aug 15, 2019, 2:46:02 PM8/15/19
to jenkinsc...@googlegroups.com
Change By: matt c
Status: Open Fixed but Unreleased
Assignee: matt c
Resolution: Fixed

matt.cengic@sherwin.com (JIRA)

unread,
Aug 27, 2019, 8:00:02 AM8/27/19
to jenkinsc...@googlegroups.com
matt c commented on Bug JENKINS-58896
 
Re: Dynamic kubernetes agents not registering to master

also, the update for the plugin on august 26 2019 fixed it in prod before we applied the rancher fix. thanks
Reply all
Reply to author
Forward
0 new messages