jenkins slave offline

[Nick Li]

jenkins-slave is offline (suspend) from jenkins webpage.

My environment:

I have jenkins running on a debian server, created a k8s cluster in gcp. installed kubernetes-plugin, configed k8s in jenkins cloud section,(using admin/k8s password.),

Then created a test pipeline job using the following script

```

podTemplate(label: 'pod-golang', 

    containers: [

        containerTemplate(

            name: 'golang',

            image: 'golang',

            ttyEnabled: true,

            command: 'cat'

        )

    ]

) {

    node ('pod-golang') {

        stage 'Switch to Utility Container'

        container('golang') {

          sh ("go version")

        }

    }

}

```

from `kubectl get pods` I can see the pod is created, but on jenkins webpage, jenkins-slave is alwasy showing offline

Any ideas

Thanks

[Carlos Sanchez]

you can check the steps in https://github.com/jenkinsci/kubernetes-plugin/#debugging to debug

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b2350dfa-07fd-4935-9d49-42c5d9d35d5f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nick Li]

Hi Carlos,

In my jenkins kubernetes cloud config section. I have a connection error now.

I confirmed the admin/k8s-password.

Error testing connection http://my-kubernetes-cluster-ip: java.net.SocketTimeoutException: connect timed out

Thanks

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

[Carlos Sanchez]

that means jenkins can't talk to my-kubernetes-cluster-ip and it wouldn't possibly launch a pod

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/60b1659d-68fd-4385-ab2f-c32f3d2a71d8%40googlegroups.com.

[Nick Li]

Hi Carlos,

How do I debug the connection issue? I was able to connect to k8s cluster using the "Test Connection" Button. But now it just doesn't work.

[Nick Li]

Hi Carlos,

could you provide more insight? Thanks very much.

Some update, I followed https://stackoverflow.com/questions/47870961/jenkins-kubernetes-plugin-not-working, I need to disable https certificate check.

then the test connection is successful.

Then I created a test pipeline job using this one https://github.com/jenkinsci/kubernetes-plugin/blob/master/examples/multi-container.groovy

When I build the job, following the Debug section

I get the following result.

$ kubectl get -a pods --watch

NAME                        READY     STATUS        RESTARTS   AGE

jenkins-slave-qcc2v-f1g3w   0/3       Terminating   0          1m

jenkins-slave-zjqw0-ksd42   2/3       Terminating   0          59s

jenkins-slave-zjqw0-qr87f   0/3       Pending   0         0s

jenkins-slave-zjqw0-qr87f   0/3       Pending   0         0s

jenkins-slave-zjqw0-qr87f   0/3       ContainerCreating   0         0s

jenkins-slave-zjqw0-qr87f   3/3       Running   0         2s

jenkins-slave-qcc2v-f1g3w   0/3       Terminating   0         1m

jenkins-slave-qcc2v-f1g3w   0/3       Terminating   0         1m

jenkins-slave-zjqw0-qr87f   2/3       Error     0         4s

jenkins-slave-zjqw0-qr87f   2/3       Terminating   0         6s

jenkins-slave-zjqw0-ksd42   0/3       Terminating   0         1m

jenkins-slave-zjqw0-r5svf   0/3       Pending   0         0s

jenkins-slave-zjqw0-r5svf   0/3       Pending   0         0s

jenkins-slave-zjqw0-r5svf   0/3       ContainerCreating   0         0s

jenkins-slave-zjqw0-r5svf   3/3       Running   0         2s

jenkins-slave-zjqw0-r5svf   2/3       Error     0         4s

jenkins-slave-zjqw0-r5svf   2/3       Terminating   0         6s

in jenkins job, console output, I see

Console Output

Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
jenkins-slave-zjqw0-ksd42 is offline

[Nick Li]

I running jenkins master outside or k8s cluster,

After running the job, I can the pod is created and running, then turned into terminated very soon,

Looks ing slave created in k8s cluster can't connect to jenkins master.

from JNLP container log I can see

Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior

Warning: SECRET is defined twice in command-line arguments and the environment variable

Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable

Feb 13, 2018 7:02:03 AM hudson.remoting.jnlp.Main createEngine

INFO: Setting up agent: jenkins-slave-n58w4-bdvl7

Feb 13, 2018 7:02:03 AM hudson.remoting.jnlp.Main$CuiListener <init>

INFO: Jenkins agent is running in headless mode.

Feb 13, 2018 7:02:03 AM hudson.remoting.Engine startEngine

WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars

Feb 13, 2018 7:02:03 AM hudson.remoting.jnlp.Main$CuiListener status

INFO: Locating server among [https://MY_JENKINS_IP/]

Feb 13, 2018 7:02:04 AM hudson.remoting.jnlp.Main$CuiListener error

SEVERE: Failed to connect to https://MY_JENKINS_IP/tcpSlaveAgentListener/: java.security.cert.CertificateException: No subject alternative names matching IP address MY_JENKINS_IP found

java.io.IOException: Failed to connect to https://MY_JENKINS_IP/tcpSlaveAgentListener/: java.security.cert.CertificateException: No subject alternative names matching IP address MY_JENKINS_IP found

at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:199)

at hudson.remoting.Engine.innerRun(Engine.java:518)

at hudson.remoting.Engine.run(Engine.java:469)

Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address MY_JENKINS_IP found

at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)

at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1959)

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)

at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)

at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)

at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)

at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)

[Nick Li]

jenkins master is running outside of k8s cluster,

from the log, looks like slave created in k8s cluster can't connect to jenkins master.

What do I need to do?

[Carlos Sanchez]

> Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address MY_JENKINS_IP found

If you don't use valid ssl certificates in the jenkins master you'll need to configure Java in the jnlp agent docker, it has nothing to do with kubernetes.

https://stackoverflow.com/questions/8443081/how-are-ssl-certificate-server-names-resolved-can-i-add-alternative-names-using?lq=1

unless you are hitting https://issues.jenkins-ci.org/browse/JENKINS-43527

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e58f2594-cec3-46df-9d51-f96c04493d0b%40googlegroups.com.

[Nick Li]

Thanks Carlos, I will check

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

[Nick Li]

Could you share some notes about "configure Java in the jnlp agent docker"?

Not sure how to do this. Thanks a lot

On Tuesday, February 13, 2018 at 5:34:43 PM UTC+8, Carlos Sanchez wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.