[JIRA] (JENKINS-39844) Unable to execute inside container

瀏覽次數：87 次
跳到第一則未讀訊息

cjyar@java.net (JIRA)

未讀,
2016年11月17日 下午6:11:022016/11/17
收件者：jenkinsc...@googlegroups.com
cjyar created an issue
 
Jenkins / Bug JENKINS-39844
Unable to execute inside container
Issue Type: Bug Bug
Assignee: Carlos Sanchez
Components: kubernetes-plugin
Created: 2016/Nov/17 11:10 PM
Environment: Jenkins 2.32
Kubernetes Plugin 0.9
Kubernetes 1.4.3
Priority: Major Major
Reporter: cjyar

I'm using Jenkinsfile syntax following the example at https://github.com/jenkinsci/kubernetes-plugin

podTemplate(label: 'test-example-build', containers: [
        containerTemplate(name: 'protobuf', image: 'nanoservice/protobuf', command: 'sleep 3600'),
        containerTemplate(name: 'jnlp', image: 'jenkinsci/jnlp-slave', args: '${computer.jnlpmac} ${computer.name}'),
]) {
    node('test-example-build') {
        stage('checkout') {
            checkout scm
        }

        container('protobuf') {
            stage('build') {
                sh "./gradlew assemble --continue"
            }
        }
    }
}

I'm seeing this in the job's console output:

[Pipeline] container
[Pipeline] {
[Pipeline] stage
[Pipeline] { (build)

[Pipeline] sh
[test_example_protobuf-4ZLX5YPVONGXNKQ5U6U3PXZSHY5ZOEH767OMBGDGDKUGOHXLRBOA] Running shell script
Executing shell script inside container [protobuf] of pod [kubernetes-bad0aa993add416e80bdc1e66d1b30fc-536045ac8bbe]
java.net.ProtocolException: Expected HTTP 101 response but was '500 Internal Server Error'
	at com.squareup.okhttp.ws.WebSocketCall.createWebSocket(WebSocketCall.java:123)
	at com.squareup.okhttp.ws.WebSocketCall.access$000(WebSocketCall.java:40)
	at com.squareup.okhttp.ws.WebSocketCall$1.onResponse(WebSocketCall.java:98)
	at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:177)
	at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container

How can I debug this to find what's causing the 500?
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

cjyar@java.net (JIRA)

未讀,
2016年11月17日 晚上7:36:022016/11/17
收件者：jenkinsc...@googlegroups.com
cjyar edited a comment on Bug JENKINS-39844
 
Re: Unable to execute inside container
I've configured a logger in Jenkins capturing ALL for the following:

* * org.apache.http
* com.squareup.okhttp
* io.fabric8
* org.csanchez

...but it doesn't turn up anything interesting:

{code}
Nov 18, 2016 12:32:30 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision

Excess workload after pending Spot instances: 1

Nov 18, 2016 12:32:30 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud

Building connection to Kubernetes host kubernetes URL https://cluster.example.com:6443/

Nov 18, 2016 12:32:30 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud

Building connection to Kubernetes host kubernetes URL https://cluster.example.com:6443/

Nov 18, 2016 12:32:30 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call

Created Pod: kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a

Nov 18, 2016 12:32:30 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$ProvisioningCallback call

Waiting for Pod to be scheduled (0/100): kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a

Nov 18, 2016 12:32:36 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesComputer

Computer KubernetesComputer name: kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a slave: KubernetesSlave name: kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a taskAccepted

Nov 18, 2016 12:32:36 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud

Building connection to Kubernetes host kubernetes URL https://cluster.example.com:6443/

Nov 18, 2016 12:32:41 AM FINE org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerStepExecution

Starting container step.

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryKubeConfig

Trying to configure client from Kubernetes config...

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryKubeConfig

Did not find Kubernetes config at: [/var/jenkins_home/.kube/config]. Ignoring.

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryServiceAccount

Trying to configure client from service account...

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryServiceAccount

Found service account ca cert at: [/var/run/secrets/kubernetes.io/serviceaccount/ca.crt].

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryServiceAccount

Found service account token at: [/var/run/secrets/kubernetes.io/serviceaccount/token].

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryNamespaceFromPath

Trying to configure client namespace from Kubernetes service account namespace path...

Nov 18, 2016 12:32:41 AM FINE io.fabric8.kubernetes.client.Config tryNamespaceFromPath

Found service account namespace at: [/var/run/secrets/kubernetes.io/serviceaccount/namespace].

Nov 18, 2016 12:32:42 AM SEVERE io.fabric8.kubernetes.client.dsl.internal.ExecWebSocketListener onFailure

Exec Failure: HTTP:500. Message:Internal Server Error

java.net.ProtocolException: Expected HTTP 101 response but was '500 Internal Server Error'
at com.squareup.okhttp.ws.WebSocketCall.createWebSocket(WebSocketCall.java:123)
at com.squareup.okhttp.ws.WebSocketCall.access$000(WebSocketCall.java:40)
at com.squareup.okhttp.ws.WebSocketCall$1.onResponse(WebSocketCall.java:98)
at com.squareup.okhttp.Call$AsyncCall.execute(Call.java:177)
at com.squareup.okhttp.internal.NamedRunnable.run(NamedRunnable.java:33)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Nov 18, 2016 12:32:44 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesComputer

Computer KubernetesComputer name: kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a slave: KubernetesSlave name: kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a taskCompleted

Nov 18, 2016 12:32:44 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate

Terminating Kubernetes instance for slave kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a

Nov 18, 2016 12:32:44 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud

Building connection to Kubernetes host kubernetes URL https://cluster.example.com:6443/

Nov 18, 2016 12:32:44 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate

Terminated Kubernetes instance for slave kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a

Nov 18, 2016 12:32:44 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesSlave _terminate

Disconnected computer kubernetes-7a49acb7e693477bafb2c55a23f6c26b-5babbafcaa1a
{code}

cjyar@java.net (JIRA)

未讀,
2016年11月17日 晚上7:36:022016/11/17
收件者：jenkinsc...@googlegroups.com
cjyar commented on Bug JENKINS-39844

I've configured a logger in Jenkins capturing ALL for the following:

  • * org.apache.http
  • com.squareup.okhttp
  • io.fabric8
  • org.csanchez

...but it doesn't turn up anything interesting:

Nov 18, 2016 12:32:30 AM INFO org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision

Excess workload after pending Spot instances: 1

Nov 18, 2016 12:32:30 AM FINE org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud

Building connection to Kubernetes host kubernetes URL https://cluster.example.com:6443/

kgraham@businessinsider.com (JIRA)

未讀,
2016年11月18日 下午1:59:012016/11/18
收件者：jenkinsc...@googlegroups.com

I am also having this issue.

  • confifirmed can auth to api from within container
 ./kubectl exec jenkinstest -- sh -c 'curl -v --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" https://kubernetes.default.svc'

expected results

however I get '401' error similar to first comment during a pipeline job.

Very strange, very hard to debug.

jenkins-ci@carlossanchez.eu (JIRA)

未讀,
2016年11月22日 凌晨3:42:012016/11/22
收件者：jenkinsc...@googlegroups.com

cjyar you are getting a 500 from kubernetes, you should check your kubernetes logs to see what's the problem

kevin graham a 401 error is totally different from what this issue is about, please open a different issue and add all the relevant logs https://github.com/jenkinsci/kubernetes-plugin#debugging
回覆所有人
回覆作者
轉寄
0 則新訊息