[JIRA] (JENKINS-60798) Setup Kubernetes CLI (kubectl) error: "FATAL: Failed to run kubectl version 1"

46 views
Skip to first unread message

salavessa@gmail.com (JIRA)

unread,
Jan 16, 2020, 11:37:07 AM1/16/20
to jenkinsc...@googlegroups.com
João Salavessa created an issue
 
Jenkins / Bug JENKINS-60798
Setup Kubernetes CLI (kubectl) error: "FATAL: Failed to run kubectl version 1"
Issue Type: Bug Bug
Assignee: Unassigned
Components: kubernetes-plugin
Created: 2020-01-16 16:36
Environment: Kubernetes 1.15.6
Kubernetes-plugin 1.23.0
Jenkins LTS 2.204.1
Priority: Blocker Blocker
Reporter: João Salavessa

In latest release (1.23.0) I'm getting the below error while trying to use the "Setup Kubernetes CLI (kubectl)" in a job where field "Certificate of authority" is empty and field "Credentials" is populated from type "Username and password":
{{}}

[...]
Building in workspace /var/jenkins_home/workspace/backups
$ kubectl version
FATAL: Failed to run kubectl version 1
java.io.IOException: Failed to run kubectl version 1
at org.csanchez.jenkins.plugins.kubernetes.KubectlBuildWrapper.setUp(KubectlBuildWrapper.java:94)
at jenkins.tasks.SimpleBuildWrapper.setUp(SimpleBuildWrapper.java:146)
at hudson.model.Build$BuildExecution.doRun(Build.java:157)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1853)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
Finished: FAILURE

The previous version (1.22.5) works fine and after downgrading all jobs started to work again.
This happened in two independent Jenkins boxes running the latest LTS Jenkins ver. 2.204.1.
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

salavessa@gmail.com (JIRA)

unread,
Jan 20, 2020, 2:46:06 PM1/20/20
to jenkinsc...@googlegroups.com
João Salavessa updated an issue
Change By: João Salavessa
{{ In latest release (1.23.0) I'm getting the below error while trying to use the "*Setup Kubernetes CLI (kubectl)*" in a job where field "_Certificate of authority_" is empty and field "_Credentials_" is populated from type "_Username and password_": }}
{ {}}
{ code :java }

[...]
Building in workspace /var/jenkins_home/workspace/backups
$ kubectl version
FATAL: Failed to run kubectl version 1
java.io.IOException: Failed to run kubectl version 1
at org.csanchez.jenkins.plugins.kubernetes.KubectlBuildWrapper.setUp(KubectlBuildWrapper.java:94)
at jenkins.tasks.SimpleBuildWrapper.setUp(SimpleBuildWrapper.java:146)
at hudson.model.Build$BuildExecution.doRun(Build.java:157)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1853)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
Finished: FAILURE{code}
The previous version (_1.22.5_) works fine and after downgrading all jobs started to work again.

This happened in two independent Jenkins boxes running the latest LTS Jenkins ver. 2.204.1.


 

After further investigation, the issue seems to be related to kubernetes config file setup. Although I didn't specify a certificate, the property _*insecure-skip-tls-verify*_ is set to *_false_*, plus the *_current-context_* is *_not set._* Sample generated file below: 
{code}
$ cat .kube2280516965942825275config
---
clusters:
- cluster:
insecure-skip-tls-verify: false
server: "https://api.k8s-cluster.dummy.com"
name: "k8s"
contexts:
- context:
cluster: "k8s"
user: "cluster-admin"
name: "k8s"
users:
- name: "cluster-admin"
user:
as-user-extra: {}
password: "XXXXXXXXXXXXX"
username: "jenkins"{code}
 

vincent@latombe.net (JIRA)

unread,
Jan 24, 2020, 9:58:03 AM1/24/20
to jenkinsc...@googlegroups.com
Vincent Latombe started work on Bug JENKINS-60798
 
Change By: Vincent Latombe
Status: Open In Progress

vincent@latombe.net (JIRA)

unread,
Jan 24, 2020, 9:59:04 AM1/24/20
to jenkinsc...@googlegroups.com
Vincent Latombe assigned an issue to Vincent Latombe
Change By: Vincent Latombe
Assignee: Vincent Latombe

vincent@latombe.net (JIRA)

unread,
Jan 24, 2020, 9:59:04 AM1/24/20
to jenkinsc...@googlegroups.com
Change By: Vincent Latombe
Status: In Progress Review

vincent@latombe.net (JIRA)

unread,
Jan 29, 2020, 3:56:03 AM1/29/20
to jenkinsc...@googlegroups.com
Change By: Vincent Latombe
Status: In Review Fixed but Unreleased
Resolution: Fixed

vincent@latombe.net (JIRA)

unread,
Jan 31, 2020, 12:21:02 PM1/31/20
to jenkinsc...@googlegroups.com
Change By: Vincent Latombe
Status: Fixed but Unreleased Resolved
Released As: kubernetes 1.23.2

salavessa@gmail.com (JIRA)

unread,
Jan 31, 2020, 1:36:02 PM1/31/20
to jenkinsc...@googlegroups.com
João Salavessa reopened an issue
 

Just tested this with version 1.23.2 and issue is still happening.

The only difference is the error message in the jenkins job (the generated config file has the same issues).


[...]
Building in workspace /var/jenkins_home/workspace/backups
$ kubectl version

ERROR: Failed to run kubectl version. Returned status code 1.
Finished: FAILURE
Change By: João Salavessa
Resolution: Fixed
Status: Resolved Reopened

vincent@latombe.net (JIRA)

unread,
Feb 1, 2020, 9:24:03 AM2/1/20
to jenkinsc...@googlegroups.com
Vincent Latombe updated an issue
Change By: Vincent Latombe
Component/s: kubernetes-credentials-plugin

vincent@latombe.net (JIRA)

unread,
Feb 1, 2020, 9:24:04 AM2/1/20
to jenkinsc...@googlegroups.com
Vincent Latombe commented on Bug JENKINS-60798
 
Re: Setup Kubernetes CLI (kubectl) error: "FATAL: Failed to run kubectl version 1"

I probably read the initial report too quickly. The problem seems to be no current context is set.

vincent@latombe.net (JIRA)

unread,
Feb 1, 2020, 9:28:02 AM2/1/20
to jenkinsc...@googlegroups.com
Change By: Vincent Latombe
Status: In Progress Review

vincent@latombe.net (JIRA)

unread,
Feb 1, 2020, 9:28:03 AM2/1/20
to jenkinsc...@googlegroups.com
 
Change By: Vincent Latombe
Status: Reopened In Progress

vincent@latombe.net (JIRA)

unread,
Feb 1, 2020, 9:32:02 AM2/1/20
to jenkinsc...@googlegroups.com

salavessa@gmail.com (JIRA)

unread,
Feb 1, 2020, 10:20:02 AM2/1/20
to jenkinsc...@googlegroups.com

Tested: job fails with the same message.

The "current-context" is now set correctly but the cluster's "insecure-skip-tls-verify" is not present (and I suppose kubectl defaults that to false).

Generated file below:

$ cat .kube4111259011631954122config
---
clusters:
- cluster:
{{ server: "https://api.internal.k8s.com"}}
{{ name: "k8s"}}


contexts:
- context:
{{ cluster: "k8s"}}
{{ user: "cluster-admin"}}
{{ name: "k8s"}}

current-context: "k8s"


users:
- name: "cluster-admin"
{{ user:}}
{{ as-user-extra: {}}}

{{ password: "......"}}
{{ username: "jenkins"}}

salavessa@gmail.com (JIRA)

unread,
Feb 1, 2020, 10:33:03 AM2/1/20
to jenkinsc...@googlegroups.com
João Salavessa edited a comment on Bug JENKINS-60798
Running _kubectl_ with the above file I get the below:

{{$ kubectl --kubeconfig=.kube4111259011631954122config version}}
{{Client Version: version.Info \ {Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}}}
{{Unable to connect to the server: x509: certificate signed by unknown authority}}

 

If I add "_insecure-skip-tls-verify: true_" to the file it works as expected:

{{$ kubectl --kubeconfig=.kube4111259011631954122config_insecure-skip-tls-verify-true version}}
{{ }}{{ Client Version: version.Info \ {Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}}}
{{Server Version: version.Info \ {Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:11:50Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}}}

salavessa@gmail.com (JIRA)

unread,
Feb 1, 2020, 10:33:03 AM2/1/20
to jenkinsc...@googlegroups.com

Running kubectl with the above file I get the below:

$ kubectl --kubeconfig=.kube4111259011631954122config version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: x509: certificate signed by unknown authority

 

If I add "insecure-skip-tls-verify: true" to the file it works as expected:

$ kubectl --kubeconfig=.kube4111259011631954122config_insecure-skip-tls-verify-true version
}}{{Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:11:50Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

salavessa@gmail.com (JIRA)

unread,
Feb 1, 2020, 10:37:03 AM2/1/20
to jenkinsc...@googlegroups.com
João Salavessa edited a comment on Bug JENKINS-60798
Running _kubectl_ with the above file I get the below:


{{$ kubectl --kubeconfig=.kube4111259011631954122config version}}
{{Client Version: version.Info
\ {Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}}}
{{ * Unable to connect to the server: x509: certificate signed by unknown authority * }}

  

 If I add " * _insecure-skip-tls-verify: true_ * " to the file it works as expected:


{{$ kubectl --kubeconfig=.kube4111259011631954122config_insecure-skip-tls-verify-true version}}
{{Client Version: version.Info
\ {Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:20:18Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}}}
{{Server Version: version.Info \ {Major:"1", Minor:"15", GitVersion:"v1.15.6", GitCommit:"7015f71e75f670eb9e7ebd4b5749639d42e20079", GitTreeState:"clean", BuildDate:"2019-11-13T11:11:50Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}}}

vincent@latombe.net (JIRA)

unread,
Feb 12, 2020, 5:01:03 AM2/12/20
to jenkinsc...@googlegroups.com

João Salavessa should be fixed in 1.23.3

vincent@latombe.net (JIRA)

unread,
Feb 12, 2020, 5:02:03 AM2/12/20
to jenkinsc...@googlegroups.com
Vincent Latombe updated an issue
Change By: Vincent Latombe
Released As: kubernetes 1.23. 2 3

vincent@latombe.net (JIRA)

unread,
Feb 12, 2020, 5:02:04 AM2/12/20
to jenkinsc...@googlegroups.com
Change By: Vincent Latombe
Status: In Review Closed
Resolution: Fixed

salavessa@gmail.com (JIRA)

unread,
Feb 12, 2020, 5:57:02 AM2/12/20
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages