[JIRA] (JENKINS-49660) Unable to configure plugin using ibm k8s

15 views
Skip to first unread message

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 3:20:03 PM2/20/18
to jenkinsc...@googlegroups.com
Steve Szabo created an issue
 
Jenkins / Bug JENKINS-49660
Unable to configure plugin using ibm k8s
Issue Type: Bug Bug
Assignee: Carlos Sanchez
Attachments: config.cert, config.token
Components: kubernetes-plugin
Created: 2018-02-20 20:19
Labels: plugin kuberenetes-plugin
Priority: Minor Minor
Reporter: Steve Szabo

I was able to configure the plugin to use my local k8s cluster however I can't get it working using my ibm k8s instance. The difference seems to be that ibm is using auth tokens and my local instance is using cert+key. See attached

io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://127.0.0.1:1234/api/v1/namespaces/jenkins/pods. Message:
 Forbidden! User user...@company.com doesn't have permission. pods is forbidden: User "system:anonymous" cannot list pods in the namespace "jenki
ns".
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:470)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:407)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:327)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:605)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:70)
    at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.addProvisionedSlave(KubernetesCloud.java:452)
    at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.provision(KubernetesCloud.java:405)
    at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:715)
    at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:320)
    at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)
    at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:809)
    at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)
    at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

 
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
Atlassian logo

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 3:27:03 PM2/20/18
to jenkinsc...@googlegroups.com
Steve Szabo commented on Bug JENKINS-49660
 
Re: Unable to configure plugin using ibm k8s

I should mention that using config.token I'm able to list pods using kubectl so the role should have the correct perms:

sszabo@xxxxxxxxxxx [/home/sszabo] $ kubectl get pods
No resources found.

jenkins-ci@carlossanchez.eu (JIRA)

unread,
Feb 20, 2018, 3:43:02 PM2/20/18
to jenkinsc...@googlegroups.com

if you run the jenkins master in k8s you should just run it using serviceAccount for authentication

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 4:29:02 PM2/20/18
to jenkinsc...@googlegroups.com

I'm not running the master in k8s however I was able to get it working with my local cluster by pointing the service endpoints to the master.

I don't think this is the issue under ibm as it gives a permission error when I test the connection from jenkins config page.

How are you authenticating from the master to k8s - you seem to be using the kubectl config.

---
kind: Endpoints
apiVersion: v1
metadata:
  name: jenkins-master
  namespace: jenkins
  labels:
    oiqclient: jenkins
subsets:
- addresses:
  - ip: 192.168.0.100
  ports:
  - port: 8080
    name: jenkins-master
- addresses:
  - ip: 192.168.0.100
  ports:
  - port: 50000
    name: jenkins-jnlp

---
kind: Service
apiVersion: v1
metadata:
  name: jenkins-master
  namespace: jenkins
  labels:
    oiqclient: jenkins
spec:
  ports:
  - name: jenkins-master
    protocol: TCP
    port: 8080
    targetPort: 8080
    nodePort: 0
  - name: jenkins-jnlp
    protocol: TCP
    port: 50000
    targetPort: 50000
    nodePort: 0

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 5:13:02 PM2/20/18
to jenkinsc...@googlegroups.com

Using the token from config.token I'm able to list pods using kubectl and curl:

sszabo@xxxxxxxxxxx [/home/sszabo/tmp] $ curl -s --cacert $KUBE_CERT -X GET -H "$KUBE_HEADER" $KUBE_MASTER/api/v1/namespaces/$KUBE_NAMESPACE/pods
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/namespaces/jenkins/pods",
    "resourceVersion": "882144"
  },
  "items": []
}

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 6:16:02 PM2/20/18
to jenkinsc...@googlegroups.com

Have you tested the plugin using a token?

token

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 7:20:02 PM2/20/18
to jenkinsc...@googlegroups.com
Steve Szabo updated an issue
 
Change By: Steve Szabo
Attachment: config.token.fixed

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 20, 2018, 7:21:03 PM2/20/18
to jenkinsc...@googlegroups.com
 
Re: Unable to configure plugin using ibm k8s

Using the basic token format works:

users:
- name: user...@company.com
  user:
    token: AAA

This version does not work:

users:
- name: user...@company.com
  user:
    auth-provider:
      name: oidc
      config:
        client-id: bx
        client-secret: bx
        id-token: AAA
        idp-issuer-url: https://iam.ng.bluemix.net/kubernetes
        refresh-token: BBB

jenkins-ci@carlossanchez.eu (JIRA)

unread,
Feb 21, 2018, 2:49:01 AM2/21/18
to jenkinsc...@googlegroups.com

typically you would create a ServiceAccount in k8s and use its token for authentication.

oidc could work if you use refresh-token as token but it has an expiration time

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 22, 2018, 9:12:01 AM2/22/18
to jenkinsc...@googlegroups.com

I plan to create a service account to manage the namespace from jenkins.

Thanks for your help on this one - cheers

steve.g.szabo@gmail.com (JIRA)

unread,
Feb 22, 2018, 9:13:02 AM2/22/18
to jenkinsc...@googlegroups.com
Steve Szabo resolved as Not A Defect
 
Change By: Steve Szabo
Status: Open Resolved
Resolution: Not A Defect

jglick@cloudbees.com (JIRA)

unread,
Jun 12, 2019, 3:34:06 PM6/12/19
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
Change By: Jesse Glick
Labels: kuberenetes- plugin plugin
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)
Reply all
Reply to author
Forward
0 new messages