Getting Jenkins CI/CD for GKE to work Using GCloud: Bug in documentation?

[Traiano Welcome]

Hi List

I'm struggling to get Jenkins CI/CD working with GKE /Kubernetes using the instructions provided by gcloud:

      https://cloud.google.com/solutions/continuous-delivery-jenkins-container-engine#deploying_a_canary_release

Specifically, at the step where I create the Jenkins deployment, the deployment pod fails to create because the disk appears to be attached already:

"Warning         FailedSync      Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "jenkins"/"jenkins-482090729-8sd51". list of unattached/unmounted volumes=[jenkins-home]"

In detail:

- K8s cluster is deployed:

bash-3.2$ gcloud container clusters list
NAME        ZONE            MASTER_VERSION  MASTER_IP       MACHINE_TYPE   NODE_VERSION  NUM_NODES  STATUS
jenkins-cd  europe-west1-b  1.6.4           104.155.53.141  n1-standard-1  1.6.4         3          RUNNING

- The jenkins image has been created:

bash-3.2$ gcloud compute images list | egrep jenkins
jenkins-home-image                                lol-prod                                                READY


- The disk appears to be READY:

bash-3.2$ gcloud compute disks list| egrep jenkins-home
jenkins-home                               europe-west1-b  10       pd-standard  READY

- However it's attached the one of the instances in the GKE cluster:

----
gcloud compute instances describe gke-lol-prod-default-pool-a8a709b6-wnd7 | less
.
.
.
 - autoDelete: false
  boot: false
  deviceName: jenkins-home
  index: 1
  interface: SCSI
  kind: compute#attachedDisk
  mode: READ_WRITE
  source: https://www.googleapis.com/compute/v1/projects/lol-prod/zones/europe-west1-b/disks/jenkins-home
  type: PERSISTENT
.
.
.
----


- Which appears to lead to the container deployment failing after I create the jenkins deployment:

Apply the deployment:

----
bash-3.2$ kubectl apply -f jenkins/k8s/

deployment "jenkins" created
service "jenkins-ui" configured
service "jenkins-discovery" configured
bash-3.2$
----

Checking on the progress of the deployment:

----
bash-3.2$ kubectl get deployments --namespace=jenkins
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
jenkins   1         1         1            0           3m
----

Pod status remains in creating forever:

---
bash-3.2$ kubectl get pods --namespace=jenkins
NAME                      READY     STATUS              RESTARTS   AGE
jenkins-482090729-8sd51   0/1       ContainerCreating   0          8m
---

Describing the deployment:

----
bash-3.2$ kubectl describe pods --namespace=jenkins
.
.
.
Events:
  FirstSeen     LastSeen        Count   From                                                    SubObjectPath   Type            Reason          Message
  ---------     --------        -----   ----                                                    -------------   --------        ------          -------
  8m            8m              1       default-scheduler                                                       Normal          Scheduled       Successfully assigned jenkins-482090729-8sd51 to gke-jenkins-cd-default-pool-6c37d3fc-jdqf
  8m            35s             5       attachdetach                                                            Warning         FailedMount     Failed to attach volume "jenkins-home" on node "gke-jenkins-cd-default-pool-6c37d3fc-jdqf" with: googleapi: Error 400: The disk resource 'projects/lol-prod/zones/europe-west1-b/disks/jenkins-home' is already being used by 'projects/lol-prod/zones/europe-west1-b/instances/gke-lol-prod-default-pool-a8a709b6-wnd7'
  6m            15s             4       kubelet, gke-jenkins-cd-default-pool-6c37d3fc-jdqf                      Warning         FailedMount     Unable to mount volumes for pod "jenkins-482090729-8sd51_jenkins(740a5566-6217-11e7-bc18-42010af001c2)": timeout expired waiting for volumes to attach/mount for pod "jenkins"/"jenkins-482090729-8sd51". list of unattached/unmounted volumes=[jenkins-home]
  6m            15s             4       kubelet, gke-jenkins-cd-default-pool-6c37d3fc-jdqf                      Warning         FailedSync      Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "jenkins"/"jenkins-482090729-8sd51". list of unattached/unmounted volumes=[jenkins-home]
.
.
.
----


My questions are:

1) Has anyone managed to get Jenkins CI/CD working with GKE following these exact instructions or is there something I'm missing?
2) What can I do to successfully get the jenkins deployment working, given the disk attachment issue is preventing that?

Many thanks in advance,
Traiano

[Traiano Welcome]

Anyone ?

--
You received this message because you are subscribed to a topic in the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kubernetes-users/nGCPdHJex3g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kubernetes-users+unsubscribe@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.

[Rodrigo Campos]

Sorry, I'm not using GKE, so probably my mail is not really useful.

Kubernetes should reconciliate the state and attach it to the proper node. Even if it's attached to some other node by kubernetes. So, I'd expect that to happen automatically. But.. just try forcing detach, if not.

Also, why is it attached to a node? Have you done it? Or was it done by kubernetes. In any case, there is probably a force detach option. If you detach it, it will probably be attached to the proper node.

Does any of this help?

You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscribe@googlegroups.com.