csi-controller: Simple, single, intelligent controller for CSI for Kubernetes

[Luis Pabon]

Hi all,

  Last Monday I discussed at the Kubernetes-CSI community meeting a PoC I was working on. The goal of the PoC is to create a single CSI controller side car to replace all other side cars on deployment. At Portworx, we realized that we had placed a lot of intelligence in our installer to determine which side-car with a specific version to use at a specific version of Kubernetes. We realized that separating the installer intelligence from the system would make it difficult to install.

  Instead, we propose csi-controller[1], a simple, single, intelligent side-car controller for CSI. It vendors in the sources for the side cars and interrogates Kubernetes and the CSI driver to determine which side car to load. We propose that this model makes it simpler to deploy, since it automatically determines what to do.

Please feel free to try it out in your deployment. Here is an example output from the csi-controller logs:

I1028 23:17:05.204711       1 feature_gate.go:216] feature gates: &{map[]}
I1028 23:17:05.204831       1 main.go:165] Version: v0.1-5-g0df2487
W1028 23:17:05.222373       1 main.go:229] Skipping CSI NODE CRD check
I1028 23:17:05.222407       1 connection.go:151] Connecting to unix:///csi/csi.sock
I1028 23:17:05.224130       1 common.go:111] Probing CSI driver for readiness
I1028 23:17:05.225893       1 main.go:251] Detected CSI driver pxd.portworx.com
I1028 23:17:05.229003       1 attacher.go:59] Kubernetes 1.13.x detected which requires the attacher process even if the CSI driver does not support it
I1028 23:17:05.229217       1 provisioner.go:44] Driver pxd.portworx.com supports provisioning
I1028 23:17:05.231177       1 controller.go:680] Using saving PVs to API server in background
I1028 23:17:05.231215       1 snapshotter.go:44] Driver pxd.portworx.com supports snapshots
I1028 23:17:05.243319       1 resizer.go:38] Driver pxd.portworx.com supports resize
I1028 23:17:05.243347       1 connection.go:151] Connecting to unix:///csi/csi.sock
I1028 23:17:05.244133       1 common.go:111] Probing CSI driver for readiness
I1028 23:17:05.247155       1 main.go:327] Using endpoints for leader election in Kubernetes version 1.13.x
I1028 23:17:05.247363       1 leaderelection.go:241] attempting to acquire leader lease  kube-system/csi-controller-leader-pxd-portworx-com...
I1028 23:17:24.892907       1 leader_election.go:172] new leader detected, current leader: px-csi-ext-77ffb6d4f9-7ldtr

I will be bringing this up at the Kubernetes SIG-Storage F2F in Kubecon San Diego for further discussion, but I would like to see if any of you can try it out on your systems and provide some feedback.

- Luis

[1] https://github.com/lpabon/csi-controller

[Gorka Eguileor]

On 28/10, Luis Pabon wrote:
> Hi all,
> Last Monday I discussed at the Kubernetes-CSI community meeting a PoC I
> was working on. The goal of the PoC is to create a single CSI controller
> side car to replace all other side cars on deployment. At Portworx, we
> realized that we had placed a lot of intelligence in our installer to
> determine which side-car with a specific version to use at a specific
> version of Kubernetes. We realized that separating the installer
> intelligence from the system would make it difficult to install.
> Instead, we propose csi-controller[1], a simple, single, intelligent
> side-car controller for CSI. It vendors in the sources for the side cars
> and interrogates Kubernetes and the CSI driver to determine which side car
> to load. We propose that this model makes it simpler to deploy, since it
> automatically determines what to do.
>

Hi Luis,

I think that's a great idea. We were facing the same situation
in Ember-CSI and it was pailful.

I love the auto detection of features in your sidecar. :-)

>
> I will be bringing this up at the Kubernetes SIG-Storage F2F in Kubecon San
> Diego for further discussion, but I would like to see if any of you can try
> it out on your systems and provide some feedback.
>
> - Luis
>
> [1] https://github.com/lpabon/csi-controller

I will definitely try it and let you know how it goes.

Cheers,
Gorka.

[Xiangqian Yu]

Hi Luis,

Is there any document I can get more color for your POC? I missed last community meeting. thanks.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-storage-wg-csi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-stora...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-storage-wg-csi/20191029124556.jyhvp7zm2wjd2inq%40localhost.

[Luis Pabon]

Hi Xiangqian,

  I do not have a document, all I have is the README.md in the repo. 

- Luis

[Michelle Au]

Hi Luis,

Thinking about this some more, I think consolidating the sidecars into one has a lot of nice benefits:

* aligning all the leaders to the same container

* sharing informers across all the sidecars

* simplifying the sharing of common code and test utilities (being able to eliminate csi-lib-utils and csi-release-tools)

However, I don't think our group currently has the bandwidth right now to maintain and test this in our CI, along with the work of deprecating and refactoring all our sidecars and test/release infrastructure to use this, while we're also still making substantial changes to some of the newer sidecars for new features. We already have many repos that have been languishing in kubernetes-csi because we don't have owners committed to maintaining them to our standards in the long term.

I also have some doubts on how the automatic K8s version detection will work when dealing with K8s alpha->beta->GA APIs and importing multiple versions of the same sidecar.

For the time being, I'm in favor of keeping the repo where it currently is, seeing if the model works well longer term, and we can consider adopting it into the SIG once it gains adoption, community support and infrastructure, and community commitment to restructuring our current sidecar infrastructure to support this.

-Michelle

To view this discussion on the web visit https://groups.google.com/d/msgid/kubernetes-sig-storage-wg-csi/CAEvoFsG1TwYYBd4kw2ACgJwdgEk4d71-vPeVMnTD0dyr9ztxpQ%40mail.gmail.com.

[Luis Pabon]

Hi Michelle,

  That is fine with me. We can continue working on it on my repo in the meantime. Also, I'm not sure importing multiple versions of the same sidecar would work either. 

We'll mature the csi-controller on my repo and provide feedback again, hopefully at the next kubecon.

- Luis