Questions on CSI migration design spec

[Sandeep P S]

Hi,

I'm from VMware and we built a new vSphere CSI driver that talks to new storage APIs in vSphere to provision and manage volumes. I'm analyzing how we can support migrating vSphere customers from in tree vSphereVolume plugin to new CSI driver. So, I went through the CSI migration design spec - https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-migration.md and I have these questions. It will be super helpful if somebody here clarified my questions:

1. It was not clear how the storage class will be translated/migrated to CSI. I did not see explanation around it. Does the migration shim hide the requirement of migrating storage class to CSI? It looked like that but some details on how it would work end to end will be helpful (see my next question)
2. Is there a doc that captures the details of the migration end to end and what every cloud provider need to do in this workflow? I saw high level explanation of the workflow, translation library/interface etc in the spec and I was wondering if this is explained in detail somewhere.
3. Has any vendor implemented migration as per the spec? If so, could you please send github links to the code?
4. vSphereVolume in tree plugin uses legacy storage API to provision volumes, whereas the new CSI uses modern cloud native storage APIs. So, the backend storage object in vSphere needs to be migrated as we migrate persistent volume objects in Kubernetes from in-tree to CSI. The spec does not talk about upgrading backend storage objects and assumes that it does not need any change, and only the Kubernetes objects needs to be translated etc, correct me if I'm wrong. Given that we need to upgrade the backed storage objects and the Kubernetes API objects (storage class, persistent volume etc), can the spec be modified to accommodate this requirement? If so, how would the end to end migration workflow look like?

Thanks,

Sandeep

[Saad Ali]

+David Zhu +Deep Debroy +Cheng Pan

--
You received this message because you are subscribed to the Google Groups "container-storage-interface-community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to container-storage-interf...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/container-storage-interface-community/42a0e251-6a27-45a4-8c59-411d75860603%40googlegroups.com.

[Pan, Cheng]

> It was not clear how the storage class will be translated/migrated to CSI. I did not see explanation around it. Does the migration shim hide the requirement of migrating storage class to CSI? It looked like that but some details on how it would work end to end will be helpful (see my next question)

The idea of CSI migration is to keep all the user facing API objects (PVC, PV, Storageclass) as it is and delegate all the APIs call to the new storage backend (CSI driver). Given that, the in-tree storage class will be kept the same, and user will be able to use the same storage class with CSI driver once migration is enabled. The actual translation logic depends on the differences between in-tree storage class and CSI storage class. For example, EBS CSI storageclass supports all the in-tree storageclass parameters (not including zone,zones since they are deprecated), so that the storageclass translation is not implemented. On the other side, GCE PD translation implemented translation for zone/zones and fstype.

>Is there a doc that captures the details of the migration end to end and what every cloud provider need to do in this workflow? I saw high level explanation of the workflow, translation library/interface etc in the spec and I was wondering if this is explained in detail somewhere.

David has a CSI migration Developer Guide that covers more details about many aspects of this. But I’m not sure if this is made public yet?

> Has any vendor implemented migration as per the spec? If so, could you please send github links to the code?

Both GCE PD, AWS EBS, Azure Disk/File and Openstack have implemented their own shims. See here for you reference

 

For your 4^th question, I think CSI migration focus on the migration of the storage driver assuming the storage backend is the same. Your problem feels include both migration of the driver as well as migration of the storage backend. Then I would say there are two different drivers, why bother migrate?

 

Thanks,

-- 

Cheng

[Sandeep P S]

> David has a CSI migration Developer Guide that covers more details about many aspects of this. But I’m not sure if this is made public yet?

It will be helpful if this is made public.

> For your 4^th question, I think CSI migration focus on the migration of the storage driver assuming the storage backend is the same. Your problem feels include both migration of the driver as well as migration of the storage backend. Then I would say there are two different drivers, why bother migrate?

The newer APIs and drivers offers useful functionalities to both vSphere admin (who manages the infrastructure) and devops/kubernetes users (who are provisioning/using volumes). So, both driver and migration of storage object is important. 

Thanks,

Sandeep

[David Zhu]

Hi Sandeep,

The CSI Driver Migration Developer Guide: https://docs.google.com/document/d/16h9vLLbqQCN83KnlPtn7qL_GB1TYwhWkoA5G4FS3KBI/edit?usp=sharing

Please comment on the doc if it is missing information or confusing and I will try to resolve it as soon as possible. Hope this helps.

As discussed in our meeting today there are some intricacies for VMWare since the new and old backends differ and are not compatible with each other. I would be happy to hop on a call and discuss this in more detail. There are a variety of ways you could approach this but I would be happy to provide my input.

Best,

David

--

David Zhu | Software Engineer | dy...@google.com | 412-436-6859

[Sandeep P S]

David,

Thanks for sending the doc, I left a comment there. For VMware storage plugin upgrade to CSI, it looks like the right place to convert legacy storage objects to new storage objects is in the vSphere specific translation library implementation. It appears like the translation library will be invoked only from external-provisioner and external-attacher, it was not clear how the library will be invoked in case of rollback (user wants to go back from CSI to in tree). Also, please share if you have any thoughts on how and where we could potentially upgrade/downgrade the vSphere storage objects.

Thanks,

Sandeep

[David Zhu]

Hi Sandeep,

Answered your comment in the doc.

The translation library is intended to be a thin in-memory API Object (struct) translator. The actual API objects in the API Server are unaffected. The translation library should be invoked only when necessary to shim structs for handoff to native in-tree/csi functions respectively. Therefore, this would not be the spot to convert legacy storage objects to new storage objects in the way you mention. It is not intended for that nor would it work the way I believe you want.

It appears like the translation library will be invoked only from external-provisioner and external-attacher

The library is also invoked in k8s/k8s in the Kube Controller Manager, PV Controller, and Volume Plugin Manager.

it was not clear how the library will be invoked in case of rollback (user wants to go back from CSI to in tree).

In case of roll-back the library will not be invoked. Since it is a thin in-memory shim if it is not invoked to translate the objects everything will simply revert to functioning normally (before Migration).

 

Also, please share if you have any thoughts on how and where we could potentially upgrade/downgrade the vSphere storage objects.

I don't have any thoughts about this. Unfortunately I believe this is an orthogonal problem to what CSI Migration is attempting to solve. The solution here is to support in-tree volume APIs indefinitely by shimming the backend - not to "upgrade" old API's or "storage objects."

Hope this answers your questions.

Best,

David