Add a ModifyVolume to CSI Controller Spec

[Rishabh Singhvi]

Hey folks,

I'm a software engineer at AWS, and one of the maintainers of the AWS EBS CSI Driver.

Certain storage providers (such as AWS/Azure), support on-line modification of volumes; however, users can't access those features through the Kubernetes control plane due to the lack of a generic 'modify volume' RPC.

We've put together a doc on how we think we can implement a 'ModifyVolume' RPC in the CSI Controller spec, and would love the CSI community's thoughts on it.

The doc can be found here.

Our eventual goal is to reach consensus within the CSI and K8s community on how to implement such a feature, and submit a KEP to do the same.

Thanks,
Rishabh

[Sandeep P S]

Hi Rishabh,

We have the same use cases in VMware vSphere where we have vSphere APIs to change the properties of the persistent volume and would like to expose that to DevOps/k8s users. This feature enhancement in CSI will greatly benefit our use cases as well. Btw, the topic of updating the volume(aka ModifyVolume in your proposal) had come up in the sig-storage meetings in the past, and IIRC the suggestion was to use a vendor-specific way to achieve that(ex CRDs). I think its worth exploring the direction you are taking here to see if this is a common requirement across cloud providers.

Thanks,

Sandeep

--
You received this message because you are subscribed to the Google Groups "container-storage-interface-community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to container-storage-interf...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/container-storage-interface-community/03d5dd61-752c-4ce1-b9be-f416456b99efn%40googlegroups.com.

[Saad Ali]

This would be a good topic to discuss at next Kubernetes Storage SIG meeting.

The challenge mutating storageClass related properties is that 1) storageClass is only supposed to apply at provision time, and 2) the specific parameters themselves are opaque to CO. 

In my head to enable mutation of something like IOPS you first need to promote IOPS to a common PVC field (like capacity) -- and doing so in a way that is consistent across all storage providers is going to be difficult.  

To view this discussion on the web visit https://groups.google.com/d/msgid/container-storage-interface-community/CAG5n6jwUGFUk_S31EeTbbmiXS7D7uC6pLUfU-mohzLVZtKu8wg%40mail.gmail.com.

[yingqi ge]

hello, i'm the maintainer of the  alibaba-cloud-csi-driver.

We also have this need about on-line modification of volumes; We currently implement it by way of crd. But it is very inconvenient and requires the installation of additional components

hope the community can actively promote this proposal~ 

Thanks

yingqi.ge

[Ripul Patel]

I agree with Saad here. In this proposal, migration is still done before coming to CO and API is for CO to merely update the storageClass in PVC, is that correct? If so, this is updating a display error in a way and nothing else.

The best end user experience would be to kick off an end to end volume migration(with new CSI APIs just like volume-expand) work-flow when storage class is changed for a given PVC and with that CO does not have to worry about any SP specific details like it does today.

Thanks,

Ripul

To view this discussion on the web visit https://groups.google.com/d/msgid/container-storage-interface-community/CABBBJP3HovsXDCn7mRj-cnHv2fFDjQwTZVzji5K%3D%2BZR7N9kvBg%40mail.gmail.com.

[Simon Croome]

Hi, I work on the Azure Container Storage team, and we're very interested in enabling this functionality too.  I'd like to help any efforts.

Our motivation is to allow dynamic, per-volume configuration of iops and throughput via the Kubernetes API.  My colleagues presented an approach a while back: https://www.youtube.com/watch?v=jfkKP4U4wik&t=1445s. 

We suggested extending the request object:

resources:
  requests:
    storage: 4Gi
    throughput: "1.2M"

    iops: "160000" 

We felt that this would be the least surprising API, but it would need agreement on which resources to add. Alternatively, the device plugin syntax could be used:

  resources:
    requests:
      azure.io/throughput: "1.2M"
      azure.io/iops:  "160000" 

The feedback at the time (2+ years ago!) was to create a ConfigMap for each tier, and use lookup rules similar to CSI secrets to apply them to PVCs.

Whatever approach is taken in Kubernetes, a ModifyVolume CSI endpoint seems valuable. It would allow vendors to write their own controllers to support different use cases (i.e. per-volume vs performance classes).

Looking forward to future discussions.

Simon

[Matthew Cary]

If the problem you're trying to solve is QoS / IO provisioning, join the conversation on the QoS doc we started a couple of weeks ago!

I think one of the key issues is to define what you're actually trying to solve. For example, above it's been suggested both to change the volume and also to add resources to the pod---but those are quite different; for example when doing it at the pod level what do you do in a ROX case where the maximum IO for a volume is exceeded?

Similarly, with modifying the volume are you talking about changing the PV or the PVC? If the PV, typically only an admin has privileges to change the cluster-level object and not an application dev. If the IO request lives on the PVC, then the application dev can change it, but practically there needs to be quota and other cluster level management. That should probably live in the storage class... but the details need to be worked out.

In the doc at the top of the thread, the suggestion is to make the storage class modifiable. This opens up some edge cases (eg is it okay to change the provisioner of a volume?), as well as preventing application developers from having fine grained control over IO requests (because they'd be limited to selecting from the set of storage classes).

Anyway because of these issues, in the QoS doc I'm trying to start by pinning down what problems we're trying to solve before designing the solution. Please join the conversation! (and I'll link the doc above into it).