DataVolume Support PVC in other namespace as a source

[Yafei Bao]

DataVolume can not use PVC of other namespace as a source. If we need to create VMs in different namespaces, we need to create a OS Image PVC in every namespace.

We could solve this problem by supporting PVC from a given namespace (e.g. kubevirt ) as a data source. We can achieve this with following method:

1. Create data volume centos in kubevirt namespace.

2. When pvc centos is ready with image content, manually create snapshot using apis from storage provider.

3. Import the snapshot into dest namespace following https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/#importing-an-existing-snapshot-with-kubernetes

4. Create PVC from newly improted VolumeSnapshot.

What do you think about this method?

[Alexander Wels]

Hi, what you are describing has been in CDI for a while now. Let me rephrase this a little bit so I am sure I understand what you are trying to do.

1. You create a DV in namespace1, let's call this DV source. This creates a matching PVC in that namespace. You import/upload a disk into that DV.

2. Now you want to have a copy of that DV in namespace2, let's call this DV workload.

3. You want to clone source from namespace1 to workload in namespace2?

If so you can use the following yaml in namespace2:

apiVersion: cdi.kubevirt.io/v1alpha1
kind: DataVolume
metadata:
  name: workload
  namespace: namespace2
spec:
  source:
    pvc:
      namespace: namespace1
      name: source
  pvc:
 storageClassName: <your storage class> #you can omit if your storage class is the default
  accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 10Gi

The user running this will need to have appropriate permissions in both namespaces for this to work, for details see [0]. Unfortunately this uses the slow cloning (copy byte by byte) method as AFAIK you cannot snapshot across namespaces. We are working with the storage sig to try and get namespace transfer for objects into kubernetes, so we can do a snapshot clone in namespace1 and then transfer it to namespace2 which should all be fast operations. So for now we are stuck with the copy byte by byte method.

Alexander

[0] https://github.com/kubevirt/containerized-data-importer/blob/master/doc/clone-datavolume.md

--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/088f345f-211b-435b-80e0-eb11d4daab4f%40googlegroups.com.

[Yafei Bao]

What I really want is no real copy actions to achieve this, it slows VM creation process.

We can not transfer VolumeSnapshot object to other namespace, but we can manually import snapshot into arbitrary namespace. See Importing an existing snapshot with Kubernetes.

If we have permission to create snapshot through storage api (not CSI), then we directly create snapshot for a volume in storage system, and then import this existing snapshot into any namespace we want.

在 2019年11月15日星期五 UTC+8下午8:45:20，Alexander Wels写道：

To unsubscribe from this group and stop receiving emails from it, send an email to kubevi...@googlegroups.com.

[Alexander Wels]

On Sun, Nov 17, 2019 at 8:51 PM Yafei Bao <baoy...@gmail.com> wrote:

What I really want is no real copy actions to achieve this, it slows VM creation process.

We can not transfer VolumeSnapshot object to other namespace, but we can manually import snapshot into arbitrary namespace. See Importing an existing snapshot with Kubernetes.

If we have permission to create snapshot through storage api (not CSI), then we directly create snapshot for a volume in storage system, and then import this existing snapshot into any namespace we want.

If you look at the "importing an existing snapshot with Kubernetes", you will notice this line: The namespace of the source VolumeSnapshot object is assumed to be the same as the namespace of the PersistentVolumeClaim object.

This is a very important line, and it prevents what you are suggesting. You cannot create a PVC from a snapshot in a different namespace, the datasource structure doesn't contain namespace.

We are working with the kubernetes storage sig, to provide a namespace transfer system to solve this issue. As soon as that is available CDI will likely implement it to do exactly what you want. Until that time, you are unfortunately stuck with the slow copy version as there is no other way to transfer the data between the namspaces.

 

To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/fe4af194-a67d-4922-be7f-f77bf36720ab%40googlegroups.com.

[Yafei Bao]

Let me clarify using Ceph for example：

1. Create datavolume centos in kubvirt 

2. Find out corresponding volume csi-vol-xxx

3. Manually create snapshot for csi-vol-xxx using ceph command

rbd snap create pool/csi-vol-xxx@test

4. Import the snapshot into dest namespace following https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/#importing-an-existing-snapshot-with-kubernetes
VolumeSnapshot and PVC both in namepsace dest

This needs perrmission to create snap directly using ceph api.

在 2019年11月18日星期一 UTC+8下午9:10:32，Alexander Wels写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/fe4af194-a67d-4922-be7f-f77bf36720ab%40googlegroups.com.

[Alexander Wels]

On Mon, Nov 18, 2019 at 10:24 PM Yafei Bao <baoy...@gmail.com> wrote:

Let me clarify using Ceph for example：

1. Create datavolume centos in kubvirt 

2. Find out corresponding volume csi-vol-xxx

3. Manually create snapshot for csi-vol-xxx using ceph command

rbd snap create pool/csi-vol-xxx@test

4. Import the snapshot into dest namespace following https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/#importing-an-existing-snapshot-with-kubernetes
VolumeSnapshot and PVC both in namepsace dest

This needs perrmission to create snap directly using ceph api.

Okay gotcha, from a technical point of view it will likely work, but you have now introduced storage provider specific information into your process. You can make a controller that does this for ceph, but maybe later you will want to use a different storage provider and now you have to alter your controller to support that provider. You are also circumventing some of the security features of kubernetes about moving data across namespaces through a side channel (the storage api snapshot). Note the byte copy also bypasses the kubernetes controls, but we added other controls.

Again if you can wait a bit, once the transfer mechanism lands in kubernetes we can do the snapshot cloning and then transfer to the correct namespace.

 

To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/9cff59c3-a14e-4f9f-8ae4-fdf405d06906%40googlegroups.com.

[Yafei Bao]

Is there relate PRs or Proposals where i can track transfer mechanism?

在 2019年11月19日星期二 UTC+8下午10:53:50，Alexander Wels写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/9cff59c3-a14e-4f9f-8ae4-fdf405d06906%40googlegroups.com.

[Alexander Wels]

On Tue, Nov 19, 2019 at 9:18 PM Yafei Bao <baoy...@gmail.com> wrote:

Is there relate PRs or Proposals where i can track transfer mechanism?

Yes, here is the kep: https://github.com/kubernetes/enhancements/pull/1112

 

To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/f93f4b73-3124-4e18-bd3e-87baa3c1c28b%40googlegroups.com.