KubeVirt VM Export/Import Format?
1,043 views
Skip to first unread message
David Vossel
Jun 16, 2021, 3:57:58 PM6/16/21
to kubevirt-dev
Hey,
An interesting topic is floating around the potential of Exporting and Importing KubeVirt virtual machines as container images. Think of something like a "container disk" that contains the VM spec as well. For us to make progress on this I'd like to remove the container image part in this theoretical example for now simply because it's just a delivery and storage mechanism. We could replace the storage mechanism with any object store (like s3 for example), and we'd still have the issue that we don't have a format for exporting/importing KubeVirt virtual machines.
OVF/OVA [1] is an open standard format for import/export of virtual machines. However, looking deeper at this standard, there's not a whole lot that is standard about it. An OVA created for one platform isn't necessarily going to work anywhere else. We could adopt the OVF format for bundling KubeVirt VirtualMachines as an OVA (vm appliance), but it's unclear what the value is. If we're talking about a non-standard standard, then why conform to the standard at all?
So I guess my question here for the community is, should we consider creating our own standard for export/import of KubeVirt virtual machines, or is there value in adopting an existing standard?
Thanks,
- David
Arik Hadas
Jun 16, 2021, 6:10:34 PM6/16/21
to David Vossel, kubevirt-dev
Hi,
My 2 cents on this from the experience we have with OVA in oVirt:
1. Yes, the OVF standard is pretty loose (so loose that in oVirt we used to have two completely different parses for OVFs we created and for OVFs from vSphere).
2. Yes, OVA that is generated by one platform cannot be consumed by another platform without conversion if they use different disk formats, hypervisors, guest tools and so on.
But I do see value in trying to align to a common standard.
The OVF specification defines the basic properties that are relevant for all platforms (the format - tar\directory, the file structure, how disks and other resources are referenced, how to describe basic VM properties like memory and CPU, OS type, etc).
While the Konveyor project looks promising, it mainly targets migration from one platform to another. It doesn't seem to fit a use case of importing just a few OVAs.
So imagine you comply with the OVF specification, then you can do the following from your platform:
1. Query the OVF from the OVA (you should be able to get it the same way, no matter what platform generated it)
2. Retrieve and present the basic information about the VM like name, memory, OS type (again, the same way no matter what platform generated it)
3. Allow the user to map the networks and storage to the one in your platform (and maybe additional changes)
4. Initiate the import
5. Add the missing features
Step 4 changes depending on the platform that generated the OVA.
If it's vSphere/Citrix and such, virt-v2v would need to be trigger to adjust the guest, convert the disks and upload them using CDI as it does for oVirt using imageio; and at the end of the process provide you with an up-to-date OVF that includes installed drivers and such to use in step 5.
Importing an OVA that was generated by oVirt should be similar to importing an OVA that will be generated by KubeVirt as both use the same hypervisor. The oVirt-specific properties that reside within the OVF in a different namespace would need to be parsed differently though.
It's not like you can't achieve that without using the same format but having the same format simplifies this process.
And if you're going with OVA, it's better to save the disks in QCOW format to preserve sparseness and optionally use the built-in compression of QCOW as the OVF specification doesn't permit compressing the whole package. But if you're converting the disks to achieve this then it might be even better to store them in VMDK that has these features and that can ease moving workloads to other virtualization platforms if one wishes to.
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAPjOJFvLB5Nmh%2B1-ZHF-iyNviEWNcihvyvboQWXsFo%2B%2BgTWCXw%40mail.gmail.com.
Mark DeNeve
Jun 21, 2021, 10:31:46 AM6/21/21
to kubevirt-dev
I would agree with Arik. While the OVA/OVF format is pretty loose, it is at least some form of standard. (insert obligatory xkcd image here: https://xkcd.com/927/ ) Anyone leveraging existing operational knowledge from oVirt, Vmware, Proxmox, Nutanix (etc) will be able to relate to this paradigm of exporting a VM and recognize the file format as they all support this concept. With respect to disk format, I agree either exporting it at qcow or vmdk would be best as they can both preserve the sparseness. No matter what format they are stored in ensuring that the files are stored optimally is best.
I do think the idea of being able to export to a OCI compliant registry could be very powerful (as well as to any arbitrary object storage), but starting "simple" and being able to just export out an image to an OVA/OVF that I can download onto my laptop/desktop will be very helpful and give kubevirt more "feature-parity" when compared with other virtualization platforms.
Mark
Adam Litke
Jun 23, 2021, 10:42:16 AM6/23/21
to David Vossel, kubevirt-dev
I think it depends on the use case(s) that you are trying to target. If we are primarily focused on mobility of VMs between different kubevirt installations (and backup) then maintaining marshal/unmarshal code for yet another format (OVF) seems unnecessary and wasteful. If we want to support conversion to/from other systems then this whole topic/feature becomes much bigger. My preference would be to support basic import/export using a native format. If we want to enable conversion, support for the kubevirt VM format could be added to libguestfs. Libguestfs already knows how to convert disk formats, OS drivers, etc. If done this way, we can combine existing, well-established tools to support a wide variety of workflows.
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAPjOJFvLB5Nmh%2B1-ZHF-iyNviEWNcihvyvboQWXsFo%2B%2BgTWCXw%40mail.gmail.com.
Arik Hadas
Jun 23, 2021, 12:04:14 PM6/23/21
to Adam Litke, David Vossel, kubevirt-dev
On Wed, Jun 23, 2021 at 5:42 PM Adam Litke <ali...@redhat.com> wrote:
On Wed, Jun 16, 2021 at 3:58 PM David Vossel <dvo...@redhat.com> wrote:Hey,An interesting topic is floating around the potential of Exporting and Importing KubeVirt virtual machines as container images. Think of something like a "container disk" that contains the VM spec as well. For us to make progress on this I'd like to remove the container image part in this theoretical example for now simply because it's just a delivery and storage mechanism. We could replace the storage mechanism with any object store (like s3 for example), and we'd still have the issue that we don't have a format for exporting/importing KubeVirt virtual machines.OVF/OVA [1] is an open standard format for import/export of virtual machines. However, looking deeper at this standard, there's not a whole lot that is standard about it. An OVA created for one platform isn't necessarily going to work anywhere else. We could adopt the OVF format for bundling KubeVirt VirtualMachines as an OVA (vm appliance), but it's unclear what the value is. If we're talking about a non-standard standard, then why conform to the standard at all?So I guess my question here for the community is, should we consider creating our own standard for export/import of KubeVirt virtual machines, or is there value in adopting an existing standard?I think it depends on the use case(s) that you are trying to target. If we are primarily focused on mobility of VMs between different kubevirt installations (and backup) then maintaining marshal/unmarshal code for yet another format (OVF) seems unnecessary and wasteful. If we want to support conversion to/from other systems then this whole topic/feature becomes much bigger. My preference would be to support basic import/export using a native format. If we want to enable conversion, support for the kubevirt VM format could be added to libguestfs. Libguestfs already knows how to convert disk formats, OS drivers, etc. If done this way, we can combine existing, well-established tools to support a wide variety of workflows.
I agree that it depends on the use case.
I'd just like to comment that I wouldn't limit the use of OVA to "conversion to/from other systems".
I think that a use case that is more likely to be required from Kubevirt in the short/mid term would be importing a third-party application from some vendor that is packaged as an appliance. It would most likely be an OVA that complies with the OVF specification and includes virtual disks in the VDSM format that users would like to deploy to their Kubernetes cluster using Kubevirt (assuming the appliance cannot be converted to container(s)).
----
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAPjOJFvLB5Nmh%2B1-ZHF-iyNviEWNcihvyvboQWXsFo%2B%2BgTWCXw%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAG8F9UGuisZJX-_TWX4gLidW51pTuXbadYEtmqSS9SeCHD4O_A%40mail.gmail.com.
Arik Hadas
Jun 23, 2021, 12:04:36 PM6/23/21
to Adam Litke, David Vossel, kubevirt-dev
On Wed, Jun 23, 2021 at 7:03 PM Arik Hadas <aha...@redhat.com> wrote:
On Wed, Jun 23, 2021 at 5:42 PM Adam Litke <ali...@redhat.com> wrote:On Wed, Jun 16, 2021 at 3:58 PM David Vossel <dvo...@redhat.com> wrote:Hey,An interesting topic is floating around the potential of Exporting and Importing KubeVirt virtual machines as container images. Think of something like a "container disk" that contains the VM spec as well. For us to make progress on this I'd like to remove the container image part in this theoretical example for now simply because it's just a delivery and storage mechanism. We could replace the storage mechanism with any object store (like s3 for example), and we'd still have the issue that we don't have a format for exporting/importing KubeVirt virtual machines.OVF/OVA [1] is an open standard format for import/export of virtual machines. However, looking deeper at this standard, there's not a whole lot that is standard about it. An OVA created for one platform isn't necessarily going to work anywhere else. We could adopt the OVF format for bundling KubeVirt VirtualMachines as an OVA (vm appliance), but it's unclear what the value is. If we're talking about a non-standard standard, then why conform to the standard at all?So I guess my question here for the community is, should we consider creating our own standard for export/import of KubeVirt virtual machines, or is there value in adopting an existing standard?I think it depends on the use case(s) that you are trying to target. If we are primarily focused on mobility of VMs between different kubevirt installations (and backup) then maintaining marshal/unmarshal code for yet another format (OVF) seems unnecessary and wasteful. If we want to support conversion to/from other systems then this whole topic/feature becomes much bigger. My preference would be to support basic import/export using a native format. If we want to enable conversion, support for the kubevirt VM format could be added to libguestfs. Libguestfs already knows how to convert disk formats, OS drivers, etc. If done this way, we can combine existing, well-established tools to support a wide variety of workflows.I agree that it depends on the use case.I'd just like to comment that I wouldn't limit the use of OVA to "conversion to/from other systems".I think that a use case that is more likely to be required from Kubevirt in the short/mid term would be importing a third-party application from some vendor that is packaged as an appliance. It would most likely be an OVA that complies with the OVF specification and includes virtual disks in the VDSM format that users would like to deploy to their Kubernetes cluster using Kubevirt (assuming the appliance cannot be converted to container(s)).
VDSM -> VMDK
Fabian Deutsch
Jun 25, 2021, 11:24:09 AM6/25/21
to Adam Litke, David Vossel, kubevirt-dev
On Wed, Jun 23, 2021 at 4:42 PM Adam Litke <ali...@redhat.com> wrote:
On Wed, Jun 16, 2021 at 3:58 PM David Vossel <dvo...@redhat.com> wrote:Hey,An interesting topic is floating around the potential of Exporting and Importing KubeVirt virtual machines as container images. Think of something like a "container disk" that contains the VM spec as well. For us to make progress on this I'd like to remove the container image part in this theoretical example for now simply because it's just a delivery and storage mechanism. We could replace the storage mechanism with any object store (like s3 for example), and we'd still have the issue that we don't have a format for exporting/importing KubeVirt virtual machines.OVF/OVA [1] is an open standard format for import/export of virtual machines. However, looking deeper at this standard, there's not a whole lot that is standard about it. An OVA created for one platform isn't necessarily going to work anywhere else. We could adopt the OVF format for bundling KubeVirt VirtualMachines as an OVA (vm appliance), but it's unclear what the value is. If we're talking about a non-standard standard, then why conform to the standard at all?So I guess my question here for the community is, should we consider creating our own standard for export/import of KubeVirt virtual machines, or is there value in adopting an existing standard?I think it depends on the use case(s) that you are trying to target.
+1
If we are primarily focused on mobility of VMs between different kubevirt installations (and backup) then maintaining marshal/unmarshal code for yet another format (OVF) seems unnecessary and wasteful. If we want to support conversion to/from other systems then this whole topic/feature becomes much bigger. My preference would be to support basic import/export using a native format.
+1 this is also the use-case I'd focus on first as well.
A loss-less way to simply export a VM from a cluster, and import it again to recreate it in the very same way.
container + yaml + sparse disks look straight forward - and probably are on the container level. Which in essence is a tarball.
But let's not assume that these container images will be straight forward to be used with i.e. container registries, simply because registries have limits, i.e. layer limits, which in turn then limit the (sum of) disk size(s). IOW you can export a VM into a container, but you can not necessarily push this container to any arbitarry registry. Even worse: This image could DOS your node.
If we want to enable conversion, support for the kubevirt VM format could be added to libguestfs. Libguestfs already knows how to convert disk formats, OS drivers, etc. If done this way, we can combine existing, well-established tools to support a wide variety of workflows.
----
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAPjOJFvLB5Nmh%2B1-ZHF-iyNviEWNcihvyvboQWXsFo%2B%2BgTWCXw%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CAG8F9UGuisZJX-_TWX4gLidW51pTuXbadYEtmqSS9SeCHD4O_A%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages