VM and VMConfig APIs

[Fabian Deutsch]

Hey,

just wanted to get the discussion started on the relationship between
VM and VMconfig.
This email is hovering around the API responsibilities, and not really
how they are implemented.

The reason for these two layers are control & convenience.

The VM API is responsible for providing the rightmechanics for all VM
related aspects.
Functional wise it should be close the libvirtd domxml, and make sure
that the features which are exposed on the VM API are implemented and
working correctly.
Thus, the responsibility of the VM API is to provide integrated and
correct working virtualization primitives.

VMConfig is building upon them, and provides a convenience API, which
is more user friendly.
It leverages the mechanics - and totally relies on their correct
implementation - of the VM API.
VMConfig API actually needs to be much more opinionated than VM API is
to provide a friendly API.

This separation has a few benefits.
VM API can exclusively focus on implementing virtualization primitves
on the cluster level correctly. This is probably much about handler,
libvirt, kubelet, pods, cgroups etc.

VMConfig on the other hand is more focused on providing a good API
suited for consumption by users directly (in letting them write simple
specs) or by a UI.

This separation seems to work out well. if we look at libvirt then we
see - regardless how "good" the API is - that the primitives it
exposes are good choosen, and a range of projects could use libvirt
for their (opinionated) use-cases.
I'm quite certain that we don't know yet of all possible use-cases for
virtualization on Kubernetes today. Thus it just sounds right to keep
the VM API exposing primitives, to allow components to consume this
API as needed.

This separation also permits a different pace for VM API and VMConfig
API, which is also a benefit.


- fabian

[Martin Sivak]

Hi,

the only part of this separation I do not understand is the exact
behavioral relationship between VMConfig and the VM.

I see two major scenarios:

- VMConfig represents the configuration of a single VM (independently
on its current state).

This case actually does not need a separate entity, just a separate
section. I can imagine a service that takes the global VM, reads the
high level VM.Config section and generates / updates the VM.Spec
section inside the same object (no extra queries to api server are
necessary).

Per-VM override is trivial as VM.Config is the object to change and
VM.Spec will be updated automatically.

Stopped VMs still look the same and keep the last VM.Spec computed for them.
Each VM is defined by its own VM object. Pools and templates are
represented outside of kubevirt (which might be necessary anyway, see
below).

- VMConfig represents a template of sorts and is separate from VM

This case allows using single VMConfig for multiple VMs (and VM
objects). But the cost is complexity:
- Any querying will have to call apiserver multiple times to
retrieve all the information
- Per VM values like MAC address assignments or PCI passthrough
device addresses will have to be kept separately in the VM object
- Some computed values (like the memory overhead) will have to use
either two sources of information or keep high level copy in the VM
object as well.

And then there comes the update scenario.. what is supposed to
happen when VMConfig is updated? Are all information supposed to be
transferred to all VMs that were created accoding to the VMConfig
object? Or just some of them? What about per-VM overrides (temporary
change in resource allocation for example - and we will need those)?


I think we need to decide between those two usage patterns and if you
are really set on the path to the second one, then you have to
describe the mechanics of updates and local overrides properly.

Martin Sivak

> --
> You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
> To post to this group, send email to kubevi...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/CA%2BPVUaSEf0pLv4inNt9eivkf-7rqzTKa9Yp56jo55RmomZ85TQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[David Vossel]

On Fri, Sep 1, 2017 at 8:20 AM, Fabian Deutsch <fdeu...@redhat.com> wrote:

Hey,

just wanted to get the discussion started on the relationship between
VM and VMconfig.
This email is hovering around the API responsibilities, and not really
how they are implemented.

The reason for these two layers are control & convenience.

The VM API is responsible for providing the rightmechanics for all VM
related aspects.
Functional wise it should be close the libvirtd domxml, and make sure
that the features which are exposed on the VM API are implemented and
working correctly.
Thus, the responsibility of the VM API is to provide integrated and
correct working virtualization primitives.

VMConfig is building upon them, and provides a convenience API, which
is more user friendly.
It leverages the mechanics - and totally relies on their correct
implementation - of the VM API.
VMConfig API actually needs to be much more opinionated than VM API is
to provide a friendly API.

Thanks for kicking off this discussion!

I've been super confused trying to come to grips with all the ideas that are floating around. We're having a bit of an issue with term overlap that is causing these discussions to be less efficient than they could be. 

I have a couple of thoughts. I'll break them up into two separate sections.

====================================

VMConfig terminology inconsistencies

====================================

We're overloading the term VMConfig/VirtualMachineConfig/VMC and we need to untangle these concepts.

- This email is focusing on the VMConfig concept containing a convenience API used to define a VM in KubeVirt.

- In this PR (https://github.com/kubevirt/kubevirt/pull/352) the convenience api is referred to as the "features" spec which is present in both the VMConfig and VM cluster objects.

- In our "Config And Instances" proposal we discuss a VMConfig being a separate cluster object from the VM object that is tasked with persisting non-running VMs between restarts. The specs provided by VM and VMConfig in this proposal are the same.

- Our glossary references a VMConfig as being a concept that will be a separate cluster object from the VM similar to the "Config and Instances" proposal.

Its difficult to discuss any of this because I have no idea what terms to use to communicate what... Further more I can't agree to anything because I don't know the scope of what I'm agreeing to.  

==================================================

Config Spec vs. Domain spec vs. Libvirt Domain XML

==================================================

I'd like to challenge why a layered API approach that utilizes a separate convenience API is necessary.  I understand this concept of a split between the API used by a UI vs the backend representation is something both OpenStack and RHV have landed on. However, I want to make sure we have a good reason to replicate this pattern in KubeVirt.

This pattern of having a simplified API mapped to a more complex representation is not used in k8s (at least in the way that's being proposed here). That simplicity is part of the reason k8s abstractions are so powerful.

If we look at a k8s representation of a container spec, the mapping between the k8s representation of a container and the docker primitives are not 1:1. 

Using that same pattern, I don't see why the mapping between the KubeVirt representation of a VM and the libvirt primitives should ever need to be 1:1.

Without requiring a 1:1 mapping, we don't need this layered API approach. We could have a single VM api that we map locally to libvirt domain xml primitives when the VM is started. 

I'd much rather us expose a single API that does not map 1:1 to libvirt domain xml rather than trying to maintain this layered relationship between a convenience API, 1:1 domain api, and libvirt domain xml. 

Can we walk through a concrete example of how this layered approach benefits us? It definitely isn't obvious to me. 

Thanks!

-- David 

[Fabian Deutsch]

Hey,

this thread went quite to quickly, but seems to have speed up in #352.
It seems that there are a range of goals that all of us try to tackle
or project into VM and VMConfig.

- Stopped VMs
- repeoducible/persistent VMs
- friendly (twoards whom?) API

To just name a few.

To find a solution I'd like to understand the use-cases and reasons
all of us have, thus I'd like to setup a face to face early next week
to exchange the points of view
See the document below for all the glory details.

To have an efficient meetup every stakeholder should make sure to
express his requirements, goals, constraints in the following document
- prior to the meeting:

https://goo.gl/7sz1vr

Greetings
- fabian

[Fabian Deutsch]

On Thu, Sep 7, 2017 at 11:01 AM, Fabian Deutsch <fdeu...@redhat.com> wrote:
> Hey,
>

> this thread went quite to quickly, but seems to have speed up in #352.
> It seems that there are a range of goals that all of us try to tackle
> or project into VM and VMConfig.
>
> - Stopped VMs
> - repeoducible/persistent VMs
> - friendly (twoards whom?) API
>
> To just name a few.
>
> To find a solution I'd like to understand the use-cases and reasons
> all of us have, thus I'd like to setup a face to face early next week
> to exchange the points of view
> See the document below for all the glory details.
>
> To have an efficient meetup every stakeholder should make sure to
> express his requirements, goals, constraints in the following document
> - prior to the meeting:
>
> https://goo.gl/7sz1vr

Hey,

this went quite good, but we were not able to get to all the discussion items.

Let's try to have a follow up to discuss the remaining topics.

Please provide your feedback on this doodle query until Wednesday, Sep
13, in case that you want to participate:

https://doodle.com/poll/zxtmpw8ihqi4t8ww

- fabian

[Fabian Deutsch]

Thursday, September 14 - tomorrow - turns out to be the time and date
which matches most people involved.

I'll be sending out an invite shortly.

- fabian

[Fabian Deutsch]

>> https://doodle.com/poll/zxtmpw8ihqi4t8ww
>
> Thursday, September 14 - tomorrow - turns out to be the time and date
> which matches most people involved.
>
> I'll be sending out an invite shortly.

Hey,

it turns out that three of us can not attend, despite the fact that we
selected the timeslot as available in the poll.

I'm sorry, but we need a second poll to find a suitable timeslot.
This time it has more options and spans the complete next week.

https://doodle.com/poll/8rv8msq9s6w8exg3

- fabian

[Fabian Deutsch]

This poll will stay open until Sunday afternoon.

- fabian

[Fabian Deutsch]

For completeness, the follow-up will take place today.
See https://doodle.com/poll/8rv8msq9s6w8exg3 for the details.

An event will also be added to the calendar.

- fabian