Hi Fabien,
Thanks for the quick reply!
> wrt qemu flags - you'll need a custom hook to configure the VM if you can not
> configure it with a custom hook to modify the domxml as needed.
> One example is here:
> https://github.com/kubevirt/kubevirt/tree/master/cmd/example-hook-sidecar
>
> You would effectively write a hook which is modifying the domxml to include
> the gl support.
I tried to create a very simple hook [1], but encountered a couple of issues along the way and though it I'd share them here:
- I had to configure a couple of overrides in Gopkg.toml, some of which were surprising.
* kubevirt.io/kubevirt doesn't seem to work properly and needs to be redirected to GitHub
* github.com/go-kit/kit needs to be pinned at a very specific commit
You can find my Gopkg.toml on GitHub [2]. It's my first foray into Go, so perhaps I missed something obvious.
Can this be simplified in the future, or would I always need to specify these overrides?
- Some libvirt options are not present in the KubeVirt API. For example, I don't think there's a way to express something like this:
<video>
<model type='virtio' heads='1' primary='yes'>
<acceleration accel3d='yes'/>
</model>
</video>
because the acceleration node doesn't exist in the kube-virt API.
If I'm not mistaken, I can modify the XML in my custom hook and add any attribute I want, but since the XML is marshalled back to Go structs, any elements/attributes which are not defined in the KubeVirt API would be ignored?
- The version of libvirt which ships with KubeVirt appears to be 4.2.0 and doesn't support the egl-headless graphics yet.
Is there an easy way to upgrade the version of libvirt?
- I tried to work around this by adding the arguments directly to qemu:commandline. I've written a unit test which shows that the hook adds the commandline arguments to the XML, but when unmarshalling the XML, these command line args seem to be missing. The source code for the unit test is here: [3].
Did I miss something there?
Thanks, and apologies for all the questions 😉.
Frederik.
[1] https://github.com/qmfrederik/kube-virt-droid-images/tree/master/android-x86-hook
[2] https://github.com/qmfrederik/kube-virt-droid-images/blob/master/android-x86-hook/Gopkg.toml
[3] https://github.com/qmfrederik/kube-virt-droid-images/blob/master/android-x86-hook/android-x86-hook_test.go#L170
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To post to this group, send email to kubevi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/VI1PR0501MB2286B0EA2CA3742A0C8C9825EADB0%40VI1PR0501MB2286.eurprd05.prod.outlook.com.
On Thu, Nov 22, 2018 at 2:33 PM Frederik Carlier <frederik...@quamotion.mobi> wrote:- The version of libvirt which ships with KubeVirt appears to be 4.2.0 and doesn't support the egl-headless graphics yet.Is there an easy way to upgrade the version of libvirt?We update it in [4] from time to time. Which version will you need? We can do an update and merge it in kubevirt master.
> -----Original Message-----
> From: kubevi...@googlegroups.com <kubevi...@googlegroups.com> On
> Behalf Of Martin Kletzander
> Sent: Thursday, November 22, 2018 3:39 PM
>
> For egl-headless you need three things:
>
[...]
>
> 3) do a little workaround by (it will be supported in libvirt, but it is not
> yet, there's a BZ for it)
>
> - setting permissions (setfacl is enough) on /dev/dri/renderD* so that qemu
> can access it
>
> - adding the DRI render node files in the cgroup_device_acl in qemu.conf
On Thu, Nov 22, 2018 at 4:38 PM Frederik Carlier <frederik...@quamotion.mobi> wrote:> -----Original Message-----
> From: kubevi...@googlegroups.com <kubevi...@googlegroups.com> On
> Behalf Of Martin Kletzander
> Sent: Thursday, November 22, 2018 3:39 PM
>
> For egl-headless you need three things:
>
[...]
>
> 3) do a little workaround by (it will be supported in libvirt, but it is not
> yet, there's a BZ for it)
>
> - setting permissions (setfacl is enough) on /dev/dri/renderD* so that qemu
> can access itHm, we will have to pass through all devices which are needed into the container. There are different options:1) a k8s device plugin, or2) an OCI hooks
Hey Fabian,
In terms of an "initial light up", yes, I got this working.
It required:
- A custom hook to set the virgl configuration [1]
- A custom launcher image [2], with a patched version of qemu [3] (looks like this patch didn't make it into 3.1-rc4) and a wrapper around virt-launcher [4] which sets the permissions on /dev/dri/renderD128
- A device plugin to assign a GPU to the compute pod [5]
- Lots of help and patience from Gerd :)
So for Android 7.1-r2, I ended up with a VMI definition which "kind of" worked [6].
I hit a couple of snags with virgl - display would be messed up after a reboot, and starting the screen capture client [7] which we use would cause the VM to crash and reboot.
I had a quick stab at trying to troubleshoot those issues, but it looks like there are a lot of moving parts involved and I'm not very familiar with the graphics stack.
Hence, I’m currently experimenting with GVT-g. The results with "bare" qemu look very promising, as well as running qemu inside a Kubernetes pod.
At the KubeVirt level, it looks like I'm missing two things:
- Making arbitrary changes to the libvirt domain (waiting for PR 1744)
--
You received this message because you are subscribed to the Google Groups "kubevirt-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubevirt-dev...@googlegroups.com.
To post to this group, send email to kubevi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kubevirt-dev/AM4PR0501MB2275CE8C4E39D2AFEFB900ABEAA80%40AM4PR0501MB2275.eurprd05.prod.outlook.com.