Bug#780093: [Pkg-libvirt-maintainers] Bug#780093: libvirt: error: unsupported configuration: QEMU 2.1.2 is too new for help parsing

[Thorsten Glaser]

On Mon, 9 Mar 2015, Guido Günther wrote:

> wrapping qemu/kvm by a skript or something?

Yes, necessarily:

tglase@tglase:~ $ virsh -c qemu:///system dumpxml MirBSD | fgrep emul
<emulator>/usr/local/bin/qemu-in-chroot</emulator>
tglase@tglase:~ $ cat /usr/local/bin/qemu-in-chroot
#!/bin/sh
exec schroot -c sid-amd64 -u root -- qemu-system-x86_64 -enable-kvm "$@"

Otherwise I cannot assign more than 2047 MiB RAM.

*But*:

tglase@tglase:~ $ virsh -c qemu:///system dumpxml SuSE_Linux_1.0 | fgrep emul
<emulator>/usr/bin/kvm</emulator>
tglase@tglase:~ $ virsh -c qemu:///system start SuSE_Linux_1.0
error: Failed to start domain SuSE_Linux_1.0
error: unsupported configuration: QEMU 2.1.2 is too new for help parsing

So this also happens for VMs created by virt-manager and
then untouched.

> Does
>
> virsh capabilities
>
> show the same error?

No, it shows a bunch of XML things (capabilities/{host,guest}/…).

bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg


--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

[Guido Günther]

On Tue, Mar 10, 2015 at 12:28:40AM +0100, Thorsten Glaser wrote:
> On Mon, 9 Mar 2015, Guido Günther wrote:
>
> > wrapping qemu/kvm by a skript or something?
>
> Yes, necessarily:
>
> tglase@tglase:~ $ virsh -c qemu:///system dumpxml MirBSD | fgrep emul
> <emulator>/usr/local/bin/qemu-in-chroot</emulator>
> tglase@tglase:~ $ cat /usr/local/bin/qemu-in-chroot
> #!/bin/sh
> exec schroot -c sid-amd64 -u root -- qemu-system-x86_64 -enable-kvm "$@"
>
> Otherwise I cannot assign more than 2047 MiB RAM.
>
> *But*:
>
> tglase@tglase:~ $ virsh -c qemu:///system dumpxml SuSE_Linux_1.0 | fgrep emul
> <emulator>/usr/bin/kvm</emulator>
> tglase@tglase:~ $ virsh -c qemu:///system start SuSE_Linux_1.0
> error: Failed to start domain SuSE_Linux_1.0
> error: unsupported configuration: QEMU 2.1.2 is too new for help parsing
>
> So this also happens for VMs created by virt-manager and
> then untouched.

Please do a:

grep usedQMP /var/cache/libvirt/qemu/capabilities/*.xml

if it doesn't find any matches do a

rm /var/cache/libvirt/qemu/capabilities/*.xml

something seems to be wrong with your capabilities parsing of
qemu. Also it would help to rund libvitd with debug level when
starting the domain and see the output.

>
> > Does
> >
> > virsh capabilities
> >
> > show the same error?
>
> No, it shows a bunch of XML things (capabilities/{host,guest}/…).

Good.
-- Guido

[Thorsten Glaser]

On Wed, 11 Mar 2015, Guido Günther wrote:

> qemu. Also it would help to rund libvitd with debug level when
> starting the domain and see the output.

sudo env LIBVIRT_DEBUG=1 /etc/init.d/libvirtd start #, thus.

This is interesting. For the VMs created by virt-manager, which
have /usr/bin/kvm as emulator, it says KVM not supported by this
target. This matches the command line:

tglase@tglase:~ $ kvm
KVM not supported for this target
No accelerator found!

This is probably an x32 issue with src:qemu.

For the VMs using qemu-in-chroot, it instead tries to start it with
-qmp unix:/var/lib/libvirt/qemu/capabilities.monitor.sock,server,nowait
and then fails to connect there. That being said, /var/lib/libvirt
is already in the list of schroot bind-mount directories, so this
should work. And indeed, if I run…

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin HOME=/ /usr/local/bin/qemu-in-chroot -S -no-user-config -nodefaults -nographic -M none -qmp unix:/var/lib/libvirt/qemu/capabilities.monitor.sock,server,nowait -pidfile /var/lib/libvirt/qemu/capabilities.pidfile

… manually, then check the “mount” output, the directory is bindmounted.

tglase@tglase:~ $ sudo ls -l /var/lib/libvirt/qemu/
total 20
srwxr-xr-x 1 root root 0 Mär 11 15:05 capabilities.monitor.sock
-rw------- 1 root root 6 Mär 11 15:05 capabilities.pidfile
drwxr-x--- 3 libvirt-qemu libvirt-qemu 4096 Okt 28 09:17 channel
drwxr-xr-x 2 root root 4096 Okt 28 09:17 dump
drwxr-xr-x 2 libvirt-qemu libvirt-qemu 4096 Okt 28 09:17 save
drwxr-xr-x 2 libvirt-qemu libvirt-qemu 4096 Okt 28 09:17 snapshot

If I connect to this from outside the chroot, I get:

tglase@tglase:~ $ sudo nc -U /var/lib/libvirt/qemu/capabilities.monitor.sock
{"QMP": {"version": {"qemu": {"micro": 2, "minor": 1, "major": 2}, "package": " (Debian 1:2.1+dfsg-11)"}, "capabilities": []}}

Does this help?

For the record, if I go into a (the same) chroot manually, start qemu
with -qmp, then connect to that inside the same chroot, I get the same
output. The presence or absence of -enable-kvm also does not change it.

bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

[Thorsten Glaser]

On Wed, 11 Mar 2015, Guido Günther wrote:

> I wonder why you still have capabilities.* sockets. Maybe these are

Nonono, these are happening *during* my tests, in which I removed
the daemonising parameter from the qemu call.

> I'm lowering the severity since the issue looks pretty much related to
> your x32 + wrapper setup.

The qemu used is in a chroot. When I ask the amd64 qemu for its
capabilities, I get an empty array, IIUC:

{
"QMP": {
"capabilities": [

],
"version": {

"package": " (Debian 1:2.1+dfsg-11)",

"qemu": {
"major": 2,
"micro": 2,
"minor": 1
}
}
}
}

(pretty-printed)

So, is this correct? Does this confuse libvirt?

I’ll send one batch of logs per PM.

[Thorsten Glaser]

severity 780093 important
thanks

On Wed, 11 Mar 2015, Guido Günther wrote:

> That said having the logs and maybe stracing the daemon should
> provide more insights.

Did you get any more insights?

In the meantime, I downgraded libvirt0 (and the three dependants
libvirt-clients libvirt-daemon libvirt-daemon-system) to 1.2.9-8
from 1.2.9-9, and my VMs start up again.

So, whatever you did in 1.2.9-9, it introduced a severe regression.

[Guido Günther]

reassign 780093 libvirt-daemon
severity 780093 normal
retitule 780093 Fails capability parsing in a bind mounted amd64 chroot on x32

On Mon, Mar 16, 2015 at 03:43:10PM +0100, Thorsten Glaser wrote:
> severity 780093 important
> thanks
sigh

>
> On Wed, 11 Mar 2015, Guido Günther wrote:
>
> > That said having the logs and maybe stracing the daemon should
> > provide more insights.
>
> Did you get any more insights?

This is a special setup that fails to parse the capabilities using a
qemu wrapper on x32 to run in a x86_64 chroot (as far as I understand
it yet). I'd be good to find out why. I'm currently lacking the time
(and internet connectivity to build chroots) to reproduce this. In any
case if somebody else wants to step in having your schroot and vm
configuration would help.

I'd be also good to have the requested logs at debug level. There
might already be the solution in there.

> In the meantime, I downgraded libvirt0 (and the three dependants
> libvirt-clients libvirt-daemon libvirt-daemon-system) to 1.2.9-8
> from 1.2.9-9, and my VMs start up again.
>
> So, whatever you did in 1.2.9-9, it introduced a severe regression.

It fixed a severe regression on since it didn't fall back to broken
help parsing preventing VMs to start on fresh installations.

Cheers,
-- Guido

[Thorsten Glaser]

On Tue, 17 Mar 2015, Guido Günther wrote:

> This is a special setup that fails to parse the capabilities using a
> qemu wrapper on x32 to run in a x86_64 chroot (as far as I understand

It also fails to parse them *without* the chroot, with…

<emulator>/usr/bin/kvm</emulator>

… which is what virt-manager creates. But that’s probably
a bug in the qemu-kvm package (plus one in virt-manager if
/usr/bin/kvm is not supposed to be used). Changing it to…

<emulator>/usr/bin/qemu-system-x86_64</emulator>

… and changing <domain type='kvm'> to <domain type='qemu'>
makes this succeed, so that’s unrelated. (Mentioning this
because I mentioned it earlier in this bugreport, so we
know this is not the problem.)

> case if somebody else wants to step in having your schroot and vm
> configuration would help.

OK. A small reproducer is here:

<domain type='kvm'>
<name>d-i-amd64</name>
<uuid>2affcd84-73f4-bbb2-2fd5-fd920b9806b7</uuid>
<memory unit='KiB'>524288</memory>
<currentMemory unit='KiB'>524288</currentMemory>
<vcpu placement='static'>1</vcpu>
<os>
<type arch='x86_64' machine='pc-1.1'>hvm</type>
<boot dev='network'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/local/bin/qemu-in-chroot</emulator>
<controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<interface type='bridge'>
<mac address='52:54:00:02:9c:44'/>
<source bridge='br0'/>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<input type='tablet' bus='usb'/>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'/>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</memballoon>
</devices>
</domain>

The other files (hopefully all relevant ones) are:

/etc/schroot/schroot.conf:

[sid-amd64]
description=Debian sid/amd64 (unstable)
type=directory
directory=/home/AMD64
root-users=libvirt-qemu,tglase,root
#personality=linux64
command-prefix=linux64,eatmydata
setup.fstab=fstab.qemu

sed 's/^X//' >etc/schroot/fstab.qemu << 'END-of-etc/schroot/fstab.qemu'
X# fstab: static file system information for chroots.
X# Note that the mount point will be prefixed by the chroot path
X# (CHROOT_PATH)
X#
X# <file system> <mount point> <type> <options> <dump> <pass>
X/proc /proc none rw,bind 0 0
X/sys /sys none rw,bind 0 0
X/dev /dev none rw,bind 0 0
X/dev/pts /dev/pts none rw,bind 0 0
X/home /home none rw,bind 0 0
X/tmp /tmp none rw,bind 0 0
X
X# It may be desirable to have access to /run, especially if you wish
X# to run additional services in the chroot. However, note that this
X# may potentially cause undesirable behaviour on upgrades, such as
X# killing services on the host.
X#/run /run none rw,bind 0 0
X#/run/lock /run/lock none rw,bind 0 0
X#/dev/shm /dev/shm none rw,bind 0 0
X/run/shm /run/shm none rw,bind 0 0
X
X/var/lib/libvirt /var/lib/libvirt none rw,bind 0 0
X/var/cache/pbuilder /var/cache/pbuilder none rw,bind 0 0
END-of-etc/schroot/fstab.qemu

> I'd be also good to have the requested logs at debug level. There
> might already be the solution in there.

I’d sent them to you already, in separate direct eMail, due to size.

> > In the meantime, I downgraded libvirt0 (and the three dependants
> > libvirt-clients libvirt-daemon libvirt-daemon-system) to 1.2.9-8
> > from 1.2.9-9, and my VMs start up again.
> >
> > So, whatever you did in 1.2.9-9, it introduced a severe regression.
>
> It fixed a severe regression on since it didn't fall back to broken
> help parsing preventing VMs to start on fresh installations.

Let’s meet in the middle and say it swapped a severe regression
with another severe regression.

bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg