Bug#929983: ipxe-qemu: virtio booting no longer works after upgrade to buster
Thorsten Glaser
Version: 1.0.0+git-20190125.36a4c85-1
Severity: serious
Justification: makes package unusable
On this virtualisation system I’ve upgraded to buster (using libvirt and
Linux-kvm), netbooting VMs no longer works:
-----BEGIN cutting here may damage your screen surface-----
SeaBIOS (version 1.12.0-1)
Machine UUID …
Press ESC for boot menu.
Booting from Hard disk...
Boot failed: not a bootable disk
No bootable device.
_
-----END cutting here may damage your screen surface-----
If I, however, change the NIC to pcnet, it works:
-----BEGIN cutting here may damage your screen surface-----
SeaBIOS (version 1.12.0-1)
Machine UUID …
iPXE (http://ipxe.org) 00:03.0 C980 PCI2.10 PnP PMM+…
Press ESC for boot menu.
Booting from Hard disk...
Boot failed: not a bootable disk
Booting from ROM...
iPXE (PCI 00:03.0) starting execution... ok
iPXE initialising devices...ok
iPXE 1.0.0+git-20190125.36a4c85-1 -- Open Source […]
-----END cutting here may damage your screen surface-----
-- System Information:
Debian Release: 10.0
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)
Kernel: Linux 4.19.0-5-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_FIRMWARE_WORKAROUND
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE=C (charmap=UTF-8)
Shell: /bin/sh linked to /bin/lksh
Init: sysvinit (via /sbin/init)
-- no debconf information
Thorsten Glaser
> Works for me using a virt-manager configured with virtio or the
> hypervisor default for netoworking with the same seabios and ipxe
Hmm.
> versions. My system was recently upgraded from stretch to buster.
This one was a fair bit older, I think it was originally a wheezy,
but I upgraded it step by step as usual.
> Could you provide more detail about your configured virtual machine?
I’ll attach the virsh dumpxml output below; I had reinstalled Debian
using an e1000 NIC and netboot in the meantime and reverted to virtio
afterwards, but I’m pretty sure this is reproducible even on other
virtualisation hosts, I will try that tomorrow.
<domain type='kvm' id='30'>
<name>veraweb-mit</name>
<uuid>52e965be-c1b7-72b7-0286-6a7bad8a014d</uuid>
<memory unit='KiB'>1048576</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
<vcpu placement='static'>1</vcpu>
<os>
<type arch='x86_64' machine='pc-1.1'>hvm</type>
<bootmenu enable='no'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<cpu mode='custom' match='exact' check='full'>
<model fallback='forbid'>Nehalem-IBRS</model>
<vendor>Intel</vendor>
<feature policy='require' name='vme'/>
<feature policy='require' name='ss'/>
<feature policy='require' name='x2apic'/>
<feature policy='require' name='tsc-deadline'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='arat'/>
<feature policy='require' name='tsc_adjust'/>
<feature policy='require' name='umip'/>
<feature policy='require' name='ssbd'/>
<feature policy='require' name='rdtscp'/>
</cpu>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/kvm</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source dev='/dev/vg-evolvis-wirt/veraweb-mit'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<boot order='1'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</disk>
<controller type='usb' index='0' model='piix3-uhci'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'>
<alias name='pci.0'/>
</controller>
<interface type='bridge'>
<mac address='52:54:00:09:55:44'/>
<source bridge='br0'/>
<target dev='vnet2'/>
<model type='virtio'/>
<boot order='2'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<source path='/dev/pts/1'/>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
<alias name='serial0'/>
</serial>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<input type='tablet' bus='usb'>
<alias name='input0'/>
<address type='usb' bus='0' port='1'/>
</input>
<input type='mouse' bus='ps2'>
<alias name='input1'/>
</input>
<input type='keyboard' bus='ps2'>
<alias name='input2'/>
</input>
<graphics type='vnc' port='5902' autoport='yes' listen='127.0.0.1'>
<listen type='address' address='127.0.0.1'/>
</graphics>
<video>
<model type='cirrus' vram='16384' heads='1' primary='yes'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</memballoon>
</devices>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+109:+117</label>
<imagelabel>+109:+117</imagelabel>
</seclabel>
</domain>
bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
**********
Mit der tarent Academy bieten wir auch Trainings und Schulungen in den
Bereichen Softwareentwicklung, Agiles Arbeiten und Zukunftstechnologien an.
Besuchen Sie uns auf www.tarent.de/academy. Wir freuen uns auf Ihren Kontakt.
**********
Thorsten Glaser
Version: 1.0.0+git-20190125.36a4c85-1
Followup-For: Bug #929983
Control: tags 929983 - moreinfo
I can confirm the issue is still there with the VM defined in libvirt.
The command from Shengjing Zhu gives the following output:
-----BEGIN cutting here may damage your screen surface-----
SeaBIOS (version 1.12.0-1)
iPXE (http://ipxe.org) 00:03.0 C980 PCI2.10 PnP PMM+7FF90020+7FED0020 C980
Booting from Hard Disk...
Boot failed: could not read the boot disk
Booting from Floppy...
Boot failed: could not read the boot disk
Booting from DVD/CD...
Boot failed: Could not read from CDROM (code 0003)
Booting from ROM...
iPXE (PCI 00:03.0) starting execution...ok
iPXE initialising devices...ok
iPXE 1.0.0+git-20190125.36a4c85-1 -- Open Source Network Boot Firmware -- http:/
/ipxe.org
Features: DNS HTTP iSCSI NFS TFTP AoE ELF MBOOT PXE bzImage Menu PXEXT
net0: 52:54:00:12:34:56 using virtio-net on 0000:00:03.0 (open)
[Link:up, TX:0 TXE:0 RX:0 RXE:0]
Configuring (net0 52:54:00:12:34:56)...... ok
net0: 10.0.2.15/255.255.255.0 gw 10.0.2.2
net0: fec0::5054:ff:fe12:3456/64 gw fe80::2
net0: fe80::5054:ff:fe12:3456/64
Nothing to boot: No such file or directory (http://ipxe.org/2d03e13b)
No more network devices
No bootable device.
-----END cutting here may damage your screen surface-----
So perhaps this is a libvirt thing?
libvirt runs the domain with:
26266 ? Sl 0:02 qemu-system-x86_64 -enable-kvm -name guest=foo,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-15-foo/master-key.aes -machine pc-1.1,accel=kvm,usb=off,dump-guest-core=off -cpu Nehalem-IBRS,vme=on,ss=on,x2apic=on,tsc-deadline=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,ssbd=on,rdtscp=on -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 8b2c4678-c507-4652-afce-de7dda4cb3e0 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=37,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot menu=on,strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/dev/vg-evolvis-wirt/foo,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,write-cache=on -netdev tap,fd=39,id=hostnet0,vhost=on,vhostfd=40 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:09:55:44,bus=pci.0,addr=0x3,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:12 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
I can confirm it to fail when run from the command line with:
qemu-system-x86_64 -enable-kvm -name guest=foo,debug-threads=on -machine pc-1.1,accel=kvm,usb=off,dump-guest-core=off -cpu Nehalem-IBRS,vme=on,ss=on,x2apic=on,tsc-deadline=on,hypervisor=on,arat=on,tsc_adjust=on,umip=on,ssbd=on,rdtscp=on -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 8b2c4678-c507-4652-afce-de7dda4cb3e0 -no-user-config -nodefaults -rtc base=utc -no-shutdown -boot menu=on,strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/dev/vg-evolvis-wirt/foo,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,write-cache=on -netdev tap,fd=39,id=hostnet0,vhost=on,vhostfd=40 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:09:55:44,bus=pci.0,addr=0x3,bootindex=1 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:12 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on
Trying to minify… success!
Adding “-machine pc-1.1,accel=kvm,usb=off,dump-guest-core=off” to
the command given by Shengjing Zhu makes it fail:
-----BEGIN cutting here may damage your screen surface-----
SeaBIOS (version 1.12.0-1)
Booting from Hard Disk...
Boot failed: could not read the boot disk
Booting from Floppy...
Boot failed: could not read the boot disk
Booting from DVD/CD...
Boot failed: Could not read from CDROM (code 0003)
No bootable device.
-----END cutting here may damage your screen surface-----
So perhaps this is a qemu thing? Or ipxe only works with USB? Or…?
-- System Information:
Debian Release: 10.3
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.19.0-8-amd64 (SMP w/8 CPU cores)
Thorsten Glaser
Version: 1.0.0+git-20190125.36a4c85-5.1
Followup-For: Bug #929983
X-Debbugs-Cc: t...@mirbsd.de
This is also broken on an up-to-date-enough sid system.
$ qemu-system-x86_64 -device virtio-net-pci,netdev=net0 -netdev user,id=net0 -nographic
boots.
$ qemu-system-x86_64 -device virtio-net-pci,netdev=net0 -netdev user,id=net0 -nographic -machine pc-1.1
fails to boot.
Downgrading *only* the package ipxe-qemu to 1.0.0+git-20161027.b991c67-1
on that very sid system…
… doesn’t make this succeed either. Huh.
-- System Information:
Debian Release: bullseye/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'oldstable-updates'), (500, 'buildd-unstable'), (500, 'unstable'), (500, 'oldstable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.9.0-4-amd64 (SMP w/2 CPU threads)
Kernel taint flags: TAINT_WARN
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Thorsten Glaser
>Downgrading *only* the package ipxe-qemu to 1.0.0+git-20161027.b991c67-1
>
>… doesn’t make this succeed either. Huh.
also doesn’t yield netboot success, so there’s something more complex
involved in between stretch and buster… so please reassign to, once it
is known, the culprit ☻
bye,
//mirabilos
--
[16:04:33] bkix: "veni vidi violini"
[16:04:45] bkix: "ich kam, sah und vergeigte"...
Michael Prokop
> Downgrading seabios along (need to remove -nographic to make that work)
> also doesn’t yield netboot success, so there’s something more complex
> involved in between stretch and buster… so please reassign to, once it
> is known, the culprit ☻
(Stumbled upon it while going through the bug reports of ipxe while
checking for new upstream versions :))
regards
-mika-
Michael Tokarev
> Hi all,
>
> [CC-ing src:debian-edu and src:qemu as they pull in src:ipxe-qemu into the key package set, so I consider them stakeholders in this RC bug.]
>
> On Fri, 12 Mar 2021 19:29:55 +0100 (CET) Thorsten Glaser <t...@debian.org> wrote:
>> So we now know without fail that there’s a change in the ipxe-qemu
>> binary package, introduced between jessie and stretch, that breaks
>> netbooting on virtio NICs for at least some qemu machine models in
>> use by libvirt guests.
>
> Is there any progress on this front? It would be a shame if we have to <stable>-ignore the bug again for bookworm.
Well, there's no progress in there, -
I weren't aware of this issue is still occurs on bookworm.
I don't have a netboot environment handy to test it, either.
Help?
/mjt
Michael Tokarev
..
> at least the bugtitle is far to unprecise. Here, I test Debian Edu bullseye really heavily and integrate FAI in Debian Edu.
>
> The FAI installer and also the diskless machines I very often boot via iPXE/QEMU. In Debian 11 (and probably beyond), PXE booting in QEMU works, for
> BIOS legacy VMs as well as for UEFI based VMs.
In this bugreport, I see it is/was broken with -machine pc-1.1. There's no indication
if it is broken with other machine types. As of qemu 5.2 (bullseye) machine types
below pc-1.3 are deprecated, and in 7.0 (current bookworm) they're removed.
/mjt
Thorsten Glaser
> On 07-05-2022 10:53, Michael Tokarev wrote:
>> In this bugreport, I see it is/was broken with -machine pc-1.1.
>> There's no indication if it is broken with other machine types. As
>> of qemu 5.2 (bullseye) machine types below pc-1.3 are deprecated, and
>> in 7.0 (current bookworm) they're removed.
with almost certainly zero clues why.
> Do you agree with this assessment of the bug you reported? If so,
> let's tag this bug with buster and bullseye as indeed I conclude it's
> not a bug in bookworm then.
I'm currently in a really bad situation to look at anything in
detail (waiting for my brain to remember the luks password of
my work laptop), please excuse brevity.
bye,
//mirabilos
--
Gestern Nacht ist mein IRC-Netzwerk explodiert. Ich hatte nicht damit
gerechnet, darum bin ich blutverschmiert… wer konnte ahnen, daß SIE so
reagier’n… gestern Nacht ist mein IRC-Netzwerk explodiert~~~
(as of 2021-06-15 The MirOS Project temporarily reconvenes on OFTC)