Problems while devices are assigned to some HVMs

[Hakisho Nukama]

Hey,

I can successfully start my Windows VM with one of my network
controllers assigned to it.
It happily displays in the device manager and complains about a missing driver.
OpenBSD also works, and has a driver shipped.

But if I assign a device to a HVM with an affected OS the startup of
this VM hangs (during driver initialisation).

Affected operation systems I've encountered with this behaviour so far:

FreeBSD-10.0-RELEASE-amd64
Ubuntu-14.04.1-desktop-amd64.iso
alpinelinux-3.0.4-x86_64.iso

Unaffected Operation systems:

OpenBSD 5.5 64 (working network)
Windows7x64 (driver not installed)
Inquisitor:
- inq-live-3.1beta2-amd64-debian.iso
- inq-live-3.0-x86_64.iso

Some excerpts of the guest-dm.log which is similar for working and failing OS:

register_real_device: Assigning real physical device 0X:00.0 ...
pt_libpci_fixup: Error: Can't open
/sys/bus/pci/devices/0000:0X:00.0/resource: I/O error
warning: pt_iomul not supported in stubdom 0X:00.0
pt_msix_init: get MSI-X table bar base XXXX0000
open(/dev/mem) -> 10
pt_msix_init: table_off = 0, total_entries = 5
pt_msix_init: errno = 5
close(10)
pt_msix_init: mapping physical MSI-X table to XXXX000
pt_msi_setup: Error: Mapping of MSI failed.
pt_enable_msi_translate: Error: MSI-INTx translation MSI setup failed, fallback
pci_intx: intx=1
register_real_device: Real physical device 0X:00.0 registered successfuly!
pt_iomem_map: Error: create new mapping failed!
pt_iomem_map: Error: remove old mapping failed!
pt_iomem_map: Error: create new mapping failed!

Best Regards,
Hakisho Nukama

[Marek Marczykowski-Górecki]

Try adding "msi=1" to xen command line (hit "e" in grub, find "multiboot
/xen..." line).

--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[Hakisho Nukama]

Done, but problem persists.

dom0 kernel: pciback 0000:04:00.0: Driver tried to write to a
read-only configuration space field at offset 0xd2, size 2. This may
be harmless, but if you have problems with your device:
1) see permissive attribute in sysfs
2) report problems to the xen-devel
mailing list along with details of your device obtained from lspci.

Trying to use xl pci-attach [domid for HVM] '04:00.0,permissive=1'
results in an attached device and this message:
libxl: warning: libxl_pci.c:202:libxl_device_pci_parse_bdf Unknown PCI
BDF option: permissive

When issuing 'echo 0000:04:00.0 > /sys/bus/pci/drivers/pciback/permissive'
the HVM hangs also.

Best Regards,
Hakisho Nukama

[Hakisho Nukama]

Booting with default boot parameters and disabling MSI-X inside
FreeBSD bootloader is a workaround for the FreeBSD hangs.
set hw.pci.enable_msix=0

Persistent setting inside /boot/loader.conf
hw.pci.enable_msix=0
-> works

hw.pci.enable_msix=0
hw.pci.enable_msi=1
-> works

hw.pci.enable_msix=0
hw.pci.enable_msi=0
-> works

hw.pci.enable_msix=1
hw.pci.enable_msi=1
-> hangs

hw.pci.enable_msix=1
hw.pci.enable_msi=0
-> hangs

hw.pci.enable_msi=0
-> hangs

Ubuntu-14.04.1 works with pci=nomsi.

Did not get kali-linux working that way.

lspci in dom0 shows me
MSI: Enable- Count=1/1 Maskable- 64bit+
and
MSI-X: Enable- Count=5 Masked-

Even with msi=1 set in xen.gz command line.

Best Regards,
Hakisho Nukama