HCL - Asrock X370 Gaming K4 BIOS P4.80

[Bjoern Christoph]

Hi all,

Some notes:

1) TPM is not working with a TPM2.0 device (as expected) so I've pulled
the chip for the time being.

2) PCIe passthrough to Windows 7 HVM works for an USB 3.0 controller
(VIA VL805) as well as for a SATA controller (ASMedia ASM1062) - this
one will however prevent booting if a hdd is attached as seabios tries
to boot from that first (no idea yet how to change that)

3) PCIe gpu passthrough to the Win7 HVM using a seconday Radeon HD5450
causes immediate lockup of the PC and reboot a second later

4) Anything else works pretty well hardware wise

BR,

Bjoern

[awokd]

Bjoern Christoph wrote on 10/23/18 5:04 PM:

> 3) PCIe gpu passthrough to the Win7 HVM using a seconday Radeon HD5450
> causes immediate lockup of the PC and reboot a second later

I know you're not asking for support, but am curious if this can be made
to work. Have you tried blacklisting that adapter in the boot options,
so Xen ignores it?

[Bjoern Christoph]

I'd LOVE to get it to work! I tried doing something like that based on some postings I found, but either I didn't do it right or it's not working.

What should I be adding and where (dom0 I guess)?

[awokd]

'Bjoern Christoph' via qubes-users wrote on 10/23/18 9:40 PM:

Find the BDF of that adapter with `lspci` in dom0, then edit your boot
options line (in `/boot/efi/EFI/qubes/xen.cfg` if using EFI boot) and
add xen-pciback.hide=(02:00.0) where `02:00.0` is the BDF you determined
earlier.

[Bjoern Christoph]

Thanks, I ended up with the following:

I added these lines to /etc/default/grub:
... rd.qubes.hide_pci=27:00.0,27:00.1 modprobe=xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(27:00.0)(27:00.1)

That way I'd get both the Video and Audio.

Then I updated grub:
grub2-mkconfig -o /boot/grub2/grub.cfg

Then I rebooted, added the pci devices to the Win7 qube and started it.

Result:
I see "Guest has not initialized the display (yet)" for a fraction of a second, then I get a BSOD from Win7 with a "memory_management" error in the qube. At least no reboot of the PC anymore ;)

If I add the devices after Win7 is running Windows tells me that the new graphic card (which is correctly identified checking the pci device ids) cannot be used due to a conflict (error 12) and a reboot is needed - which fails like before.

You can find the logs here:
https://pastebin.com/GXqvHduq

[awokd]

'Bjoern Christoph' via qubes-users wrote on 10/24/18 3:31 PM:

> Thanks, I ended up with the following:
>
> I added these lines to /etc/default/grub:
> ... rd.qubes.hide_pci=27:00.0,27:00.1 modprobe=xen-pciback.passthrough=1 xen-pciback.permissive xen-pciback.hide=(27:00.0)(27:00.1)

This part in particular "modprobe=xen-pciback.passthrough=1
xen-pciback.permissive" should not be necessary with Qubes and defeats a
good portion of its security, but makes sense for a shotgun
troubleshooting approach now and can be tightened up later.

> That way I'd get both the Video and Audio.

Good call.

> Then I updated grub:
> grub2-mkconfig -o /boot/grub2/grub.cfg
>
> Then I rebooted, added the pci devices to the Win7 qube and started it.
>
> Result:
> I see "Guest has not initialized the display (yet)" for a fraction of a second, then I get a BSOD from Win7 with a "memory_management" error in the qube.

Continuing the same troubleshooting approach, try also enabling
permissive and no-strict-reset modes for both those AMD video & audio
devices per https://www.qubes-os.org/doc/assigning-devices/#r40-1.
Also, for now try limiting your pass-through testing to just one device
at a time- i.e. see if video works by itself, then audio. Once both work
separately, then try together again. Should only need to update in Qube
Settings, not the boot options.

> At least no reboot of the PC anymore ;)
> If I add the devices after Win7 is running Windows tells me that the new graphic card (which is correctly identified checking the pci device ids) cannot be used due to a conflict (error 12) and a reboot is needed - which fails like before.

Agree, it's good progress! I wonder if there's some tweak needed to
Windows and/or the AMD driver itself. Try switching to AMD's driver pack
if using the built-in ones, or vice-versa.

> You can find the logs here:
> https://pastebin.com/GXqvHduq
>

Mind including the Xen log (/var/log/xen/console/hypervisor.log)?

[Bjoern Christoph]

Am Mittwoch, 24. Oktober 2018 23:41:42 UTC+2 schrieb awokd:

> Continuing the same troubleshooting approach, try also enabling
> permissive and no-strict-reset modes for both those AMD video & audio
> devices per https://www.qubes-os.org/doc/assigning-devices/#r40-1.
> Also, for now try limiting your pass-through testing to just one device
> at a time- i.e. see if video works by itself, then audio. Once both work
> separately, then try together again. Should only need to update in Qube
> Settings, not the boot options.

Tried, didn't help.

> > At least no reboot of the PC anymore ;)
> > If I add the devices after Win7 is running Windows tells me that the new graphic card (which is correctly identified checking the pci device ids) cannot be used due to a conflict (error 12) and a reboot is needed - which fails like before.
>
> Agree, it's good progress! I wonder if there's some tweak needed to
> Windows and/or the AMD driver itself. Try switching to AMD's driver pack
> if using the built-in ones, or vice-versa.

I doubt that will help - this happens when Windows first tries to access the graphic card I assume as we're less than a second into boot - there is no driver loaded yet, this is probably standard VGA mode at this point and that should always work no?

For now I'll stop working on this, I have other things... but it's good to see that things are improving a lot with Qubes :)