Black Screen when installing 4.0.3 & 4.1 on AMD Ryzen 4750U

[dylang...@gmail.com]

When trying to install Qubes 4.0.3 or 4.1 (Test ISO) into a Ryzen 4750U based laptop, I see xen output, it relinquishes vga to dom0, then black.

I managed to enabled logging, where it prints dom0's dmesg super slow, line by line.

efifb: cannot reserve video memory at 0x60000000
efi: EFI_MEMMAP is not enabled

Then it crashes but it continues to come all the way to Anaconda confirming it's a GPU/Memory Issue

This seems to be related to UMA / iGPU Memory Management, unsure if it's just that Kernel Support for this CPU is spotty, Fedora 32 Live USB boots perfectly.

This is 100% a Display/GPU issue has everything else comes up smitten, I just don't see anything / the framebuffer isn't passed from Xen to dom0.

[Dylanger Daly]

The Device is a Lenovo X13 (AMD).

Does anyone have any ideas? I've tried essentially all variations on the "UEFI Troubleshooting" guide

[Chris Laprise]

On 8/5/20 6:54 AM, Dylanger Daly wrote:
> The Device is a Lenovo X13 (AMD).

You have good taste in laptops. :)

>
> Does anyone have any ideas? I've tried essentially all variations on the
> "UEFI Troubleshooting" guide

I have a Thinkpad T14 with a Ryzen 4750U on its way. These new Ryzen
APUs are becoming popular... its going to become a real sticking point
to get them working with Qubes!

A user named Claudia had a long thread about installing Qubes on a
previous gen Ryzen...

https://groups.google.com/d/msgid/qubes-users/ae710f3839c46261d257d7d188da32b9%40disroot.org

Also worth noting is that some Ryzen 4000 power management drivers have
just landed in Linux 5.8.

>
> On Wednesday, August 5, 2020 at 4:04:32 PM UTC+10 dylang...@gmail.com wrote:
>
> When trying to install Qubes 4.0.3 or 4.1 (Test ISO) into a Ryzen
> 4750U based laptop, I see xen output, it relinquishes vga to dom0,
> then black.
>
> I managed to enabled logging, where it prints dom0's dmesg super
> slow, line by line.
>
> |
> efifb:cannot reserve video memory at 0x60000000
> efi:EFI_MEMMAP isnotenabled
> |
>
>
> Then it crashes but it continues to come all the way to Anaconda
> confirming it's a GPU/Memory Issue
>
> This seems to be related to UMA / iGPU Memory Management, unsure if
> it's just that Kernel Support for this CPU is spotty, Fedora 32 Live
> USB boots perfectly.
>
> This is 100% a Display/GPU issue has everything else comes up
> smitten, I just don't see anything / the framebuffer isn't passed
> from Xen to dom0.

--
Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Chris Laprise]

Another user had similar symptoms as you with a Ryzen 3200G, apparently
not resolved...

https://groups.google.com/d/msgid/qubes-users/75241cee-4e22-4a63-b81e-46581ede9308%40googlegroups.com

[Dylanger Daly]

> You have good taste in laptops. :)

Haha thank you, as do you, the T14 was second on my list, yeah I suspect there will be plenty of Qubes users on these devices they tick a lot of boxes, I've never used Qubes with >4 Cores so it'll be a nice experience.

 > Also worth noting is that some Ryzen 4000 power management drivers have

just landed in Linux 5.8.

Hmm, wonder if I should try building a 4.1 ISO with a Linux 5.8 Kernel, it's interesting because Xen is able to write to the framebuffer just fine, I think it's dom0 that isn't able to remap it so it stays at an address Xen had it configured for, it almost smells like an IOMMU/Memory Mapping issue, not necessarily GPU. 

There's UEFI Options for the UMA Framebuffer size of 512MB, 1GB and 2GB I've tried all variants unsuccessfully.

I don't think it's a Xen issue because I tried simply moving my current laptop's NVMe, when I entered my LUKs Password (Blind) I could see LEDs on the keyboard initialize so I think 4.0.3 does indeed work fine.

I don't think there's a migration path for 4.0.3 - 4.1 (Backup & Restore) yet, I don't think the Qubes team have even signed any 4.1 ISOs yet either so I'd rather 4.0.3 but I'll take anything I can get at this point.

[Chris Laprise]

On 8/5/20 7:29 PM, Dylanger Daly wrote:
> Hmm, wonder if I should try building a 4.1 ISO with a Linux 5.8 Kernel,
> it's interesting because Xen is able to write to the framebuffer just
> fine, I think it's dom0 that isn't able to remap it so it stays at an
> address Xen had it configured for, it almost smells like an IOMMU/Memory
> Mapping issue, not necessarily GPU.

My Thinkpad T14 arrived and Qubes 4.0.3 installer behaves the same on
the T14 as what you reported.

With Ubuntu upgraded to kernel 5.8.0 to fix broken suspend & brightness
and system running hot; now its great.... extremely fast, cool and
quiet. (Yes, I upgraded kernel bc the existing one had.)

I'm going to experiment with moving a couple of my Qubes VMs over to the
Ubuntu install under KVM (using VM Manager app). I've already got an LVM
thin pool setup and re-provisioning OS root snapshots to specific VMs
before they boot as if they were templates.

>
> There's UEFI Options for the UMA Framebuffer size of 512MB, 1GB and 2GB
> I've tried all variants unsuccessfully.
> I don't think it's a Xen issue because I tried simply moving my current
> laptop's NVMe, when I entered my LUKs Password (Blind) I could see LEDs
> on the keyboard initialize so I think 4.0.3 does indeed work fine.

FYI release notes for both Xen 4.13 and 4.14 mention additional support
for new AMD Epyc processors. I interpret this as a server-oriented way
of expressing support for certain generations of AMD processors, though
I don't know how close Ryzen and Epyc are in terms of operation.

The Qubes 4.1 tree appears to have Xen 4.13 and Linux 5.7, currently.

>
> I don't think there's a migration path for 4.0.3 - 4.1 (Backup &
> Restore) yet, I don't think the Qubes team have even signed any 4.1 ISOs
> yet either so I'd rather 4.0.3 but I'll take anything I can get at this
> point.

I feel the same way. I would love to run Qubes on my T14 but I have a
feeling that Linux 5.7 won't cut it and I'm not experienced enough with
qubes builder to confidently upgrade either Linux or Xen. I did make a
sloppy attempt with ISO Master to replace the Qubes 4.0.3 installer ISO
kernel with the Ubuntu 5.8.0 kernel but due to my ignorance about the
format I couldn't get it to initiate the boot process.

[Dylanger Daly]

> I'm going to experiment with moving a couple of my Qubes VMs over to the
Ubuntu install under KVM (using VM Manager app).

Nice, I've had the same thought with Fedora Silverblue, but Qube's qvm- etc tools make everything so much easier.

> The Qubes 4.1 tree appears to have Xen 4.13 and Linux 5.7, currently.

Indeed R4.1 is using Xen 4.13.1 last commit was back on May 7th 2020, I can't seem to see any AMD/Ryzen specific commits that are newer than this date, I was looking at cherry picking Xen commits related to AMD/Ryzen however I can't find any.

Any and all AMD Related commits I can find were made in 2019 and are included in the current Xen 4.13.1

So perhaps this is actually a dom0/Linux Kernel issue? Surely if it were Xen they'd have something committed by now?

[Chris Laprise]

On 8/11/20 1:13 AM, Dylanger Daly wrote:
> > I'm going to experiment with moving a couple of my Qubes VMs over to the
> Ubuntu install under KVM (using VM Manager app).
>
> Nice, I've had the same thought with Fedora Silverblue, but Qube's qvm-
> etc tools make everything so much easier.
>
> > The Qubes 4.1 tree appears to have Xen 4.13 and Linux 5.7, currently.
>
> Indeed R4.1 is using Xen 4.13.1

> <https://github.com/xen-project/xen/commits/RELEASE-4.13.1> last commit

> was back on May 7th 2020, I can't seem to see any AMD/Ryzen specific
> commits that are newer than this date, I was looking at cherry picking
> Xen commits related to AMD/Ryzen however I can't find any.
>
> Any and all AMD Related commits I can find were made in 2019 and are
> included in the current Xen 4.13.1
>
> So perhaps this is actually a dom0/Linux Kernel issue? Surely if it were
> Xen they'd have something committed by now?
> On Monday, August 10, 2020 at 8:21:05 PM UTC+10 Chris Laprise wrote:

I've experimented a bit with Ubuntu and some different kernels. I was
not able to get 5.7.0 kernel graphics to work at all (not even
vga/framebuffer), while 5.8.0 and 5.8.1 work beautifully and 'lshw'
confirms the amdgpu driver is being used to drive the display. The
default 5.4.0 kernel won't recognize the gpu but graphics do work
without acceleration.

I don't know why graphics are broken in 5.7 (dmesg didn't show any
vga/amdgpu errors) when that is the first kernel to receive AMD Renoir
support. It could be that 5.8 has a fixed amdgpu driver, or that the
presence of power management drivers is required to properly drive the
Vega gpu.

[Dylanger Daly]

> It could be that 5.8 has a fixed amdgpu driver

Yeah you could have a point here, looks like 5.8 was pushed to qubes-linux-kernel 12 hours ago.

I'll try compile this and see how it goes.

[lama...@gmail.com]

On Thursday, August 6, 2020 at 1:29:49 AM UTC+2 Dylanger Daly wrote:

> You have good taste in laptops. :)

Haha thank you, as do you, the T14 was second on my list, yeah I suspect there will be plenty of Qubes users on these devices they tick a lot of boxes, I've never used Qubes with >4 Cores so it'll be a nice experience.

Yes, better support for new Ryzen CPU's would be nice. I would like an 4800H in my next machine, they're so fast.
Maybe newer kernel versions like the kernel-latest package can be included as optional kernel versions in the Qubes ISO. That would make it a lot more user friendly to install Qubes on newer hardware without having to manually create an ISO with a newer kernel.

[Dylanger Daly]

So I managed to wrestle with Qube's build system and compile kernel-latest 5.8.1 after installing the resulting rpm I'm still observing the same black screen bug.

Linux dom0 5.8.1-1.qubes.x86_64 #1 SMP Wed Aug 19 11:21:32 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

This bug must be something IOMMU / Memory Management related.

> Yes, better support for new Ryzen CPU's would be nice.

I'm sure this issue will be fixed within a few weeks, AMD's new Laptop CPUs are all the rage right now, support shouldn't be too far behind, in the meantime I'll monitor the master branch for Xen and watch for any AMD specific commits.

Fedora 32 USB boots just fine, that ISO is on 5.6 or something, so I'm fairly sure it's Xen related.

[Chris Laprise]

On 8/19/20 8:52 AM, Dylanger Daly wrote:
> So I managed to wrestle with Qube's build system and compile
> kernel-latest 5.8.1 after installing the resulting rpm I'm still
> observing the same black screen bug.
>

> /Linux dom0 5.8.1-1.qubes.x86_64 #1 SMP Wed Aug 19 11:21:32 UTC 2020
> x86_64 x86_64 x86_64 GNU/Linux/

Thanks for trying this.

>
> This bug must be something IOMMU / Memory Management related.
>
> > Yes, better support for new Ryzen CPU's would be nice.
>
> I'm sure this issue will be fixed within a few weeks, AMD's new Laptop
> CPUs are all the rage right now, support shouldn't be too far behind, in
> the meantime I'll monitor the master branch for Xen

> <https://github.com/xen-project/xen/commits/master>and watch for any AMD
> specific commits.

I am thinking of ways to make a standard Linux KVM environment more
Qubes-like just in case this takes months or forever. My short list is:

1. Secure copy+paste

2. Auto snap-back (like read-only) for guest root

3. Isolated NICs via passthrough

4. Split GPG

Probably a good place to get tips for these would be Whonix forum, since
they also use non-Qubes virtualization.

[Dylanger Daly]

I'll start posting in the new forum