Install errors on Thinkpad P1 (aka X1 Extreme) with R4.0 and R4.0.1-rc2
Eric Duncan
Anaconda installer gets past the initial setup, partitions the drive, etc and reboots. After first reboot, when selecting which qubes to configure, the system starts to configure all the Qubes.
It is upon completion of this step, just before the system switches to the login screen, that the error message pops up:
/usr/bin/qvm-start sys-firewall failed
stdout: ""
stderr: "Start failed: internal error: Unable to reset PCI device 0000:00:1f:6:
no FLR, PM reset or bus reset available, see /var/log/libvirt/libxl/libxl-driver.log for details.
"
Click OK switches a black screen, and the system become unresponsive. Only a hard reset gets it to reboot, at which it boots up to the LUKS password, I enter it, and the system boots to a black screen again - unresponsive.
I've tried flipping various options in BIOS to no avail.
I suspect it's the Nvidia graphics. However, I can't get the installer to boot past Xen with Hybrid graphics - Xen pauses for 5 minutes or something, and goes black.
NOTE: I got the same error on a Thinkpad X1 Tablet 3rd Gen using R4.0.1-rc2. However, switching to R4.0 RTM did not get the error and the system installed normally.
WARNING (to anyone else installing on a P1/X1 Extreme/P52, etc): To install, you must switch to discrete graphics in BIOS (no hybrid). But, DO NOT DO THIS unless you have BIOS 1.17 or later or you will BRICK YOUR THINKPAD! Known issue across most latest-generation Thinkpads these days with discrete graphics.
Achim Patzner
> It is upon completion of this step, just before the system switches to the login screen, that the error message pops up:
>
> /usr/bin/qvm-start sys-firewall failed
> stdout: ""
> stderr: "Start failed: internal error: Unable to reset PCI device 0000:00:1f:6:
> no FLR, PM reset or bus reset available, see /var/log/libvirt/libxl/libxl-driver.log for details.
> "
sys-net (I do not really see any reason it should be assigned to sys-
firewall... are you sure?) it needs to get set to no-strict-reset=true
and permissive=true; take a look at qubes-os.org.
> Click OK switches a black screen, and the system become unresponsive. Only a hard reset gets it to reboot, at which it boots up to the LUKS password, I enter it, and the system boots to a black screen again - unresponsive.
(nouveau.modesetting=0).
> I suspect it's the Nvidia graphics. However, I can't get the installer to boot past Xen with Hybrid graphics - Xen pauses for 5 minutes or something, and goes black.
the iGPU anyway, booting faster, using (at least on my machine) less
energy and you do not meed to modify the kernel command line. And on
kernel-latest my system is working with all cores.
> WARNING (to anyone else installing on a P1/X1 Extreme/P52, etc): To install, you must switch to discrete graphics in BIOS (no hybrid). But, DO NOT DO THIS unless you have BIOS 1.17 or later or you will BRICK YOUR THINKPAD!
off secure boot did that for me. Switching to dGPU is only causing
problems if you do not wait on the next reboot for the system to
reinitialize the device tree in ME (and thus starting with empty ACPI
tables) by resetting it at just the right time during the 30 seconds
this would take.
Achim
awokd
Eric Duncan
On Friday, January 4, 2019 at 5:16:16 PM UTC-5, Achim Patzner wrote:
>
> The good old I219-LM problem... Before assigning (or after 8-) it to
> sys-net (I do not really see any reason it should be assigned to sys-
> firewall... are you sure?) it needs to get set to no-strict-reset=true
> and permissive=true; take a look at qubes-os.org.
I think you mis-understand. I am not assigning/attaching anything. This is doing installation only that generates the error.
> Use the dGPU or take care of turning off the nouveau driver completely
> (nouveau.modesetting=0).
>
> Why don't you use the nVidia GPU instead? It is definitely faster than
> the iGPU anyway, booting faster, using (at least on my machine) less
> energy and you do not meed to modify the kernel command line. And on
> kernel-latest my system is working with all cores.
>
That is not ideal. The main reasons to disable the dGPU is for battery life.
Under Arch, I can do this (not efficiently have you) with bbswitch. However, I am not making it that far with Qubes as I cannot boot into the system after installation.
> > WARNING (to anyone else installing on a P1/X1 Extreme/P52, etc): To install, you must switch to discrete graphics in BIOS (no hybrid). But, DO NOT DO THIS unless you have BIOS 1.17 or later or you will BRICK YOUR THINKPAD!
>
> That didn't break mine. Turning on Thunderbolt BIOS support and turning
> off secure boot did that for me. Switching to dGPU is only causing
> problems if you do not wait on the next reboot for the system to
> reinitialize the device tree in ME (and thus starting with empty ACPI
> tables) by resetting it at just the right time during the 30 seconds
> this would take.
>
First of all, what laptop do you have? "That didn't break mine" indicates you also have a P1? Please share your experience installing Qubes to a fully functional state (including suspend - not even us Arch guys have figured out suspend yet with the ACPI tables on the P1).
It is an extremely well known bricking problem with all modern (early to late 2018) Lenovo Thinkpad models:
https://www.reddit.com/r/thinkpad/comments/a2g0k4/warning_do_not_change_from_hybrid_graphics_to/
(and dozens of threads on the Lenovo forums)
If you have an early 2018 Thinkpad, you're pretty much ok. My warning applies to all current models.
It got so bad Lenovo actually pulled the last 2 or 3 BIOS from just about every Thinkpad and Yoga series from the downloads site. The 1.15 my P1 came with is long gone, as well as the 1.13 version that was there.
Only 1.10 and 1.17 are available now - and 1.17 I can confirm does not brick it.
Eric Duncan
On Monday, January 7, 2019 at 12:49:52 AM UTC-5, awokd wrote:
>
> Try temporarily disabling your wifi card for the install.
>
Good idea! I disabled it, along with just about every device I could find. It generated a different GUI error during the Post installation qubes configuration screen, with everything disabled and disconnected:
['/usr/bin/qvm-start', 'sys-firewall'] failed:
stdout: ""
stderr: "PCI device dom0:00_14.3 does not exist"
Once logged in, I see that Fedora in dom0 isn't detecting the 4K resolution like it has on my Thinkpad Yoga 3rd gen and Thinkpad Tablet 3rd gen devices - everything is tiny text.
Running lspci shows me that 00_14.3 is described as the network controller (it has an onboard network card, along with the wifi card). Given, the PCIe device IDs could change since I have been disabling/enabling various devices at this point. But the error would seem to point to the onboard network card, even though it is Disabled in the bios.
Now, I've noticed that none of the VMs startup (sys-firewall, sys-net, sys-usb).
$ sudo systemctl status qube...@sys-net.service
This shows the exact same error message as above. Going into the bios and re-enabling everything, generates the original error in the first message in this thread.
00:1f.6 points to "Ethernet controller"
00:14.3 points to "Network controller"
Going back into the bios and disabling "Wireless" and "Ethernet" devices (under Security), sys-usb is now able to start. However, sys-net and sys-firewall gives the exact same "00:14.3 does not exist" error on starting any VM.
However, now it does not show under lspci with everything disable. Nor does 14.3 show up under any Qubes settings as attached to either sys-net or sys-firewall.
I did another full install with everything disabled, and the same error of 14.3 does not exist keeps showing up, preventing sys-net and sys-firewall from starting.
awokd
disabled. Check sys-net's Qube Setting/Devices and remove 14.3 from the
right if it's there. Leave 1f.6/Ethernet enabled and assigned if it's
not causing problems, then try to start sys-net.
Eric Duncan
> Weird, not sure how it can detect that 14.3 device with it supposedly
> disabled. Check sys-net's Qube Setting/Devices and remove 14.3 from the
> right if it's there. Leave 1f.6/Ethernet enabled and assigned if it's
> not causing problems, then try to start sys-net.
It was a combination of me enabling/disabling multiple things.
I got it going - though I've exhausted all the time I had to spend on Qubes on this laptop... I'm going to make a HCL with the information as well to help the next person, or when I get more time this summer to spend days/weeks with it again.
From a fresh install...
* Make sure you have BIOS 1.17 or later for the Thinkpad P1. Anything earlier than that WILL BRICK your laptop with these instructions.
* BIOS: switch to Discrete Graphics (nvidia only for now).
* BIOS: Under Security -> I/O Port Access, disable "Ethernet LAN". This is the "I219-LM" onboard ethernet controller that causes the 00:1f.6 PCI error mentioned above, about how the sys-* VMs won't start because it can't reset it.
Now, boot the media and install Qubes 4.0.1 normally. I wipe the entire disk and all Windows partitions personally, never a dual boot so I can't help with that.
After the first install phase, it will reboot into the Post Install phase screen (where it asks you which VMs you want to configure). Go ahead and select what you want.
However, the Post Install steps will fail at the end with the original error stated in the OP, or something close to it (perhaps 00:14.3 as well). Just ignore this for now. Press OK to "Finish Installation."
Note: Do not step away from the laptop at this time. If you do, and the 10 minute timeout occurs after the post install/error message displays, the screensaver will kickin - and you won't get back. It was a number of ACPI errors in the logs. Power-off (hold power for 4 seconds) and start the Post Install phase again.
You will get prompted to login. Go ahead and login. Or reboot for good measure.
You will notice none of the sys-firewall, sys-usb, nor sys-net are running. The sys-net may start now, manually, but the others will continue to error. To confirm the error, open a Terminal in dom0 and run:
$ sudo systemctl status qube...@sys-net.service
It should the errors I listed above, such as "PCI device dom0:00_14.3 does not exist" and when you run an "lspci" you don't see it either. There's nothing to do on this error. We're going to re-enable the Ethernet controller now. The 14.3 device is the Wireless card btw, but don't worry about this error - it will resolve itself when you fix the next error below.
Reboot into BIOS and re-Enable the "Ethernet LAN" you disabled earlier above. F10, Save and reboot back into Qubes. If you press ESC after the disk encryption key, you'll see that sys-net nor sys-firewall starts (and maybe sys-usb may fail as well). That's fine, we're going to fix that now.
Now that you have the I219-LM ethernet controller re-enabled, if you run the systemctl status command again you'll see a different error - the one I originally posted in the OP above, "Unable to reset PCI device 0000:00:1f:6".
To fix this one error, that also happens to resolve the 0:14.3 not found error, you need to detach the "1f:6" device from sys-net, and reattach it with special permissions:
$ qvm-pci detach sys-net dom0:00_1f.6
$ qvm-pci attach --persistent --option permissive=true \
--option no-strict-reset=true sys-net dom0:00_1f.6
After this, you can qvm-start sys-net and then sys-firewall. You may want to reboot just for good measure.
I think I also detached the reattached the 00:14.3 during my debugging. But I currently see I don't have the permissive nor no-strict-reset flags set on my 14.3 device - so that's not needed to get sys-net to start.
The reason sys-firewall give the same error on "Unable to reset PCI device 0000:00:1f:6" when trying to start them is, I can only guess, sys-firewall is dependent on sys-net running - and if sys-net fails to start with the error, that error message propagates up to the originally started vm, sys-firewall - showing the same error. Makes sense, as when the error message changes, they both had the same error under systemctl logging (and journalctl).
I then spent several days trying to get Nvidia drivers loaded to no avail. Also, trying to get just the Intel card working (hybrid graphics) but I've hit a brick wall there - Xen just freezes on the initial splash screen of the 4 lines showing what kernel is loading. The system hard locks up there, with no logs. I highly prefer the Intel drivers for battery life.
At this point, I'm just throwing Arch Linux on it and will come back and revisit it when I have time. It's too bad... I purposely bought this laptop specifically for Qubes.
awokd
> I then spent several days trying to get Nvidia drivers loaded to no avail. Also, trying to get just the Intel card working (hybrid graphics) but I've hit a brick wall there - Xen just freezes on the initial splash screen of the 4 lines showing what kernel is loading. The system hard locks up there, with no logs. I highly prefer the Intel drivers for battery life.
> At this point, I'm just throwing Arch Linux on it and will come back and revisit it when I have time. It's too bad... I purposely bought this laptop specifically for Qubes.
>
Achim Patzner
> Eric Duncan wrote on 1/13/19 9:08 PM:
>
> > I then spent several days trying to get Nvidia drivers loaded to no avail. Also, trying to get just the Intel card working (hybrid graphics) but I've hit a brick wall there - Xen just freezes on the initial splash screen of the 4 lines showing what kernel is loading. The system hard locks up there, with no logs. I highly prefer the Intel drivers for battery life.
talking about a desktop replacement machine (with the P1 being the
smallest of the current generation but even that one will suck the
battery dry before you finished watching a movie) and intending to use
it without a power supply is a rather limited experience.
> A nouveau.modeset=0 might help there.
generation P systems offer you "dGPU and iGPU active" or "dGPU only"
(unlike the Px0 and Px1 where it was "iGPU" or "dGPU and iGPU").
But it's not a problem in itself; the dGPU is using less energy than
the iGPU running on "intel" or "modesetting" (and in my case it's a
P3200). And with kernel-latest installed you get all your cores working
vs. 1 core on a P52.
> That is too bad; I thought those Thinkpads usually worked well with Qubes.
constantly comparing what Qubes can do with an Arch on a second
(identical) machine I have around. The worst complaint I might want to
make is that nouveau does not have control of screen brightness (unlike
intel/modesetting) and acpi_backlight=vendor thinkpad-
acpi.brightness_enable=1 is not working as thinkpad-acpi doesn't know
the hardware yet. Both network interfaces (and Bluetooth) are working
on the P52 so they will be working on the P1 if correctly set up.
If there are any Qubes-related problems I'm very sure they will be
found and fixed sooner or later.
Achim