DispVM broken in 3.2?

[qubes...@riseup.net]

DispVM functionality does no longer work for me in Q3.2 since yesterday.

First I assumed the template is broken but after reinstalling the fedora-26 template
from the rpm repo and recreating the DVM I get the very same error.

Has anyone else the same problem after installing the latest dom0 updates?

from the dom0 yum.log the latest installed updates were (2018-03-15):

libgcc-5.3.1-6.qubes1.fc23.x86_64
2001:xen-licenses-4.6.6-37.fc23.x86_64
2001:xen-libs-4.6.6-37.fc23.x86_64
2001:xen-hypervisor-4.6.6-37.fc23.x86_64
2001:xen-runtime-4.6.6-37.fc23.x86_64
libstdc++-5.3.1-6.qubes1.fc23.x86_64
2001:xen-hvm-4.6.6-37.fc23.x86_64
2001:xen-4.6.6-37.fc23.x86_64
qubes-gpg-split-dom0-2.0.28-1.fc23.x86_64
libgomp-5.3.1-6.qubes1.fc23.x86_64


I also tried a reboot but that did not change anything either.


This is the error I get when trying to start xterm (or anything else) in a DispVM:


echo xterm| /usr/lib/qubes/qfile-daemon-dvm qubes.VMShell dom0 DEFAULT red
time=1521287074.5, qfile-daemon-dvm init
time=1521287074.51, creating DispVM
time=1521287075.04, collection loaded
time=1521287075.05, VM created
time=1521287075.06, VM starting
time=1521287075.06, creating config file
time=1521287075.06, calling restore
Traceback (most recent call last):
File "/usr/lib/qubes/qfile-daemon-dvm", line 217, in <module>
main()
File "/usr/lib/qubes/qfile-daemon-dvm", line 205, in main
dispvm = qfile.get_dvm()
File "/usr/lib/qubes/qfile-daemon-dvm", line 167, in get_dvm
return self.do_get_dvm()
File "/usr/lib/qubes/qfile-daemon-dvm", line 103, in do_get_dvm
dispvm.start()
File "/usr/lib64/python2.7/site-packages/qubes/modules/01QubesDisposableVm.py", line 193, in start
domain_config, libvirt.VIR_DOMAIN_SAVE_PAUSED)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4405, in restoreFlags
if ret == -1: raise libvirtError ('virDomainRestoreFlags() failed', conn=self)
libvirt.libvirtError: internal error: libxenlight failed to restore domain 'disp1'

[Unman]

Have you made sure that you have updated the packages in the template
and regenerated the DispVM?
That's a common source of these problems and fixed it for me.

[qubes...@riseup.net]

> Have you made sure that you have updated the packages in the template
> and regenerated the DispVM?
> That's a common source of these problems and fixed it for me.

Yes I've updated and regenerated the dispvm after the dom0 update.

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Same here. Updating and regenerating did not resolve the issue.
Tracking:

https://github.com/QubesOS/qubes-issues/issues/3711

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAlqtcQcACgkQ203TvDlQ
MDCQpw/+KBAcs3K4hqLv9XrgbSM3q7FyiDbNtvM/bd78v3/OI9Dmd74HW1R+irUP
jbDCpunKhMJs8y/a0i7urrYPRSQ4Yy2j3QT4OduZj1YjqvxvtUzSD7i6NNdWgrV9
oCsKLjrZBvTtSMv2TbqlyMIbCwuvvUCUvCqN7478iGw0mNUvdjk0BJEwRqFmcPKK
oHH/NKI9Ay8azjuFGu0l6Z+L1m73FYuJ8imch9GgAucJqx+YjMvX4cMfUoqMgu/m
YougHWC9Ddyq4VFZ3tojrIaANOHRZeU2DzWVK58I/iDxMmWKHCV8xfD6iZLKT9VG
zrONSZrAcbEUY+CBdEu45wcmhs7U6MD5DBx4LSXdVwF5yBYXPpuqcLiVgkhS6ro/
LBpSgGRQdHs8ePAr4BdaKA9Y/VRHjm8TNtHW5AriQ+kwnQtPl6VHjTNKOxn/xRoc
02MMDXQXkd2YPHcTYqSGpp61tuhBUgMO06+I09kjxzGIyGf0CFGUY47CDRUTvpXJ
/IeXbu+ag3DvDezg18vkKo/WAALJyhSpnyBgk1LWGAohvk/t4AtKGeSZP3hTUTXJ
l/UbnZEeJ2hPMq6Vb0X4qBzewioe74Qw0G263F5giYeOzBBDcUPt46+e8MulWmEW
tEt3Y4lx3p3TnyasWK4slASj+sulKwX/BQ/t7yVygyEoJPKi+UQ=
=WmKY
-----END PGP SIGNATURE-----

[qubes...@riseup.net]

Andrew David Wong:

> On 2018-03-17 11:40, qubes...@riseup.net wrote:
>
>>> Have you made sure that you have updated the packages in the
>>> template and regenerated the DispVM? That's a common source of
>>> these problems and fixed it for me.
>
>> Yes I've updated and regenerated the dispvm after the dom0 update.
>
>
> Same here. Updating and regenerating did not resolve the issue.
> Tracking:
>
> https://github.com/QubesOS/qubes-issues/issues/3711

thank you for filing it on gh!

Assuming the latest updates introduced this regression:

What would be the step to downgrade to the previous version to
get dispVMs working again?

qubes-dom0-update --action=downgrade xen
something like this or is it necessary to specify
all packets individually?

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

`dnf history undo` or `dnf history rollback` might work, depending on
which is appropriate:

https://unix.stackexchange.com/q/349238

However, I don't know whether this is the optimal way, or whether
it'll break anything, since these are Xen packages. There's also a
concern that downgrading might render the machine vulnerable to
Spectre SP2 again.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAlqtdJIACgkQ203TvDlQ
MDC8Mw//fwCHZ9CeTUEHEA024XJ7WysII+FZvjk8DviLiRfFjaY1wGEYryY32uvo
5pejGc4l6kjnTW1Lolcn7AA7Ne4TpWXP6B7cl2SAWcHPm2+ka9E9lX8gy8S+16rt
dY7uVxawFFQr2MwsW0bPiv8y510k7VjnAvchWWA6UEAYdW5PytrFNKWEIxxttxOc
v+aEwAM5D399vES7++VgknQ9kKgaccN+ymLxdxjhYz666DdrWtBuB7YJDEnXKFWl
7Jux0W26loz1E3ADr/SOapjmxbmSQe/J4xcffpL4NOn6peP6VCtZqLFNzKM0FIVo
aqUQLAuM8dGLOD7I4MjAW0Rxjg65UZV8G/dzY+RcUA7Qls0nw03ywLdnG5HwS9oS
cJel/R8Bbvw7KXHrstENRx34i+UMmLGjvJoC6KAgS194aUzA0touODWlrQyPGaT+
gbRpSKAeDa0tXOX61VOobCFJGFm7oEDEV9l6DH+ln92XQNIwaNcbIBhazC9Zd4xi
zIJB0IkylrjahLZRuwpKNV8kibGgI62uZoq8kcqBQPZ65WC/8n7jyKYafOvXizwS
NGzo7QlQ2js/OcJW/FrxiR1t2+12ObNi8BkwoE5Srf8I2bceKMADNuqQEFxKsExI
0AL0Zd6cZ56fjf1QnIbcVuKvM7vFFZxl7zLIyJo2RER6dhZnUsg=
=KsgK
-----END PGP SIGNATURE-----

[awokd]

On Sat, March 17, 2018 7:57 pm, qubes...@riseup.net wrote:

> What would be the step to downgrade to the previous version to
> get dispVMs working again?
>
> qubes-dom0-update --action=downgrade xen something like this or is it
> necessary to specify all packets individually?

Depending on how attached you are to Fedora, you might want to temporarily
try the Debian template instead of downgrading Xen?

I'm still on -36 so can't confirm...

[qubes...@riseup.net]

awokd:

I get the same error after switching to debian-9 as the dispVM template.

So switching to debian-9 for dispVM is unfortunately _not_ a possible workaround for this.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I think it wouldn't help. It looks like something is broken in Xen, not
in template.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAlqm/ksACgkQ24/THMrX
1yyvcgf+MPUg0EmPMiT3RhPz2D2lz+CNrVlWsqgnlUPYc2RSYDAZeMZiSM/JUEOp
eAwYYDekz47iWeoVgiuwvv28swlTMMw71kNQcgBR8ZndoBLjPFR/oLG9yr5RP/bI
HhO8gb5TlZai7WWV8/TDbiJ0FHR2WczqD5oXBkcr1+lB6YxsiK6eXmnHbiTRSnvU
8SINDfVoeJhJuXKjUuFK7DddP+Bdoq7UfdRwg1ajqiowrZgItHhxt3Bq0Si+M1rF
Ppo5Hw/vW3LWuy+q1EkhooQX+0hglOaElnbgWyes0iyt04QJdPYShvoQ2qyUnvfK
I+r3ncgZoVmXFDfpDXE0VcixKMt32A==
=4fa7
-----END PGP SIGNATURE-----

[awokd]

On Sat, March 17, 2018 8:13 pm, qubes...@riseup.net wrote:

> I get the same error after switching to debian-9 as the dispVM template.
>
>
> So switching to debian-9 for dispVM is unfortunately _not_ a possible
> workaround for this.

I saw Marek confirmed it's a Xen issue too. Thanks for trying it anyways,
and for the heads up!

[Unman]

You're using Testing arent you?

You could downgrade to the 36 version of xen-libs and runtime etc. That
should fix your problem until 6-38 is released

[qubes...@riseup.net]

>>>> What would be the step to downgrade to the previous version to
>>>> get dispVMs working again?
>>>>
>>>> qubes-dom0-update --action=downgrade xen something like this or is it
>>>> necessary to specify all packets individually?
>>>
>>> Depending on how attached you are to Fedora, you might want to temporarily
>>> try the Debian template instead of downgrading Xen?
>>
>>
>> I get the same error after switching to debian-9 as the dispVM template.
>>
>> So switching to debian-9 for dispVM is unfortunately _not_ a possible workaround for this.
>
> You're using Testing arent you?

Yes, to be precise:

qubes-dom0-current-testing is _not_ enabled.

qubes-dom0-security-testing _is_ enabled.

> You could downgrade to the 36 version of xen-libs and runtime etc. That
> should fix your problem until 6-38 is released

What would be the recommended way of downgrading? (and wouldn't it be
better to revert all packages of the given upgrade?)

qubes-dom0-update --action=downgrade xen-libs xen-runtime xen-hvm

Maybe lets add generic downgrade instructions for such cases (dom0) to the documentation?

[Unman]

I would just downgrade the Xen packages to start.

There are already instructions here:
https://www.qubes-os.org/doc/software-update-dom0/

Look at the section on downgrading a specific package.

[qubes...@riseup.net]

I used the following commands in dom0 to restore dispVM functionality:


qubes-dom0-update xen-libs-4.6.6-36.fc23.x86_64 xen-hypervisor-4.6.6-36.fc23.x86_64 xen-runtime-4.6.6-36.fc23.x86_64 xen-hvm-4.6.6-36.fc23.x86_64 xen-4.6.6-36.fc23.x86_64 xen-license-4.6.6-36.fc23.x86_64

dnf downgrade xen-libs-4.6.6-36.fc23.x86_64 xen-hypervisor-4.6.6-36.fc23.x86_64 xen-runtime-4.6.6-36.fc23.x86_64 xen-hvm-4.6.6-36.fc23.x86_64 xen-4.6.6-36.fc23.x86_64 xen-license-4.6.6-36.fc23.x86_64


(downgrading the packages in the template was not necessary)

[cooloutac]

I still use fedora 26 for my dispvm. from time to time it breaks. even when using debian. has always been a problem for me since using qubes. the solution for me is just to simply delete the DVM template. Using the qubes manager in 3.2, select view and show inactive hidden vms. then simply delete the fedora dvm. Then recreate it with qvm-create-default-dvm fedora-26.

[cooloutac]

I also use a cloned fedora-26. I doubt this makes a difference though.

[cooloutac]

xl info shows i'm on xen 4.6.6 is what what you guys are on?

One thing that does bother i've noticed past few month is from time to time, especially if I load it right after machine boots, The dispvm will use 100% cpu and be freezing on me and working slow. And if I kill it, from that point on as soon as I load it it has the yellow triangle. Until I reboot the whole machine. Just seeing that yellow triangle bothers me.

To lessen this i try to give time for qubes to fully boot before I load the dispvm. But it will still happen at times, though much much less frequently, maybe depending on the page I visit.

I don't know what this means though I'd put it out there.

[cooloutac]

I should add the dispvm is more likely to freeze with 100% cpu use when loading from the dom0 appmenu option loading firefox.

Almost never happened when loading chrome from an appvm terminal in a dispvm. But now its freezing at 100% cpu use everytime I try to attach a usb to it. Whether loading the firefox app from main menu or loading chrome one from an appvm terminal...

[cooloutac]

On Thursday, March 22, 2018 at 5:47:25 PM UTC-4, cooloutac wrote:
> I should add the dispvm is more likely to freeze with 100% cpu use when loading from the dom0 appmenu option loading firefox.
>
> Almost never happened when loading chrome from an appvm terminal in a dispvm. But now its freezing at 100% cpu use everytime I try to attach a usb to it. Whether loading the firefox app from main menu or loading chrome one from an appvm terminal...

wanted to add that adding more cores to the dispvm doesn't help.