q4rc4 very slow. VMs take 23 - 33 seconds to start

[pixel fairy]

[user@dom0 ~]$ time qvm-start personal

real 0m23.517s
user 0m0.182s
sys 0m0.065s
[user@dom0 ~]$ time qvm-start alpha

real 0m23.801s
user 0m0.191s
sys 0m0.056s
[user@dom0 ~]$ time qvm-start alphax

real 0m32.831s
user 0m0.193s
sys 0m0.059s

starting with debug turned on takes 46 seconds. it shows a console window with

SeaBIOS
Machine UUID
Booting from ROM...
Probing EDD...

15 seconds for the console window to come up, with the first 3 lines
8 seconds later for Probing EDD to come up
23 seconds after that for the VM to start and the console window to go blank.

[Chris Laprise]

Is this Debian or Fedora? If the latter, can you try Fedora?

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[pixel fairy]

Fedora. just tried debian. 44.286s seconds.

[pixel fairy]

On Wednesday, February 14, 2018 at 4:58:06 PM UTC-8, pixel fairy wrote:
> Fedora. just tried debian. 44.286s seconds.

Forgot the hardware.

i7-6700, 64gigs ddr4, supermicro c7z170-sq, onboard intel graphics.

[Chris Laprise]

What virt_mode is used? Default is pvh; Try switching the mode to hvm
(and this let you use debug mode). Then there are logs in dom0
/var/log/qubes for each VM.

On the VM side you can try 'systemd-analyze blame' for start timings,
also 'journalctl' and 'dmesg'.

[pixel fairy]

pvh. the hvm ones took even longer. looked at a couple systemd-analyze, one of them had 10s for dkms and 40 for qubes-update-check, even though that one only took 25s to boot, at least according to dom0. could whatever tells dom0 a guest is up have run before that?

will play with this more and get back to you.

turns out the qvm-pref debug doesnt matter in boot time. its hvm that takes around 40 seconds, and pvh that takes around 25.

a standalone hvm with no os installed took 16 seconds to "start"

this started happening after installing 4.0rc4 over 4.0rc3. had to qvm-prefs the restored vms to pvh. at first i thought it was just the performance hit from mitigating speculation vulnerabilities, but others were reporting better performance in rc4 than rc3.

[WillyPillow]

I also noticed similar slowdowns. I have no experience with rc3 (upgraded directly from 3.2), but your numbers seem to councide with my experience. Hardware is also pretty similar, namely Z170/6700K.

--WillyPillow
----------
https://blog.nerde.pw/
PGP fingerprint = 6CCF 3FC7 32AC 9D83 D154  217F 1C16 C70E E7C3 1C8
----------

[David Hobach]

Got 13s with pvh on a laptop last built in 2012 or so, i.e. your timings
are rather odd indeed...

[awokd]

On Thu, February 15, 2018 2:24 am, pixel fairy wrote:

> at first i thought it was just the
> performance hit from mitigating speculation vulnerabilities, but others
> were reporting better performance in rc4 than rc3.

Maybe they are on AMD. :)

To rule out mitigations, I know Linux has an option to turn it off. Does
Xen have something similar? Try that and see if it restores your
performance.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Yes, there is "xpti=false" option for Xen command line (xen.gz option in
grub, or options= line in xen.cfg for UEFI).

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAlqHcg4ACgkQ24/THMrX
1ywm/gf/TBoqkz8zMaJS+oEvq3ezpydA9vc+NZVvxv+qYr4OKDq+BNIbhwRRV90Y
kPrEsBTLPhBhjA39RB37gjv3tkLj1pUMg/mZRFENbS6/QU3zIuOgylBdwLIGdTQf
sz56zdU9ypAYJfFGZhpj68c7PGyukG7pCdvG5c0kezwG30Xls1Cn221Nwxsc1zt1
kuIUehQuSKcv+v5upX6WS0rPii0Q6pUF734QtHaT41hLsFmdnXPc9bvMK9sEzwH7
EV4eiLzv3l3XiHlpB7y4I6THQnQSbjTdpU8eXN5LY7XLxrHqUUlfxQdXDgprKAFA
Tu/KTSFt9B9GFbv+uz+sGE5MiUmY5A==
=96W1
-----END PGP SIGNATURE-----

[pixel fairy]

On Friday, February 16, 2018 at 4:07:10 PM UTC-8, Marek Marczykowski-Górecki wrote:

> Yes, there is "xpti=false" option for Xen command line (xen.gz option in
> grub, or options= line in xen.cfg for UEFI).

tried that by editing the multiboot /xen-4.8.3.gz line while booting. no change. maybe its a different change between rc3 and rc4. seems like a stretch, but one that only affects supermicro c7z motherboards?

[awokd]

Try the Linux equivalent to disable mitigations in your template kernel
options too maybe?

[WillyPillow]

-------- Original Message --------

I did some timing, and for PVH, my Fedora 26 VMs take about 30s, while Debian 9 takes about 45s. For HVM, qrexec usually times out (did not bother to adjust the timeout).

Just to clarify, my mobo is an ASRock Z170, not a Supermicro one.

[David Hobach]

On 02/17/2018 05:26 AM, WillyPillow wrote:
> I did some timing, and for PVH, my Fedora 26 VMs take about 30s, while Debian 9 takes about 45s. For HVM, qrexec usually times out (did not bother to adjust the timeout).

P.S.: I recall I had similar issues with HVM in 4.0rc1, cf.
https://github.com/QubesOS/qubes-issues/issues/3149

So I think only your pvh timings are strange.