qvm-open-in-vm (Q4.0rc2)

[Frédéric Pierret (fepitre)]

Hi,
I'm experiencing something strange in Qubes 4.0rc2: when I try to qvm-open-in-vm several images in the same vm it seems to does not work. For e.g. from work appvm:

qvm-open-in-vm '$default' appel.jpg (then I choose e.g. personal), It opens great and I keep the image open in personal, and then I do the same with appel2.jpg but the image does not appear and does not exist in /tmp/work-*

Doing this with two differents targets VM or with e.g. PDF or TXT works great. Can someone test it please (just before trying to eventually deeply debug...)?

[Yuraeitha]

I second this, it's also an issue I'm having on Qubes. Though was this reported on github yet? is it a common or more rare problem?

I encounter this issue only in some AppVM's and Templates, and oddly, sometimes the ones working stop working, though only for a while. I do not yet know how to reproduce this odd behaviour.Tthough some of the Templatres or AppVM's are permanently broken and there is no way to start applications from the VM.

qvm-run -q -a --service -- 'AppVM' qubes.StartApp+'Application'
or
qvm-run AppVM Application

Sometimes one work while the other doesn't, and other times both don't work.
Sometimes I manage to run VM-Terminal with the above commands, and through that I can start the desired application in the VM.

So not all applications can be started either. Frankly, this bug doesn't make a whole lot of sense, it's so dynamic.

Some templates like Fedora-25 or various random AppVM's, oddly never seem to stop working with the qvm-run, while contradictionary some AppVM running on the working Fedora template, might stop working. So too are some templates that really don't work, like Debian, where oddly the AppVM based on the broken template, seems to work.

This mess of a contradictions, or seemingly catch-22 paradoxes, is not something I've made sense out of yet. But one thing is for sure, it's frustratingly annoying.

It'd be nice to know some workarounds if anyone has some temporary solutions or ideas until it gets fixed.
I've tried changing them from HVM to PV, but running with PV only aggrivates the problem, and it has never solved the issue for me.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I can confirm this behavior. (I guess) it is because the opening the
second file in fact open it in the same viewer/editor instance. And that
command exit immediately after sending a message to main (first)
instance. So qvm-open-in-vm (in fact, /usr/lib/qubes/vm-file-editor)
thinks you already closed the file and remove it from the target VM.
This is very similar to more general issue we have with gnome-terminal
in DispVM - the command you use to open app/file do not wait until you
close app/file.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJaAgksAAoJENuP0xzK19csi/AH+gOZ1aUZVuLv6cKAjNBSckdK
o7hwZBhdwwyeeBJ3NYdl0R1VDXWOSK+RMdh5jPXnXVdJHjJb2CXJ2nwrofWwP8C0
Kv4O3scER6Iq7iFOns/0C9pvXAVs9FI0T9rZ98kGy9isIugDDIJJR2ie7hvKBhg1
BeSXLFIBNPv3TERRxOqiZpOkWh7+YmDZWsIpTX1nldsEZDzWxvkbKbOK+fg7RhY1
AkBJFjNTbNPIhwIqXVBIBBB7Ygu4J0TNP6k7f9eWYqCJrGgc4ykAglK/UhUf4Cwx
4lyhRWveNt4td5bGBOCslnIQNQYdztViIN/ubxAcQMcIk4kaXHaqR19zc1MxNEE=
=M00S
-----END PGP SIGNATURE-----

[Yuraeitha]

Does that mean the qvm-run issues run on freshly started AppVMs and Templates, isn't related to the original topic issue with qvm-open-in-vm?

All the VM domains start just fine, and even if shutting down and starting again, it rarely fixes the issue, some never even work despite restart. It's also really, really weird and puzzling that this only applies to some Templates and AppVMs.

My apologies if this isn't related to the original topic, at which case my post was off-topic. But if there is any chance its related information, then I'll post it here.

I've just tried running Qubes 4 on my other hardware system, which seems not to have this issue. Perhaps it's hardware specific or software condition specific. Either way, just attempting to add information to the original post, not to hijack it.

[Yuraeitha]

On Tuesday, November 7, 2017 at 7:27:48 PM UTC, Marek Marczykowski-Górecki wrote:

I spoke too soon, it's not fixed on the Qubes 4 install on the second hardware machine.
Again, this is just adding information to the original post, I'm not seeking a solution in this thread.

Rather, everything is still working (which is a big improvement to the first machine in my first post with similar but much more aggravating version of the issue on the first Q4 machine), but eventually stops working too on the second Q4 machine after a shutdown or two of the same AppVM, similar to what you described, yet slightly different too.

What is different however, this is on entire VM domain level and not app/file level inside a VM (macro vs. micro, yet similar issue it seems). Both the Qubes widget, 'xl list' rapport that the VM domain has shutdown, but if entering the VM Settings, it says in the device menu for example, that the VM still has not shutdown.
Basically, Qubes thinks the VM has shutdown, but it doesn't seem to have been, not properly at least. No crash either, it was shutdown normally when it was running.

Only quick fix to this, seems to be restarting Qubes itself altogether, and every non-working AppVM can start again.

Isn't all this possibly related? Maybe there is something deeper going on?

[Frédéric Pierret (fepitre)]

Thank you for your feedback

Le mercredi 8 novembre 2017 06:20:12 UTC+1, Yuraeitha a écrit :

Does that mean the qvm-run issues run on freshly started AppVMs and Templates, isn't related to the original topic issue with qvm-open-in-vm?

All the VM domains start just fine, and even if shutting down and starting again, it rarely fixes the issue, some never even work despite restart. It's also really, really weird and puzzling that this only applies to some Templates and AppVMs.

My apologies if this isn't related to the original topic, at which case my post was off-topic. But if there is any chance its related information, then I'll post it here.

No problem, for me it is related. I also experience "random" behavior and such problems you described in your previous post. So there is something to dig in.

[Yuraeitha]

It's good to know I'm not alone with this issue, although at the same time I wish it was only my machine so no one else had to experience this stressing issue. I feel like its frankly the most annoying Qubes 4 bug to date, nothing else I experienced is as annoying as this one >.< It's like putting a big yummy carrot in front of you on a stick, it's allmoooost working, but nope, not getting it. Frustrating when that close after hours of trying to fix it, and it fails at the last step, especially, and indeed, given its random nature, haha

No disrespect meant to the developers, I'm aware this is a big job, a lot of fundamental code was changed between 3.2 and 4, and that Qubes 4 is still in testing phase as well. Obviously bugs like these are natural to happen, and I'm definitely not complaining (if anything, Qubes 4 is an amazing job done), it's just a natural to expect, but frustrating kind of bug to encounter.

[Yuraeitha]

On Tuesday, November 7, 2017 at 7:27:48 PM UTC, Marek Marczykowski-Górecki wrote:

Marek, I possibly found a way to reproduce some of the issues here regarding the Qubes tools issues in the templates/AppVM's that is causing haywire on some Qubes 4 systems out there (and a way to solve it too). It's a user mistake, and quite frankly, quite an embarrassing one at that. Nonetheless, perhaps it might be a good idea to help make it harder for other users to repeat the same mistake, if feasible or desired of course.

It's triggered by restoring Qubes 3.2. templates in Qubes 4 (face-palm moment, I know), which without knowing the code I'm guessing is different between the Qubes 3.2. and Qubes 4 releases. I did not think much of it at the time, and it was easier to just restore everything, like go take a break or sleep while it works, instead of typing -x AppVM -x Template, etc. exceptions into the command line. I mean, if the computing time is irrelevant and you got plenty of disk space, it may be tempting to just restore it all while you go and do something else. Perhaps it'd be good to have something that stops the restoration of Qubes 3.2. templates, or a message that warns people in place before it goes out to people that might make same user mistake.

Somewhere down the line, one of the old Fedora templates from my 3.2. system was not deleted after restoration, and same user mistake happened on more than one machine.

After I re-linked all my AppVM's over to Qubes 4 fedora templates, it has for now worked pretty smoothly for several hours now.
I'm not sure if having to delete the old template was part of the solution, or if it made any difference, but I deleted it right away before testing that, perhaps I should have, but you probably know if it matters or not.

Creating shortcut issues in the Xfce4 menu is still for some AppVM's broken, but I'm suspecting creating a new AppVM and manually transfer userdata over might be able to fix that (Will try when I have the time). Seemingly old backup AppVM's are the ones with broken shortcuts? I haven't had time to test this, but it appears like that.

Also, Chris Laprise over at https://groups.google.com/forum/#!topic/qubes-users/Dxe8-0vaIuk found an easy way to solve the re-install of debian-8 issue. But this did not solve the qvm-run or qvm-open-in-vm issues for debian, at least not for my system, and possibly maybe others out there.

In short.
- Deleting 3.2. fedora templates and re-linking to Qubes 4 fedora templates only, appears to fix the qvm-run / qvm-open-in-vm issues in fedora templates.
- Re-installing Debian works with provided solution in the link from Chris Laprise posts over at the link above, but qvm-run or qvm-open-in-vm are still broken in debian.
- Some systems have a working debian template, while other systems do not. Is this hardware specific issue perhaps? Random? or different setting during Qubes 4 install maybe?

[Yuraeitha]

On Tuesday, November 7, 2017 at 7:27:48 PM UTC, Marek Marczykowski-Górecki wrote:

Update:
The new released dom0 updates seems to have done the trick, debian-8 now works flawlessly. Much appreciated!
No discovered delay or issues either for qvm-run or qvm-open-in-vm in debian.8.

Debian-9 works too, but it needs to sit idle for a minute or two when started up, before qvm-run works. Also pressing the qvm-run impatiently or too early, seems like it can bug out and then nothing starts at all. Which means the VM has to be shutdown and re-started before trying again. So in regard of debian-9, start it up, be patient for a minute or two, before starting anything. The exact same applies to any App-VM running on the debian.9 template. Debian-8 works normal and is quick. If others are in the same situation, II'd stick with debian-8 for now.

Also for anyone trying to get debian-9 running with this delay, then don't trust the circular status thingy in the Qubes widget in regard to whether debian-9 is ready or not. The extra minute or two, is after it's done starting according to the widget, and not from the point you first started it. The amount of time may also vary from machine to machine?


What was done on my side:
- I rebooted all of Qubes after the new dom0 update, (just to be sure everything was in proper place).
- Unlink any debian-8 templates.
- sudo dnf remove qubes-template-debian-8
- sudo qubes-dom0-update qubes-template-debian-8
- Profit.



Further details:
I'm not 100% sure it was the updates, but given it did not work before the updates, and worked afterwards, with no other chances, I strongly assume its the updates that fixed it.

Also, here are some more complicated details, that may or may not matter.
At the time of the update, I did not have debian-8 installed, which I had removed the day before during my own attempts to fix it. I only had debian-9 installed when the updates came out. After running the dom0 updates, and rebooting, debian-9 still did not work. So I went ahead and installed the debian-8, which to my pleasant surprise, unlike the many previous attempts, now worked right out of the box after the install, and seemingly for now flawlessly at that.
However, debian-9 did still not work, so I re-installed debian-9. When I started debian-9, it did still not work, but I stayed with it for a bit, trying to see if I could get it working. I found out that debian-9 has a time-lag, or delay, before it works after starting up. And if spamming the qvm-run, it'll stop working entirely. debian-9 must run idle after it has booted up, for a minute or two. But once there, it works flawlessly too (it seems, did not try debian-9 much).

Now I do not know if these templates really required a re-install to fix them, after all, I had debian-9 at the time of the updates. As you must see by now, I did not test whether debian-9 had a time delay before trying to re-install it, and debian-8 was not installed.

I tried to update the debian-9 template too, at this current time, it did not fix the delay. I presume it's the Qubes tools in debian-9 that needs updating, but debian-8 is fine for now.

So for anyone else out there, if the update did not fix it, try re-install the template, and try stick for debian-8 until debian.9 is working. Presumably, debian-9 may also work for some, and not for others? Is worth considering.

Again, thanks a lot for the update. With debian-8 now working, and all user mistake caused old fedora templates removed, and all old AppVM's that was broken replaced with new Qubes 4 AppVM's, and all files transfered, everything seems to be working correctly. Including the Xfce4 shortcuts, albeit I haven't tested whether the shortcuts work extensively yet on different AppVM's.