Automatically mount block devices when appvm launches

[Dave C]

I'm setting up Qubes (R3 rc1) on a machine with two drives.  Qubes boots from an SSD.  I've divided the other (HD) into a few partitions.

I'd like to automatically mount one of these partitions when say the personal appvm starts up.

Currently, I need to run `qvm-block` in dom0, and then mount in the app vm.  This is a bit more complex than mounting a USB drive.  With USB, qubes presents a UI to associate the drive with an appvm.  With non-USB block devices I believe `qvm-block` is the only way.

I'm wondering if there is some way to make those two steps (`qvm-block` and `mount`) happen automatically.  Thanks!

[conp...@gmail.com]

You can create an alternative config file, add the device there (using phy driver) and then start it with qvm-start --custom-config=
Make sure the partition is not mounted in dom0 and as it is a partition you will have to think of a way to automount it in the vm. I guess the easiest place is in /rw/config/rc.local

[tom...@gmail.com]

Hi, here's my way:
- In storage VM I've created '/rw/config/fstab' with content that will be appended to /etc/fstab on each boot. It contains UUIDs of partitions with target mount point. Note the 'auto' attribute:
---
[user@storage config]$ cat /rw/config/fstab
# This should have been appended to /etc/fstab on boot

#Disk1
UUID=3f3564db-7df4-40af-d067-33ed2c049b65 /s/disk1 reiserfs auto,noatime,users,exec 0 2
UUID=a4e9a4fc-ef0a-962d-3cf2-a6fdfa35a00b /s/disk2 ext3 auto,noatime,users,exec 0 2
---

File /rw/config/rc.local creates mount points and adds UUIDs
#!/bin/bash
mkdir -m 770 -p /s/disk1
mkdir -m 770 -p /s/disk2

cat /rw/config/fstab >> /etc/fstab

Then in dom0 we have similar script:
#!/bin/bash

qvm-start storage

list=`qvm-block -l | grep -v attached`

awk '
BEGIN{
attach["HDID"]="dummy";
attach["ST320ABC"]="dummy";
}

{
if( ($1 ~ /sd.$/) && ($4 > 0) && attach[$2]){
print "Attaching",$1,$2;
system("qvm-block -a storage " $1);
}
}
' <<< "$list"

qvm-run storage 'sudo mount -a'
----
attach[] contains the names of white-listed devices to be attached to storage - this is 2nd column of table output by qvm-block command.
After devices are assigned to storageVM, auto-mount is executed in the VM.

I have similar script that unmounts and detaches devices from the storageVM.

Currently I have to find suitable place and time to run that script. Suggestions are welcomed.

Tomhet

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

If you want to call it automatically at VM startup, you can create
qrexec service for that, and call it from /rw/config/rc.local within VM:

1. In dom0:
/etc/qubes-rpc/my-block-attach - place your script here, make sure it
doesn't read anything from stdin, as it is controlled by (potentially
compromised) VM.

/etc/qubes-rpc/policy/my-block-attach - add a policy allowing that
particular VM to call it:

my-vm-name dom0 allow
$anyvm $anyvm deny

2. In VM, call the service from /rw/config/rc.local, before mounting
(move 'mount -a' to rc.local too):

qrexec-client-vm dom0 my-block-attach

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXVy6kAAoJENuP0xzK19cs+xsH/RoaAHUojOpv0WSjkcW7AKTF
1f/wUkwsS5Twr9cQW1jCX/fXMieEs/o6dRCOkPHVfQTnyLTV+QJHv6pJPJJ7Z3Gy
nlZ1aolu5iohwqGLZoaRO3NkbzFvM0yVyA872IrqpRGmb2w04nRUOh0mUYvy/Iri
9f86GPUA+oCEC7BoW9p5NaR174ZZzhV4p8K+z4l7+TbymM4halcyD10lANenGvMQ
snhsY2hyeFE8oqY30KxAfggdRusWA9GAfi+CaUyO3dhaMS+6iXHJI16b9T2l/3lk
/INFnhnEu2UZEoYdi4tLwSEWkwpNLDBc7iXMzcOKnFpZVNT15VSSarOu38SOJQk=
=qs6k
-----END PGP SIGNATURE-----

[tom...@gmail.com]

Thanks for details, Marek !

> If you want to call it automatically at VM startup, you can create qrexec service for that

Cool, this was my first idea, but I was unable to implement it.
Q1: If taking this way I'll need to have opposite (umount & detach) operation at VM shutdown (to avoid refreshing block devices).
Where should I attach (in appVm) the opposite shutdown script (injected by rc.local)?

Q2: If taking the opposite way (dom0-initiated process), how/where should attach/mount script be called from?
- systemd service that depends on some (which) qubes service?
- /etc/init.d/rc.local (not sure if necessary qubes stuff will be already available)
- something else

thanks,
Tomhet

P.S. An alternative could be event listeners for qubes events 'onVmStartComplete'/'beforeVmShutdown', if such exist & I knew python.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Jun 07, 2016 at 02:07:52PM -0700, tom...@gmail.com wrote:
> Thanks for details, Marek !
>
> > If you want to call it automatically at VM startup, you can create qrexec service for that
> Cool, this was my first idea, but I was unable to implement it.
> Q1: If taking this way I'll need to have opposite (umount & detach) operation at VM shutdown (to avoid refreshing block devices).

Umount should happen automatically by system scripts and when you
shutdown the VM, detach shouldn't matter (as long it's shutdown of the
VM _to_ which device is attached, not the one exposing it).

> Where should I attach (in appVm) the opposite shutdown script (injected by rc.local)?

If you really need to, I guess you should add a systemd service to
shutdown.target. Or a normal one and use ExecStop instead of (in
addition to?) ExecStart.

> Q2: If taking the opposite way (dom0-initiated process), how/where should attach/mount script be called from?
> - systemd service that depends on some (which) qubes service?
> - /etc/init.d/rc.local (not sure if necessary qubes stuff will be already available)
> - something else

Currently there is no easily accessible place in dom0 to plug VM
startup/shutdown scripts.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXV0LOAAoJENuP0xzK19cs7G0H/1iP8VKTWQaf802etp526IKV
7HQmf6qpBsAXwqUnhaMCOgrW3Dsg2O5Z15ihqsG7fxo2OdBPZFNsPOspfjNDRy72
3JfYbGIsrSutF2dAJGjo22bGnT3a6xKjAx+7EfOStxzQ7Yl1JJMwXqDhMIhgqQ3O
a2Q3f87eGE1b7oERGUj+/JQga/yaywPXmSrVoFK153sN5UhtrOpOuVv2ViNbn+hP
HAQZ3FE9BtkumV+IWCYmr3K7DsCGpQc0cc7MkK79y150MqDlmXyYk639AIdCbB2w
ebgXl1HuODNf9Ju/VnaQ2/vczQ3pg3r3YHmeMpRjpKIv3L9lCV7Cm8uJfISQAiI=
=SJpG
-----END PGP SIGNATURE-----

[tom...@gmail.com]

Hi Marek,

Mounting as you advised works like charm when doing it via Qubes VM Manager:
"qrexec-client-vm dom0 storage.block.attach" from rc.local of appvm.

But when setting 'storage' VM to auto-start I find such things in the log (and no storage is attached:
----
125626-Jun 12 18:27:11 dom0 qvm-start[2666]: Waiting for VM's qrexec agent......connected
125709-Jun 12 18:27:11 dom0 systemd[1]: Started Start Qubes VM storage.
125774:Jun 12 18:27:12 dom0 storage.block.attach-storage[3328]: *** Running this tool as root is strongly discouraged, this will lead you in permissions problems.
125930:Jun 12 18:27:12 dom0 storage.block.attach-storage[3328]: Retry as unprivileged user.
126015:Jun 12 18:27:12 dom0 storage.block.attach-storage[3328]: ... or use --force-root to continue anyway.
126116-Jun 12 18:27:13 dom0 qvm-start[2668]: Waiting for VM's qrexec agent......connected
126199-Jun 12 18:27:13 dom0 systemd[1]: Started Start Qubes VM sys-whonix.
----

Obviously this is the appvm->dom0 call.
Calling "qrexec-client-vm dom0 storage.block.attach" as "user" does not help.
I'm not sure which "this tool" is (called command) and how can I run it as non-root (or with --force option) during dom0 boot time.

Any ideas how to workaround it?

thanks,
Tom

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

When VM is autostarted, all services from it are started as root. This
is some obscure interaction of how VM is started, which will be improved
in Qubes 3.2. For now, you can add --force-root option to qvm-block
command in your script.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXXb91AAoJENuP0xzK19csqeIH/2eIn1SN45FSD4ImyBwwSndX
VkXbDOzCM/Yxi1a0wjZNqvxOsnleljTNkRb4nFru/l5NUWx6oYzXQ9cK8epHBPPk
s08aZaMY5N9KDRDebK6ucmZjYFHAKMhUMVHHUPT8dXWJyg8eUZTXW5T+5mN2wmfb
Ynjx1kpsq0ZcTroAwsqgeNZfY1cKRV44UOFqsTtXlvxllviVOfTRjkvv7kJqdkC1
Mgn19NcO/DbxZ2Rgid+38NZtZCcX1QDuYKcKavrArP8K63Ju88eEFZksMdPIncy1
LwSFlL0RhY/bAqLFmd5UeSPSIpd91j82lqkWOKE4CxP5mvPqH3qZzKNJ8wVPzfQ=
=p8Oq
-----END PGP SIGNATURE-----

[tom...@gmail.com]

Hi Marek,

In Qubes Manager's VM context menu 'Shutdown VM' seems to do equivalent of 'virsh shutdown VM', i.e. power button press. I'd like to catch it, so my shutdown script is executed also via Qubes Manager shutdown.

In /var/lib/qubes/vm-templates/storage/storage.conf I have " <on_poweroff>destroy</on_poweroff>".

This file is re-generated on vm start. Where is value for "on_poweroff" taken from (so I can set it to "shutdown")?
Seems /etc/sysconfig/libvirt-guests is not the right place.

regards,
tom

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Mon, Jun 27, 2016 at 03:47:14PM -0700, tom...@gmail.com wrote:
> Hi Marek,
>
> In Qubes Manager's VM context menu 'Shutdown VM' seems to do equivalent of 'virsh shutdown VM', i.e. power button press. I'd like to catch it, so my shutdown script is executed also via Qubes Manager shutdown.
>
> In /var/lib/qubes/vm-templates/storage/storage.conf I have " <on_poweroff>destroy</on_poweroff>".
>
> This file is re-generated on vm start. Where is value for "on_poweroff" taken from (so I can set it to "shutdown")?

First of all, this is not the setting you're looking for. This one is
about what should happen after shutting down the VM system[1].

But it should be possible to trigger qrexec service at VM shutdown very
similar to the startup one. There is no rc.local equivalent for
shutdown, but it's easy to add systemd unit for that. Something like
this:

[Unit]
Description=My shutdown actions
After=qubes-dvm.service qubes-mount-dirs.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/true
ExecStop=-/usr/bin/qrexec-client-vm dom0 ...

[Install]
WantedBy=multi-user.target

The easiest way would be to save it in template for example in
/etc/systemd/system/my-shutdown.service, then enable it with `systemctl
enable`. And setup appropriate qrexec service to allow service call only
from selected VM(s). This is why I've prefixed the command above with
"-" - to not fail the execution when service call is refused.

But you can also save it in /rw/config/ and move in place from rc.local.

[1] http://libvirt.org/formatdomain.html#elementsEvents

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXcbAGAAoJENuP0xzK19cstdIH/3A6/K4EDv8S7Qr/MVagc4Wo
DDKum1TzwZCNSu6oBDuWyBterALFygRurOMNyGChfWgpIS1nP95A/wgNs0DTmNu3
TsxsEv7IlF83KblD8sDPExGgEiv66b6X9u6PdCi/ZnOQTuzeDItgARFCeKP/ig/1
suXYtQzCLZ2cRcg8t2FBXkjhwWTBswgbAkUwHEN6cf5TTWwkO7IR6ok2JZlrVCu8
oksShQw8gR1ZBv85z0RHFkpVFPhsMzCtGQIH6GBs7ClS76kIy1+a4REV2I/hv5Li
7wAFpQuIsqZpnjLG3+oBP2CrYj2zwp7r9HyhFp5/JDSZ2d+dzvlVHJ9GYeyMNZI=
=GjMu
-----END PGP SIGNATURE-----

[tom...@gmail.com]

Man, this is amazing! It finally works!

I spend over 20 hours fighting with systemd (depending on shutdown target, all or some qubes targets, doing PreStart=- but nothing worked) and in desperation I tried nasty hack of replacing /usr/sbin/shutdown with my script, but it was not called when shutting down from qubes manager.

Here's final service:
---------
[Unit]
Description=Detach block devices
After=qubes-dvm.service qubes-mount-dirs.service qubes-qrexec-agent.service

[Service]
Type=oneshot
RemainAfterExit=true

ExecStart=/bin/true
ExecStop=-/usr/local/bin/detach-devices.sh

KillMode=none
TimeoutSec=65

[Install]
WantedBy=multi-user.target
-----------

In detach-devices.sh I put two 'sleep 5' commands around 'umount' to be sure future commands for stopping Samba / NFS will have time to complete and it works.

Great! Much thanks!