[HOWTO] use 2nd drive partition as 'home' drive.

[Tom Zander]

Lots of things changed in Qubes4, and I think I am finding out things lots of
others will need to find out at one point too.
So for them, as well as for my own memory, I'll write some howto emails.


The task;

as I run Qubes on a machine with a relatively small SSD and large spinning
disk, I want to make my homedir (/home/user) be completely stored on the
spinning disk.
I have two main Qubes which require storage. A Private and a Work qube.
Each gets one partition on my 2TB drive.
I assume you already partitioned and did everything you need with the drive,
it should be available to dom0.

1) Create and start a Qube "Work".
2) open a terminal in the Work qube.
3) do an ls /dev/xv*

4) Start a terminal for dom0;
5) run in dom0 in a terminal;
a) qvm-block
this shows a listing of drives with their names. Mine is; "dom0:sdb1"
b) qvm-block a --persistent -- Work dom0:sdb1

The 'persistent' part here is a new 4.0 feature, seems undocumented but it
means you only ever have to do the add once. Futher reboots and restarts of
the Qube will automatically re-attach the drive.

6) in the terminal for Work, rerun the ls from step 3 and check which device
was added. Possibly "xvdi"
7) edit (as root) the file /rw/config/rc.local and add this line;
mount /dev/xvdi /rw/home/user/
Using the device you found in 6 instead of xvdi should it be different.

8) make the /rw/config/rc.local file executable.
You can do this by running;
sudo chmod 755 /rw/config/rc.local

9) Now shutdown and restart the Work qube and start a new terminal
10) (optionally) in the terminal type;
chown user.user /home/user

All done!

known issue; it looks like the rc.local isn't always finished executing when
the first app is started. This looks like a bug to me.
So if your first app is firefox, for instance, you won't get your personal
settings (plugins/bookmarks) until you start it the second time :-(
My suggestion; make this qube autostart on login.

[Connor Page]

I hope you do understand that there is no encryption in what you propose.

[Tom Zander]

On Monday, 11 December 2017 15:10:17 GMT Connor Page wrote:
> I hope you do understand that there is no encryption in what you propose.

Thats why I wrote;

> I assume you already partitioned and did everything you need with the
> drive, it should be available to dom0.

I cowerdly leave the full-disk encryption details to be done by people before
they start the howto :-)

[Chris Laprise]

On 12/11/2017 04:12 AM, 'Tom Zander' via qubes-users wrote:
> Lots of things changed in Qubes4, and I think I am finding out things lots of
> others will need to find out at one point too.
> So for them, as well as for my own memory, I'll write some howto emails.
>
>
> The task;
>
> as I run Qubes on a machine with a relatively small SSD and large spinning
> disk, I want to make my homedir (/home/user) be completely stored on the
> spinning disk.

I think the intended way to do this is still to use qubes storage pools
as before, but its changed in R4.0. You would create a new lvm thin pool
(perhaps inside a new volume group) then use qvm-pool to add it to
Qubes. Then you can specify it when using qvm-create etc.

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Connor Page]

I agree with Chris. Data specific to a qube should be stored on one of that qube’s volume. Backups work then.

so in short, first create a qubes storage pool
qvm-pool --add <poolname> <driver> <options>

qvm-create -P <poolname> <all the usual options>

if you go for a thin pool, create it first and use volume group and thin pool names as options for qvm-pool.

P.S. I’m not sure lvm backend operates properly. File-based backend can also be used instead. Just mount the secondary drive in dom0 and use the old trusty file driver if worried.

[Tom Zander]

On Tuesday, 12 December 2017 16:18:25 CET Connor Page wrote:
> so in short, first create a qubes storage pool
> qvm-pool --add <poolname> <driver> <options>

In the spirit of a “howto”, can you fill in the actual values to allow one to
add a second drive as the ‘private’ (home) partition *only* of a Qube?

> if you go for a thin pool, create it first and use volume group and thin
> pool names as options for qvm-pool.

As the storage pools doc is missing readability, I have to say I have no
clue what a “thin pool” is.
What a “volume group” is.

Last, how does one create a btrfs filesystem on their “home” drive when using
this pool concept?

> P.S. I’m not sure lvm backend operates properly. File-based backend can
> also be used instead. Just mount the secondary drive in dom0 and use the
> old trusty file driver if worried.

Using a file is going to cause lots of fragmentation and adds an unneeded
layer that will just be able to introduce issues.
What is the benefit of using pools?

Doing a backup of a 1TB homedir can be done without the backup tool too ;)
--
Tom Zander
Blog: https://zander.github.io
Vlog: https://vimeo.com/channels/tomscryptochannel

[Connor Page]

sudo lvcreate -L <size> --type thin-pool --thinpool <thin_pool_name> <vg_name>
qvm-pool --add <qubes-pool-name> lvm_thin -o volume_group=<vg_name>,thin_pool=<thin_pool_name>

qvm-create -P <qubes-pool-name> .......
or
qvm-clone -P <qubes-pool-name> <old> <new>
set desired private image size using standard tools.

this will put the private volume in the new thin pool. private volumes are mounted in /rw

btrfs may be tricky but you can try in-place conversion in a test vm. don’t forget to check fstab. I don’t remember if the fs type is hardcoded there.

[Chris Laprise]

On 12/12/2017 10:39 AM, 'Tom Zander' via qubes-users wrote:
> On Tuesday, 12 December 2017 16:18:25 CET Connor Page wrote:
>> so in short, first create a qubes storage pool
>> qvm-pool --add <poolname> <driver> <options>
> In the spirit of a “howto”, can you fill in the actual values to allow one to
> add a second drive as the ‘private’ (home) partition *only* of a Qube?

>
>> if you go for a thin pool, create it first and use volume group and thin
>> pool names as options for qvm-pool.
> As the storage pools doc is missing readability, I have to say I have no
> clue what a “thin pool” is.
> What a “volume group” is.

Adding LVM objects can get convoluted. A thin pool is like a collection
of disk partitions that can easily grow and shrink as data is
added/removed; its a bit like a filesystem. The (supposed) advantage
over using an actual filesystem to hold disk images is speed; for
complex volume setups typical on servers, its also more flexible.

Overview and howto for LVM:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lvm_definition

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/lv_overview#thinprovisioned_volumes


But if you use the file driver instead of LVM you won't need the above
and setup will be simplified (see last).

>
> Last, how does one create a btrfs filesystem on their “home” drive when using
> this pool concept?

If you want btrfs for the private volume, you can probably just
'qvm-block attach' the volume (see output of 'qvm-volume') to a
different, trusted appVM and do 'mkfs.btrfs /dev/xvdi'. Then detach it.
When you start the target VM it should auto populate the new fs with a
/home dir, etc. Alternately you could try btrfs-convert on it.

Keep in mind that Btrfs is like a simple-to-use LVM system itself, so
using it on top of LVM or another filesystem can be slow. The other way
around, normal VM fs/disk images on Btrfs is what I successfully used
for over a year on 3.2 until I upgraded; it was flexible but demanded
more CPU cycles and metadata use could get tricky.

>
>> P.S. I’m not sure lvm backend operates properly. File-based backend can
>> also be used instead. Just mount the secondary drive in dom0 and use the
>> old trusty file driver if worried.
> Using a file is going to cause lots of fragmentation and adds an unneeded
> layer that will just be able to introduce issues.
> What is the benefit of using pools?
>
> Doing a backup of a 1TB homedir can be done without the backup tool too ;)

An easier option is to use the file driver/backend as Connor suggests,
even if it may be a little slower. Then preparation is reduced to
encryption + formatting. Assuming your extra drive is /dev/sdb with one
partition...

$ sudo cryptsetup luksFormat /dev/sdb1
$ sudo cryptsetup luksOpen /dev/sdb1 newdisk-luks
$ sudo mkfs.ext4 /dev/mapper/newdisk-luks
$ sudo mkdir /mnt/newdisk
$ sudo mount /dev/mapper/newdisk-luks /mnt/newdisk
$ sudo chown root:qubes /mnt/newdisk
$ qvm-pool -a newpool file -o 1,/mnt/newdisk

Create a vm using newpool:
$ qvm-create -p VOLUME_NAME=newpool  -t debian-9 newvmname

Getting this volume back online after reboot:
$ sudo cryptsetup luksOpen /dev/sdb1 newdisk-luks
$ sudo mount /dev/mapper/newdisk-luks /mnt/newdisk

To have it come online automatically, add entries in /etc/crypttab and
/etc/fstab respectively.

[Connor Page]

I’ll disagree with comparison of btrfs to lvm. there is a very significant difference between btrfs and lvm. btrfs is like a namespace and lvm volumes are block devices. one can put a namespace on a block device. but yes, layers and layers of metadata processing required.

BTW, has anyone started a btrfs driver for storage pools? I think it could very tricky if at all possible.

[Tom Zander]

related;
https://github.com/QubesOS/qubes-issues/issues/3334