Storage pools

[Chris Laprise]

Does anyone have a primer on Qubes storage pools in 3.2? I'm talking
about the facility controlled by /etc/qubes/storage.conf.

The capability looks intriguing but there isn't anything that explains
it as a real feature of Qubes 3.2.

Chris

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I'd ask Kalkin (cc-ed). This feature is really used only in Qubes 4.0,
the one in 3.2 is some preliminary version of it.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYSYPsAAoJENuP0xzK19csXfkH/iU2yIdH/x00hpuyGIhMY+rt
HWFbvHyyN8gaOEfHmwCVAjuACz8sPRbbXZN6GDMgZonwENuXNaRs/RVopuzKVP5C
7IWKgOXRcMx1bpiSx60xwOkVkj1wHsYY00F3QFl0hlWDqgExdyRwG1jPERW6Irg8
TEJHuaT0iT4vo/lGFZr6SSrt6v8nfOr8VDbxSnHPwXdVkI+4WsvioXKi75hKNvhs
t8zJa3Grkik42WQoJkfo0HYvxASgULPAa2v4HDvIEgC74+OY+++uRAbhKWEitHvT
oXFngAMIi5SyWsa7j9uNL5WGPwdtVT8qNZxP02lv9Xyesoy9AZuNmoHlwvEHdZc=
=1Otq
-----END PGP SIGNATURE-----

[Bahtiar `kalkin-` Gadimov]

Hi,

On 12/08/2016 04:07 AM, Chris Laprise wrote:
> Does anyone have a primer on Qubes storage pools in 3.2? I'm talking
> about the facility controlled by /etc/qubes/storage.conf.

Qubes OS R 3.2 introduced the concept of storage drivers & pools.
This feature was a first step towards a saner storage api, which is
heavily rewritten in R4. A storage driver provides a way to store vm
images in a QubesOS system. Currently the default driver is `xen` which
is the default way of storing volume images as files in a directory tree
like `/var/lib/qubes/`.

A pool storage driver can be identified either by the driver name with
the `driver` key or by the class name like this
`class=qubes.storage.xen.XenStorage`. Because R3.2 doesn't use python
setup_hooks, to actually use a short driver name for a custom storage
driver, you have to patch qubes-core-admin. You can use the `class`
config key instead, when your class is accessible by `import` in python.

A pool (in R3.2) is a configuration information which can be referenced
when creating a new vm. Each pool is saved in `storage.conf`, it has a
name, a storage driver and some driver specific configuration attached.

When installed, the system has the—as you can see from the content of
`/etc/qubes/storage.conf`—a pool named `default`. It uses the driver
`xen`. The default pool is special in R3.2 it will add
`dir_path=/var/lib/qubes` configuration value from
`defaults[pool_config]`, if not over written.


Currently the only supported driver out of the box is `xen`. The benefit
of pools (besides that you can write an own storage driver i.e. for
btrfs) in R3.2 is that you can store your domains in multiple places.

You can add a pool like this to `storage.conf`:

```
[foo]
driver=xen
dir_path=/opt/qubes-vm
```

Now when creating a new vm on the command line a you may pass `-Pfoo`
argument to qvm-create to have the vm images stored in pool `foo`. See
also `qvm-create --help`

While the current API is not as clean and beautiful as the R4 API, it
allows you to write your own storage drivers i.e. for btrfs today.

If you have any further questions, just ask!

kalkin-

PS: There is also a patch set providing a `lvm-thin` driver maintained
by me for R3.2
https://github.com/kalkin/qubes-core-admin/tree/lvm3/core/storage

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2016-12-11 22:07, Bahtiar `kalkin-` Gadimov wrote:
> Hi,
>
> On 12/08/2016 04:07 AM, Chris Laprise wrote:
>> Does anyone have a primer on Qubes storage pools in 3.2? I'm
>> talking about the facility controlled by
>> /etc/qubes/storage.conf.
>
> Qubes OS R 3.2 introduced the concept of storage drivers & pools.

> [...]

Thanks, kalkin-!

Added to the docs: https://www.qubes-os.org/doc/storage-pools/

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYTlNNAAoJENtN07w5UDAwKJIP/iZYK8+gnGnqxp0Ft6EpxtRY
3SiHuk9Txe3lCevx5jnsFUy6Y8VuDdYFaMNPRGHAdtSLFQh8VBvyaZKxtADG28Iy
A0r4w79jZWeacHCaJeQ4YfmOR+FmoO0MIwl/U7g3r3nmdgdWINTIT7ilST13OXTQ
PqnEBS1XHroXRmTtjS9THtk2XT6iJmlq6MVlQQXhar70tTUY0DHBZnCOnAPKdJOY
6k0IVfsVJraQWivYD0y5nonI0E38WulEkyMMdnimLPwuzvCRX5hocpHjvYiI1Wqs
atazNf9hUCD+fvCW4L4v3pEJxO/jBwOMSxM00zhm5VSGw7e15ppIel496Xo/tDve
+Ot0Lo3iQPAOLk5lWDm8/VoiF3MduWxF46WEQK4mf/CYbp+df1wyvJD/sUr0o93B
W5YZQThIehwpcdB+jeSusnMcnTRCXdeze/tdatsUfFZIAkmVhqFCtfmpEVK5YDP4
MSzB9ZnxDrsFHAAu/OWytRyXe0Cn7a55IjRssPc+XHLeJQzZ4HyaDL9GDnTiRkA2
6gzN/024daBRApPcn+ahIMlZnFi6/1FUMRVal4m78n6U0houmbobxRQlRIa8GyEv
IOHlWi9dAjNhc5Ycc4ZGV8QWEexJs973tPT0IKcpcgM13D1eSl1vcjs3GiMDn8k/
I+ZmYDQrtyW1VEpk8Y0B
=34Dk
-----END PGP SIGNATURE-----

[Chris Laprise]

On 12/12/2016 01:07 AM, Bahtiar `kalkin-` Gadimov wrote:
> Currently the only supported driver out of the box is `xen`. The
> benefit of pools (besides that you can write an own storage driver
> i.e. for btrfs) in R3.2 is that you can store your domains in multiple
> places.
>

Thanks! That aspect--using multiple places--is what the term "pools"
suggested to me, so I'm glad it can be used that way. It seems like
users could use different pools to enable different backup procedures
(based on btrfs subvolumes and btrfs-send, for example).

The only question for me is how existing VMs could be moved to other
pools. (Thinking it would be a nice feature if Qubes would automatically
recognize any VM dir placed within a known pool.)

Chris