Addressing the long shutdown time with LVM thin volumes

[Jinoh Kang]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

TL;DR: I'm growingly getting tired of #5426 [1].

Every time I shut down a domain, I have to endure _seconds_ of freezing
in every other VMs of the same LVM Thin pool just because I have discard
passdown enabled. A quickly wearing SSD is the last thing I want,
however.

Other than performance, this can also be a security issue. Notice how
any VM can reliably tell whenever other VMs in the same pool are being
destroyed.

The thin pool driver (dm-thin) is to blame.

Commit 34fbcf6257eb ("dm thin: range discard support") [2] sort of
remedied this, yet the core persistent-data framework still manages to
insist in deleting ranges only _one_ node at a time. Rebalancing the
large btree a thousand times isn't exactly one of the things you'd
expect to be done in no time.

Possible solutions would be:

- - Patch dm-thin. Now to implement this properly, dm-thin must first
slice the range, have it ref'd somewhere and then issue discards (so
that it can handle power disruptions).
- - Switch to reflinks. BTRFS has been around 10+ years, so I assume it's
stable enough?

[1] https://github.com/QubesOS/qubes-issues/issues/5426
[2] https://git.kernel.org/torvalds/c/34fbcf6257eb3f39a5b78a4f51b40f881b82033b

- --
Sincerely,
Jinoh Kang
-----BEGIN PGP SIGNATURE-----

iQJMBAEBCAA2FiEEzGktrvc/U3kFXd9AGlqQRGyEq/UFAmAdcUIYHGppbm9oLmth
bmcua3JAZ21haWwuY29tAAoJEBpakERshKv1OpcQAMbSMdezVlgnNEwylDMWWnoV
IG7mAX7EMfLVEFmdTK7BAJh4xPf+6kW+YYttimvXD5gUGFylctOoYgGmoCWResfK
bPfY70Psu/GOEhsGOGaJXD+nV512MMbiCA2B2AsOn6ukW/HLDs9A3VAWGSOfJWJK
axHQ1O90CTvX+j/nmBrXYvzGARTXblqLMIsVuPIB7Dx8NgEXVNBR0PDUmCXEi+D/
YJU4qiF7HbUOh633IKxx8Yv40VzI3SLgVQtydQX3YwzRkExinGnx8yCVIMy5XywJ
Kr9+TnbjaOJhEqZT248UUA88PYTqNaIO/j0Xb5YTL+zpYJVSCaoMeFMGaSRs1rAN
LrPkVfB73ZqqIVfbie3Bio8nMRdU9baaBF57SHgqUhvc+hXN9Ifr8ygJTMTC1osj
ddM5vz/Qa8/+EkyuBBojG5FVkWUna166Vcw2xScTczRsfHdLG6Cn4lesLMQzW2PI
wxAsFs6AEJVtvkfU8dMSGHCOi53C9M8YCIvICFHsDOJVvLM6/clCVJj6ghsUL/Gb
yMh730L7zwNGlP3HqfxwSZuhlbfrGs+A/QF3fjWBRpFVOqcMT7YkzIUZNr4f2kph
FRXfJuFdwsW1pKTIrevGbX57JBP07M2hx1jWGRdZ2nIRra7/sks6cxVCA1etYsBX
P2gxMgcOCdMuRQ/zA3Oz
=l94N
-----END PGP SIGNATURE-----

[donoban]

On 2/5/21 5:24 PM, Jinoh Kang wrote:
> Possible solutions would be:
>
> - Patch dm-thin. Now to implement this properly, dm-thin must first
> slice the range, have it ref'd somewhere and then issue discards (so
> that it can handle power disruptions).
> - Switch to reflinks. BTRFS has been around 10+ years, so I assume it's
> stable enough?
>

I've been using BTRFS for a few years now and haven't had any problems
except some CPU bottleneck using send/receive, but probably you are not
interested in that functionality.

[Peter Todd]

I'd like to see btrfs added for its checksums as well. I've personally had
`btrfs scrub` catch hardware problems before more data got silently corrupted.

--
https://petertodd.org 'peter'[:-1]@petertodd.org

[donoban]

On 2/11/21 6:06 AM, Peter Todd wrote:
> I'd like to see btrfs added for its checksums as well. I've personally had
> `btrfs scrub` catch hardware problems before more data got silently corrupted.

It isn't default but is official supported [1]. The only problem is that
it could be a little tricky in the anaconda partitioning window. It is
really simple, like setting auto scheme and then select btrfs, but I
remember doing something pretty different and ending with a unbootable
system.

https://github.com/QubesOS/qubes-issues/issues/2340

[Rusty Bird]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Peter Todd:

It's not the default, but Qubes already supports this since R4.0.1:

If the Btrfs layout is selected (instead of LVM Thin) during Qubes
installation, a 'file-reflink' storage driver pool will be created
automatically.

Rusty
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEhLWbz8YrEp/hsG0ERp149HqvKt8FAmAlDtpfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDg0
QjU5QkNGQzYyQjEyOUZFMUIwNkQwNDQ2OUQ3OEY0N0FBRjJBREYACgkQRp149Hqv
Kt/RhA//QoMuAJEGB4P7np2wbJ2YP2noR8puPPgAlxpZ7o9ZsgQVOMH4rn/O03fz
YJoVYxaOIxEttjRvZa2trjTcM0zWlxgnVKXkio11rMFACLkJJb5bICtUri77ynFM
O8aUI8VgY5erI8zivP0H9NlWi/I93cgd8TFEJjlgLgPlIs22NzhfCrmbIy5Pa/SL
ppW/mn+v6hzvsIII9fwj8Dv3meDVS/0UTjpeAWjizPaIX41DIZwtUbwfw/wxsHWm
CzIVQKC/jRsgFjc66WsGvK+L0Sg4zwtKjJBjexTRhsoYvejoGPn6vq8kXCXk8Qt6
QR841VKgyV+svqx0muZTaibTW/bIHI09mjAkJpuPerPGSaobBZ2dOmYIZRpkTHQb
WWnTrLfP3onWPQMrGmKiXBP/2o07i91AU6aOVjn6UfHcdVzISHdLUB4EaKVgoXpb
UU1VtON0oNEhvoRHO62lEdbbIkdA1abAYLYiHb5BfS3kdUcf3qJcKw1R7IxngWf0
qDoz7SCgwhDMwGweglH57mFvqUBFvMqIVmkWLZ9oL7iIivb+ke3zvH5HZR9x7w81
/O6fSW4b2NQeBPE0RjiqrQnV67PsLOF3QG7+asUl53u3zUnBIuNoD+PFCuiVvgnF
aKGnFZHsWxBAwVxpZ28tFK3D8BQpYKgChxsHN9E+MQSG8+XynI8=
=7V+3
-----END PGP SIGNATURE-----

[Peter Todd]

On Thu, Feb 11, 2021 at 11:02:51AM +0000, Rusty Bird wrote:
> Peter Todd:
> > On Fri, Feb 05, 2021 at 06:59:05PM +0100, donoban wrote:
> > > On 2/5/21 5:24 PM, Jinoh Kang wrote:
> > > > - Switch to reflinks. BTRFS has been around 10+ years, so I assume it's
> > > > stable enough?
> > >
> > > I've been using BTRFS for a few years now and haven't had any problems
> > > except some CPU bottleneck using send/receive, but probably you are not
> > > interested in that functionality.
> >
> > I'd like to see btrfs added for its checksums as well. I've personally had
> > `btrfs scrub` catch hardware problems before more data got silently corrupted.
>
> It's not the default, but Qubes already supports this since R4.0.1:
>
> If the Btrfs layout is selected (instead of LVM Thin) during Qubes
> installation, a 'file-reflink' storage driver pool will be created
> automatically.

I didn't realize this was still an option. Just tried it out on a new install,
and looks like it still works. Thanks!