Btrfs best practices

[Jorge P. de Morais Neto]

Hi! I own a Dell Inspiron 5570 laptop with 1 TB SATA HDD, a new 250 GB
NVMe SSD¹ and 16 GiB RAM. I seek reliability, durability, performance
and power efficiency.

I do weekly duplicity backups to external 1.5 TB USB3 HDD. I'll start
also daily rsyncing some of the SSD data to the SATA HDD.

The SSD will have 50 GB extra over provisioning and a 200 GB partition,
besides the special UEFI partition. The SATA HDD will start with 16 GiB
swap partition then a big partition. I'll put system and /home on the
SSD but all XDG user dirs² on the HDD, and tmpfs on /tmp. All three
drives will have Btrfs with space_cache=v2, noatime, zstd compression
and reasonable free breathing space.

I use Gnome and:
- GNU Emacs
- notmuch and offlineimap (I may switch to mbsync)
- GNU IceCat, Mozilla Firefox and ungoogled-chromium
- Gajim and GNU Jami
- Gnome Boxes or Virtual Machine Manager running a VM with 2 GiB RAM and
one .qcow2 disk image currently weighting 24 GB.
- mpv
- Nextcloud (always running but rarely syncing changes)

I use Debian stable with only official repositories, including
bullseye-backports. I manually installed GNU Guix package manager and
have 163 packages on main Guix profile.

* Doubts
** Backported kernel
Should I use a backported kernel as Btrfs [wiki][] recommends? I worry
that bullseye-backports comes from Debian testing with poor security.

[wiki]: https://btrfs.wiki.kernel.org/index.php/Getting_started#Before_you_start

** Strong compression during install
For lifetime and space saving, I intend to install Debian to the SSD
with compress-force=zstd:12, but then adopt compress-force=zstd. Thus
the installation will be slow---I'll do something else while the
installer works---but the installed system will be efficient, right?

** HDD Compression
Both HDD have a lot of already-compressed data: videos, audio, photos
and compressed archives and disk images; compress-force would force
Btrfs to recompress it all, only to discard the recompressed data and
store the original. Therefore compress=zstd:4 would be better, right?

** Fragmentation
Is fragmentation a concern? Is the [Gotchas][] article accurate?

[Gotchas]: https://btrfs.wiki.kernel.org/index.php/Gotchas#Fragmentation

** Subvolumes
What about
https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Additional_subvolumes ?

** Swappiness
Most performance-critical data will be on the SSD, so there will be much
less need for RAM caches; therefore I should decrease swappiness
(especially if I put swap on the SSD), right? By how much?

* Footnotes
¹ A 250 GB WD Blue SN550 rated for 150 TBW.
² See the xdg-user-dir manpage.

Kindest regards,
Jorge

--
- Many people hate injustice but few check the facts; this causes more
injustice. Ask me about <https://stallmansupport.org>
- I am Brazilian. I hope my English is correct and I welcome feedback.
- https://www.defectivebydesign.org
- https://www.gnu.org

[Eduardo M KALINOWSKI]

On 16/12/2021 14:13, Jorge P. de Morais Neto wrote:
> I'll put system and /home on the

> SSD but all XDG user dirs² on the HDD [snip]

I don't have that manpage installed, but if you're refering to
~/.config, ~/.local, etc, these are exactly the kinds of things that
should be on the SSD - it'll help with application startup times as
files in those directories are read when applications start.

Put large files (photos, music, video, etc) on the HD. For everything
else, prefer the SSD.

As for the other questions, I cannot help specifically, but as a general
advice, don't overthink it and don't bother trying to optimize
everything up to the smallest details. Unless you have some very
specific use, default settings are good enough, you should'nt notice any
different in day to day use.

--
The best way to get rid of worries is to let them die of neglect.

Eduardo M KALINOWSKI
edu...@kalinowski.com.br

[Jorge P. de Morais Neto]

Hi.

Em [2021-12-16 qui 14:55:23-0300], Eduardo M KALINOWSKI escreveu:

> On 16/12/2021 14:13, Jorge P. de Morais Neto wrote:
>> I'll put system and /home on the SSD but all XDG user dirs² on the
>> HDD [snip]
>
> I don't have that manpage installed, but if you're refering to
> ~/.config, ~/.local, etc, these are exactly the kinds of things that
> should be on the SSD - it'll help with application startup times as
> files in those directories are read when applications start.

No, I was referring to:

DESKTOP
DOWNLOAD
TEMPLATES
PUBLICSHARE
DOCUMENTS
MUSIC
PICTURES
VIDEOS

> As for the other questions, I cannot help specifically, but as a
> general advice, don't overthink it and don't bother trying to optimize
> everything up to the smallest details. Unless you have some very
> specific use, default settings are good enough, you should'nt notice
> any different in day to day use.

Thank you for the advice! I will think about it. I must not return to
the days when I riced Gentoo for hours every week trying to gain a tiny
bit of extra performance. There must be moderation.

Regards

--
- Many people hate injustice but few check the facts; this causes more
injustice. Ask me about <https://stallmansupport.org>

- Please adopt free/libre formats like PDF, Org, LaTeX, ODF, Opus, WebM and 7z.
- Libre apps for AOSP (Replicant, LineageOS, etc.) and Android: F-Droid
- https://www.gnu.org/philosophy/free-sw.html "What is free software?"

[Jorge P. de Morais Neto]

Hi. I must add the information that I use zswap:

GRUB_CMDLINE_LINUX_DEFAULT="quiet zswap.enabled=1 zswap.zpool=z3fold zswap.compressor=lzo-rle"

- Free Software Supporter: https://www.fsf.org/free-software-supporter
- If an email of mine arrives at your spam box, please notify me.

[piorunz]

I'd like to add question to this topic.

Is it safe & recommended to run autodefrag mount option in fstab?
I am considering two machines here, normal desktop which has Btrfs as
/home, and server with VM and other databases also btrfs /home. Both
Btrfs RAID10 types. Both are heavily fragmented. I never defragmented
them, in fact.

Running manual defrag on server machine, like:
sudo btrfs filesystem defrag -v -t4G -r /home
takes ages and can cause 120 second timeout kernel error in dmesg due to
service timeouts. I prefer to autodefrag gradually, overtime, mount
option seems to be good for that.

My current fstab mounting:

noatime,space_cache=v2,compress-force=zstd:3 0 2

Will autodefrag break COW files? Like I copy paste a file and I save
space, but defrag with destroy this space saving?
Also, will autodefrag compress files automatically, as mount option
enforces (compress-force=zstd:3)?

Any suggestions welcome.

--
With kindest regards, Piotr.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀⠀⠀

[Andy Smith]

Hello,

On Wed, Jan 26, 2022 at 08:38:41PM +0000, piorunz wrote:
> My current fstab mounting:
>
> noatime,space_cache=v2,compress-force=zstd:3 0 2
>
> Will autodefrag break COW files? Like I copy paste a file and I save
> space, but defrag with destroy this space saving?

Yes, I believe so. That is, if you have files sharing extents (like
if you de-duplicated them or did a "cp --reflink=always") and defrag
touches those files, it will undo the extent sharing.

> Also, will autodefrag compress files automatically, as mount option
> enforces (compress-force=zstd:3)?

Yes.

> Any suggestions welcome.

If you're not relying on extent sharing then I'd think that
automated defrag is okay.

Cheers,
Andy

--
https://bitfolk.com/ -- No-nonsense VPS hosting

[Jorge P. de Morais Neto]

Hello! I think I should inform this list about my choices so far:

Em [2021-12-16 qui 14:13:05-0300], Jorge P. de Morais Neto escreveu:

> Should I use a backported kernel as Btrfs [wiki][] recommends? I worry
> that bullseye-backports comes from Debian testing with poor security.

I'm just using bullseye's kernel (5.10 LTS).

> For lifetime and space saving, I intend to install Debian to the SSD
> with compress-force=zstd:12, but then adopt compress-force=zstd. Thus
> the installation will be slow---I'll do something else while the
> installer works---but the installed system will be efficient, right?

I'm still using compress=zstd:12 and it's performing well. Notice I
went from "compress-force=zstd:12" to just "compress=zstd:12". That is
because of:

Using the forcing compression is not recommended, the heuristics are
supposed to decide that and compression algorithms internally detect
incompressible data too.[1]

Btrfs contains an internal heuristics that determines if some data
is compressible so that it doesn't try to compress data that isn't
compressible as this wastes CPU time. The compress-force mount
option bypasses this heuristics in order to gain better compression
ratios. A downside is that this increases fragmentation with
non-compressible files.[2]

1: https://btrfs.readthedocs.io/en/latest/Compression.html "Compression —
BTRFS documentation"
2: https://wiki.tnonline.net/w/Btrfs/Compression#The_compress-force_mount_option
"Btrfs/Compression - Forza's ramblings"

In the near future I intend to reduce this strong compression level (12)
to something more usual, in order to reduce power usage.

> Is fragmentation a concern? Is the [Gotchas][] article accurate?

I now have little to worry about fragmentation, because:

1. I dedicated a raw partition to my qemu-KVM virtual machine, bypassing
Btrfs.
2. I moved the caches of ungoogled-chromium, GNU IceCat, Firefox,
Evolution and GNU Guix to the HDD, because they (especially the web
browser caches) were writing too much temporary data to the SSD.
Thus, if they ever become too fragmented, I can now just defrag them,
without the danger of wearing the SSD.
3. I made a script to find heavily fragmented files (using compsize's
output) and so far I have nothing to worry about.

> ** Subvolumes

I read https://en.opensuse.org/SDB:BTRFS and laid out subvolumes
according to this fstab excerpt:

LABEL=SSD / btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@rootfs 0 0
LABEL=SSD /var/cache btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@var-cache 0 0
LABEL=SSD /var/backups btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@var-backups 0 0
LABEL=SSD /var/mail btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@var-mail 0 0
LABEL=SSD /var/spool btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@var-spool 0 0
LABEL=SSD /var/tmp btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@var-tmp 0 0
LABEL=HDD /var/log btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@var-log 0 0
LABEL=SSD /var/lib/libvirt/images btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@libvirt-images 0 0
LABEL=HDD /var/cache/apt/archives btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@apt-archives 0 0
LABEL=HDD /root btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@~root 0 0
LABEL=SSD /home btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@home 0 0
LABEL=SSD /home/cache btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@home-cache 0 0
LABEL=HDD /home-HDD btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@home 0 0
LABEL=HDD /home-HDD/cache btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@home-cache 0 0
LABEL=SSD /gnu btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@guix-store 0 0
LABEL=SSD /var/guix btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@guix-var 0 0
LABEL=SSD /usr/local btrfs noatime,space_cache=v2,compress=zstd:12,subvol=@usr-local 0 0

Rationale:
1. In the future I could snapshot @rootfs before certain system
operations (say, large upgrades). If I then rollback the system to a
snapshot, I'll still want the latest logs, user data, cache,
libvirt-images etc., so these should be outside the @rootfs
subvolume. Also, including them in snapshots would be very expensive
because some of these directories have too much variable data.
2. If I snapshot @home (probably for backup) I don't want to snapshot
user cache (see above).
3. Some user data should be on the HDD, such as videos, music, pictures,
downloads etc. They are large files that would fill the SSD; and
their usage characteristics don't require SSD performance.
4. Also some of the user cache should be on the HDD (such as web browser
caches), to avoid wearing the SSD.

> ** Swappiness

I have not messed with swappiness.

Speaking of swap, I dedicated a 16 GiB swap partition at the start of
the HDD.

All is working well, except for some error messages I get when shutting
down. See my email from Thu, 20 Jan 2022 08:57:35 -0300 with subject
'"Failed unmounting "/{root,var/cache} error messages when shutting down'
(Message-ID <87zgnq6...@disroot.org>).

Kind regards,