Bug#1010485: plocate PRUNE_BIND_MOUNTS incompatible with btrfs subvolumes

[Julian Andres Klode]

Package: plocate
Version: 1.1.15-1
Severity: normal
X-Debbugs-Cc: jul...@ubuntu.com

plocate considers all mount points of a btrfs as bind mounts,
so when I have a somewhat usual case of subvolumes, e.g.

/dev/mapper/ubuntu--vg-root / btrfs defaults,compress=zstd,subvol=@ 0 1
/dev/mapper/ubuntu--vg-root /var/cache/squid-deb-proxy btrfs defaults,subvol=@squid 0 2
/dev/mapper/ubuntu--vg-root /var/log btrfs defaults,subvol=@log 0 2
/dev/mapper/ubuntu--vg-root /var/cache/apt/archives btrfs defaults,subvol=@apt 0 2
/dev/mapper/ubuntu--vg-root /mnt btrfs defaults 0 2

plocate does not find anything as it considers / to be a bind mount and
then updatedb exits immediately. Special casing / also would not work,
it would not locate anything in the other subvolumes.

Originally reported at https://bugs.launchpad.net/ubuntu/+source/plocate/+bug/1968190


-- System Information:
Debian Release: bookworm/sid
APT prefers kinetic
APT policy: (500, 'kinetic'), (500, 'jammy-security')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.15.0-25-generic (SMP w/8 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages plocate depends on:
ii libc6 2.35-0ubuntu3
ii libgcc-s1 12-20220319-1ubuntu1
ii libstdc++6 12-20220319-1ubuntu1
ii liburing2 2.1-2build1
ii libzstd1 1.4.8+dfsg-3build1

plocate recommends no packages.

Versions of packages plocate suggests:
ii powermgmt-base 1.36
ii systemd-sysv 249.11-0ubuntu3

-- Configuration Files:
/etc/updatedb.conf changed [not included]

-- no debconf information

--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en

[Julian Andres Klode]

On Mon, May 02, 2022 at 05:46:42PM +0200, Steinar H. Gunderson wrote:

> On Mon, May 02, 2022 at 05:23:08PM +0200, Julian Andres Klode wrote:
> > plocate considers all mount points of a btrfs as bind mounts,
> > so when I have a somewhat usual case of subvolumes, e.g.
>

> Hi,
>
> It considers them as bind mounts because btrfs implements subvolumes
> as bind mounts. This is documented in updatedb.conf(5) (which refers to
> btrfs-subvolume(8) and shows a workaround), so I'm closing this.

It says to make / as subvolume as well, which seems incomplete, as
mounted at / is the subvolume @ (that's the standard Ubuntu layout);
the actual non-subvolume / is only mounted at /mnt for managing
snapshots (later /var/lib/apt-btrfs-snapshot).

updatedb needs to parse /proc/mounts, look for subvol= flag, and
for each subvol= traverse the shortest path (or first path it sees).

Essentially this is broken for btrfs users now unless they only
mount once. Which doesn't *always* work for various reasons, e.g.
if you want to snapshot / but not /var/log, you have to have them
be subvolumes /@ and /@log such that you backup the former. If you
had / and /var/log were a direct subvolume, you'd lose it on restore.

[Steinar H. Gunderson]

On Mon, May 02, 2022 at 06:47:46PM +0200, Julian Andres Klode wrote:
> It says to make / as subvolume as well, which seems incomplete, as
> mounted at / is the subvolume @ (that's the standard Ubuntu layout);

That would be to make it at @root or similar.

> updatedb needs to parse /proc/mounts, look for subvol= flag, and
> for each subvol= traverse the shortest path (or first path it sees).

There's no way you can meaningfully say that /a is shorter than /b.
And the order Linux gives you a directory entry back is pretty much random,
so “first it sees“ would be a nightmare for incremental updatedb.

If you don't want bind mounts pruned (and that includes btrfs subvolumes
on the inside of each other), you'll need to turn off the bind mount pruning
option. Or you'd have to fix btrfs so that it doesn't implement subvolumes
as bind mounts :-)

/* Steinar */
--
Homepage: https://www.sesse.net/

[Julian Andres Klode]

On Mon, May 02, 2022 at 07:02:14PM +0200, Steinar H. Gunderson wrote:
> On Mon, May 02, 2022 at 06:47:46PM +0200, Julian Andres Klode wrote:
> > It says to make / as subvolume as well, which seems incomplete, as
> > mounted at / is the subvolume @ (that's the standard Ubuntu layout);
>
> That would be to make it at @root or similar.

I fail to see how naming it @root instead of @, or @screwedup for that
matter makes a difference.

>
> > updatedb needs to parse /proc/mounts, look for subvol= flag, and
> > for each subvol= traverse the shortest path (or first path it sees).
>
> There's no way you can meaningfully say that /a is shorter than /b.

There's a word for confusing the issue with corner cases that I don't
remember that perfectly describes what's happening here.

> And the order Linux gives you a directory entry back is pretty much random,
> so “first it sees“ would be a nightmare for incremental updatedb.

It's not a problem to parse /proc/mounts, sort it and consider the
shortest/first path the canonical location of a "bind mount". Then all
the normal cases of having a btrfs fs mounted at like /, /.snapshots or
whatever, /var/log will just work as you expect them to, and if you
mount the same subvolume in multiple places, the behavior is still odd,
it would index /a and not /b, but those are niche cases in comparison,
and the behavior is still better than not indexing at all.

>
> If you don't want bind mounts pruned (and that includes btrfs subvolumes
> on the inside of each other), you'll need to turn off the bind mount pruning
> option. Or you'd have to fix btrfs so that it doesn't implement subvolumes
> as bind mounts :-)

There are no btrfs subvolumes inside of each other here. At least not in
the sense that the subvolume is inside the subvolume, all subvolumes are
at the / level of the file system (which is outside the filesystem
hierarchy mounted at /, but might be mounted temporarily for
snapshotting purposes).

They are inside each other in the sense that we mount those top-level
subvolumes below the /@ subvolumes.

This is a significant regression for users coming from mlocate. They had
working locate, and after they migrate to plocate, their databases are
empty and they spent hours trying to figure out why. That's the reason
I (work me) reported it in Ubuntu and put it on the team backlog to work on.

If you prefer to keep plocate broken with btrfs in Debian, that's your
choice. It's not a choice I agree with; and I'm happy carrying a delta
for in Ubuntu to make things work for our users.

[Steinar H. Gunderson]

On Tue, May 03, 2022 at 07:59:48PM +0200, Julian Andres Klode wrote:
> I fail to see how naming it @root instead of @, or @screwedup for that
> matter makes a difference.

Try it. :-)

> This is a significant regression for users coming from mlocate. They had
> working locate

No, did they not. The mlocate updatedb code had broken bind mount detection
after the switch from /etc/mtab to /proc/mountinfo, which specifically broke
btrfs subvolumes:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=746943

> If you prefer to keep plocate broken with btrfs in Debian,

Please stop misrepresenting the issue. plocate isn't broken with btrfs,
but you can configure btrfs subvolumes in a way such that plocate
(and basically any other tool that tries to skip bind mounts) will not work
in the default configuration.

[Julian Andres Klode]

On Tue, May 03, 2022 at 08:59:31PM +0200, Steinar H. Gunderson wrote:
> On Tue, May 03, 2022 at 07:59:48PM +0200, Julian Andres Klode wrote:
> > I fail to see how naming it @root instead of @, or @screwedup for that
> > matter makes a difference.
>
> Try it. :-)

I tried something else...

>
> > This is a significant regression for users coming from mlocate. They had
> > working locate
>
> No, did they not. The mlocate updatedb code had broken bind mount detection
> after the switch from /etc/mtab to /proc/mountinfo, which specifically broke
> btrfs subvolumes:
>
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=746943
>
> > If you prefer to keep plocate broken with btrfs in Debian,
>
> Please stop misrepresenting the issue. plocate isn't broken with btrfs,
> but you can configure btrfs subvolumes in a way such that plocate
> (and basically any other tool that tries to skip bind mounts) will not work
> in the default configuration.

Removing the /mnt mount of the / root volume makes it work:

$ grep btrfs /proc/self/mountinfo
32 1 0:28 /@ / rw,relatime shared:1 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=256,subvol=/@
143 32 0:28 /@apt /var/cache/apt/archives rw,relatime shared:79 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=273,subvol=/@apt
146 32 0:28 /@squid /var/cache/squid-deb-proxy rw,relatime shared:81 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=274,subvol=/@squid
142 32 0:28 /@log /var/log rw,relatime shared:83 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=344,subvol=/@log
2734 32 0:28 /@/snap /snap rw,relatime shared:1 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=256,subvol=/@
4560 32 0:28 /@/var/snap /var/snap rw,relatime shared:1 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=256,subvol=/@
140 32 0:28 / /mnt rw,relatime shared:77 - btrfs /dev/mapper/ubuntu--vg-root rw,compress=zstd:3,ssd,space_cache,subvolid=5,subvol=/

It's not clar to me why that would make a difference but this is
all very confusing and needs further investigation. Like it only
reads mountinfo and stats / why would the existence of /mnt make
a difference to it - did it read "/" on the LHS and took it too mean
a bind mount of "/@" because "/@" is mounted at "/"? Clearly subvolumes
create confusing paths.

That /mnt mountpoint is anormal, but normally apt-btrfs-snapshot
mounts to a temporary directory temporarily to create a snapshot
(however they get leftover sometimes); whereas snapper uses a
different layout where / is the / and snapshots are kept in
/.snapshots or something, so it's not affected.