merged /usr considered harmful (was Re: Bits from the Technical Committee)

[Thorsten Glaser]

Sean Whitton dixit:

>* #978636 move to merged-usr-only?
>
> We were asked to decide whether or not Debian 'bookworm' should
> continue to support systems which are not using the merged-usr
> filesystem layout. We decided that support should not continue beyond
> Debian 'bullseye'.

What? WHAT? ＷＨＡＴ？

> The decision is captured here:
> <https://bugs.debian.org/978636#178>

No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
Debian is becoming untenable. Years ago, I had hoped it won’t.

bye,
//mirabilos
--
Yes, I hate users and I want them to suffer.
-- Marco d'Itri on gmane.linux.debian.devel.general

[Guillem Jover]

On Wed, 2021-07-14 at 19:54:56 +0000, Thorsten Glaser wrote:
> Sean Whitton dixit:
> >* #978636 move to merged-usr-only?
> >
> > We were asked to decide whether or not Debian 'bookworm' should
> > continue to support systems which are not using the merged-usr
> > filesystem layout. We decided that support should not continue beyond
> > Debian 'bullseye'.
>
> What? WHAT? ＷＨＡＴ？
>
> > The decision is captured here:
> > <https://bugs.debian.org/978636#178>
>
> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> Debian is becoming untenable. Years ago, I had hoped it won’t.

I've been meaning to send a note about this for some time now, but
as I feel it keeps getting ignored, it always seems a bit pointless.

But in any case, given that merged-usr-via-aliased-dirs is not really
supported by dpkg anyway, it is broken by design [B], I have no
intention whatsoever to break any of my systems with such layout going
forward, I'm thus planning to spend any necessary volunteer time
implementing any fix, workaround or solution required to avoid having
to use it, in detriment of other Debian volunteer time. I already
started some time ago with dpkg-fsys-usrunmess(8), present already in
the upcoming bullseye release.

[B] <https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F>

Thanks,
Guillem

[Thorsten Glaser]

Guillem Jover dixit:

>I've been meaning to send a note about this for some time now, but
>as I feel it keeps getting ignored, it always seems a bit pointless.

Yeah, I saw this popping up multiple times in that bugreport ☹

>But in any case, given that merged-usr-via-aliased-dirs is not really
>supported by dpkg anyway, it is broken by design [B], I have no

Time for another GR? Leaving Debian behind?

>intention whatsoever to break any of my systems with such layout going

Yeah, this totally caught me by surprise. I guess I’ll have to
figure out how to switch *all* my systems from sid to bullseye
*fast*, now. This also effectively ends debian-ports work for me…

Not amused,

[Sean Whitton]

Hello Guillem,

Just to confirm, when you say "merged-usr-via-aliased-dirs", you mean
what I would get if I typed 'debootstrap bullseye /foo', right?

I would like to note that the TC decision did not specify any particular
implementation of merged-/usr. It was just about whether to continue to
try to support both.

--
Sean Whitton

[Jonathan Carter]

Hi Sean

On 2021/07/15 09:04, Sean Whitton wrote:
> Just to confirm, when you say "merged-usr-via-aliased-dirs", you mean
> what I would get if I typed 'debootstrap bullseye /foo', right?
>
> I would like to note that the TC decision did not specify any particular
> implementation of merged-/usr. It was just about whether to continue to
> try to support both.

I think a more detailed explanation/expansion/clarification on what
exactly this means (and ideally also the rationale behind) that in the
bug report would be appreciated.

thanks,

-Jonathan

[Marc Haber]

On Thu, 15 Jul 2021 09:56:18 +0200, Jonathan Carter <j...@debian.org>
wrote:

Can we please delay this discussion until after the release? I don't
think we can afford an additional time sink at the moment. Please, get
bullseye out of the door first, then decide how many existing users
we're going to drive away from Debian in the next round.

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

[Luca Boccassi]

Hi,

As it has been said and written many times already, in reality this is
not broken by design at all and in fact it is the only successful
strategy that has been deployed by other distros - it's what is being
called merged-usr-via-moves-and-symlink-farms that is broken. We can
say this with certainty because both approaches have been tried, so
there's actual real-world data to look at. Just go look at the absolute
mess that Suse got themselves into by following that solution - a 10-
years-long massive half-done-never-finished headache that took an
inordinate amount of work to back out of, and move to the actual
working solution that everybody else are using - merged-usr-via-
aliased-dirs. On the other hand Fedora/RHEL had a smooth and simple
transition using merged-usr-via-aliased-dirs, and that was the end of
it.

Dpkg has some very minor bugs that rpm/dnf/yum/zypper/whatever do not
suffer from. So what? It is perfectly normal as it's software and all
softwares have bugs. They could be fixed, worked around, or ignored,
like all other bugs.

--
Kind regards,
Luca Boccassi

[Jonathan Carter]

On 2021/07/15 10:47, Marc Haber wrote:
> Can we please delay this discussion until after the release?

To be clear, I was requesting further details from the TC, not a
re-evaluation or further discussion.

-Jonathan

[Sean Whitton]

Hello,

You're right, it would have been good if our Bits mail had linked to
some other messages in that thread rather than just the statement of the
result of the vote. I'm sorry we didn't spot that before sending it.

Additionally, as someone else was kind enough to point out to me
off-list, my statement that "the TC decision did not specify any
particular implementation of merged-/usr" was rather misleading. What I
should have written was that we did not specify any particular
implementation of the *migration* to merged-/usr for existing systems.

ISTM that "merged-usr-via-aliased-dirs" could refer to the migration
path implemented by the usrmerge package, and/or simply the replacement
of the directories /lib, /bin etc. with symlinks. To the extent that it
refers to the latter, the TC decision does indeed specify that we will
implement merged-/usr using merged-usr-via-aliased-dirs.

Here are some useful messages we should have linked to:
<https://bugs.debian.org/978636#128>
<https://bugs.debian.org/978636#143>
<https://bugs.debian.org/978636#153>

and of course
<https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html>

--
Sean Whitton

[Thorsten Glaser]

Marc Haber dixit:

>think we can afford an additional time sink at the moment. Please, get

While that’s true…

>Can we please delay this discussion until after the release? I don't

… we can’t afford to: the TC discussion becomes valid as soon as
bullseye is released, which is in two weeks, and the people who
want to make Debian into a Fedora/RedHat/systemd-OS derivative
are going to rely on it and make our lives harder immediately.
They have proven time and time again, cf. #921012, #964139, that
they’re interested in actively breaking nōn-systemd users’ cases,
even despite policy to the contrary (basically ignoring the latter
until they managed to change policy to their likes).

>bullseye out of the door first, then decide how many existing users
>we're going to drive away from Debian in the next round.

*sigh* and isn’t that true…

I *really* don’t get why…

ⓐ these things aren’t done in a derivative that’s *really* close
to Debian proper but can do all the funky new stuff, preserving
support for the old stuff in Debian itself, and…

ⓑ usrmerge is “needed” anyway; we were working rather well with the
previous (and rather-recently-introduced) model of “either /usr
must be on the same filesystem as / or you must use an initrd to
mount it”.

Frankly, usrmerge is WORSE then what we had before, in all possible
ways. People are going to write unportable scripts and programs, for
example. Symlink farming and moving is also not going to cut it (also,
rules like “if ed(1) is installed, /bin/ed must be able to call it”
is where other distros failed during moving stuff to /usr).

Why is there such heavy incentive to break things that work, for any
price?

Disappointed, and having spent a day crossgrading and moving from
sid to bullseye,
//mirabilos
PS: Please keep me in Cc, I’m not subscribed here, too high-volume
PPS: This is really draining energy. I just refused adopting a package
I use and which is rather useful because I just can’t any more.
And there is flooding and… stuff.
--
22:59⎜<Vutral> glaub ich termkit is kompliziert | glabe nicht das man
damit schneller arbeitet | reizüberflutung │ wie windows │ alles evil
zuviel bilder │ wie ein spiel | 23:00⎜<Vutral> die meisten raffen auch
nicht mehr von windows | 23:01⎜<Vutral> bilderbücher sind ja auch nich
wirklich verbreitet als erwachsenen literatur ‣ who needs GUIs thus?

[Marc Haber]

On Fri, 16 Jul 2021 01:44:54 +0000 (UTC), Thorsten Glaser
<t...@debian.org> wrote:
>Marc Haber dixit:
>>think we can afford an additional time sink at the moment. Please, get
>
>While that’s true…

You conveniently snipped the "I don't" which turns your quote into the
opposite that I wanted to say.

[Thomas Goirand]

On 7/15/21 5:08 AM, Thorsten Glaser wrote:
> Guillem Jover dixit:
>
>> I've been meaning to send a note about this for some time now, but
>> as I feel it keeps getting ignored, it always seems a bit pointless.
>
> Yeah, I saw this popping up multiple times in that bugreport ☹
>
>> But in any case, given that merged-usr-via-aliased-dirs is not really
>> supported by dpkg anyway, it is broken by design [B], I have no
>
> Time for another GR?

Please don't! This isn't funny...

Thomas

[Thomas Goirand]

On 7/14/21 9:54 PM, Thorsten Glaser wrote:
> Sean Whitton dixit:
>
>> * #978636 move to merged-usr-only?
>>
>> We were asked to decide whether or not Debian 'bookworm' should
>> continue to support systems which are not using the merged-usr
>> filesystem layout. We decided that support should not continue beyond
>> Debian 'bullseye'.
>
> What? WHAT? ＷＨＡＴ？
>
>> The decision is captured here:
>> <https://bugs.debian.org/978636#178>
>
> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> Debian is becoming untenable. Years ago, I had hoped it won’t.
>
> bye,
> //mirabilos

Hi Thorsten,

Sent privately, I hope you don't mind.

I very much like you, and appreciate the work you're doing in Debian.
However, I would not like at all if this is the start of yet-another
drama in Debian. We had enough of this.

Instead of complaining about change, it'd be nice if instead, you were
embracing it, and tried to contribute fixes for the parts that you're
maintaining in Debian. I'm not asking you to be enthusiastic about
changes you don't like, but maybe you could at least tried to understand
others are trying to improve things, rather than destroying your work.

Merging binaries in /usr and getting rid of /bin and /sbin, at the end,
WILL be an improvement. Debian cannot be the last distro not doing the
move, I hope you understand that.

Also, I'm having a hard time understanding why moving binaries around
should just break any system, especially if we have workarounds, like
symlink and/or bind mounts or the like. If non-systemd setups are having
issues, please just fix them... The end result *will* be a better Debian.

Last thing, the discussion happened a long time ago, you had plenty of
time to react, especially before the TC voted. Reacting so late, with
such emotion is really uncalled for.

Cheers,

Thomas Goirand (zigo)

[The Wanderer]

On 2021-07-16 at 03:18, Marc Haber wrote:

> On Fri, 16 Jul 2021 01:44:54 +0000 (UTC), Thorsten Glaser
> <t...@debian.org> wrote:
>
>>Marc Haber dixit:
>>
>>>think we can afford an additional time sink at the moment. Please, get
>>
>>While that’s true…
>
> You conveniently snipped the "I don't" which turns your quote into the
> opposite that I wanted to say.

No, he didn't; he reordered the quoted lines (moving this one ahead of
the rest), so that he could put this one line of his reply where it
would make the most sense relative to the rest of his reply, without
also having to adjust the within-each-line quoting etc.. The "I don't"
is in the first line which you yourself didn't quote.

The result was slightly confusing to me at first glance, but I figured
it out within a minute.

If you'd chosen to chide him for reordering lines in a way which
(presumably inadvertently) produced an initially-misleading result, that
would be one thing, but I don't think it's appropriate to accuse him of
snipping out context when he didn't do so.

--
The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw

[Philip Hands]

Thorsten Glaser <t...@debian.org> writes:

...

> I *really* don’t get why…
>
> ⓐ these things aren’t done in a derivative that’s *really* close
> to Debian proper but can do all the funky new stuff, preserving
> support for the old stuff in Debian itself, and…

This issue has been explored very thoroughly, so if you've not noticed
that happening then I suspect that you decided to mostly ignore it.

The TC ended up deciding this because it's perfectly possible for
reasonable people to agree on the facts, but weigh the importance of the
various implications of those facts and available options differently.

> Frankly, usrmerge is WORSE then what we had before, in all possible
> ways.

By saying that, in that way, you give the impression that you assume
that the honestly held opinions of a lot of your fellow developers are
wrong and/or stupid.

Even if you think that, saying it as part of your argument tends to
force people make a snap decision on which side of the argument more
fools are standing, and then to pick sides, which entrenches opinion,
and so is pretty counter-productive if your intention is to persuade.

Making these 'unagreeable' decisions is the reason we need a TC.

Sadly there are always going to be winners and losers in these cases,
otherwise they wouldn't make it to the TC. Deciding to do nothing is a
decision too.

Do you think the TC didn't take that job seriously enough?

Do you think the TC considered insufficient evidence?

Did you not submit your most persuasive evidence for some reason?

I would hope that you are at least willing to admit that opinions on
this subject can differ even when the facts are generally agreed.

Also, that while you might be upset that your opinion didn't align with
the decision made by the TC, that the people making that decision were
devoting their best efforts to doing what they hope will serve the
project well in the long run.

Cheers, Phil.
--
|)| Philip Hands [+44 (0)20 8530 9560] HANDS.COM Ltd.
|-| http://www.hands.com/ http://ftp.uk.debian.org/
|(| Hugo-Klemm-Strasse 34, 21075 Hamburg, GERMANY

[Pierre-Elliott Bécue]

Thorsten Glaser <t...@debian.org> writes:
>>But in any case, given that merged-usr-via-aliased-dirs is not really
>>supported by dpkg anyway, it is broken by design [B], I have no
>
> Time for another GR? Leaving Debian behind?

Threats of leaving are not fine and are just making any discussion
pointless.

I refuse (and I recommend no one accepts) to enter debate with someone
putting their involvment in the balance as a way to force their opinion
in.

--
PEB

[Thomas Goirand]

On 7/16/21 10:09 AM, Thomas Goirand wrote:
> On 7/14/21 9:54 PM, Thorsten Glaser wrote:
>> Sean Whitton dixit:
>>
>>> * #978636 move to merged-usr-only?
>>>
>>> We were asked to decide whether or not Debian 'bookworm' should
>>> continue to support systems which are not using the merged-usr
>>> filesystem layout. We decided that support should not continue beyond
>>> Debian 'bullseye'.
>>
>> What? WHAT? ＷＨＡＴ？
>>
>>> The decision is captured here:
>>> <https://bugs.debian.org/978636#178>
>>
>> No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
>> Debian is becoming untenable. Years ago, I had hoped it won’t.
>>
>> bye,
>> //mirabilos
>
> Hi Thorsten,
>
> Sent privately, I hope you don't mind.

It was sent to the list, never mind, there's nothing private here, just
a call to be more constructive...

Cheers,

Thomas Goirand (zigo)

[Magissia]

In this case, this page should be updated to reflect the fact it is not
broken.

https://wiki.debian.org/Teams/Dpkg/MergedUsr

Nol'

[Thorsten Glaser]

Johannes Drexl dixit:

>embrace getting rid of /sbin and /bin. FHS 3.0 explicitely states that
>/usr is allowed to be not only on a separate partition, but even on a
>network device shared by other machines:

This hasn’t been true on Debian for a while (partially due to the
systemd/usrmerge proponents, partially due to the difficulty of
deciding what needs to be present). You have had to either ensure
/usr is mounted by an initrd or it’s not on a separare filesystem
for a while, already (and that was fine with me, it’s the usrmerge
crap that totally pointlessly breaks things).

And no, I’m not going to embrace every unnecessary change thrown my way.


Magissia dixit:

>In this case, this page should be updated to reflect the fact it is not
>broken.

No; usrmerge is broken from the PoV of Debian’s package manager.
Running usrmerge breaks assumptions done by dpkg; you can probably
do it with RPM or something, but it’s not supported with dpkg.

bye,
//mirabilos
--
Gestern Nacht ist mein IRC-Netzwerk explodiert. Ich hatte nicht damit
gerechnet, darum bin ich blutverschmiert… wer konnte ahnen, daß SIE so
reagier’n… gestern Nacht ist mein IRC-Netzwerk explodiert~~~
(as of 2021-06-15 The MirOS Project temporarily reconvenes on OFTC)

[Simon McVittie]

On Sat, 17 Jul 2021 at 20:34:41 +0000, Johannes Drexl wrote:
> /usr is allowed to be not only on a separate partition, but even on a
> network device

This has been discussed at considerable length before, but I'll try to
recap:

A separate or network-mounted /usr is possible in Debian, whether
merged-/usr is used or not, but only if /usr is already mounted
by the time the system hands over from the initramfs (usually generated by
initramfs-tools, or sometimes dracut or another alternative) to the main
system (which starts systemd, sysvinit or some other init system as the
main pid 1 and proceeds from there). Merged /usr is actually better for
this configuration than unmerged /usr in some ways: it lets you share
*all* the non-modifiable files between machines, not just the ones that
are in /usr (as opposed to /lib, /bin, /sbin).

This is not a new requirement: Debian >= 9 (2017) officially did not support
mounting /usr halfway through the main system boot process. Older Debian
releases aimed to support /usr being mounted during boot (for example
during the rcS sequence in sysv-rc) but it frequently didn't actually
work in practice, particularly if combined with network mounts.

The reason why mounting /usr during main-system boot is no longer
supported is that it often didn't work. A major reason why it often didn't
work is that depending on how the system is configured, mounting /usr can
require networking; but bringing up networking can require arbitrarily
many programs and libraries (networking infrastructure like ifupdown or
NetworkManager often has either a plugin architecture, or an arbitrary
hooks mechanism that executes programs of the sysadmin's choice that
will often have dependencies in /usr, or both).

Similarly, udevd needs to run early in the main system boot to bring up
/dev, but udev rules can run arbitrary programs, some of them in /usr;
and for the main system, those programs often need data from /usr/share.

More generally, if we took every library and every program that could
conceivably be a dependency of /usr and moved them from /usr into the
root filesystem, then the root filesystem would become increasingly large
over time, negating any benefit that you hoped to gain by mounting /usr
separately. Over time, this approach would tend towards the layout that
the Debian Hurd port briefly tried to use, which was the opposite of
merged-/usr (you could call it "merged rootfs"), with /usr as either a
symlink to /, or containing only symlinks to /lib, /bin and so on.

Instead, we require everything that is needed in *your* configuration
(not necessarily in *anyone's* configuration, just yours!) to be part
of the initramfs, which is generated per-machine.

Effectively, the requirement to mount /usr before pivoting from initramfs
to main system means that instead of a small rootfs (which can be used for
recovery) and a larger /usr, we have a small initramfs (which can be used
for recovery) and a large main system.

One key advantage of this is that decisions about what to include in
the rootfs (of non-merged-/usr systems) have to be made globally for all
of Debian, but decisions about what to include in the initramfs can be
made per-machine, allowing some otherwise impossible situations to be
resolved. If your /usr is mounted via NFS, but bringing up my network
requires /usr (perhaps for a VPN), in the old model with a small rootfs
and a larger /usr it was impossible for us both to have what we needed:
we could not bring up networking both before /usr (as you would have
needed) and after /usr (as I would have needed).

> /usr is allowed to be [...] on a network device shared by other machines

The FHS may allow this, but it has significant practical problems,
unrelated to merged-/usr, on machines that receive security/stable updates
(which I would hope by now should mean all machines). Updates to Debian
packages can touch both mutable per-machine files (/etc, /var) and
immutable/shareable per-(package,version) files (/usr, /lib*, /bin, /sbin).
If a machine's /usr is not in sync with its /etc and /var, then it is likely
to work incorrectly: at a minimum, asking dpkg which packages and versions
are installed will give you an answer that does not match what is actually
in /usr.

On non-merged-/usr systems, the machine's /usr and {/lib*,/bin,/sbin}
must also be kept in sync: for example, there is no guarantee that
/usr/bin/dbus-daemon (in /usr) will work correctly with a mismatched
version of /lib/x86_64-linux-gnu/libdbus-1.so.3 (on the rootfs of a
non-merged-/usr system). On merged-/usr systems, it is impossible for
those directories to become out-of-sync, so this particular requirement
becomes trivial to achieve.

A more robust approach to sharing /usr between machines (or containers)
might be to give each machine (or container) its own private /usr or even
its own private root filesystem, and then carry out file- or block-level
deduplication on the storage backend (for example, if /usr is NFS-shared
and is stored on btrfs on the NFS server, then reflinks could be used to
make files with identical content share storage).

If the FHS allows something, that also does not mean "all FHS-compliant
operating systems must allow this"; it just means "programs designed to
run on FHS-compliant operating systems must not assume this can't happen".
For example, the FHS allows /run and /var/run to be separate, but on
Debian they are always synonymous (because making them distinct would
only cause problems, without providing any benefit). If Debian did not
support a network-mounted /usr, then asking for network-mounted /usr to be
supported would be a reasonable feature request, but the lack of that
feature would not make Debian any less FHS-compliant.

smcv

[Geert Stappers]

Summary: let go, let go

On Sat, Jul 17, 2021 at 09:13:57PM +0000, Thorsten Glaser wrote:
>
> And no, I’m not going to embrace every unnecessary change thrown my way.

None of us does embraces every unnecessary change.
We all choose our battles wisely.

> No; usrmerge is broken from the PoV of Debian’s package manager.
> Running usrmerge breaks assumptions done by dpkg; you can probably
> do it with RPM or something, but it’s not supported with dpkg.

Hence a change.


Groeten
Geert Stappers
--
Silence is hard to parse

[Marco d'Itri]

On Jul 18, Simon McVittie <sm...@debian.org> wrote:

> If a machine's /usr is not in sync with its /etc and /var, then it is likely
> to work incorrectly: at a minimum, asking dpkg which packages and versions

But in my experience (with shared-/usr containers) this works great as
long as everything is aligned to the same major Debian release.

> are installed will give you an answer that does not match what is actually
> in /usr.

I link /var/lib/dpkg/ to somewhere in /usr/, and I think that this is
something that we should do anyway.

--
ciao,
Marco

[Bastian Blank]

On Sun, Jul 18, 2021 at 11:13:37AM +0200, Marco d'Itri wrote:
> On Jul 18, Simon McVittie <sm...@debian.org> wrote:
> > If a machine's /usr is not in sync with its /etc and /var, then it is likely
> > to work incorrectly: at a minimum, asking dpkg which packages and versions
> But in my experience (with shared-/usr containers) this works great as
> long as everything is aligned to the same major Debian release.

So we can just merge it and make it break even less likely?

Bastian

--
A princess should not be afraid -- not with a brave knight to protect her.
-- McCoy, "Shore Leave", stardate 3025.3

[Stephan Verbücheln]

On Sun, 2021-07-18 at 11:13 +0200, Marco d'Itri wrote:
> I link /var/lib/dpkg/ to somewhere in /usr/, and I think that this is
>

What? No matter whether we merge “/bin” or not, “/usr” should stay
read-only.

Regards

[Andrey Rahmatullin]

On Debian /usr is changed only when /var/lib/dpkg is changed (ideally).

--
WBR, wRAR

[Marco d'Itri]

The dpkg database IS read-only as long as you are not installing or
removing packages, which you cannot do anyway if /usr is read-only.
In other words, the read-only state of the dpkg database is tightly
coupled with the read-only state of /usr.

The only corner case which would require some more thinking is how to
handle changing diversions of files in /etc in the systems with the
read-only /usr, and for the time being "don't do that" could be
a totally valid solution.

--
ciao,
Marco

[Stephan Verbücheln]

Thank you for the explanation. I think it covers most use cases.
However, it does not cover packages that do not actually install
programs but only perform changes to /etc or install something to /opt,
is that correct?

Regards

[Andrey Rahmatullin]

While, AFAIK, it's indeed only Debian Policy stopping you from not
shipping /usr/share/doc/*/copyright, and that and common sense stopping
you from not shipping /usr/share/doc/*/changelog, that's just yet another
case of something broken that we probably are not required to support.

--
WBR, wRAR

[Andrey Rahmatullin]

On Sun, Jul 18, 2021 at 10:11:22PM +0500, Andrey Rahmatullin wrote:
> While, AFAIK, it's indeed only Debian Policy stopping you from not
> shipping /usr/share/doc/*/copyright, and that and common sense stopping
> you from not shipping /usr/share/doc/*/changelog, that's just yet another
> case of something broken that we probably are not required to support.

... even though these files really belong with other dpkg metadata in
/var/lib/dpkg...

--
WBR, wRAR

[Svante Signell]

On Wed, 2021-07-14 at 23:40 +0200, Guillem Jover wrote:
> On Wed, 2021-07-14 at 19:54:56 +0000, Thorsten Glaser wrote:
> > Sean Whitton dixit:
> > > * #978636 move to merged-usr-only?
> > >
> > > We were asked to decide whether or not Debian 'bookworm' should
> > > continue to support systems which are not using the merged-usr
> > > filesystem layout. We decided that support should not continue
> > > beyond Debian 'bullseye'.
> >
> > What? WHAT? ＷＨＡＴ？
> >
> > > The decision is captured here:
> > > <https://bugs.debian.org/978636#178>
> >
> > No reason provided either. This stinks. I’m v̲e̲r̲y̲ disappointed.
> > Debian is becoming untenable. Years ago, I had hoped it won’t.
>
> I've been meaning to send a note about this for some time now, but
> as I feel it keeps getting ignored, it always seems a bit pointless.
>
> But in any case, given that merged-usr-via-aliased-dirs is not really
> supported by dpkg anyway, it is broken by design [B], I have no
> intention whatsoever to break any of my systems with such layout
> going forward, I'm thus planning to spend any necessary volunteer
> time implementing any fix, workaround or solution required to avoid

> having to use it, in detriment of other Debian volunteer time. I
> alreadystarted some time ago with dpkg-fsys-usrunmess(8), present
> already inthe upcoming bullseye release.
>
[B]
https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
>

Since the dpkg developer and maintainer Guillem considers merged /usr
broken by design, maybe Debian should consider to use some other
package management software for the peace of mind for people involved
in the project? Maybe guix could be usable?

[Helmut Grohne]

On Fri, Jul 16, 2021 at 01:15:37PM +0200, Magissia wrote:
> In this case, this page should be updated to reflect the fact it is not
> broken.
>
> https://wiki.debian.org/Teams/Dpkg/MergedUsr

Logic does not quite work that way. Just because we selected that way of
doing things doesn't imply it would not be broken. Arguably, if it
wasn't broken, there would not have been a need to defer the decision to
the CTTE as there would have been consensus on choosing the non-broken
way already. The CTTE was tasked to choose the less broken one of two
broken options and it did.

>From a dpkg pov, the aliasing technique continues to be broken and the
solution continues to manifest as unintuitive behaviour and related
issues. Calling that broken sounds fair to me.

The thing that makes me a little grumpy about this is that the
proponents of the /usr merge continue stuffing ever more fingers into
their ears when faced with problems. I for one wouldn't care as much if
the /usr merge wasn't that much of a time sink to me. For instance, I
happened to be one of the first affected users of dpkg-shlibdeps ceasing
to work. I'd really like to ignore the whole mess and leave it to
others (and that includes not opposing it).

I also note that there is a subtle difference between those who disagree
with the /usr merge and those that disagree with its implementation
strategy. The former ones seems like a very small minority to me. As a
result, reiterating the "why" is kinda pointless as there already seems
to be consensus on that front. As for the how, there seems to be rough
consensus that all implementation strategies are broken one way or
another. We just get to pick the least painful one and deferred the
choice to the CTTE.

Maybe someone could fix #858331 et al and we can move on?

Helmut

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 5:31 p.m., Svante Signell wrote:
> Hi, is it OK to forward your mail to debian-devel. I don't think
> mailing to debian-user will have any effect on this issue?
>
Sure ! Honestly it's my mistake to have sent it to debian-user.
I get everything in one mailbox. I need to have this sorted out and use
one mailbox for each mailing list.

Thanks

--
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 5:07 p.m., Andy Smith wrote:
> Hello,
>
> On Sun, Jul 18, 2021 at 04:31:11PM -0400, Polyna-Maude Racicot-Summerside wrote:
>> My personal opinion is that Debian is going into a mostly "we got the
>> best idea in the world but forgot that not everyone implement things the
>> same way".
>
> I recommend understanding the issue before putting forth an opinion.
>
Maybe I shall correct what I said as it may be misunderstood.

Without regard to this specific issue, I seem to have noticed that the
Debian project has made some change in good faith, based on some
technological conception of what is the best. But in the end, most user
seem not to need those change and be a source of problem for most of them.

I've had hell of a mess to understand the change to SystemD.

Not so long ago I even learned that it caused a fork for a Debian
derivative called Devuan.

For me it was a huge departure from the KISS philosophy and I feel that
this "usrmerge" is going the same way.

I will take time to review what you have gave me as link and see what
will be my opinion after this.

>> I currently have a different partition for my /usr and this has been the
>> case since the end of 1990s when I started on Linux. Maybe I'm wrong but
>> I like it this way.
>>
>> Will the merge-usr cause myself problem ?
>
> No. Not as long as you use an initramfs created by any of the
> supported Debian tools like initramfs-tools or dracut, which you
> will do unless you have gone out of your way to do something
> different.
>
I've did many Debian install, using the standard installation software,
creating those partition from the "expert" choice of install mode. And
never had problem, I believe that the initramfs tools must be doing
their job.

> And regardless of merged-/usr, /usr on separate partition has not
> been supported in Debian without an initramfs since the stretch
> release in 2017.
>
I've used Stretch with a separate /usr without much problem, as said above.

> I think all of this is quite clearly explained in:
>
> https://salsa.debian.org/md/usrmerge/raw/master/debian/README.Debian
>
> which is linked from:
>
> https://wiki.debian.org/UsrMerge
>
> If you think it's not then you should probably raise a bug against
> the usrmerge package with your suggested edits/clarifications.
>
> Andy

[Polyna-Maude Racicot-Summerside]

Hello (Hi) !

On 2021-07-18 5:07 p.m., Andy Smith wrote:
> Hello,
>

> I think all of this is quite clearly explained in:
>
> https://salsa.debian.org/md/usrmerge/raw/master/debian/README.Debian
>
> which is linked from:
>
> https://wiki.debian.org/UsrMerge
>
> If you think it's not then you should probably raise a bug against
> the usrmerge package with your suggested edits/clarifications.
>

I've read the link above plus the link to
https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/

So if I get it right...

One partiton for /boot
One partition for /usr
One partition for /usr/local (if you feel like it)
One / partiton that will contain not much stuff other than config files ?
One partition for /var (if you feel)
One partition for /opt, /srv ...
One partition for /tmp

The root partition can be small as 16 Gb as it won't contain much ?

Am I getting this right ?

Here's my actual partition table (2TB HD)

udev 16G 0 16G 0% /dev
tmpfs 3.2G 2.1M 3.2G 1% /run
/dev/sda3 184G 70G 105G 40% /
tmpfs 16G 163M 16G 2% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
/dev/sda5 262G 50G 199G 21% /var
/dev/sda4 175G 86M 166G 1% /tmp
/dev/sda8 88G 60M 83G 1% /usr/local
/dev/sda7 92G 3.6G 84G 5% /opt
/dev/sda10 92G 60M 87G 1% /srv
/dev/sda1 487M 3.3M 483M 1% /boot/efi
/dev/sda11 733G 649G 47G 94% /home

I currently have a 70G usage of the root partition.
Running Buster.

[Marco d'Itri]

On Jul 19, Polyna-Maude Racicot-Summerside <deb...@polynamaude.com> wrote:

> So if I get it right...

Except for /boot/, which may be required for technical reasons, there
is no need to further partition your file system unless you actually
have reasons to do it.

> One partiton for /boot
> One partition for /usr
> One partition for /usr/local (if you feel like it)
> One / partiton that will contain not much stuff other than config files ?
> One partition for /var (if you feel)
> One partition for /opt, /srv ...
> One partition for /tmp

If you are aiming for overcomplexity then I think that you forgot /home.

> The root partition can be small as 16 Gb as it won't contain much ?

If you create a partition for everything else as described here then
/ will only contain /etc and /root, so even 1 GB will be enough.
But again, I do not really recommend this.

I do not recommend to partition general purpose systems with less than
a few hundreds of GBs of allocated disk space.

--
ciao,
Marco

[Polyna-Maude Racicot-Summerside]

Hi,

Here's my actual config (with 2TB) and yes I have a separate /home

What is tmpfs and why is it set to 3.2 GB ?
And /dev have 16G free ? Where does this come from...
I'm wasting some space with /tmp !

Filesystem Size Used Avail Use% Mounted on

udev 16G 0 16G 0% /dev
tmpfs 3.2G 2.1M 3.2G 1% /run
/dev/sda3 184G 70G 105G 40% /
tmpfs 16G 163M 16G 2% /dev/shm
tmpfs 5.0M 4.0K 5.0M 1% /run/lock
/dev/sda5 262G 50G 199G 21% /var
/dev/sda4 175G 86M 166G 1% /tmp
/dev/sda8 88G 60M 83G 1% /usr/local
/dev/sda7 92G 3.6G 84G 5% /opt
/dev/sda10 92G 60M 87G 1% /srv
/dev/sda1 487M 3.3M 483M 1% /boot/efi
/dev/sda11 733G 649G 47G 94% /home

tmpfs 3.2G 84K 3.2G 1% /run/user/118
tmpfs 3.2G 124K 3.2G 1% /run/user/1000

[Svante Signell]

Again, everybody is just hiding, I wonder from who, the big wolf??
Who is hen? Anybody having the courage to reply to this list about this
issue, not only workarounds/diversions?

[Russ Allbery]

Svante Signell <svante....@gmail.com> writes:

> Again, everybody is just hiding, I wonder from who, the big wolf?? Who
> is hen? Anybody having the courage to reply to this list about this
> issue, not only workarounds/diversions?

I'm not discussing the issue on the list because I think the current
direction in which Debian is heading seems reasonable and forward progress
is being made, so there doesn't seem to be anything to argue about or any
point in doing so.

The folks who have been upset about this direction for years are still
upset and I'm sorry that they're still upset and that we haven't found
some approach that makes everyone happy. But so far as I can tell, there
have been no new substantive arguments and no changes in direction, so
there doesn't seem to be anything new to reply about, particularly given
that I'm not involved in the implementation and therefore am not involved
in analyzing any of the technical concerns they raise.

"Courage" seems to me like an odd label to put on any of this.

--
Russ Allbery (r...@debian.org) <https://www.eyrie.org/~eagle/>

[Russ Allbery]

Russ Allbery <r...@debian.org> writes:

> I see that you have your system configured to store /tmp on your disk.
> This is generally not recommended these days. Storing /tmp in tmpfs is
> much faster for some applications and automatically achieves the desired
> and standard /tmp behavior of clearing it on reboot. About the only
> reason not to use tmpfs is if you have a very memory-constrained system
> and don't want to use any member at all for memory-backed file systems.

Argh, that should have been "and don't want to use any *memory* at all for
memory-backed file systems."

Apologies for the additional message, but that word substitution was
sufficiently confusing that I'm not sure the intended meaning was clear.

[Russ Allbery]

Polyna-Maude Racicot-Summerside <deb...@polynamaude.com> writes:

> Here's my actual config (with 2TB) and yes I have a separate /home

> What is tmpfs and why is it set to 3.2 GB ?

tmpfs is a RAM-backed temporary file system that is automatically used for
paths like /run and /dev/shm that are supposed to be cleared on each
reboot and hold only small files (or memory references, in the case of
/dev/shm).

I see that you have your system configured to store /tmp on your disk.
This is generally not recommended these days. Storing /tmp in tmpfs is
much faster for some applications and automatically achieves the desired
and standard /tmp behavior of clearing it on reboot. About the only
reason not to use tmpfs is if you have a very memory-constrained system
and don't want to use any member at all for memory-backed file systems.

> And /dev have 16G free ? Where does this come from...

The size of the udev file system is essentially meaningless.

> I'm wasting some space with /tmp !

I agree with the other feedback that you are overpartitioning your disk.
I used to do this back when I was first learning UNIX in the 1990s because
it seems like a good idea and it does isolate one part of the system from
another if it uses an excessive amount of space. But what I found in
practice, and what almost everyone who does this eventually finds in
practice, is that this much partitioning drastically reduces the long-term
flexibility of the system. It requires you predict in advance what parts
of the system will grow, and when you guess wrong, you end up with
symlinks trying to move directories from a partition with no free space to
another partition with free space, with all the complexity and breakage
that can cause.

There are some technical reasons to separate /boot if you are using a file
system for other partitions that isn't suitable for early boot (or if
you're using cryptsetup or other file system layers). /boot/efi is always
a separate partition because of how it works. Apart from those two
special cases, the only reason to put something on a separate file system
is if you have a clear and compelling reason why you expect a given file
system to run out of space and you want to ensure that it cannot take
space from other parts of the system.

This can be a good justification for putting /home on a separate partition
*if* you are running a multi-user system. But otherwise, separating out
things like /var or /usr/local or /opt or /srv is more likely to cause you
long-term headaches than it is to do anything useful.

[Andy Smith]

Hi,

On Sun, Jul 18, 2021 at 05:54:33PM -0400, Polyna-Maude Racicot-Summerside wrote:
> On 2021-07-18 5:07 p.m., Andy Smith wrote:
> > I recommend understanding the issue before putting forth an opinion.
> >
> Maybe I shall correct what I said as it may be misunderstood.

It's unclear to me why you've taken my reply to your mail on
debian-user, and instead sent it to me privately and also to
debian-devel.

As neither of us are Debian Developers and you don't seem to have
done much research regarding merged-/usr, I don't feel it is
on-topic for me to continue that conversation with you on
debian-devel.

I recommend doing some more reading of these threads and the
relevant wiki pages and if things are still unclear then asking
user-level questions only on debian-user.

Thanks,
Andy

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-18 7:21 p.m., Russ Allbery wrote:
> Polyna-Maude Racicot-Summerside <deb...@polynamaude.com> writes:
>
>> Here's my actual config (with 2TB) and yes I have a separate /home
>
>> What is tmpfs and why is it set to 3.2 GB ?
>
> tmpfs is a RAM-backed temporary file system that is automatically used for
> paths like /run and /dev/shm that are supposed to be cleared on each
> reboot and hold only small files (or memory references, in the case of
> /dev/shm).
>
> I see that you have your system configured to store /tmp on your disk.
> This is generally not recommended these days. Storing /tmp in tmpfs is
> much faster for some applications and automatically achieves the desired
> and standard /tmp behavior of clearing it on reboot. About the only
> reason not to use tmpfs is if you have a very memory-constrained system
> and don't want to use any member at all for memory-backed file systems.
>

I had the belief that some software used /tmp for temporary file that
may grow many GB (example DVD creation).

I have 32 GB

[Guillem Jover]

On Thu, 2021-07-15 at 10:13:47 +0100, Luca Boccassi wrote:
> As it has been said and written many times already, in reality this is
> not broken by design at all and in fact it is the only successful
> strategy that has been deployed by other distros - it's what is being
> called merged-usr-via-moves-and-symlink-farms that is broken. We can
> say this with certainty because both approaches have been tried, so
> there's actual real-world data to look at. Just go look at the absolute
> mess that Suse got themselves into by following that solution - a 10-
> years-long massive half-done-never-finished headache that took an
> inordinate amount of work to back out of, and move to the actual
> working solution that everybody else are using - merged-usr-via-
> aliased-dirs. On the other hand Fedora/RHEL had a smooth and simple
> transition using merged-usr-via-aliased-dirs, and that was the end of
> it.

Yes, a single (a couple!?) data point(s) of a strategy used in another
unrelated distribution, with completely different packaging stacks and
ecosystems, which was done very poorly, has been presented repeatedly.
I'm not sure why that has much value TBH.

The above seems to also be confusing how and if a design has been
deployed, with its inherent (short and long term) properties. A badly
performed *deployment* for a better design does not make its properties
bad, in the same way that *deploying* a flawed design faster does not
make its properties better…

What I've also said multiple times, is that
merged-usr-via-moves-and-symlink-farms could have been implemented in
a fully automated way, by debhelper, w/o requiring any maintainer scripts,
all with full cooperation and managed by dpkg, with .debs shipping
actual tracked pathnames, if it had not been for the mess required
by merged-usr-via-aliased-dirs. :/

> Dpkg has some very minor bugs that rpm/dnf/yum/zypper/whatever do not
> suffer from. So what? It is perfectly normal as it's software and all
> softwares have bugs. They could be fixed, worked around, or ignored,
> like all other bugs.

If by "very minor bugs" you mean that f.ex.:

* dpkg, dpkg-divert, or update-alternatives are unable to detect file
conflicts and thus might allow silent overwrites of random stuff on
disk,
* when moving files across packages and across aliased directories,
these pathnames might end up disappearing depending on the unpack
order,
* dpkg-deb -x on the root directory (yes, people use this to recover
systems) with any .deb that contains files on real directories under
«/», will replace the symlinked directories with real ones,

then, yes, I guess "very minor" indeed. But then I completely object
to this being classified as bugs in dpkg, as this has been shoved in
disregarding that dpkg does *not* support this, and it would require
new *features* to be implemented, so this "transition" is founded on
assuming features that do not exist, or completely going behind
dpkg's back, which sounds great I guess…

The problem in general is that this layout introduces unreliability
and silently induced breakage stemming from this flawed filesystem
layout, which is going to affect the people that are going to benefit
the less from its properties, and are the less experience to deal with
it.

I've tried to update the <https://wiki.debian.org/Teams/Dpkg/MergedUsr>
with more explicit information, in case some people might have missed
that from previous instances of this discussion, and added an initial
table of properties for both proposals, to avoid cluttering and repeating
it on the list, and to ease updating it.




We do actually have experience with "an aliased" layout from symlinked
/usr/share/doc/ directories, where those are for optional/removable parts
of the filesystem, and the symlinks are even known and managed by dpkg.
And those have been a major and constant source of packaging bugs.

And here we are getting the project installing by default systems that
are fighting the packaging system and going on its back, to enable a
filesystem design layout mostly experienced admins will benefit from
in very special deployment conditions, where final users can very
easily suffer from its introduced unreliability (from 3rd-party repos
or locally built packages, etc, f.ex.).

Because the above has been brought up before and the proponents are well
aware of these, I'm afraid at this point the only thing that comes to
mind is negligence, TBH. :/

But *shrug*, have at it, I'm tired of the continued and complete
disregard of the unreliability issues and subversion of the packaging
system, which are supposed to be pillars of our project, every single
time. I just replied so that people that might not want to be forced
into this stuff know that there's going to be a way out.

Regards,
Guillem

[Russ Allbery]

Polyna-Maude Racicot-Summerside <deb...@polynamaude.com> writes:

> I had the belief that some software used /tmp for temporary file that
> may grow many GB (example DVD creation).

> I have 32 GB

It should not, or at least it should let you specify a different path,
because using tmpfs for /tmp is very common these days. (/var/tmp is
available if one needs a file system more likely to be on a larger disk.)

[Marc Haber]

On Mon, 19 Jul 2021 03:36:59 +0200, Guillem Jover <gui...@debian.org>
wrote:

>If by "very minor bugs" you mean that f.ex.:
>
> * dpkg, dpkg-divert, or update-alternatives are unable to detect file
> conflicts and thus might allow silent overwrites of random stuff on
> disk,
> * when moving files across packages and across aliased directories,
> these pathnames might end up disappearing depending on the unpack
> order,
> * dpkg-deb -x on the root directory (yes, people use this to recover
> systems) with any .deb that contains files on real directories under
> «/», will replace the symlinked directories with real ones,
>
>then, yes, I guess "very minor" indeed. But then I completely object
>to this being classified as bugs in dpkg, as this has been shoved in
>disregarding that dpkg does *not* support this, and it would require
>new *features* to be implemented, so this "transition" is founded on
>assuming features that do not exist, or completely going behind
>dpkg's back, which sounds great I guess…

In an ideal world, would the package manager not be a service utility
to SUPPORT policy and adapt to changing environment contitions instead
being a showstopper for innovation?

Who is the dpkg maintainer to challenge the decisions of the entire
project? I fully understand that there is only ONE dpkg maintainer,
but a utility THIS central to the entire ecosystem not being team
maintained is a HUGE part of the problem.

And no, I cannot help and no, you wouldn't want me to write a single
line of code in a package THIS central.

>The problem in general is that this layout introduces unreliability
>and silently induced breakage stemming from this flawed filesystem
>layout, which is going to affect the people that are going to benefit
>the less from its properties, and are the less experience to deal with
>it.

Would it not be dpkg's job to work around these flaws? It's not that
every other component of a Debian system are perfect.

>And here we are getting the project installing by default systems that
>are fighting the packaging system and going on its back, to enable a
>filesystem design layout mostly experienced admins will benefit from
>in very special deployment conditions, where final users can very
>easily suffer from its introduced unreliability (from 3rd-party repos
>or locally built packages, etc, f.ex.).

I must be missing some thing, but isn't it the experienced admins
facing reinstalls of vital systems when we finally move to a
completely merged /usr, because these usually currently have /usr on a
dedicated mountpoint?

>Because the above has been brought up before and the proponents are well
>aware of these, I'm afraid at this point the only thing that comes to
>mind is negligence, TBH. :/

on both sides of the conflict, yes.

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

[Marc Haber]

On Sun, 18 Jul 2021 16:21:24 -0700, Russ Allbery <r...@debian.org>
wrote:

>I agree with the other feedback that you are overpartitioning your disk.

It is especially evident in the df output where there are two-digit
amounts of gigabytes free on most of those HUGE partitions.

>I used to do this back when I was first learning UNIX in the 1990s because
>it seems like a good idea and it does isolate one part of the system from
>another if it uses an excessive amount of space. But what I found in
>practice, and what almost everyone who does this eventually finds in
>practice, is that this much partitioning drastically reduces the long-term
>flexibility of the system. It requires you predict in advance what parts
>of the system will grow, and when you guess wrong, you end up with
>symlinks trying to move directories from a partition with no free space to
>another partition with free space, with all the complexity and breakage
>that can cause.

Nowadays, with LVM, file systems can easily grow, even online. I have
stopped putting /usr on a dedicated file system as the usrmerge began
to show up on the horizon, but I still use dedicated file systems for
/home and /var.

I am NOT looking forward having to manually convert legacy systems to
merged /usr and I do sincerely hope that Debian will choose a way to
get away without throwing away systems that have just a small /, still
supporting a dedicated /usr as long as it's mounted by initramfs. I am
not sure whether we ever issued a clear statement about that.

>There are some technical reasons to separate /boot if you are using a file
>system for other partitions that isn't suitable for early boot (or if
>you're using cryptsetup or other file system layers). /boot/efi is always
>a separate partition because of how it works. Apart from those two
>special cases, the only reason to put something on a separate file system
>is if you have a clear and compelling reason why you expect a given file
>system to run out of space and you want to ensure that it cannot take
>space from other parts of the system.

I also believe that smaller file systems are unlikely to break and
that a system that can boot up to a ssh-able state even with a broken
file system is way easier to fix. We have taken a huge step back in
that regard with systemd since the systemd rescue mode requiring the
"real" root password even for minor startup failures is way more
unfriendly than what we had before. Many installations closely guard
the root password for real emergencies (I have been working on big
installations for years and have NEVER seena case where the "real"
root password was actually used - it is usually easier to rebuild
affected systems from scratch).

>This can be a good justification for putting /home on a separate partition
>*if* you are running a multi-user system. But otherwise, separating out
>things like /var or /usr/local or /opt or /srv is more likely to cause you
>long-term headaches than it is to do anything useful.

I disagree for /var (maybe just for /var/log or parts of /var/lib).

[Stephan Verbücheln]

Nowadays you can also have BTRFS subvolumes, which does not require you
to define sizes in advance. In that case it is nice for snapshotting to
have separate subvolumes for things like home directories.

Regards

[Gunnar Wolf]

Sorry to single you out here, Marc -- This goes to many people. This
goes, in fact, to the discussion itself.

Marc Haber dijo [Mon, Jul 19, 2021 at 07:12:24AM +0200]:

> In an ideal world, would the package manager not be a service utility
> to SUPPORT policy and adapt to changing environment contitions instead
> being a showstopper for innovation?
>
> Who is the dpkg maintainer to challenge the decisions of the entire
> project? I fully understand that there is only ONE dpkg maintainer,
> but a utility THIS central to the entire ecosystem not being team
> maintained is a HUGE part of the problem.
>
> And no, I cannot help and no, you wouldn't want me to write a single
> line of code in a package THIS central.

> (...)

While I agree with what you write here (will answer on a separate
mail), I'll ask you -and everybody- to please moderate the tone. It is
frustrating to speak in different wavelengths and not be able to hear
one another, but we are not going to get anywhere if we just SHOUT
LOUDER using our same wavelength. We must find some alternate
frequencies to get to a constructive situation.

[Gunnar Wolf]

As I said, on a separate mail...

Marc Haber dijo [Mon, Jul 19, 2021 at 07:12:24AM +0200]:

> In an ideal world, would the package manager not be a service utility
> to SUPPORT policy and adapt to changing environment contitions instead
> being a showstopper for innovation?
>
> Who is the dpkg maintainer to challenge the decisions of the entire
> project? I fully understand that there is only ONE dpkg maintainer,
> but a utility THIS central to the entire ecosystem not being team
> maintained is a HUGE part of the problem.
>
> And no, I cannot help and no, you wouldn't want me to write a single
> line of code in a package THIS central.

> (...)

> Would it not be dpkg's job to work around these flaws? It's not that
> every other component of a Debian system are perfect.

FWIW, I mostly agree with what you say here -- If the project decides
to a new standard (and, in this case, it has via the TC's decision --
which can of course be repealed by GR, if things come to that),
packages that behave differently... are buggy and should be fixed.

Of course, dpkg is a very special case for obvious reasons; I did try
to reach out to Guillem when we started discussing the bug at the TC,
and was disheartened by his harsh reply basically negating all
possibility of discussion because his non-belief in the TC itself.

There are technical issues that this migration will bring, and yes,
there is a nonzero chance some users will be bitten by the dissonance
between dpkg and reality. But after two TC bug resolutions (#914897
and #978636), and after lots of bytes have been spilled by various
people, I can only see work has to be put into making possibly
problematic cases less likely -- rebuilding and checking packages
don't ship files in the root directories will cover a great deal of
the distance. If aliasing the directories via symlinks is so messy,
well... we should focus on the root cause, and fix the rasons for it
to be broken as much as we can. The symlinks could probably be an
unconsequential footnote if this is done right.

[Marco d'Itri]

On Jul 19, Marc Haber <mh+debi...@zugschlus.de> wrote:

> I am NOT looking forward having to manually convert legacy systems to
> merged /usr and I do sincerely hope that Debian will choose a way to
> get away without throwing away systems that have just a small /, still
> supporting a dedicated /usr as long as it's mounted by initramfs. I am

They cannot be supported without keeping a lot of unneeded complexity
around, but there is no reason to reinstall them: you can move / inside
/usr instead. You may use either sash or a live CD image.

> I also believe that smaller file systems are unlikely to break and
> that a system that can boot up to a ssh-able state even with a broken
> file system is way easier to fix. We have taken a huge step back in

And "apt install grml-rescueboot" is even better.
Also, with merged-/usr you may keep your *whole* OS in a read only file
system.

--
ciao,
Marco

[Stephan Lachnit]

On Mon, Jul 19, 2021 at 3:37 AM Guillem Jover <gui...@debian.org> wrote:
> What I've also said multiple times, is that
> merged-usr-via-moves-and-symlink-farms could have been implemented in
> a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> all with full cooperation and managed by dpkg, with .debs shipping
> actual tracked pathnames, if it had not been for the mess required
> by merged-usr-via-aliased-dirs. :/

Maybe I get this wrong, but I don't think this conflicts with the
decision from the TC.

Until Debian 12 gets released, we have a lot of time for a transition.
Maybe we should start discussing the transition and less whether or
not we do it, as this has been decided now anyway.

We could start with collecting the packages that install to /bin*
instead of /usr/bin, and adjust the packaging so that they don't do
that. Of course, we would need to add a maintainer script that detects
un-merged usr and creates a symlink. Actually, I think it would be
enough to just let debhelper detect if a package installs to /bin, and
adjust the package automatically. For packages not using debhelper,
lintian can add a warning if the package installs to /bin. After all
packages that installed to /bin have been rebuilt, nothing would
install to /bin except for symlinks. At this point, it shouldn't
matter if you run a merged usr system or not, or am I forgetting
something? IMHO it would make way more sense to discuss how to merge
usr once the packages are fixed.

Anyway, I think the discussion made clear that we shouldn't
immediately start with merging usr once bullseye is released, and I
wouldn't interpret the TC decision as such.

Regards,
Stephan

* using /bin as an example, same goes for /lib etc

[Simon McVittie]

On Mon, 19 Jul 2021 at 11:33:49 +0200, Stephan Lachnit wrote:
> We could start with collecting the packages that install to /bin*
> instead of /usr/bin, and adjust the packaging so that they don't do

> that. [...] At this point, it shouldn't

> matter if you run a merged usr system or not, or am I forgetting
> something?

This would have part of the practical effect of merged-/usr, but not all
of it. It could still be a useful step forwards, but we should not view
it as being entirely equivalent to merged-/usr.

What we have now on unmerged-/usr systems, using /bin/bash and
/usr/bin/perl as examples of Essential programs that use the two paths:

bash perl
/bin/foo physical location (does not exist)
/usr/bin/foo (does not exist) physical location

What we would have on unmerged-/usr systems if we do as you suggest:

bash perl
/bin/foo exists via symlinks (does not exist)
/usr/bin/foo physical location physical location

Merged-/usr for comparison:

bash perl
/bin/foo exists via symlinks exists via symlinks
/usr/bin/foo physical location physical location

As you can see from those tables, a package that hard-codes /usr/bin/bash
is currently considered broken, but would work with either your proposal
or merged-/usr. Conversely, a package that hard-codes /bin/perl would
still be considered broken, would *not* work with your proposal, but
would work on merged-/usr systems.

(Obviously the same applies when considering hard-coded paths in /sbin,
/lib, etc. instead of /bin, in particular the ELF interpreters like
/lib64/ld-linux-x86-64.so.2 that are hard-coded into every
dynamically-linked executable)

Meanwhile, various shared libraries are also moving from /lib to
/usr/lib. One potentially serious problem with that on non-merged-/usr
systems is that we still don't understand why the bugs discussed
in https://bugs.debian.org/911225 and https://bugs.debian.org/949395
happened, and a similar thing could potentially happen to /lib libraries
other than GLib. The script that GLib uses to work around this is generic,
and could be used in other affected packages if required, but it's larger
and more fragile than I'm really comfortable with.

(Merged-/usr systems cannot suffer from bugs like the ones discussed in
#911225, because the paths involved are the same directory.)

smcv

[Michael Biebl]

Am 19.07.21 um 07:23 schrieb Marc Haber:

> I am NOT looking forward having to manually convert legacy systems to
> merged /usr and I do sincerely hope that Debian will choose a way to
> get away without throwing away systems that have just a small /, still
> supporting a dedicated /usr as long as it's mounted by initramfs. I am
> not sure whether we ever issued a clear statement about that.

I think this is a misunderstanding. Files from / would be moved to /usr.
So the only way this could fail is, if your /usr partition was too
small.That's still a possibility for existing systems, but a much
smaller one then moving files from /usr to /. Typically a separate /usr
partition is larger then /.

>> There are some technical reasons to separate /boot if you are using a file
>> system for other partitions that isn't suitable for early boot (or if
>> you're using cryptsetup or other file system layers). /boot/efi is always
>> a separate partition because of how it works. Apart from those two
>> special cases, the only reason to put something on a separate file system
>> is if you have a clear and compelling reason why you expect a given file
>> system to run out of space and you want to ensure that it cannot take
>> space from other parts of the system.
>
> I also believe that smaller file systems are unlikely to break and
> that a system that can boot up to a ssh-able state even with a broken
> file system is way easier to fix. We have taken a huge step back in
> that regard with systemd since the systemd rescue mode requiring the
> "real" root password even for minor startup failures is way more
> unfriendly than what we had before.

I assume you are referring to the sulogin issue here [1], i.e. whether
we require a root password on an emergency failure or not.
Fwiw, this is mostly me being paranoid and not handing out root shells.
This has nothing to do with merged-/usr.


Michael


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802211

[Michael Biebl]

Hi Guillem

Am 19.07.21 um 03:36 schrieb Guillem Jover:

> What I've also said multiple times, is that
> merged-usr-via-moves-and-symlink-farms could have been implemented in
> a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> all with full cooperation and managed by dpkg, with .debs shipping
> actual tracked pathnames

I'm convinced this view is way too naive and not implementable in
practice (and yes, openSUSE is a data point that confirms that)

What you propose is, that each and every package does its /usr-merge
transition on its own. This only works, if packages are independent
(enough) so this actually works.
Unfortunately this is not the case. Take PAM for example. You can't just
recompile src:pam and have debhelper automatically move all files to
/usr. This would break all packages that install a PAM plugin. You have
a transition here, involving many packages.
Same is true for udev rules, systemd service files, basically every
package that provides interfaces/hooks to other packages is affected.
So it's not that simple unfortunately. You can't fully automate that.

According to
apt-file search -x '^/(lib|bin|sbin)'
on my Debian sid/amd64 system, we have 1747 packages shipping 24583
files in those directories. There are *many* such entangled transitions
hidden in there, so I fear this is not manageable.
As Luca pointed out, even distros with a much stricter governance model
were not able to do that.

The /usr-merge transition as described and decided on in the TC bug,
seems to me is the only viable way forward.

Yes, it does break dpkg -S, but your idea of using a list of mapped
paths as in [1] seems like an entirely reasonable approach to solve this.

Once we have this global switch to merged-usr, packages can bit by bit,
completely independent, update their debian/rules to use --prefix=/usr
and after a few years, we don't have any packages anymore installing any
files in /. We could aid this process with a lintian check that flags
packages that install files in /(sbin|bin|lib)/.




Regards,
Michael


[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=858331#33

[Johannes Schauer Marin Rodrigues]

Quoting Michael Biebl (2021-07-19 15:10:42)

> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames
> I'm convinced this view is way too naive and not implementable in
> practice (and yes, openSUSE is a data point that confirms that)
>
> What you propose is, that each and every package does its /usr-merge
> transition on its own. This only works, if packages are independent
> (enough) so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just
> recompile src:pam and have debhelper automatically move all files to
> /usr. This would break all packages that install a PAM plugin. You have
> a transition here, involving many packages.
> Same is true for udev rules, systemd service files, basically every
> package that provides interfaces/hooks to other packages is affected.
> So it's not that simple unfortunately. You can't fully automate that.
>
> According to
> apt-file search -x '^/(lib|bin|sbin)'
> on my Debian sid/amd64 system, we have 1747 packages shipping 24583
> files in those directories.

more precisely, on amd64 unstable:

/bin 247 files
/lib{32,64,x32} 83 files
/lib/firmware 2379 files
/lib/live 115 files
/lib/modules 17500 files
/lib/systemd 2221 files
/lib/udev 614 files
/lib/x86_64-linux-gnu 343 files
/lib/* 441 files
/sbin 547 files

So most files come from /lib/modules, where only 14 packages are involved,
/lib/systemd which will be fixed by an update to dh_installsystemd, and
/lib/firmware where only 36 packages are involved. The remainder then doesn't
sound so scary anymore as it only involves 656 unique packages and not 1747.
And again many of those remaining packages will be fixed by an update to
debhelper, correct? Given that 90% of source packages use dh, that would reduce
the number to a very manageable size.

> There are *many* such entangled transitions
> hidden in there, so I fear this is not manageable.
> As Luca pointed out, even distros with a much stricter governance model
> were not able to do that.
>
> The /usr-merge transition as described and decided on in the TC bug,
> seems to me is the only viable way forward.
>
> Yes, it does break dpkg -S, but your idea of using a list of mapped
> paths as in [1] seems like an entirely reasonable approach to solve this.
>
> Once we have this global switch to merged-usr, packages can bit by bit,
> completely independent, update their debian/rules to use --prefix=/usr
> and after a few years, we don't have any packages anymore installing any
> files in /. We could aid this process with a lintian check that flags
> packages that install files in /(sbin|bin|lib)/.

So what what is actually the roadmap after the bullseye release? What is the
way forward? Should I rather file bugs with patches against individual packages
to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
already have a debhelper patch to do that move for us?

This would mean that we only have to bear with the problems mentioned by
guillem until that move is complete, correct?

And another question: once there are no more files shipped by any package in
/(sbin|bin|lib)/ we can let base-file create the symlinks?

Thanks!

cheers, josch

[Marc Haber]

On Mon, 19 Jul 2021 15:19:32 +0200, Michael Biebl <bi...@debian.org>
wrote:

>Am 19.07.21 um 07:23 schrieb Marc Haber:
>> I am NOT looking forward having to manually convert legacy systems to
>> merged /usr and I do sincerely hope that Debian will choose a way to
>> get away without throwing away systems that have just a small /, still
>> supporting a dedicated /usr as long as it's mounted by initramfs. I am
>> not sure whether we ever issued a clear statement about that.
>
>I think this is a misunderstanding. Files from / would be moved to /usr.
>So the only way this could fail is, if your /usr partition was too
>small.That's still a possibility for existing systems, but a much
>smaller one then moving files from /usr to /. Typically a separate /usr
>partition is larger then /.

Right, that sounds much easier. It's still an Open Heart Operation,
especially for systems that I don't have out of band access for, which
is rather common for smaller installations.

>I assume you are referring to the sulogin issue here [1], i.e. whether
>we require a root password on an emergency failure or not.

Yes. From my point of view, this is taking away a freedom from the
local admin.

In an ideal world, boot failure behavior would be locally
configurable.

A mis-booted sysv system, if I remember correctly (I have been a
mostly happy systemd user for already quite some time), could be told
to "just try to continue and show me how far you get", which in the
vast majority of cases led to a regular login prompt from which the
user-login-plus-sudo routine just worked. This didn't hand out any
more root shells than the current way of stopping dead and refusing to
do anything without the "real" root password, as far as I understand.
I probably don't have enough experience to have the final call on
that. But it's just a pet peeve of mine that I can easily live with.

>Fwiw, this is mostly me being paranoid and not handing out root shells.

It's good to be paranoid by default. It's bad to force that paranoia
on the local admin who might have a choice to move to a different
distribution. But alas, the others do it the same way. So it's just
freedom lost.

>This has nothing to do with merged-/usr.

I never said it has.

[Simon McVittie]

On Mon, 19 Jul 2021 at 16:41:42 +0200, Johannes Schauer Marin Rodrigues wrote:
> Should I rather file bugs with patches against individual packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/

As discussed in previous iterations of the ongoing merged-/usr megathread,
I don't think this can be the whole solution, because some packages have
paths outside /usr that are part of their Essential functionality.
Prominent examples include /bin/bash and /lib64/ld-linux-x86-64.so.2.
See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118 for
full details.

smcv

[Niels Thykier]

Johannes Schauer Marin Rodrigues:

> Quoting Michael Biebl (2021-07-19 15:10:42)

>> [...]

>>
>> According to
>> apt-file search -x '^/(lib|bin|sbin)'
>> on my Debian sid/amd64 system, we have 1747 packages shipping 24583
>> files in those directories.
>
> more precisely, on amd64 unstable:
>
> /bin 247 files
> /lib{32,64,x32} 83 files
> /lib/firmware 2379 files
> /lib/live 115 files
> /lib/modules 17500 files
> /lib/systemd 2221 files
> /lib/udev 614 files
> /lib/x86_64-linux-gnu 343 files
> /lib/* 441 files
> /sbin 547 files
>
> So most files come from /lib/modules, where only 14 packages are involved,

and debhelper's dh_installmodules that will need to be tweaked to look
into /usr/lib/modules as well.
But I have no idea if/when we would be ready for that and I will not
be changing debhelper until we are ready to move these bits to /usr.

Likewise for udev, here dh_installudev still uses /lib/udev and will
continue to do so until I know that udev also checks /usr/lib/udev.


If you are involved in udev or/and /lib/modules and know that they are
ready to move to the relevant directory under /usr, then please feel
free to file a bug against debhelper requesting that the /usr directory
will be used in the future.
Please do also note in that bug report whether you also want debhelper
to automatically migrate existing files from /lib to /usr/lib. This
should only be the case where we expect (almost) zero breakage from an
automatic "unordered" transition.

(NB: Please use a bug report as I will first be implementing this after
the bullseye release and rely to this thread is likely to be lost in my
inbox)

> /lib/systemd which will be fixed by an update to dh_installsystemd, and

Indeed, I have heard this request and the systemd maintainers confirmed
to me that systemd in bullseye checks both /lib/systemd and
/usr/lib/systemd.

I have a branch lying around trying to support this. The main key
feature missing is the migration of /lib/systemd -> /usr/lib/systemd
(which needs to handle that both exist and merge them into one somehow).

> /lib/firmware where only 36 packages are involved. The remainder then doesn't
> sound so scary anymore as it only involves 656 unique packages and not 1747.
> And again many of those remaining packages will be fixed by an update to
> debhelper, correct? Given that 90% of source packages use dh, that would reduce
> the number to a very manageable size.
>

Indeed, about 90% of all packages uses dh according to trends.

Though for good measure I thought I would explicitly answer the proposal
(from another mail in this thread) that debhelper could move everything
from /lib to /usr/lib:

No, I will not support an unconditional move from /lib to /usr/lib
via debhelper during bookworm.

There are already two distinct examples in this thread for how this
could break things. I will be sticking to "targeted" migrations where
the major consumers have informed me that they are ready to support the
migration.


>> [...]

>>
>> Once we have this global switch to merged-usr, packages can bit by bit,
>> completely independent, update their debian/rules to use --prefix=/usr
>> and after a few years, we don't have any packages anymore installing any
>> files in /. We could aid this process with a lintian check that flags
>> packages that install files in /(sbin|bin|lib)/.
>
> So what what is actually the roadmap after the bullseye release? What is the
> way forward? Should I rather file bugs with patches against individual packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
> already have a debhelper patch to do that move for us?
>

For some, debhelper will your problem but there will packages that will
need manual migration. As I see it, there will *not* be "the debhelper
patch to fix them all" - even if there will /some/ debhelper patchers
that will fix "most of them".

Related, I have no intention of supporting / maintaining a rewrite of
"--prefix/PREFIX" parameters to configure/make/cmake or whatever. (Not
saying anyone proposed it - but mentioning it as another "there will
definitely be manual clean up" data point).

~Niels

[Guillem Jover]

On Mon, 2021-07-19 at 15:10:42 +0200, Michael Biebl wrote:
> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames

> What you propose is, that each and every package does its /usr-merge
> transition on its own. This only works, if packages are independent (enough)
> so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just
> recompile src:pam and have debhelper automatically move all files to /usr.
> This would break all packages that install a PAM plugin. You have a
> transition here, involving many packages.
> Same is true for udev rules, systemd service files, basically every package
> that provides interfaces/hooks to other packages is affected.
> So it's not that simple unfortunately. You can't fully automate that.

Not at all. pam or whatever we transition via cooperation from dpkg,
would be kept compiling using the directories it currently uses, and
debhelper would simply move the objects on the .deb from «/» to «/usr»,
and create the compat symlinks. That means pam, and any plugins migrated
or not, would still be available in the current pathnames causing no
breakage. This part would be done automatically.

And as I've said elsewhere, removal of the compat symlinks would
require manual intervention (at the maintainer discretion), at some
later time, to modify the configured installation paths (which is
something that cannot be automated in debhelper, due to packaging
overrides and similar), at which point it can be decided whether to
declare say a local pam flag day, or add the needed Breaks and similar,
or add support to pam to look in both pathnames to avoid the two
previous items.

> According to
> apt-file search -x '^/(lib|bin|sbin)'
> on my Debian sid/amd64 system, we have 1747 packages shipping 24583 files in
> those directories. There are *many* such entangled transitions hidden in
> there, so I fear this is not manageable.

See my comment above, and josch reply.

> As Luca pointed out, even distros with a much stricter governance model were
> not able to do that.

Well if they did it poorly, no wonder, I guess.

> The /usr-merge transition as described and decided on in the TC bug, seems
> to me is the only viable way forward.

I obviously disagree.

> Yes, it does break dpkg -S, but your idea of using a list of mapped paths as
> in [1] seems like an entirely reasonable approach to solve this.

Did you miss the section in the mail you are replying where I mention
that f.ex. dpkg tools (dpkg, dpkg-divert, u-a) missing to then detect
file conflicts, or files disappearing on moves, or dpkg-deb -x (or tar)
destroying merged-/usr-via-aliased-dirs systems?

All these, including dpkg -S (which BTW also breaks after paths have
been moved, as when say bash ships as /usr/bin/bash, then dpkg -S will
also fail to find the expected /bin/bash), are currently affecting any
system being installed by the default installer, or ones having been
switched by the usrmerge hack.

> Once we have this global switch to merged-usr, packages can bit by bit,
> completely independent, update their debian/rules to use --prefix=/usr and
> after a few years, we don't have any packages anymore installing any files
> in /. We could aid this process with a lintian check that flags packages
> that install files in /(sbin|bin|lib)/.

The same can be said about my proposal, except that then dpkg is kept
in the loop, the transition is done safely by dpkg as part of individual
package upgrades, instead of having to use something off-band and as
disturbing as usrmerge during upgrade, on dpkg's back w/o any
cooperation from it…

Regards,
Guillem

[Guillem Jover]

On Mon, 2021-07-19 at 16:41:42 +0200, Johannes Schauer Marin Rodrigues wrote:
> So what what is actually the roadmap after the bullseye release? What is the
> way forward? Should I rather file bugs with patches against individual packages
> to move their files from /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we
> already have a debhelper patch to do that move for us?

Unfortunately, when the supporters of the merged-/usr-via-aliased-dirs
pushed their approach into the distribution, that meant that package
stopped being able to ship compatibility symlinks under «/», and those
needed to be "handled" in maintscripts (by reimplementing poorly and
unsafely what dpkg is supposed to do). This means dpkg is not in the
loop and cannot perform a safe upgrade moving these pathnames safely,
as long as merged-/usr-via-aliased-dirs is supported.

This is all kinds of nasty, because a proposal that has a complete
disregard for the packaging system, and is founded on assumptions
about features that do not even exist, is also forcing that design
breakage into a solution that would otherwise require no such hacks.
:(

> This would mean that we only have to bear with the problems mentioned by
> guillem until that move is complete, correct?

No. Even if today we magically could get all .debs shipping object
directly under «/usr» instead of «/», and had the aliased symlinks in
place, the problems I've mentioned would pretty much be in place.

Regards,
Guillem

[Guillem Jover]

On Tue, 2021-07-20 at 11:31:37 +0200, Guillem Jover wrote:
> Unfortunately, when the supporters of the merged-/usr-via-aliased-dirs
> pushed their approach into the distribution, that meant that package
> stopped being able to ship compatibility symlinks under «/», and those
> needed to be "handled" in maintscripts (by reimplementing poorly and
> unsafely what dpkg is supposed to do). This means dpkg is not in the
> loop and cannot perform a safe upgrade moving these pathnames safely,
> as long as merged-/usr-via-aliased-dirs is supported.

Sorry, how sloppy of me, let me qualify that last word, to avoid any
confusion, "supported". As in inflicting that into unsuspecting users,
and in having to bear and pay the price for its current existence, not
as in an approach that is supported by the packaging system.

Regards,
Guillem

[Andreas Metzler]

On 2021-07-20 Guillem Jover <gui...@debian.org> wrote:
> On Mon, 2021-07-19 at 16:41:42 +0200, Johannes Schauer Marin Rodrigues wrote:
>> So what what is actually the roadmap after the bullseye release?
>> What is the way forward? Should I rather file bugs with patches
>> against individual packages to move their files from
>> /(sbin|bin|lib)/ to /usr/(sbin|bin|lib)/ or do we already have a
>> debhelper patch to do that move for us?

> Unfortunately, when the supporters of the merged-/usr-via-aliased-dirs
> pushed their approach into the distribution, that meant that package
> stopped being able to ship compatibility symlinks under «/», and those
> needed to be "handled" in maintscripts (by reimplementing poorly and
> unsafely what dpkg is supposed to do). This means dpkg is not in the
> loop and cannot perform a safe upgrade moving these pathnames safely,
> as long as merged-/usr-via-aliased-dirs is supported.

[...]

Hello,
Isn't this kind of crying over spilt milk? I also wish we never had ended
up with the buster/bullseye state where both unmerged and
merged-/usr-via-aliased-dirs are fully supported. However there is now a
huge number of merged-/usr-via-aliased-dirs installations out there and
we cannot make them magically disappear. Undoing
merged-/usr-via-aliased-dirs would be very error-prone while afaiui we
have a relatively simple plan to get a clean merged /usr in bookworm or
bookworm +1:

1. Make merged-/usr-via-aliased-dirs the only supported layout and make
this information available to apt. (Like we did for multi-arch-support.)
2. After that individual packages can safely move files from / to /usr,
pre-depending on merged-usr-support.

cu Andreas
--
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

[Svante Signell]

On Tue, 2021-07-20 at 13:41 +0200, Andreas Metzler wrote:
>
> Hello,
> Isn't this kind of crying over spilt milk? I also wish we never had
> ended up with the buster/bullseye state where both unmerged and
> merged-/usr-via-aliased-dirs are fully supported. However there is
> now a huge number of merged-/usr-via-aliased-dirs installations out
> there and we cannot make them magically disappear. Undoing
> merged-/usr-via-aliased-dirs would be very error-prone while afaiui
> we have a relatively simple plan to get a clean merged /usr in
> bookworm or bookworm +1:

According to the dpkg developer and maintainer Guillem users can still
rescue their systems from merged-/usr-via-aliased-dirs with the aid of
dpkg-fsys-usrunmess(8), see
https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F

I for one will always use that whenever I (accidentally?) install
Debian in the future. All my old installations does not carry this bug.
Or is there some easier way to avoid merged-/usr except using --no-
merged-usr in debootstrap with new installations as written in
https://wiki.debian.org/Teams/Dpkg/MergedUsr?

[Marc Haber]

On Tue, 20 Jul 2021 14:47:04 +0200, Svante Signell
<svante....@gmail.com> wrote:
>According to the dpkg developer and maintainer Guillem users can still
>rescue their systems from merged-/usr-via-aliased-dirs with the aid of
>dpkg-fsys-usrunmess(8), see
>https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F

The naming of the utility alone gives me the impression that we have a
dpkg maintainer who has gone to war with the rest of the distribution.
I am not sure whether Debian should accept that.

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-20 3:30 p.m., Marc Haber wrote:
> On Tue, 20 Jul 2021 14:47:04 +0200, Svante Signell
> <svante....@gmail.com> wrote:
>> According to the dpkg developer and maintainer Guillem users can still
>> rescue their systems from merged-/usr-via-aliased-dirs with the aid of
>> dpkg-fsys-usrunmess(8), see
>> https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
>
> The naming of the utility alone gives me the impression that we have a
> dpkg maintainer who has gone to war with the rest of the distribution.
> I am not sure whether Debian should accept that.
>

I think that the option "usrunmess" says pretty much the state of mind
regarding the person who named it this way.

Something like "usrunmerge" would be more realistic.
> Greetings
> Marc

[Svante Signell]

On Tue, 2021-07-20 at 15:34 -0400, Polyna-Maude Racicot-Summerside
wrote:

> Hi,
>
> On 2021-07-20 3:30 p.m., Marc Haber wrote:
> > On Tue, 20 Jul 2021 14:47:04 +0200, Svante Signell
> > <svante....@gmail.com> wrote:
> > > According to the dpkg developer and maintainer Guillem users can
> > > still rescue their systems from merged-/usr-via-aliased-dirs with
> > > the aid of dpkg-fsys-usrunmess(8), see
> > > https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Does_dpkg_support_merged-.2Fusr-via-aliased-dirs.3F
> >
> > The naming of the utility alone gives me the impression that we
> > have a dpkg maintainer who has gone to war with the rest of the
> > distribution. I am not sure whether Debian should accept that.
> >
> I think that the option "usrunmess" says pretty much the state of
> mind regarding the person who named it this way.
>
> Something like "usrunmerge" would be more realistic.

It is really stunning that the Debian project, including the TC
overrides the dpkg developer and maintainer Guillem, and still using
dpkg for package management. Maybe Debian should switch to some other
software, like rpm-based used by Fedora or even guix used by GNU?? Or
perhaps the Debian project should come to senses with the force of
things upon users and developers/maintainers without their approval
(consensus-free)?

Debian, the Universal Operating System was used some years ago!

[Holger Levsen]

On Tue, Jul 20, 2021 at 11:15:33PM +0200, Svante Signell wrote:
[...]

> Debian, the Universal Operating System was used some years ago!

Svante, fine. You are unhappy with Debian since years, you're not using it
anymore, you are not contributing, this is debian-devel@ not debian-rant@,
so please STFU.


--
cheers,
Holger

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄

The upcoming clima apocalypse is the big elephant in every room now.

[Svante Signell]

On Tue, 2021-07-20 at 21:51 +0000, Holger Levsen wrote:
> On Tue, Jul 20, 2021 at 11:15:33PM +0200, Svante Signell wrote:
> [...]
> > Debian, the Universal Operating System was used some years ago!
>
> Svante, fine. You are unhappy with Debian since years, you're not
> using it anymore, you are not contributing, this is debian-devel@ not
> debian-rant@, so please STFU.

Hi Holger, I would have expected a reply like this from you. I do still
use Debian, some of my boxes are still Debian-based. Soon they will
probably be converted to Devuan though. I do still contribute to
Debian, mainly to Debian GNU/Hurd and Debian GNU/kFreeBSD. As long as
these ports are still alive within Debian I won't go away.

Holger (and others), please consider the arguments and facts given in
the latest postings to this thread, these issues are serious and not
something you just can easily close with an STFU argument to me!

Thanks!

[Holger Levsen]

On Wed, Jul 21, 2021 at 12:09:52AM +0200, Svante Signell wrote:
> Hi Holger, I would have expected a reply like this from you. I do still
> use Debian, some of my boxes are still Debian-based. Soon they will
> probably be converted to Devuan though. I do still contribute to
> Debian, mainly to Debian GNU/Hurd and Debian GNU/kFreeBSD.

ok, cool! & sorry for assuming you didn't contribute!

> Holger (and others), please consider the arguments and facts given in
> the latest postings to this thread, these issues are serious and not
> something you just can easily close with an STFU argument to me!

well, sigh, your last mail (to which I replied) was just ranting,
which I still think is wrong here. (Once ok, twice too/maybe, ...)

however, this ship has sailed, there has been a proper discussion by
a proper process and if you don't like it, please start a GR, as this
is the way to overrule the TC, if there is a majority for it.

writing mails to debian-devel will not change anything. and repeating
a discussion every other month without new arguments will just annoy
people as one can see here.

--
cheers,
Holger

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org
⢿⡄⠘⠷⠚⠋⠀ OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
⠈⠳⣄

Words may inspire but only action creates change.

[Polyna-Maude Racicot-Summerside]

Hi,

> Hi Holger, I would have expected a reply like this from you. I do still
> use Debian, some of my boxes are still Debian-based. Soon they will
> probably be converted to Devuan though. I do still contribute to
> Debian, mainly to Debian GNU/Hurd and Debian GNU/kFreeBSD. As long as
> these ports are still alive within Debian I won't go away.
>

I'd love to know more about what made you switch to Devuan and what
could I also benefit from using Debian/kFreeBSD.
I used to have boxes on FreeBSD long time ago (6.0 I think).

Thanks

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-07-20 5:51 p.m., Holger Levsen wrote:
> On Tue, Jul 20, 2021 at 11:15:33PM +0200, Svante Signell wrote:
> [...]
>> Debian, the Universal Operating System was used some years ago!
>
> Svante, fine. You are unhappy with Debian since years, you're not using it
> anymore, you are not contributing, this is debian-devel@ not debian-rant@,
> so please STFU.
>

You are so nice with people who have a different opinion than yours.
This is really a example of someone who's mature.

[Polyna-Maude Racicot-Summerside]

Hi,

> It is really stunning that the Debian project, including the TC
> overrides the dpkg developer and maintainer Guillem, and still using
> dpkg for package management. Maybe Debian should switch to some other
> software, like rpm-based used by Fedora or even guix used by GNU?? Or
> perhaps the Debian project should come to senses with the force of
> things upon users and developers/maintainers without their approval
> (consensus-free)?
>
> Debian, the Universal Operating System was used some years ago!
>

It's really the kind of problem that make Debian look like a crazy non
sense operating system. People using program options as a way to shout
their opinion to the world...
It's a problem with many FOSS project that benefits mainly from
volunteer work. Because they need to job done, they can't have any
coercion against developer (as a difference, your boss can impose you
things). This may be a good thing as it promote creativity and sometime
allow emergence of better practice.
But in other case, most of the time, it just give a bad look.

Reminds me of when I worked with the Tiki CMS project. I was using node
with some tools to do pre-processing of JS code and HTML validation. So
I added my package.json to the distribution, in case other developer
want it.
Ended up with a 3 month useless discussion regarding if this would give
a bad impression, that we need to use node for doing development.
Later on I was working on a plugin that treated huge amount of data. So
I introduced Vue.JS, again a three month discussion with people saying
it's a overhead, even after explaining that you can't always do server
side processing and serve all the data at a time. Even if people didn't
understand a word about the global concept and the system as a whole,
they halted the project.
So I just let go my participation.
Now 4 years later, they are integrating Vue.JS and Node doing code
validation while development.

One of the main reason some people don't want to invest time in FOSS
project is exactly because of that type of toxic situation that make
everyone look like crazy nut head.

Sometime we also have to accept choices we dislike (hey, we teach this
to kids).

[Brian Thompson]

On Tue, 2021-07-20 at 21:13 -0400, Polyna-Maude Racicot-Summerside
wrote:

> Ended up with a 3 month useless discussion regarding if this would
> give
> a bad impression, that we need to use node for doing development.
> Later on I was working on a plugin that treated huge amount of data.
> So
> I introduced Vue.JS, again a three month discussion with people
> saying
> it's a overhead, even after explaining that you can't always do
> server
> side processing and serve all the data at a time. Even if people
> didn't
> understand a word about the global concept and the system as a whole,
> they halted the project.
> So I just let go my participation.
> Now 4 years later, they are integrating Vue.JS and Node doing code
> validation while development.
>
> One of the main reason some people don't want to invest time in FOSS
> project is exactly because of that type of toxic situation that make
> everyone look like crazy nut head.

I find it pretty crazy that you think FOSS is toxic, when in reality
the only toxic thing about this situation was when you labeled a
discussion as "useless". They didn't want to use your technology stack
four years ago. Now they do. Throwing a temper tantrum when you don't
get your way isn't doing anyone else any favors. 

P.S. You may be the biggest hippocrit in these mailing lists.

--
BT

[Polyna-Maude Racicot-Summerside]

Hi,

You either cherrypicked or only took what you wanted in what I said.

So I'll do it again...

I was asked by one of the project manager to implement a new plugin that
would allow some machine learning and data processing (using chart with
plotly.js using database on a CMS).
To do so efficiently, I had to use some type of client side processing
and decided to use Vue.JS.

This didn't change nothing for anyone except me.

But all the developers started saying it wasn't good because it would
project the idea that we need to use Node to do development (false) or
that Vue.JS is needed for the CMS (false).

Most of the person talking about this were people who didn't have a clue
what they were talking about. Mostly people doing translation for text
string, HTML/CSS front end themes and such. Most of the time, when we
talked about PHP they would say "that they ain't developer".

For you information, I didn't throw a tamper tantrum, I simply felt that
my time would be better invested somewhere I could use my time in a
positive manner.

Having to explain that "It's not because there's a package.json file
that you'll need Node ecosystem. As there's already a .gitla-ci.yml and
you don't need to have a Gitlab runner on your PC to do development".
Having to explain that sending all the data at one time, when processing
thousands of record is not a possibility if you want to have a system
that can be scaled up. And much more...
This is time consuming and doesn't help much going a project forward.

It wasn't a technology stack that I've chosen but the one that was
requested by the project manager who paid me. But I had to deal with
those questions on the mailing list.

And I did get tired so went somewhere else...

So we we're in front of people that (like many) had a opinion but didn't
have much to support it.

I don't know many business we're we pass more time discussing about
solution than implementing them. Unless, I got it wrong, and FOSS
project are some type of IT consultant box ?

I come from health science and we do stuff, don't smoke butterfly powder
thinking about what would be the best in a ideal world. I don't think
about what's the best solution for a ideal world.

And if I was running Windows for the computer used for medical purpose
in my office, there was one reason : I didn't have nothing that made if
efficient for me to go Linux. All the advantage were overwhelmed by the
time consuming of process changing, staff training or simply because no
such solution existed.

If all the time wasted in fork was used to making project grow better
then it'd be long ago that we wouldn't see as many Windows.

> P.S. You may be the biggest hippocrit in these mailing lists.
>

You don't have a clue who I am...
So let me put you in the box where it says "I talk about what I don't
know and think I'm soooo great".

[Marc Haber]

On Tue, 20 Jul 2021 23:15:33 +0200, Svante Signell
<svante....@gmail.com> wrote:
>It is really stunning that the Debian project, including the TC
>overrides the dpkg developer and maintainer Guillem, and still using
>dpkg for package management. Maybe Debian should switch to some other
>software, like rpm-based used by Fedora or even guix used by GNU??

This suggestion doesnt get any smart by repeating it.

[Simon McVittie]

On Tue, 20 Jul 2021 at 21:16:57 -0400, Polyna-Maude Racicot-Summerside wrote:
> I'd love to know more about what made you switch to Devuan

You're welcome to have this discussion privately, but please take it
off-list. A few days before the provisional release date for Debian 11
is not the time to light more fires on -devel.

Back to the topic of merged-/usr, the first step in any reasonable plan
to move on from the current situation - whether in favour of merged-/usr
or against it - is going to be "get Debian 11 released". So let's do that?

At this point in the release cycle in particular, I would ask anyone
who is not able to contribute directly to getting a high-quality Debian
11 release to contribute by trying not to distract the people who are
making it happen.

Thanks,
smcv

[Thorsten Glaser]

Andreas Metzler dixit:

>1. Make merged-/usr-via-aliased-dirs the only supported layout and make
>this information available to apt. (Like we did for multi-arch-support.)
>2. After that individual packages can safely move files from / to /usr,
>pre-depending on merged-usr-support.

This will still break “dpkg -S $(which programname)”, which I use a lot.
And tons of other stuff. All aliasing schemes will. Just keep supporting
unmerged filesystems and requiring /usr to be there at the time control
is handed to init(8).


A little bit more positively, as I recently switched my sid systems to
bullseye, I was wondering which packages I now have to manually down‐
grade; additionally, a lot of “dust” they had accumulated over time.
I am vaguely aware of aptitude having parts of this but don’t use it,
so here we are:

https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/debian-dev/aptcheck;hb=HEAD

This little tool (the shellsnippets.git repository is also mirrored
to github for those who prefer there) asks dpkg for the status of all
packages (or those listed as arguments), complains about all which are
not ii or hi, and for those it checks (via apt-cache policy as that was
easier/more straightforward than apt-cache showpkg) whether the installed
version is available from any repository and up-to-date (ignoring back‐
ports{,-sloppy}); neighbouring versions which *are* in repositories are
shown as well (both bpo and not; for both, the respective highest one).

This script needs…

https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/progress-bar;hb=HEAD
(also on github in the same repository, or in MirBSD CVS)

… to be in the same or parent directory to display the progress bar
which makes the long wait (I had 100% CPU utilisation on one core by
apt-cache alone) bearable. You can redirect stdout still, though ☻

It found a surprising amount of packages I hope the maintainers filed
unblock requests for ;-)

Improvements welcome… that don’t involve rewriting this in a programming
language beginning with b or p anyway ;-) also, comments.

I can imagine it being useful in all sorts of situations, not just the
one I’m currently using it for. (Also, “what packages I use were removed
from Debian?” etc.)

Development was sponsored by ⮡ tarent solutions GmbH

Enjoy,
//mirabilos
--
Infrastrukturexperte • tarent solutions GmbH
Am Dickobskreuz 10, D-53121 Bonn • http://www.tarent.de/
Telephon +49 228 54881-393 • Fax: +49 228 54881-235
HRB AG Bonn 5168 • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

[Barak A. Pearlmutter]

Seriously? Being able to type

dpkg -S $(which cat)

instead of

dpkg -S $(which cat|sed 's:^/usr::')

is the big user-level pain point?

People seem pretty worked up over this, but honestly I'm not
understanding why. We already have $PATH which (let's be honest) is an
ancient crappy workaround for the original Unix sin of not keeping all
the executables in one place. (And analogous wheel-reinventing goo for
/lib vs /usr/lib vs /usr/lib/x86_64-linux-gnu, etc.) Given that,
moving them around isn't supposed to be a big fuss. Oh, but there are
also shebang #!/bin/foo issues and other hard coding. The shebang is
already such a mess that people use #!/usr/bin/env foo, just to get
foo searched in $PATH. (85 of the 890 scripts in /usr/bin/* on the
machine I'm typing this on use a /usr/bin/env trampoline.) Nobody
would ever consider having /bin/foo and /usr/bin/foo be different
programs, that would be madness. The TC basically concluded that the
distinction between /bin and /usr/bin, etc, had totally broken down
and there's no advantage to keeping them distinct, plus initrd is the
new /bin. (Pretty much the same argument as in the design of plan9.)
I'm not seeing much argument against that, except for nostalgia.

It seems that dpkg is ground zero for problems that occur during the
transition itself, and for the problem of handling the transition
gracefully. I for one am enormously impressed with the quality of
dpkg: its amazing robustness, the way it's evolved to support hooks
and such and really been the cornerstone of the distribution. So I
think we should take the dpkg maintainer's concerns seriously. But
frankly I don't quite understand them.

--Barak Pearlmutter.

[Marco d'Itri]

On Jul 22, "Barak A. Pearlmutter" <b...@debian.org> wrote:

> People seem pretty worked up over this, but honestly I'm not
> understanding why.

Because they are scared by new things, hence they oppose merged-/usr
(and systemd before that, and udev even before...).
But since they cannot admit that, then they need to rationalize by
inflating the effects of minor bugs or usability issues.

Basically all new systems are installed with merged-/usr since Debian
10, and since obviously this has not caused noticeable issues (or at
least, any which could not be fixed) then we must treat with skepticism
any such claims.

--
ciao,
Marco

[Guillem Jover]

[ Barak, I appreciate your mail, but *sigh*, it's still frustrating,
as pretty much many of the mails related to this, as they keep
ignoring what has been said, and I feel like talking to a wall. :/ ]

On Thu, 2021-07-22 at 11:48:34 +0100, Barak A. Pearlmutter wrote:
> Seriously? Being able to type
>
> dpkg -S $(which cat)
>
> instead of
>
> dpkg -S $(which cat|sed 's:^/usr::')
>
> is the big user-level pain point?

No. As I've mentioned before, f.ex.:

* dpkg, dpkg-divert, or update-alternatives are unable to detect file
conflicts and thus might allow silent overwrites of random stuff on
disk (3rd-party/local packages where we have no control over, or
even packages from old Debian releases come to mind),
* when moving files across packages and across aliased directories,
these pathnames might end up disappearing depending on the unpack
order (new/old Debian or 3rd-party/local packages too),
* dpkg-deb -x on the root directory (yes, people use this to recover
systems) with any .deb that contains files on real directories under
«/» (3rd-party/local packages), will replace the symlinked directories
with real ones,
* dpkg-statoverride used with aliased pathnames that exist on disk
but unknown to dpkg, will fail to apply the overrides (itself and
dpkg on unpack), this could have security implications f.ex.,
* dpkg-triggers used with aliased pathnames that exist on disk but
unknown to dpkg, will fail to activate triggers,

Also, yes, anything using dpkg-query is now broken, take for example the
cruft package, the problem is that there are way more things depending
on this interface.

But the aliasing problem is a general one. Take the /etc/shells
mentioned recently on d-d, the admin needs to remember to always add
two entries (/ and /usr) to make sure things will match by any random
program accessing the file for its checks.


Regarding your dpkg-query example above, using which(1) and assuming
a PATH set to prefer /usr directories (which AFAIR is the current
default) would mean that once all objects have moved in the .debs into
/usr, the former would always work, but at that point the latter would
not. Also that assumes people would use which(1) or similar, or even
fully canonicalize the pathname, which might or might not have anything
to do with what dpkg knows about on its database.

> People seem pretty worked up over this, but honestly I'm not
> understanding why. We already have $PATH which (let's be honest) is an
> ancient crappy workaround for the original Unix sin of not keeping all
> the executables in one place. (And analogous wheel-reinventing goo for
> /lib vs /usr/lib vs /usr/lib/x86_64-linux-gnu, etc.) Given that,
> moving them around isn't supposed to be a big fuss. Oh, but there are
> also shebang #!/bin/foo issues and other hard coding. The shebang is
> already such a mess that people use #!/usr/bin/env foo, just to get
> foo searched in $PATH. (85 of the 890 scripts in /usr/bin/* on the
> machine I'm typing this on use a /usr/bin/env trampoline.) Nobody
> would ever consider having /bin/foo and /usr/bin/foo be different
> programs, that would be madness. The TC basically concluded that the
> distinction between /bin and /usr/bin, etc, had totally broken down
> and there's no advantage to keeping them distinct, plus initrd is the
> new /bin. (Pretty much the same argument as in the design of plan9.)
> I'm not seeing much argument against that, except for nostalgia.

In case this is not clear, and as I've mentioned also countless times
already, I have no problem with merging contents of directories from /
into /usr, I've done that for all packages I maintain (where no compat
symlinks were required). I do have a huge problem with the approach
that has been forced into the distribution disregarding the packaging
system and going on its back, though.

You might want to take a peek at
<https://wiki.debian.org/Teams/Dpkg/MergedUsr>.


Can any of the above be avoided? Well certainly, in the same way you
can probably cross a mine field safely, you'll just need to always be
aware of any of these things, and verify everything you install, and
all pathnames you use, etc. I'm not sure how this is in any reasonable
universe an acceptable way to expect end users to use the system TBH.


Thanks,
Guillem

[Wouter Verhelst]

On Mon, Jul 19, 2021 at 03:10:42PM +0200, Michael Biebl wrote:
> Hi Guillem

>
> Am 19.07.21 um 03:36 schrieb Guillem Jover:
> > What I've also said multiple times, is that
> > merged-usr-via-moves-and-symlink-farms could have been implemented in
> > a fully automated way, by debhelper, w/o requiring any maintainer scripts,
> > all with full cooperation and managed by dpkg, with .debs shipping
> > actual tracked pathnames

> I'm convinced this view is way too naive and not implementable in practice
> (and yes, openSUSE is a data point that confirms that)

>
> What you propose is, that each and every package does its /usr-merge
> transition on its own. This only works, if packages are independent (enough)
> so this actually works.
> Unfortunately this is not the case. Take PAM for example. You can't just
> recompile src:pam and have debhelper automatically move all files to /usr.
> This would break all packages that install a PAM plugin. You have a
> transition here, involving many packages.

Why?

Nobody is saying the old path should cease to function. The whole point
of a symlink farm is that *YOU ADD A SYMLINK* to replace the old path.

So then you have /lib/*/security/pam_foo.so ->
/usr/lib/*/security/pam_foo.so and your old-pam plugin will still work
with new-pam (and vice versa) and there is no need for a transition.

I've suggested previously that we can easily make it RC for bookworm to
have a file outside a limited set of directories (/etc and /usr would be
OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
This is easy to detect with a lintian check and reasonably easy to
implement, and would not confuse dpkg *at all*.

But whenever I bring this up, I hear people say "oh but suse tried it
and failed" (well suse aren't using dpkg and there's no reason to assume
we'll have the same problem, why don't we try?) or "oh but the /usr/doc
transition that worked that way 20 years ago took forever" (that was 20
years ago, our tooling is way more advanced these days) or "oh but that
will break bash and you can't upgrade safely without bash" (true, but
bash is just the one package and we already have /bin/sh be a symlink
and that never made upgrades fail permanently so I don't see how
usrmerge is somehow special).

I've grown tired of the whole discussion, and the "we must go forward
and only our way will work and your ideas are stupid just shut up
already" mentality the proponents of usrmerge seem to have.

I can understand the use case for usrmerge, and I won't cry over my
/bin/sh being essentially the same file as /usr/bin/sh -- but I long for
the good old days of Debian where we did things the right way, not
whichever is the fastest, because that way, things would *work* in *all*
cases, not just the cases that the proponents of some new feature care
about.

It took us forever to implement the /usr/doc transition, but it was
finished and nobody's machine broke.

It took us a fairly large time to implement multiarch, but we did it and
it works *way* better than in the RPM world.

I fail to see why usrmerge is so special that it can't wait until we do
things the right way.

Sure, there are technical issues with doing things the right way, and we
should deal with them. But just throwing them under the carpet and
deciding they're only a problem for other people isn't going to help
anyone.

--
w@uter.{be,co.za}
wouter@{grep.be,fosdem.org,debian.org}

[Simon McVittie]

On Thu, 22 Jul 2021 at 15:53:32 +0200, Wouter Verhelst wrote:
> I've suggested previously that we can easily make it RC for bookworm to
> have a file outside a limited set of directories (/etc and /usr would be
> OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
> This is easy to detect with a lintian check and reasonably easy to
> implement

I don't think that works in general without breaking some of Debian's
axioms around Essential packages, as previously described here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118

I have a longer mail written with possible ways forward, which I'm
deliberately not sending right now, because the first step in all of these
plans is "release Debian 11" and I don't want to distract the people who
are making that happen (any more than has already happened).

smcv

[Luca Boccassi]

Not only all Debian systems, but all Ubuntu systems since 2019 (iirc)
and other derivatives too. Still waiting for the sky to fall.

--
Kind regards,
Luca Boccassi

[Wouter Verhelst]

On Thu, Jul 22, 2021 at 03:20:05PM +0100, Simon McVittie wrote:
> On Thu, 22 Jul 2021 at 15:53:32 +0200, Wouter Verhelst wrote:
> > I've suggested previously that we can easily make it RC for bookworm to
> > have a file outside a limited set of directories (/etc and /usr would be
> > OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
> > This is easy to detect with a lintian check and reasonably easy to
> > implement
>
> I don't think that works in general without breaking some of Debian's
> axioms around Essential packages, as previously described here:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118

Yes. Those arguments didn't convince me then, and they don't convince me
now.

A package in the essential set could work around the issue by moving a
file around and creating a necessary symlink in preinst rather than
shipping things. The set of Essential packages is small however, and
most packages can ship a compat symlink.

I didn't say we *should* ship compat symlinks; I said we should make
antyhing that is *not* a compat symlink in a particular set of
directories be RC.

> I have a longer mail written with possible ways forward, which I'm
> deliberately not sending right now, because the first step in all of these
> plans is "release Debian 11" and I don't want to distract the people who
> are making that happen (any more than has already happened).

This is so exhausting.

Yes, I know the release is close, and yes, I know that some people are
immensely busy working on that. I want to help them do so in any way I
can, but they're not *required* to read -devel, and "they might read
this and get distracted" seems like a pretty poor argument.

I'm not busy with the release. Are you? If not, you *can* actually come
up with an argument right now, and I promise not to insist on any
decision being made until the release happens, so that those
hypothetical people who *are* busy with the release can still chip in
later if they choose to do so.

Meanwhile, we can still discuss this.

[Andreas Metzler]

On 2021-07-27 Wouter Verhelst <wou...@debian.org> wrote:
> On Thu, Jul 22, 2021 at 03:20:05PM +0100, Simon McVittie wrote:
>> On Thu, 22 Jul 2021 at 15:53:32 +0200, Wouter Verhelst wrote:
>>> I've suggested previously that we can easily make it RC for bookworm to
>>> have a file outside a limited set of directories (/etc and /usr would be
>>> OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
>>> This is easy to detect with a lintian check and reasonably easy to
>>> implement

>> I don't think that works in general without breaking some of Debian's
>> axioms around Essential packages, as previously described here:
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118

> Yes. Those arguments didn't convince me then, and they don't convince me
> now.

> A package in the essential set could work around the issue by moving a
> file around and creating a necessary symlink in preinst rather than
> shipping things. The set of Essential packages is small however, and
> most packages can ship a compat symlink.

> I didn't say we *should* ship compat symlinks; I said we should make
> antyhing that is *not* a compat symlink in a particular set of
> directories be RC.

[...]

Hello Wouter,

I will bite.

just for context: Simon said in #978636 that e.g. coreutils
a) cannot ship both /usr/bin/mv and /bin/mv (the latter a symlink) in
the tarfile since /bin _might_ be a symlink to /usr/bin but
b) it needs to provide /bin/mv in unpacked, unconfigured state.

Simon then said we needed a flag day where the aliasing-symlinks /bin -->
/usr/bin are either guaranteed to exist or forbidden. Once that is know
essential packages either ship both /usr/bin/mv and /bin/mv (the latter
a symlink) or only ship /usr/bin/mv (with no symlink required.)

Afaiu you are suggesting to do somethink like this instead and
immediately post bulleye release.
----------------------------------------
preinst upgrade|install
if aliasing-symlinks /bin --> /usr/bin
# do nothing
else
mv /bin/mv /usr/bin/mv
ln -s /usr/bin/mv /bin/mv
fi
Plus corresponding error handling code in postrm abort install.
----------------------------------------

I just do not get the benefit. It seems rather complicated with
potential for breakage in corner cases and unnecessary since we (CTTE)
have essentially decided that there is going to be a cutoff date
pre-bookworm-release whereupon package maintainers can rely on the
existence of aliasing-symlinks and can simply move the file without any
maintainerscripts. It seems to be a waste of work to write
complicated maintainerscripts that are only needed as long as we need to
handle both usrmerge-d and non-usrmerge-d systems.

[Guillem Jover]

On Tue, 2021-07-27 at 11:44:32 +0200, Wouter Verhelst wrote:
> On Thu, Jul 22, 2021 at 03:20:05PM +0100, Simon McVittie wrote:
> > On Thu, 22 Jul 2021 at 15:53:32 +0200, Wouter Verhelst wrote:
> > > I've suggested previously that we can easily make it RC for bookworm to
> > > have a file outside a limited set of directories (/etc and /usr would be
> > > OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
> > > This is easy to detect with a lintian check and reasonably easy to
> > > implement
> >
> > I don't think that works in general without breaking some of Debian's
> > axioms around Essential packages, as previously described here:
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118
>
> Yes. Those arguments didn't convince me then, and they don't convince me
> now.

Ack, these are very contrived.

> A package in the essential set could work around the issue by moving a
> file around and creating a necessary symlink in preinst rather than
> shipping things. The set of Essential packages is small however, and
> most packages can ship a compat symlink.

Yes, along those lines. To try to do something resembling dpkg's safe
behavior, we'd need to do in preinst, something like:

- if /usr/foo does not exist:
- copy /foo to /usr/foo
- replace /foo with a symlink to /usr/foo

Then dpkg would replace /usr/foo with the new version. Of course this
is all kinds of suboptimal, as the .debs will still not ship the actual
symlinks and it's trying to replicate what dpkg is designed and supposed
to do to handle Essential packages safely, even when doing this kind of
switch, where it will delay symlink installation as the last step… but
we cannot do that right now due to the incorrect restrictions imposed by
merged-/usr-via-aliased-dirs. :(

I have very deep and strong regrets about having removed compat symlinks
under /, to make it possible for people that wanted to locally use
the usrmerge hack. I guess the lesson learned with this episode, is
that in the future, similar stuff cannot be let through, when people
promise this will not be pushed into the distro, and it's just for
local deployments and similar, or we end up with this kind of mess. :/

> > I have a longer mail written with possible ways forward, which I'm
> > deliberately not sending right now, because the first step in all of these
> > plans is "release Debian 11" and I don't want to distract the people who
> > are making that happen (any more than has already happened).
>
> This is so exhausting.

Indeed, very. The impression I'm getting is that instead of stopping the
bleeding effect, this is being let fester to the point any option will
be terrible, so anything, regardless of its badness will seem acceptable
to make progress at that point.

> Yes, I know the release is close, and yes, I know that some people are
> immensely busy working on that. I want to help them do so in any way I
> can, but they're not *required* to read -devel, and "they might read
> this and get distracted" seems like a pretty poor argument.
>
> I'm not busy with the release. Are you? If not, you *can* actually come
> up with an argument right now, and I promise not to insist on any
> decision being made until the release happens, so that those
> hypothetical people who *are* busy with the release can still chip in
> later if they choose to do so.

Yes, I'd say this is one of Debian's development fallacies. You know
you are dealing with strong arguments when this comes up. Also the
proponents are not worried now that this is being shoved down by
force due to the involvement of the TC…

Thanks,
Guillem

[Wouter Verhelst]

On Tue, Jul 27, 2021 at 02:13:33PM +0200, Andreas Metzler wrote:
> On 2021-07-27 Wouter Verhelst <wou...@debian.org> wrote:
> > On Thu, Jul 22, 2021 at 03:20:05PM +0100, Simon McVittie wrote:
> >> On Thu, 22 Jul 2021 at 15:53:32 +0200, Wouter Verhelst wrote:
> >>> I've suggested previously that we can easily make it RC for bookworm to
> >>> have a file outside a limited set of directories (/etc and /usr would be
> >>> OK, but notably /bin /lib and /sbin wouldn't be) that is not a symlink.
> >>> This is easy to detect with a lintian check and reasonably easy to
> >>> implement
>
> >> I don't think that works in general without breaking some of Debian's
> >> axioms around Essential packages, as previously described here:
> >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=978636#118
>
> > Yes. Those arguments didn't convince me then, and they don't convince me
> > now.
>
> > A package in the essential set could work around the issue by moving a
> > file around and creating a necessary symlink in preinst rather than
> > shipping things. The set of Essential packages is small however, and
> > most packages can ship a compat symlink.
>
> > I didn't say we *should* ship compat symlinks; I said we should make
> > antyhing that is *not* a compat symlink in a particular set of
> > directories be RC.
> [...]
>
> Hello Wouter,
>
> I will bite.

Cool.

> just for context: Simon said in #978636 that e.g. coreutils
> a) cannot ship both /usr/bin/mv and /bin/mv (the latter a symlink) in
> the tarfile since /bin _might_ be a symlink to /usr/bin but
> b) it needs to provide /bin/mv in unpacked, unconfigured state.
>
> Simon then said we needed a flag day where the aliasing-symlinks /bin -->
> /usr/bin are either guaranteed to exist or forbidden. Once that is know
> essential packages either ship both /usr/bin/mv and /bin/mv (the latter
> a symlink) or only ship /usr/bin/mv (with no symlink required.)
>
> Afaiu you are suggesting to do somethink like this instead and
> immediately post bulleye release.
> ----------------------------------------
> preinst upgrade|install
> if aliasing-symlinks /bin --> /usr/bin
> # do nothing
> else
> mv /bin/mv /usr/bin/mv

That should be a copy (mv is too dangerous)

> ln -s /usr/bin/mv /bin/mv

This can be "ln -sf" to make it atomic.

> fi
> Plus corresponding error handling code in postrm abort install.
> ----------------------------------------

Yes, but for packages in the Essential set only. For other packages, we
can make it much simpler.

> I just do not get the benefit. It seems rather complicated with
> potential for breakage in corner cases and unnecessary since we (CTTE)
> have essentially decided that there is going to be a cutoff date
> pre-bookworm-release whereupon package maintainers can rely on the
> existence of aliasing-symlinks and can simply move the file without any
> maintainerscripts. It seems to be a waste of work to write
> complicated maintainerscripts that are only needed as long as we need to
> handle both usrmerge-d and non-usrmerge-d systems.

I'm not worried about the support for both usrmerge'd and not usrmerge'd
systems.

I'm worried about systems being written to completely bypass the dpkg
database. It's being pushed forward "because we broke things in the past
and now the only way to fix it is to break even more things". That's BS.

I'm convinced there is a way that we can move forward which does *not*
require bypassing the dpkg database. I think that such a way *should* be
preferential, and the complete lack of even a desire to discuss things
with the dpkg maintainer in ways that the dpkg maintainer thinks is a
reasonable way forward is distressing for me.

[Andrey Rahmatullin]

On Tue, Jul 27, 2021 at 03:25:48PM +0200, Wouter Verhelst wrote:
> I'm worried about systems being written to completely bypass the dpkg
> database.

Like alternatives and things that create files in postinst?

--
WBR, wRAR

[Simon Richter]

Hi,

On 7/27/21 11:44 AM, Wouter Verhelst wrote:

> A package in the essential set could work around the issue by moving a
> file around and creating a necessary symlink in preinst rather than
> shipping things. The set of Essential packages is small however, and
> most packages can ship a compat symlink.

In debootstrap (which is the important use case for Essential packages
and their constraints), all Essential packages are unpacked first, and
then, individually, their preinst is run, the files unpacked again (this
time from dpkg), and then we're in normal dpkg land, although in a chroot.

So the concept of a preinst script for an Essential package is wobbly at
best. For debootstrap --foreign, this might be even more complicated.

Also, take care when moving shell commands from a shell script: the bash
shell at least keeps a cache of commands to paths so it doesn't have to
do a full path search every time. A shell script that calls

mv /bin/cp /usr/bin/cp
ln -s ../usr/bin/cp bin/cp
mv /bin/ln /usr/bin/ln
ln -s ../usr/bin/ln bin/ln

could fall over because it cached the location of "ln" as /bin/ln in the
beginning, then after the move cannot find it anymore. That needs at
least a "hash -d ln".

Simon

[Wouter Verhelst]

The alternatives system doesn't bypass the dpkg database. It creates
extra symlinks on the system that do not exist in the dpkg database.

Creating files in postinst doesn't bypass the dpkg database. It creates
extra files on the system that do not exist in the dpkg database.

Creating a system that tells dpkg that files exist in one place but
where in reality they're in a different place does bypass the dpkg
database.

[Guillem Jover]

On Tue, 2021-07-27 at 16:26:34 +0200, Simon Richter wrote:
> On 7/27/21 11:44 AM, Wouter Verhelst wrote:
> > A package in the essential set could work around the issue by moving a
> > file around and creating a necessary symlink in preinst rather than
> > shipping things. The set of Essential packages is small however, and
> > most packages can ship a compat symlink.
>
> In debootstrap (which is the important use case for Essential packages and
> their constraints), all Essential packages are unpacked first, and then,
> individually, their preinst is run, the files unpacked again (this time from
> dpkg), and then we're in normal dpkg land, although in a chroot.

The installation bootstrap is currently "undefined behavior" and it's
not specified by policy. The properties of the Essential set do not
apply there. See <https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap>.

> So the concept of a preinst script for an Essential package is wobbly at
> best.

Not really. Bootstrapping has always been done with strings and tape.
Of course, having to unnecessarily add more maintainer scripts to
handle something that dpkg can do perfectly fine on its own, would regress
the progress we have been making to make the installation bootstrapping
automatable and definable. But that seems less worse than the breakage
induced by the merged-/usr-via-aliased-dirs layout. :/

> For debootstrap --foreign, this might be even more complicated.

Only a few paths are expected to be hardcoded, nothing that couldn't
be special-cased by debootstrap along all other bootstrapping
knowledge it contains which would ideally be contained in their
respective packages anyway.

But this indeed is, having to pile hack over hack from the original
broken foundation.

> Also, take care when moving shell commands from a shell script: the bash
> shell at least keeps a cache of commands to paths so it doesn't have to do a
> full path search every time. A shell script that calls
>
> mv /bin/cp /usr/bin/cp
> ln -s ../usr/bin/cp bin/cp
> mv /bin/ln /usr/bin/ln
> ln -s ../usr/bin/ln bin/ln
>
> could fall over because it cached the location of "ln" as /bin/ln in the
> beginning, then after the move cannot find it anymore. That needs at least a
> "hash -d ln".

As has been mentioned, this is completely unsafe and does not map to
what dpkg would be doing.

Regards,
Guillem

[Wouter Verhelst]

On Tue, Jul 27, 2021 at 04:26:34PM +0200, Simon Richter wrote:
> Also, take care when moving shell commands from a shell script: the bash
> shell at least keeps a cache of commands to paths so it doesn't have to do a
> full path search every time. A shell script that calls
>
> mv /bin/cp /usr/bin/cp
> ln -s ../usr/bin/cp bin/cp
> mv /bin/ln /usr/bin/ln
> ln -s ../usr/bin/ln bin/ln
>
> could fall over because it cached the location of "ln" as /bin/ln in the
> beginning, then after the move cannot find it anymore. That needs at least a
> "hash -d ln".

This is why I said to use cp, not mv, when moving the file...

[Sam Hartman]

>>>>> "Wouter" == Wouter Verhelst <wou...@debian.org> writes:

Wouter> I'm convinced there is a way that we can move forward which
Wouter> does *not* require bypassing the dpkg database. I think that
Wouter> such a way *should* be preferential, and the complete lack
Wouter> of even a desire to discuss things with the dpkg maintainer
Wouter> in ways that the dpkg maintainer thinks is a reasonable way
Wouter> forward is distressing for me.

Yeah, like extending dpkg to be able to tell dpkg that /bin and /sbin
are aliases and have it deal with that. I think that adding that
extension to dpkg is going to be simpler (technically) than getting the
handling right to move things in essential packages. Your corrections
(copy instead of mv, atomic symlink) are in my mind just the beginning
in terms of how complicated that's going to be. I noticed that neither
of you took a stab at the error handling for abort-upgrade.

We'd either need to do that for each essential package, or try and come
up with something (in debhelper?) that is a useful abstraction. In
practice we'd probably find that we needed a combination.

So, even though I think the extensions to dpkg will also be complicated,
at a purely technical level, I think they are less complicated.


I understand technical complexity is only part of the picture.
I understand the dpkg maintainer might make extending dpkg politically
challenging. I also agree that there are things we could have done
better throughout this process in terms of being respectful in our
decision making, giving people a chance to voice their opinions, but
ultimately letting a decision be made and all falling in on that
decision (or standing aside if we cannot) once that has been done.

I think the areas for improvement in decision making are broad here.
I'll pick examples from both sides.

During the discussion of the debootstrap decision to default to merged
/usr, several people pointed to a debian-devel thread and claimed that
thread came to a consensus in favor of merged /usr.
That was not obvious to me as someone who read the referenced thread.
More over, since no one chose to summarize the discussion, people didn't
have an opportunity to confirm they were on the same page or raise
objections if they felt concerns they raised had not been addressed.


Today though, I think we are approaching (or have passed) a point where
a decision has been made and people need to fall in (or stand aside) and
respect our processes.
If you don't feel that your concerns were adequately addressed, one
constructive thing you can do is help us develop better processes for
the future.

--Sam

[Andreas Metzler]

On 2021-07-27 Wouter Verhelst <wou...@debian.org> wrote:
> On Tue, Jul 27, 2021 at 02:13:33PM +0200, Andreas Metzler wrote:

[...]

Hello Wouter,

I think we complicated things enormously and caused real breakage by
trying to support both setups. This has already caused considerable work
without longterm gain and is preventing us to reach an unbroken state
(dpkg knowning the correct paths on all systems) again. That is what I
see as goal.

The maintainerscript setup for symlinking looks like a lot of work and
muddles the whole situation even more, there are more files/symlinks
dpkg does not know about and our systems diverge even more. I really do
not get how that is a step forward.

> It's being pushed forward "because we broke things in the past
> and now the only way to fix it is to break even more things". That's BS.

[...]

When you say "break more things" you are thinking of the social effect
(alienating Guillem)? I am not aware of any plans for new technical
breakage.

cu Andreas

PS: As you can probably tell English is not my native language so please
take the whole mail with a grain of salt if I did not manage to hit the
correct level of politeness.

[Calum McConnell]

> Of course, having to unnecessarily add more maintainer scripts to
> handle something that dpkg can do perfectly fine on its own

TL;DR: merged-usr-via-symlink-farms cannot be done without changing dpkg,
and since the quote above seems to indicate you'd be willing to do that,
why not just change dpkg to support aliased dirs?

--------------------

Lets look at going forward. We have a problem (Debian supports a mixture
of merged-usr and unmerged-usr layouts). We need to solve this problem,
since the headaches it produces are far worse than either layout on its
own. Let's assume the eventual solution is going to be merged-usr:an

Fact: A significant portion of supported Debian installations currently
use usrmerge, with symlinks between /bin -> /usr/bin, /lib -> /usr/lib,
etc

Any path forward must allow for that. The decisions that caused that fact
to be true are irrelevant: whether or not that fact is a good or bad thing
doesn't matter. What matters is that it is true, and so we need to work
around it, even if the path forward we choose involves reverting it.

It doesn't matter if merged-usr-via-aliased-dirs would have been better if
we'd just done it from the start. It doesn't matter if we could have made
that transition with a small change to debhelper and a release cycle. It
doesn't even matter if the problems we are facing now would never have
occurred with that plan. The fact is true: systems using merged-usr-via-
aliased-dirs exist, and we need to figure out how to go onwards from here.

One way to move forward despite that fact is to mandate running dpkg-fsys-
usrunmess on every system that has a merged-usr layout, and then move
forward from there. However, that script is hardly battle-tested, and
that solution is completely unacceptable to usrmerge proponents. So lets
look at other solutions that would work for everyone.

Another way forward is to transition existing systems without merged-usr
to a merged-usr-via-symlink-farms. To accomplish this, we need the
ability to create symlinks in installations that are not usrmerge, but to
not create those links in installations that are. That requires either
maintainer scripts or a change to dpkg. You just criticized maintainer
scripts, so I would assume that they are not your favorite solution.
Furthermore, others have pointed out that essential packages need to work
before maintainer scripts are executed.  This behavior is codified in
policy: 

> "Since dpkg will not prevent upgrading of other packages while an essential package is in an unconfigured state, all essential packages must supply all of their core functionality even when unconfigured".

In other words, using maintainer scripts to create the symlinks that
enable the core functionality of these packages during configuration is a
no-go without a revision to policy and a change to dpkg (which might be
impossible). That means the symlinks would need to be included in the
package declaratively: but simply shipping them would break the existing
merged-usr installations. We've already established that un-merging and
then re-merging every installation isn't going to happen: so we'll need to
get the file references in place using a method that isn't shipping two
aliasing files and doesn't require maintainer scripts.

Now, shipping the file in /bin, and then eventually moving it to /usr/bin
and replacing it with a symlink as soon as you can would work, but that
isn't a solution. It means that we will always have packages that ship
files in /bin, because there is no migration path out of that, short of
completely redefining 'essential'. In thirty years, the bash package will
still contain this cludge. That is not a suitable long term solution, and
I think you agree.

Simply modifying dpkg to automatically produce symlinks from /bin to
/usr/bin at unpack time is not enough either: dpkg isn't necessarily the
tool doing the unpacking, and so other tools would need to be modified,
each one of them checking for a merged-usr and then accounting for it if
needed. This solution is actually viable: it would lead to a working
system, and not break the essential guarantee.

However...

Achieving this would require changing every tool that is responsible for
unpacking a Debian package. That isn't just dpkg and debootstrap: there
are many others. mmdebstrap and cdeboostrap come to mind. This approach
thus requires significant changes to at least four distinct tools. It
would also require at least two upgrade cycles before maintainers could
actually rely on the system being merged-usr: one cycle to get the dpkg
change out, and another to ensure that all packages have been changed to
ship their files in /usr. That is four years of waiting.

Of course, one could drop that to two years if you made the dpkg change a
little bit more aggressive. Since we already have dpkg creating
compatibility symlinks, why not have it also handle the file move? Simply
treat all files shipped to /{bin,sbin,lib} as actually being shipped to
/usr/{bin,sbin,lib}, and create symlinks accordingly. But that raises an
important question.

If we are willing to do that, why not just tweak dpkg to support merged-
usr-via-aliased-dirs? As far as I can tell, the problems with the layout
boil down to "we're letting packages treat the folders as separate, but in
the layout they aren't". Since we are already changing dpkg to make it
treat the folders as equivalent (which we need to do to avoid a long and
painful upgrade cycle), why not just save a few hundred symlinks and use
the aliased-dirs layout?

Thanks for making it through my castle of text,
Calum McConnell

[Steve Cotton]

Am Tue, Jul 27, 2021 at 09:23:48AM -0600 schrieb Sam Hartman:
> So, even though I think the extensions to dpkg will also be complicated,
> at a purely technical level, I think they are less complicated.
>
> I understand technical complexity is only part of the picture.
> I understand the dpkg maintainer might make extending dpkg politically
> challenging.

I took a look at the changelog and open issues of dpkg. The issues with
symlinked dirs were known about 15 years ago. There's a lack of offers of help
in those issues, and it seems a lack of volunteers to join the dpkg team.

When he says "it would require new *features* to be implemented", I can
understand the grumpyness when that means "new features that people have been
calling bugs for 15 years, but are still asking when they'll be implemented
without helping implement them".

I don't know the people in this thread personally, there's surely more that
has been said in person. However, just from the debate in this thread and the
BTS, it seems the "politics" might be simply be a lack of volunteers compared
to demands.

Steve

[Simon Richter]

Hi,

On 7/27/21 7:23 PM, Calum McConnell wrote:

> Of course, one could drop that to two years if you made the dpkg change a
> little bit more aggressive. Since we already have dpkg creating
> compatibility symlinks, why not have it also handle the file move? Simply
> treat all files shipped to /{bin,sbin,lib} as actually being shipped to
> /usr/{bin,sbin,lib}, and create symlinks accordingly. But that raises an
> important question.

This might be doable with diversions, but probably only for a closed set
of names. The Essential set is closed, but I'd suspect that there are
quite a few datacenter deployments (and deployment tools) that add
custom packages to debootstrap.

Simon

[Guillem Jover]

On Tue, 2021-07-27 at 13:23:46 -0400, Calum McConnell wrote:
> > Of course, having to unnecessarily add more maintainer scripts to
> > handle something that dpkg can do perfectly fine on its own
>
> TL;DR: merged-usr-via-symlink-farms cannot be done without changing dpkg,

In my mind that's "false", but whatever yes… given that the reversion does
not appear forthcoming, other new features might indeed be needed in case
people do not want to be forced into this train wreck in slow motion.
And as I mentioned on my first reply in this thread I'm prepared to
devote any necessary volunteer time to implement such things in detriment
of other Debian work if necessary.

> and since the quote above seems to indicate you'd be willing to do that,
> why not just change dpkg to support aliased dirs?

I've mentioned this multiple times now, firstly because I think it
would still be a broken layout. Secondly, because there are different
types of dpkg features, some can be used right away, some others will
still need at least two release cycles to be usable. The features that
I think might be needed to be able to avoid being forced into using
merged-usr-via-aliased-dirs, such as registering arbitrary files are of
the former category. The features that would be needed to add support
for merged-usr-via-aliased-dirs would be the latter.

> Another way forward is to transition existing systems without merged-usr
> to a merged-usr-via-symlink-farms. To accomplish this, we need the
> ability to create symlinks in installations that are not usrmerge, but to
> not create those links in installations that are. That requires either
> maintainer scripts or a change to dpkg. You just criticized maintainer
> scripts, so I would assume that they are not your favorite solution.

Having to add maintainer script usage would be non-ideal, but would be
better than being forced to use merged-usr-via-aliased-dirs.

> Furthermore, others have pointed out that essential packages need to work
> before maintainer scripts are executed.  This behavior is codified in
> policy: 

> > "Since dpkg will not prevent upgrading of other packages while an
> > essential package is in an unconfigured state, all essential packages
> > must supply all of their core functionality even when unconfigured".

That's not what policy says. "unconfigured" does not mean that
*maintainer scripts* have not been executed.

> In other words, using maintainer scripts to create the symlinks that
> enable the core functionality of these packages during configuration is a
> no-go without a revision to policy and a change to dpkg (which might be
> impossible). That means the symlinks would need to be included in the
> package declaratively: but simply shipping them would break the existing
> merged-usr installations. We've already established that un-merging and
> then re-merging every installation isn't going to happen: so we'll need to
> get the file references in place using a method that isn't shipping two
> aliasing files and doesn't require maintainer scripts.

This is based on an incorrect premise. In addition, as I've also said
elsewhere bootstrapping is outside the realms of debian-policy anyway.

> Simply modifying dpkg to automatically produce symlinks from /bin to
> /usr/bin at unpack time is not enough either: dpkg isn't necessarily the
> tool doing the unpacking, and so other tools would need to be modified,
> each one of them checking for a merged-usr and then accounting for it if
> needed. This solution is actually viable: it would lead to a working
> system, and not break the essential guarantee.

This is still based on an incorrect premise. And even though I'd find
what you propose to be a major kludge, all bootstrappers I know of,
always end up running dpkg over all .debs to do a proper installation
of any possibly manually unpacked package. And not to mention that
bootstrappers that support merged-usr-via-aliased-dirs have required
implementing an explicit kludge for it.

Regards,
Guillem

[Marco d'Itri]

Implementations with real /bin /sbin /lib* directories and symlink farms
are not useful because they would negate the major benefits of
merged-/usr, i.e. the ability of sharing and independently updating
/usr.

--
ciao,
Marco

[Guillem Jover]

Yes, that major benefit that is completely broken by design and
unsupported anyway, because /etc and /var can also easily get out
of sync. If you rely on this then you are on your own anyway…

Also nothing prevents sharing /usr with symlink farms, iff all systems
are in sync, which they must anyway.

So that argument is pretty void of substance and founded on a base
of unreliability… but what's new.

Guillem