Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bug#1051582: Policy 9.3 (Starting system services) is largely obsolete
2 views
Skip to first unread message
Russ Allbery
Sep 9, 2023, 10:20:04 PM9/9/23
to
Package: debian-policy
Version: 4.6.2.0
Severity: important
X-Debbugs-Cc: r...@debian.org
As part of reviewing #1039102, I took a detailed look at Policy 9.3
on system services and realized that it is largely obsolete and is
not followed by most Debian packages that provide system services.
Specifically:
* There is no real documentation of systemd units except in the
introduction, and most of the section describes init scripts as
if they were the only way to start a service.
* All packages are told they must use update-rc.d, but systemd units
no longer use update-rc.d. They instead use deb-systemd-helper in
ways that are not documented in Policy and I believe are maintained
primarily in debhelper.
* All packages are told they should use invoke-rc.d, but systemd units
no longer use invoke-rc.d. They instead use deb-systemd-invoke,
which also supports the policy-rc.d interface.
* The policy-rc.d interface is undocumented.
This part of Policy is also a bit of a mess of deleted sections due to
a desire to avoid section renumbering.
I started a rewrite of this section, but quickly ran into the question
of how to document the correct invocations of deb-systemd-helper and
deb-systemd-invoke. After thinking about this for a while, the
conclusion I reached was that documenting this in Policy is both extra
make-work that we do not have the resources to keep up with, and serves
no real purpose for nearly every reader of Debian. debhelper is already
maintaining the correct code to set up systemd services in close
cooperation with the systemd and init-system-helpers maintainers, that
code contains temporary workarounds and other fixes that Policy is not
going to keep up with, and it's hard to justify duplicating that work in
a Policy statement.
I therefore would like to propose a first: I think Policy should simply
say that any package that provides a system service should use debhelper
and rely on dh_installsystemd to put the appropriate commands in its
maintainer scripts. (We can then discuss whether we should do the same
for init scripts and dh_installinit, although its stanzas are simpler.)
Previously we have tried to avoid doing this, and have maintained the
principle that debhelper is simply an *implementation* of Policy, and it
still falls to Policy to describe what debhelper should do. However, I
think it is now abundantly obvious that debhelper has considerably more
development resources available to it than Policy has writers, debhelper
developers are quite rightfully not waiting for things to be added to
Policy before they can be implemented, and essentially every Debian
package that does anything non-trivial uses debhelper now. I also don't
see any truly useful purpose served by dumping a wad of shell code into
Policy that essentially no one will use because it's what debhelper would
add for them.
I have some other questions about the rewrite of these sections (such as
how hard we should try to retain section numbering), but I think we should
resolve this question first, since it has dramatic effects on what text
we should write.
-- System Information:
Debian Release: trixie/sid
APT prefers unstable
APT policy: (990, 'unstable'), (500, 'unstable-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 6.4.0-4-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
debian-policy depends on no packages.
Versions of packages debian-policy recommends:
ii libjs-sphinxdoc 5.3.0-7
Versions of packages debian-policy suggests:
pn doc-base <none>
-- no debconf information
Version: 4.6.2.0
Severity: important
X-Debbugs-Cc: r...@debian.org
As part of reviewing #1039102, I took a detailed look at Policy 9.3
on system services and realized that it is largely obsolete and is
not followed by most Debian packages that provide system services.
Specifically:
* There is no real documentation of systemd units except in the
introduction, and most of the section describes init scripts as
if they were the only way to start a service.
* All packages are told they must use update-rc.d, but systemd units
no longer use update-rc.d. They instead use deb-systemd-helper in
ways that are not documented in Policy and I believe are maintained
primarily in debhelper.
* All packages are told they should use invoke-rc.d, but systemd units
no longer use invoke-rc.d. They instead use deb-systemd-invoke,
which also supports the policy-rc.d interface.
* The policy-rc.d interface is undocumented.
This part of Policy is also a bit of a mess of deleted sections due to
a desire to avoid section renumbering.
I started a rewrite of this section, but quickly ran into the question
of how to document the correct invocations of deb-systemd-helper and
deb-systemd-invoke. After thinking about this for a while, the
conclusion I reached was that documenting this in Policy is both extra
make-work that we do not have the resources to keep up with, and serves
no real purpose for nearly every reader of Debian. debhelper is already
maintaining the correct code to set up systemd services in close
cooperation with the systemd and init-system-helpers maintainers, that
code contains temporary workarounds and other fixes that Policy is not
going to keep up with, and it's hard to justify duplicating that work in
a Policy statement.
I therefore would like to propose a first: I think Policy should simply
say that any package that provides a system service should use debhelper
and rely on dh_installsystemd to put the appropriate commands in its
maintainer scripts. (We can then discuss whether we should do the same
for init scripts and dh_installinit, although its stanzas are simpler.)
Previously we have tried to avoid doing this, and have maintained the
principle that debhelper is simply an *implementation* of Policy, and it
still falls to Policy to describe what debhelper should do. However, I
think it is now abundantly obvious that debhelper has considerably more
development resources available to it than Policy has writers, debhelper
developers are quite rightfully not waiting for things to be added to
Policy before they can be implemented, and essentially every Debian
package that does anything non-trivial uses debhelper now. I also don't
see any truly useful purpose served by dumping a wad of shell code into
Policy that essentially no one will use because it's what debhelper would
add for them.
I have some other questions about the rewrite of these sections (such as
how hard we should try to retain section numbering), but I think we should
resolve this question first, since it has dramatic effects on what text
we should write.
-- System Information:
Debian Release: trixie/sid
APT prefers unstable
APT policy: (990, 'unstable'), (500, 'unstable-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 6.4.0-4-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
debian-policy depends on no packages.
Versions of packages debian-policy recommends:
ii libjs-sphinxdoc 5.3.0-7
Versions of packages debian-policy suggests:
pn doc-base <none>
-- no debconf information
Edward Little
Sep 10, 2023, 9:20:05 PM9/10/23
to
Please remove the following email address: e.lit...@gmail.com
Sam Hartman
Sep 10, 2023, 10:10:04 PM9/10/23
to
>>>>> "Russ" == Russ Allbery <r...@debian.org> writes:
Russ> I therefore would like to propose a first: I think Policy
Russ> should simply say that any package that provides a system
Russ> service should use debhelper and rely on dh_installsystemd to
Russ> put the appropriate commands in its maintainer scripts. (We
Russ> can then discuss whether we should do the same for init
Russ> scripts and dh_installinit, although its stanzas are simpler.)
For a variety of reasons, I support this.
--Sam
Russ> I therefore would like to propose a first: I think Policy
Russ> should simply say that any package that provides a system
Russ> service should use debhelper and rely on dh_installsystemd to
Russ> put the appropriate commands in its maintainer scripts. (We
Russ> can then discuss whether we should do the same for init
Russ> scripts and dh_installinit, although its stanzas are simpler.)
For a variety of reasons, I support this.
--Sam
Santiago Vila
Sep 11, 2023, 7:00:04 AM9/11/23
to
El 10/9/23 a las 4:09, Russ Allbery escribió:
here. One of them is our ability (or lack thereof) of policy to catch up with
real development done elsewhere. Another one is the fact that policy does
not mandate any given implementation.
I believe that by choosing the wording appropriately, we can still keep this
desired property of Policy while still not mandating any given implementation.
(i.e. we could essentially say "you should do the same thing
as dh_installsystemd does", but in an elegant way).
Thanks.
> I therefore would like to propose a first: I think Policy should simply
> say that any package that provides a system service should use debhelper
> and rely on dh_installsystemd to put the appropriate commands in its
> maintainer scripts. (We can then discuss whether we should do the same
> for init scripts and dh_installinit, although its stanzas are simpler.)
Hello. I don't like this approach, and I believe we are mixing two different things
> say that any package that provides a system service should use debhelper
> and rely on dh_installsystemd to put the appropriate commands in its
> maintainer scripts. (We can then discuss whether we should do the same
> for init scripts and dh_installinit, although its stanzas are simpler.)
here. One of them is our ability (or lack thereof) of policy to catch up with
real development done elsewhere. Another one is the fact that policy does
not mandate any given implementation.
I believe that by choosing the wording appropriately, we can still keep this
desired property of Policy while still not mandating any given implementation.
(i.e. we could essentially say "you should do the same thing
as dh_installsystemd does", but in an elegant way).
Thanks.
Santiago Vila
Sep 11, 2023, 7:10:04 AM9/11/23
to
I wrote:
> I believe that by choosing the wording appropriately, we can still keep this
> desired property of Policy while still not mandating any given implementation.
Sorry, that was terribly worded. I meant that we can avoid the hassle of
> I believe that by choosing the wording appropriately, we can still keep this
> desired property of Policy while still not mandating any given implementation.
documenting everything dh_installsystemd does and at the same time not
*formally* mandating the use of dh_installsystemd.
Thanks.
Bill Allombert
Sep 11, 2023, 10:40:04 AM9/11/23
to
On Mon, Sep 11, 2023 at 12:47:15PM +0200, Santiago Vila wrote:
> El 10/9/23 a las 4:09, Russ Allbery escribió:
> > I therefore would like to propose a first: I think Policy should simply
> > say that any package that provides a system service should use debhelper
> > and rely on dh_installsystemd to put the appropriate commands in its
> > maintainer scripts. (We can then discuss whether we should do the same
> > for init scripts and dh_installinit, although its stanzas are simpler.)
>
> Hello. I don't like this approach, and I believe we are mixing two different things
> here. One of them is our ability (or lack thereof) of policy to catch up with
> real development done elsewhere. Another one is the fact that policy does
> not mandate any given implementation.
I agree. The issue is that we need to document what dh_installsystemd should do,
> El 10/9/23 a las 4:09, Russ Allbery escribió:
> > I therefore would like to propose a first: I think Policy should simply
> > say that any package that provides a system service should use debhelper
> > and rely on dh_installsystemd to put the appropriate commands in its
> > maintainer scripts. (We can then discuss whether we should do the same
> > for init scripts and dh_installinit, although its stanzas are simpler.)
>
> Hello. I don't like this approach, and I believe we are mixing two different things
> here. One of them is our ability (or lack thereof) of policy to catch up with
> real development done elsewhere. Another one is the fact that policy does
> not mandate any given implementation.
otherwise we will not be able to distinguish between bug in dh_installsystemd and
intended behaviour, and we will end up freezing dh_installsystemd to avoid introducing
problem by breaking incidental interfaces.
Cheers,
--
Bill. <ball...@debian.org>
Imagine a large red swirl here.
Sam Hartman
Sep 11, 2023, 11:30:04 AM9/11/23
to
>>>>> "Santiago" == Santiago Vila <san...@debian.org> writes:
Santiago> El 10/9/23 a las 4:09, Russ Allbery escribió:
Santiago> mixing two different things here. One of them is our
Santiago> ability (or lack thereof) of policy to catch up with real
Santiago> development done elsewhere. Another one is the fact that
Santiago> policy does not mandate any given implementation.
The question in my mind is whether it is valuable to support multiple
implementations, and I think the answer is no.
In the past, there was not a preference for using debhelper. So, back
then, we did need to support multiple implementations of debian/rules,
and we needed to specify the things debhelper did.
Having a specified interface like policy is expensive. I know; I've
spent a good chunk of my life working on various protocols and
standards.
Having specified interfaces brings value when you need multiple
implementations and in a few other cases, like where you need to plan
for certain forms of extensibility or isolation.
There's a part of this where we do need an interface. We will have
multiple packages wanting their debian/rules to install systemd units.
So, we do need to at least say or imply that if you stick systemd units
in the right place and call dh_installsystemd, then your systemd units
will be properly handled.
But today, we have a policy preference for using debhelper. We do not
need multiple implementations of debhelper. There does sort of need to
be an interface between debhelper and systemd if for no other reason
than to allow for systemd to change and to control which details are
hard-coded in maintainer scripts. But if we agree that we do not need
multiple implementations of debhelper, the policy team does not need to
be part of defining that interface. We can be more efficient by not
being involved in that process. Also, we can do a better job of
focusing on the interface that the project does care about (how to tell
debhelper to install your systemd units) and not confuse people with the
details between debhelper and systemd.
Also, by explicitly acknowledging that the deb-systemd-helper and
deb-systemd-invoke interfaces are effectively between debhelper and
systemd, those interfaces can be simpler because they do not need to
allow for multiple implementations of debhelper or systemd.
In effect by making this decision, we are strengthening our preference
for saying packages should use debhelper. For a significant class of
packages (those with service units) there really will be no easy
alternative. And if we go down that route, over time, we will probably
prefer debhelper more and more, and it will be less possible to generate
a policy-compliant package that does not use debhelper.
I think that's desirable.
I think we can still find ways to experiment with new more declarative
ways of handling package building. But I think that having more
uniformity in the cases where we have a single solution that has broad
support will make things easier for everyone.
Put another way, having multiple implementations is often very expensive
in terms of interface complexity, testing complexity, and especially
complexity that developers need to deal with.
In this instance, I do not think that cost is justified.
--Sam
Santiago> El 10/9/23 a las 4:09, Russ Allbery escribió:
>> I therefore would like to propose a first: I think Policy should
>> simply say that any package that provides a system service should
>> use debhelper and rely on dh_installsystemd to put the
>> appropriate commands in its maintainer scripts. (We can then
>> discuss whether we should do the same for init scripts and
>> dh_installinit, although its stanzas are simpler.)
Santiago> Hello. I don't like this approach, and I believe we are
>> simply say that any package that provides a system service should
>> use debhelper and rely on dh_installsystemd to put the
>> appropriate commands in its maintainer scripts. (We can then
>> discuss whether we should do the same for init scripts and
>> dh_installinit, although its stanzas are simpler.)
Santiago> mixing two different things here. One of them is our
Santiago> ability (or lack thereof) of policy to catch up with real
Santiago> development done elsewhere. Another one is the fact that
Santiago> policy does not mandate any given implementation.
The question in my mind is whether it is valuable to support multiple
implementations, and I think the answer is no.
In the past, there was not a preference for using debhelper. So, back
then, we did need to support multiple implementations of debian/rules,
and we needed to specify the things debhelper did.
Having a specified interface like policy is expensive. I know; I've
spent a good chunk of my life working on various protocols and
standards.
Having specified interfaces brings value when you need multiple
implementations and in a few other cases, like where you need to plan
for certain forms of extensibility or isolation.
There's a part of this where we do need an interface. We will have
multiple packages wanting their debian/rules to install systemd units.
So, we do need to at least say or imply that if you stick systemd units
in the right place and call dh_installsystemd, then your systemd units
will be properly handled.
But today, we have a policy preference for using debhelper. We do not
need multiple implementations of debhelper. There does sort of need to
be an interface between debhelper and systemd if for no other reason
than to allow for systemd to change and to control which details are
hard-coded in maintainer scripts. But if we agree that we do not need
multiple implementations of debhelper, the policy team does not need to
be part of defining that interface. We can be more efficient by not
being involved in that process. Also, we can do a better job of
focusing on the interface that the project does care about (how to tell
debhelper to install your systemd units) and not confuse people with the
details between debhelper and systemd.
Also, by explicitly acknowledging that the deb-systemd-helper and
deb-systemd-invoke interfaces are effectively between debhelper and
systemd, those interfaces can be simpler because they do not need to
allow for multiple implementations of debhelper or systemd.
In effect by making this decision, we are strengthening our preference
for saying packages should use debhelper. For a significant class of
packages (those with service units) there really will be no easy
alternative. And if we go down that route, over time, we will probably
prefer debhelper more and more, and it will be less possible to generate
a policy-compliant package that does not use debhelper.
I think that's desirable.
I think we can still find ways to experiment with new more declarative
ways of handling package building. But I think that having more
uniformity in the cases where we have a single solution that has broad
support will make things easier for everyone.
Put another way, having multiple implementations is often very expensive
in terms of interface complexity, testing complexity, and especially
complexity that developers need to deal with.
In this instance, I do not think that cost is justified.
--Sam
Bill Allombert
Sep 11, 2023, 4:20:04 PM9/11/23
to
On Mon, Sep 11, 2023 at 09:21:56AM -0600, Sam Hartman wrote:
> >>>>> "Santiago" == Santiago Vila <san...@debian.org> writes:
>
> Santiago> El 10/9/23 a las 4:09, Russ Allbery escribió:
> >> I therefore would like to propose a first: I think Policy should
> >> simply say that any package that provides a system service should
> >> use debhelper and rely on dh_installsystemd to put the
> >> appropriate commands in its maintainer scripts. (We can then
> >> discuss whether we should do the same for init scripts and
> >> dh_installinit, although its stanzas are simpler.)
>
> Santiago> Hello. I don't like this approach, and I believe we are
> Santiago> mixing two different things here. One of them is our
> Santiago> ability (or lack thereof) of policy to catch up with real
> Santiago> development done elsewhere. Another one is the fact that
> Santiago> policy does not mandate any given implementation.
>
> The question in my mind is whether it is valuable to support multiple
> implementations, and I think the answer is no.
But we do: we support debhelper 13.11.4 and debhelper 13.11.6.
> >>>>> "Santiago" == Santiago Vila <san...@debian.org> writes:
>
> Santiago> El 10/9/23 a las 4:09, Russ Allbery escribió:
> >> I therefore would like to propose a first: I think Policy should
> >> simply say that any package that provides a system service should
> >> use debhelper and rely on dh_installsystemd to put the
> >> appropriate commands in its maintainer scripts. (We can then
> >> discuss whether we should do the same for init scripts and
> >> dh_installinit, although its stanzas are simpler.)
>
> Santiago> Hello. I don't like this approach, and I believe we are
> Santiago> mixing two different things here. One of them is our
> Santiago> ability (or lack thereof) of policy to catch up with real
> Santiago> development done elsewhere. Another one is the fact that
> Santiago> policy does not mandate any given implementation.
>
> The question in my mind is whether it is valuable to support multiple
> implementations, and I think the answer is no.
Even if we support a single implementation, we still need to know what is
expected of it.
Russ Allbery
Sep 11, 2023, 4:40:03 PM9/11/23
to
Bill Allombert <ball...@debian.org> writes:
> But we do: we support debhelper 13.11.4 and debhelper 13.11.6. Even if
> we support a single implementation, we still need to know what is
> expected of it.
Policy already requires a single implementation of quite a lot of tools,
> But we do: we support debhelper 13.11.4 and debhelper 13.11.6. Even if
> we support a single implementation, we still need to know what is
> expected of it.
does not specify a version, and does not specify how they do what they do.
Bugs are handled as bugs against the packages that supply those tools, and
what they do, except at a very high level, is opaque to Policy. Examples
include update-rc.d, invoke-rc.d, adduser, and ldconfig, as well as the
obvious example of dpkg (which doesn't really count).
Now, I realize that debhelper feels conceptually different than those, and
I think to some extent it *is* conceptually different. Those are all
tools that do a single thing, whereas debhelper is an infrastructure that
has a lot of implications for how the package is defined and assembled.
It is a big step to go from requiring that all packages use a specific
tool to requiring that packages of a specific type use a particular
packaging framework. That's why I wanted to have a discussion about that
first.
The alternative is to adopt deb-systemd-invoke and deb-systemd-helper as
tools similar to update-rc.d and invoke-rc.d and specify how packages
should call them. This is indeed what I had started doing. But when I
started writing the language for this, it quickly became obvious that
those tools were written for debhelper and debhelper embeds a lot of
information about how to call them. (deb-systemd-invoke, for an obvious
example, has a very minimal man page.)
The invocation added by dh_installsystemd is also not entirely trivial.
Here is an example from wpasupplicant, reformatted a bit to make it more
readable in a mail message (and also note that my system has several
versions of this, so it's clearly under active development):
# Automatically added by dh_installsystemd/13.11.6
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] \
|| [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
# The following line should be removed in trixie or trixie+1
deb-systemd-helper unmask 'wpa_supplicant.service' >/dev/null || true
# was-enabled defaults to true, so new installations run enable.
if deb-systemd-helper --quiet was-enabled 'wpa_supplicant.service'; then
# Enables the unit on first installation, creates new
# symlinks on upgrades if the unit file has changed.
deb-systemd-helper enable 'wpa_supplicant.service' >/dev/null || true
else
# Update the statefile to add new symlinks (if any), which need to be
# cleaned up on purge. Also remove old symlinks.
deb-systemd-helper update-state 'wpa_supplicant.service' >/dev/null || true
fi
fi
# End automatically added section
# Automatically added by dh_installsystemd/13.11.6
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] \
|| [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
if [ -d /run/systemd/system ]; then
systemctl --system daemon-reload >/dev/null || true
if [ -n "$2" ]; then
_dh_action=restart
else
_dh_action=start
fi
deb-systemd-invoke $_dh_action 'wpa_supplicant.service' >/dev/null || true
fi
fi
# End automatically added section
This is postinst, which is the most complex, but prerm and postrm snippets
are also required.
This felt like kind of a lot to put into Policy. There is a line that
apparently won't be needed in the future, there's an update-state concept
that's internal to the tool, and the logic differs based on whether this
is a package upgrade or a new installation. We can put this all into
Policy, but I would just be blindly copying and pasting code from
debhelper, and I'm very dubious that it is going to stay accurate and up
to date (as witnessed by the fact that the existing snippets in Policy for
using update-rc.d and invoke-rc.d do not match what debhelper does and are
likely broken for packages that also include systemd units or packages
that need to care about DPKG_ROOT).
Also, honestly, nearly all packagers think of debhelper as an opaque tool
that everyone uses, and I have a larger concern that this makes Policy
much less useful for a large part of its intended audience than it could
be. Quite a lot of Policy is of the general format "here's a bunch of
complex things you need to do, wait, never mind, just use debhelper, go
see its documentation for the configuration files you should use instead"
and some of the rest of Policy is "here's a bunch of complex things you
need to do but if you follow these instructions instead of using debhelper
your package will be buggy." This is not ideal!
I think there's a lot of appeal of having a thorough specification for
what debhelper is supposed to be doing. It would enable future
competition around better packaging helpers (and I do think debhelper will
not be the last word). But I also want to be realistic about whether
we're really capable of maintaining that specification.
--
Russ Allbery (r...@debian.org) <https://www.eyrie.org/~eagle/>
Sam Hartman
Sep 11, 2023, 4:40:03 PM9/11/23
to
>>>>> "Bill" == Bill Allombert <ball...@debian.org> writes:
Bill> But we do: we support debhelper 13.11.4 and debhelper 13.11.6.
Bill> Even if we support a single implementation, we still need to
Bill> know what is expected of it.
At that level, I think the answer is roughly that if you call
dh_installsystemd, then any units in the package installation directory,
or any units matching certain file patterns in the debian directory will
be installed and if appropriate enabled.
I understand there's some work to do:
* do we support units in /lib/systemd/system, /usr/lib/systemd/system,
or both.
* What are those filename patterns for finding service units under
debian?
* You might implicitly call dh_installsystemd via dh
I think Russ might be arguing that we should actually push all that out
to the Debhelper documentation. I'd support that, although I understand
that makes the debhelper 13.4 vs 13.6 issue more relevant. I'd also
support describing a bit of the interface between debian/rules and
debhelper wrt systemd units if it allowed us to move forward.
My argument is that even if we support multiple versions of debhelper,
we do not need to define the interface between debhelper and systemd in
policy: we do not need to specify deb-systemd-helper and
deb-systemd-invoke.
--Sam
Bill> But we do: we support debhelper 13.11.4 and debhelper 13.11.6.
Bill> Even if we support a single implementation, we still need to
Bill> know what is expected of it.
At that level, I think the answer is roughly that if you call
dh_installsystemd, then any units in the package installation directory,
or any units matching certain file patterns in the debian directory will
be installed and if appropriate enabled.
I understand there's some work to do:
* do we support units in /lib/systemd/system, /usr/lib/systemd/system,
or both.
* What are those filename patterns for finding service units under
debian?
* You might implicitly call dh_installsystemd via dh
I think Russ might be arguing that we should actually push all that out
to the Debhelper documentation. I'd support that, although I understand
that makes the debhelper 13.4 vs 13.6 issue more relevant. I'd also
support describing a bit of the interface between debian/rules and
debhelper wrt systemd units if it allowed us to move forward.
My argument is that even if we support multiple versions of debhelper,
we do not need to define the interface between debhelper and systemd in
policy: we do not need to specify deb-systemd-helper and
deb-systemd-invoke.
--Sam
Russ Allbery
Sep 17, 2023, 6:00:04 PM9/17/23
to
Niels Thykier <ni...@thykier.net> writes:
> I had a look at the introduction section of Policy to check who the
> target audience is. I cannot find an explicit mention of the target
> audience. I suspect there is a conflict here on the content because we
> have different audiences in mind for the Policy and the Policy does not
> seem to be explicit here.
> If the target audience is package maintainers, then 100% of all debian
> contributors should read it. Then we need "simple and easy to
> understand" language and examples, because we want *everybody* to
> understand this. I agree with Russ that long winded sections of "here
> is how you do this manually but really just use debhelper" is directly
> counter productive to this audience.
> If it is cross-package integration, are we targeting the ones
> integrating or the ones providing the integration points? If it is
> both, then for more complex topics, it would make sense to split the
> topic into two. One for consumer and one for the provider of the
> integration, because for the consumer it will probably boil down to
> "install path at $X in your deb and run tool $Y" and then the consumer
> can skip the provider section as "not relevant".
> If the target audience is tool-chain maintainers, then only 5-10 people
> need to read the policy and the entire document + related process seems
> completely over-engineered. But then, for the same reason, I suspect
> this is an unlikely target audience for the Policy.
Ooo, this is a great framing of the problem. I have a lot of thoughts
about this. Unfortunately, I'm not sure if they're actionable thoughts
since my grand vision requires someone to sit down and do some serious
Policy restructuring and produce a radically different document. But
maybe if I write them all down and enough people feel similarly, it would
be worth doing. I would love to work on this, I am just afraid that it is
the sort of project that I would start and never finish and thus never
accomplish anything of use.
I think that ideally Policy targets all three of your audiences, but not
at the same time, and not with the same priority.
I have a lot of problems with the current structure of Policy, to the
extent that it sometimes interferes with my motivation for working on it,
and one of the big ones is that it doesn't follow any structured design
pattern for documentation, such as Divio [1]. It may have at one point,
but iteratively over the years, and due to some decisions like trying to
retain section numbering, we've diverged from that. Even if it were
trying to be only one type of documentation, it doesn't consistently stay
in that lane.
[1] https://documentation.divio.com/
I think my ideal structure of Policy would have three major parts.
First, there would be Policy for Debian packagers. This would focus on
the things someone packaging for Debian needs to know, and would be
organized roughly around task. Example sections here would be:
* Choosing an archive area
* Files on the file system (FHS, ownership, permissions, etc.)
* Writing a debian/control file
* Writing a changelog
* Writing a copyright file
* Packaging a shared library
* Packaging a system service
* Using maintainer scripts
In other words, this section would consist primarily of Divio how-to
guides, with some intermixed Divio explanations. (Tutorials I think are
outside the scope of Policy and are better handled elsewhere, such as in
debhelper documentation.)
(This is very rough, this is not the right order, etc. This is just to
provide a feel of the nature of this section.)
This section would assume that you're using a packaging helper and not
tell you how to do things that the packaging helper is going to do for
you. There's an awkward line here between describing what to do and being
debhelper documentation, but the basic idea is that this would tell you
what you are aiming for, and then your packaging helper documentation will
tell you where to put configuration files to achieve that.
Second section would be Policy the reference manual for interfaces in the
distribution. Sections here would be:
* Complete list of control fields and their meanings.
* Specifications for the .dsc and .changes files (which packagers mostly
never need to care about, but tool maintainers do).
* The detailed reference documentation on all the ways maintainer scripts
can be called.
* Specification for the symbols and shlibs files.
This is all Divio reference stuff. Whenever we have a comprehensive
explanation of the details of something, it goes here, and then we
liberally link to the reference section from the packaging-focused how-to
section for more details.
Finally, there is the reference manual for toolchain maintainers. My
position on this is that it's best-effort documentation and should
probably be a non-normative appendix where toolchain maintainers are
relatively free to just make changes without going through a very formal
process as long as those changes reflect reality. This is, by nature,
going to be incomplete and possibly out of date, but I do think the
project should have *somewhere* outside of any specific package where
people can write down the details of, oh, the other options to
Rules-Requires-Root that we aren't currently using. But we would stick a
lot of disclaimers on it and make it clear that this is internals that
only 10-20 people really need to know about.
I don't know if we can get here, but personally I think it would be a
massive improvement over where we are now, even if we spend the next ten
years sorting out structural problems with how we moved things around.
But it's a *ton* of work, so realistically it's never going to happen
unless it excites other people as much as it does me.
Anyway, that's a very long-winded answer to Niels's question. I think
Policy should primarily have an audience of packagers, including packagers
who need to coordinate cross-package integrations, secondarily have an
audience of tool makers who need a reference manual for Debian's file
formats and integrations, and then have a deprioritized tertiary audience
of toolchain maintainers.
> I had a look at the introduction section of Policy to check who the
> target audience is. I cannot find an explicit mention of the target
> audience. I suspect there is a conflict here on the content because we
> have different audiences in mind for the Policy and the Policy does not
> seem to be explicit here.
> If the target audience is package maintainers, then 100% of all debian
> contributors should read it. Then we need "simple and easy to
> understand" language and examples, because we want *everybody* to
> understand this. I agree with Russ that long winded sections of "here
> is how you do this manually but really just use debhelper" is directly
> counter productive to this audience.
> If it is cross-package integration, are we targeting the ones
> integrating or the ones providing the integration points? If it is
> both, then for more complex topics, it would make sense to split the
> topic into two. One for consumer and one for the provider of the
> integration, because for the consumer it will probably boil down to
> "install path at $X in your deb and run tool $Y" and then the consumer
> can skip the provider section as "not relevant".
> If the target audience is tool-chain maintainers, then only 5-10 people
> need to read the policy and the entire document + related process seems
> completely over-engineered. But then, for the same reason, I suspect
> this is an unlikely target audience for the Policy.
Ooo, this is a great framing of the problem. I have a lot of thoughts
about this. Unfortunately, I'm not sure if they're actionable thoughts
since my grand vision requires someone to sit down and do some serious
Policy restructuring and produce a radically different document. But
maybe if I write them all down and enough people feel similarly, it would
be worth doing. I would love to work on this, I am just afraid that it is
the sort of project that I would start and never finish and thus never
accomplish anything of use.
I think that ideally Policy targets all three of your audiences, but not
at the same time, and not with the same priority.
I have a lot of problems with the current structure of Policy, to the
extent that it sometimes interferes with my motivation for working on it,
and one of the big ones is that it doesn't follow any structured design
pattern for documentation, such as Divio [1]. It may have at one point,
but iteratively over the years, and due to some decisions like trying to
retain section numbering, we've diverged from that. Even if it were
trying to be only one type of documentation, it doesn't consistently stay
in that lane.
[1] https://documentation.divio.com/
I think my ideal structure of Policy would have three major parts.
First, there would be Policy for Debian packagers. This would focus on
the things someone packaging for Debian needs to know, and would be
organized roughly around task. Example sections here would be:
* Choosing an archive area
* Files on the file system (FHS, ownership, permissions, etc.)
* Writing a debian/control file
* Writing a changelog
* Writing a copyright file
* Packaging a shared library
* Packaging a system service
* Using maintainer scripts
In other words, this section would consist primarily of Divio how-to
guides, with some intermixed Divio explanations. (Tutorials I think are
outside the scope of Policy and are better handled elsewhere, such as in
debhelper documentation.)
(This is very rough, this is not the right order, etc. This is just to
provide a feel of the nature of this section.)
This section would assume that you're using a packaging helper and not
tell you how to do things that the packaging helper is going to do for
you. There's an awkward line here between describing what to do and being
debhelper documentation, but the basic idea is that this would tell you
what you are aiming for, and then your packaging helper documentation will
tell you where to put configuration files to achieve that.
Second section would be Policy the reference manual for interfaces in the
distribution. Sections here would be:
* Complete list of control fields and their meanings.
* Specifications for the .dsc and .changes files (which packagers mostly
never need to care about, but tool maintainers do).
* The detailed reference documentation on all the ways maintainer scripts
can be called.
* Specification for the symbols and shlibs files.
This is all Divio reference stuff. Whenever we have a comprehensive
explanation of the details of something, it goes here, and then we
liberally link to the reference section from the packaging-focused how-to
section for more details.
Finally, there is the reference manual for toolchain maintainers. My
position on this is that it's best-effort documentation and should
probably be a non-normative appendix where toolchain maintainers are
relatively free to just make changes without going through a very formal
process as long as those changes reflect reality. This is, by nature,
going to be incomplete and possibly out of date, but I do think the
project should have *somewhere* outside of any specific package where
people can write down the details of, oh, the other options to
Rules-Requires-Root that we aren't currently using. But we would stick a
lot of disclaimers on it and make it clear that this is internals that
only 10-20 people really need to know about.
I don't know if we can get here, but personally I think it would be a
massive improvement over where we are now, even if we spend the next ten
years sorting out structural problems with how we moved things around.
But it's a *ton* of work, so realistically it's never going to happen
unless it excites other people as much as it does me.
Anyway, that's a very long-winded answer to Niels's question. I think
Policy should primarily have an audience of packagers, including packagers
who need to coordinate cross-package integrations, secondarily have an
audience of tool makers who need a reference manual for Debian's file
formats and integrations, and then have a deprioritized tertiary audience
of toolchain maintainers.
Russ Allbery
Sep 20, 2023, 3:40:03 AM9/20/23
to
Niels Thykier <ni...@thykier.net> writes:
> Russ Allbery:
>> Ooo, this is a great framing of the problem. I have a lot of thoughts
>> about this. Unfortunately, I'm not sure if they're actionable thoughts
>> since my grand vision requires someone to sit down and do some serious
>> Policy restructuring and produce a radically different document. But
>> maybe if I write them all down and enough people feel similarly, it
>> would be worth doing. I would love to work on this, I am just afraid
>> that it is the sort of project that I would start and never finish and
>> thus never accomplish anything of use.
> Indeed, it is definitely the thing I would personally prefer to
> pre-align on before adventuring on something of this scale myself (were
> I in your shoes), so I totally feel your concern about actionability.
I do to some extent want people to encourage me to work on this if they
think it would be awesome, since people being happy with it would be what
would make all the work worthwhile (although I will probably also need
help). :)
> Interesting choice with the mixing. I was of the understanding that the
> idea was one should try to avoid mixing documentation styles according
> to Divio. Admittedly, I find it hard to fully stick to exactly one type
> of style, so I would be happy if I had just overlooked the "advice on
> mixing".
So, I have to admit that I have not read Divio in detail although I have
been pointed at it many times and have had the high-level concepts
explained to me. I've read bits and pieces, but I'm not sure what it says
about mixing.
But in terms of what makes an effective Policy document, I think a general
structure of an explanation section introducing a part of packaging
followed by how-to sections for the underlying tasks would work.
> As for the content: The "How-to" style would make sense for the target
> audience. I am less clear if all of these headlines makes up a "Policy"
> as they sounds like something you could find in a "Debian Packaging 101"
> guide.
I think that about a quarter of Policy currently is already how-to text.
For example, look at Policy 3.4:
https://www.debian.org/doc/debian-policy/ch-binary.html#s-descriptions
This is a disguised how-to document on how to write a good package
description. There's a ton of stuff in Policy like this. What
distinguishes it from a Debian Packaging 101 guide is that the how-to goes
into *way* more depth than a 101 guide: edge cases, exceptions, advanced
bits of packaging, etc.
The main problem from the perspective of helping the typical packager, is
that this is mixed in with oodles of advice that is irrelevant to anyone
except debhelper maintainers. To take another short example, look at the
section on symbolic links:
https://www.debian.org/doc/debian-policy/ch-files.html#symbolic-links
Half of that section is a specification for the packaging helper, which
will fix symbolic links to follow those rules. The rest is sort of a
how-to (mixed in with some basic shell command advice).
> Side question: Does Policy add anything on the specification for
> `symbols` and `shlibs` files as a reference document that is not covered
> by dpkg's documentation for these formats? I assume the "symbols guide"
> (on /when/ to use symbols and when not too) would go in the previous
> section.
Well, one can read the two side-by-side and see, and I'm biased as the
person who wrote it, but I think it does.
Policy duplicates dpkg documentation quite a lot, so this is a question
worth asking, but I do think Policy has a good answer. The value that
Policy adds over the dpkg documentation generally falls into three
categories:
1. Policy is much more opinionated about what features supported by dpkg
should be used in packages and how they should be used. There are
sometimes things dpkg *can* do that we don't do. A bunch of this is
how-to, but I think some of this is reference.
2. Policy is sometimes a lot more verbose and offers worked examples, and
sometimes benefits from the additional formatting tools available in
Sphinx. This could be imported into the dpkg documentation to some
extent, of course, but as it stands I think there's value there.
3. dpkg documentation has to cover the complete operation of dpkg, whereas
Policy, even in reference sections and packaging helper sections,
should only need to cover the surface area that's visible to packaging
at the lowest level.
I agree that this is a source of some duplication of effort, although in
my experience it's not the part that takes the most time. (But I haven't
written triggers or multiarch documentation for Policy yet, so maybe it's
part of the problem.)
> What is it you would see go into this section [packaging helper
> specification] that is not the previous section?
Literally everything in Policy that you shouldn't think about because the
packaging helpers will cover it. For example, the symlink
canonicalization stuff mentioned above. How to run update-rc.d in
maintainer scripts. How to compress man pages. All that stuff that
Policy is riddled with right now that's basically a specification for
debhelper. I don't want any of that stuff even in reference ideally,
because the person packaging doesn't need to know any of it 99% of the
time so it just eats up cognitive space unless you're digging into what
debhelper is doing.
But I don't want to *delete* it, because there is that 1% case where you
have some really weird packaging problem and debhelper is doing the wrong
thing.
I think this is distinct from the pure reference stuff like, say, the
specification for how maintainer scripts work or the list of all the
fields in .dsc and .changes files. You *do* have to know that stuff if
you're doing something advanced or if you're writing other tools that
interact with Debian packages that aren't packaging helpers, so the
audience is a fair bit larger.
> I think it would take the Policy editors to have consensus on this as a
> direction and then go step by step from there. Maybe turn the current
> Policy into "Section II" as starting point and extract stuff into
> "Section I" piece by piece.
Yeah, I was thinking of something like that.
> Lintian maintainer will love you for breaking all the section numbers
> though! :D (I don't have a good solution for that problem).
What I want to do is give sections persistent short codes that are easy to
find and search for somehow and ideally should also serve as HTML anchors.
Maybe we can do something with a Sphinx extension.
> In that narrative, I think it would make sense for Policy to reference our
> primary packaging toolchain a lot more - at least in Section I. As a
> toolchain writer, my primary concern here is inertia.
> I would prefer that the references enable the tool to evolve over
> time. As an example, "This section assumes debhelper-compat (= 13)"
> could work because that phrase implies that things can change in a
> future compat level.
Yup, that makes sense. I also want to leave space for entirely new
packaging helpers to crop up. We've been in a fallow period for that for
a while, with everything converging on debhelper, but my (informed :P)
guess is that this is somewhat cyclical. Definitely a detail that we'd
need to work out.
> Russ Allbery:
>> Ooo, this is a great framing of the problem. I have a lot of thoughts
>> about this. Unfortunately, I'm not sure if they're actionable thoughts
>> since my grand vision requires someone to sit down and do some serious
>> Policy restructuring and produce a radically different document. But
>> maybe if I write them all down and enough people feel similarly, it
>> would be worth doing. I would love to work on this, I am just afraid
>> that it is the sort of project that I would start and never finish and
>> thus never accomplish anything of use.
> pre-align on before adventuring on something of this scale myself (were
> I in your shoes), so I totally feel your concern about actionability.
I do to some extent want people to encourage me to work on this if they
think it would be awesome, since people being happy with it would be what
would make all the work worthwhile (although I will probably also need
help). :)
> Interesting choice with the mixing. I was of the understanding that the
> idea was one should try to avoid mixing documentation styles according
> to Divio. Admittedly, I find it hard to fully stick to exactly one type
> of style, so I would be happy if I had just overlooked the "advice on
> mixing".
So, I have to admit that I have not read Divio in detail although I have
been pointed at it many times and have had the high-level concepts
explained to me. I've read bits and pieces, but I'm not sure what it says
about mixing.
But in terms of what makes an effective Policy document, I think a general
structure of an explanation section introducing a part of packaging
followed by how-to sections for the underlying tasks would work.
> As for the content: The "How-to" style would make sense for the target
> audience. I am less clear if all of these headlines makes up a "Policy"
> as they sounds like something you could find in a "Debian Packaging 101"
> guide.
I think that about a quarter of Policy currently is already how-to text.
For example, look at Policy 3.4:
https://www.debian.org/doc/debian-policy/ch-binary.html#s-descriptions
This is a disguised how-to document on how to write a good package
description. There's a ton of stuff in Policy like this. What
distinguishes it from a Debian Packaging 101 guide is that the how-to goes
into *way* more depth than a 101 guide: edge cases, exceptions, advanced
bits of packaging, etc.
The main problem from the perspective of helping the typical packager, is
that this is mixed in with oodles of advice that is irrelevant to anyone
except debhelper maintainers. To take another short example, look at the
section on symbolic links:
https://www.debian.org/doc/debian-policy/ch-files.html#symbolic-links
Half of that section is a specification for the packaging helper, which
will fix symbolic links to follow those rules. The rest is sort of a
how-to (mixed in with some basic shell command advice).
> Side question: Does Policy add anything on the specification for
> `symbols` and `shlibs` files as a reference document that is not covered
> by dpkg's documentation for these formats? I assume the "symbols guide"
> (on /when/ to use symbols and when not too) would go in the previous
> section.
Well, one can read the two side-by-side and see, and I'm biased as the
person who wrote it, but I think it does.
Policy duplicates dpkg documentation quite a lot, so this is a question
worth asking, but I do think Policy has a good answer. The value that
Policy adds over the dpkg documentation generally falls into three
categories:
1. Policy is much more opinionated about what features supported by dpkg
should be used in packages and how they should be used. There are
sometimes things dpkg *can* do that we don't do. A bunch of this is
how-to, but I think some of this is reference.
2. Policy is sometimes a lot more verbose and offers worked examples, and
sometimes benefits from the additional formatting tools available in
Sphinx. This could be imported into the dpkg documentation to some
extent, of course, but as it stands I think there's value there.
3. dpkg documentation has to cover the complete operation of dpkg, whereas
Policy, even in reference sections and packaging helper sections,
should only need to cover the surface area that's visible to packaging
at the lowest level.
I agree that this is a source of some duplication of effort, although in
my experience it's not the part that takes the most time. (But I haven't
written triggers or multiarch documentation for Policy yet, so maybe it's
part of the problem.)
> What is it you would see go into this section [packaging helper
> specification] that is not the previous section?
Literally everything in Policy that you shouldn't think about because the
packaging helpers will cover it. For example, the symlink
canonicalization stuff mentioned above. How to run update-rc.d in
maintainer scripts. How to compress man pages. All that stuff that
Policy is riddled with right now that's basically a specification for
debhelper. I don't want any of that stuff even in reference ideally,
because the person packaging doesn't need to know any of it 99% of the
time so it just eats up cognitive space unless you're digging into what
debhelper is doing.
But I don't want to *delete* it, because there is that 1% case where you
have some really weird packaging problem and debhelper is doing the wrong
thing.
I think this is distinct from the pure reference stuff like, say, the
specification for how maintainer scripts work or the list of all the
fields in .dsc and .changes files. You *do* have to know that stuff if
you're doing something advanced or if you're writing other tools that
interact with Debian packages that aren't packaging helpers, so the
audience is a fair bit larger.
> I think it would take the Policy editors to have consensus on this as a
> direction and then go step by step from there. Maybe turn the current
> Policy into "Section II" as starting point and extract stuff into
> "Section I" piece by piece.
Yeah, I was thinking of something like that.
> Lintian maintainer will love you for breaking all the section numbers
> though! :D (I don't have a good solution for that problem).
What I want to do is give sections persistent short codes that are easy to
find and search for somehow and ideally should also serve as HTML anchors.
Maybe we can do something with a Sphinx extension.
> In that narrative, I think it would make sense for Policy to reference our
> primary packaging toolchain a lot more - at least in Section I. As a
> toolchain writer, my primary concern here is inertia.
> I would prefer that the references enable the tool to evolve over
> time. As an example, "This section assumes debhelper-compat (= 13)"
> could work because that phrase implies that things can change in a
> future compat level.
Yup, that makes sense. I also want to leave space for entirely new
packaging helpers to crop up. We've been in a fallow period for that for
a while, with everything converging on debhelper, but my (informed :P)
guess is that this is somewhat cyclical. Definitely a detail that we'd
need to work out.
Bill Allombert
Sep 20, 2023, 1:00:04 PM9/20/23
to
Hello Russ,
In my view the main purpose of policy is to allow interoperability by defining
interfaces between packages.
We used to have a separate Packaging Manual, but it has been merged with
Policy a long time ago. The intent was to reduce duplication which lead to
outdated information.
However, this directly lead to the structural problem we have now.
While it would be OK for a Packaging Manual to say "just use debhelper"
that does not help the debhelper developers that need to know what it
is expected of debhelper.
In my view the main purpose of policy is to allow interoperability by defining
interfaces between packages.
We used to have a separate Packaging Manual, but it has been merged with
Policy a long time ago. The intent was to reduce duplication which lead to
outdated information.
However, this directly lead to the structural problem we have now.
While it would be OK for a Packaging Manual to say "just use debhelper"
that does not help the debhelper developers that need to know what it
is expected of debhelper.
0 new messages