Debian Configuration Packaging System
Timothy G Abbott
tentatively packaged as the config-package-dev package) for creating
Debian configuration packages. By configuration packages, we mean
packages that configure an existing Debian system by applying dpkg-divert
to configuration files. Our configuration package system makes the
process of creating configuration packages efficient.
Our system is targeted at site defaults (i.e. configuration for a
university or a company), though it is useful for smaller scales as well.
It has some support for multiple layers of site defaults, e.g. MIT, CSAIL
(an MIT lab), and a research group within CSAIL might all use it to
configure their machines [1].
The configuration package system is documented in detail at
<http://debathena.mit.edu/config-packages/>; there are links from there to
the complete source code and compiled Debian packages. The license is GPL
(the same as that for CDBS itself).
Since this package is adding a new feature to Debian itself, we think our
system should be discussed before we submit an ITP bug. There are some
changes to Debian that would enhance the effectiveness of this system,
(such as having all packages include md5sums and making ucf interact well
with dpkg-divert'ed configuration files), which should perhaps be
discussed in this context as well.
We would appreciate any questions, comments, or feedback.
-Tim Abbott and Anders Kaseorg
[1] A version of config-package-dev has been in use as part of the
Debathena Project (http://debathena.mit.edu/) at MIT for a few years now.
Debathena is an enhanced port of Athena (MIT's cross-platform computing
environment) from RHEL 4 and Solaris 10 to Debian (and Ubuntu). It's been
adopted by MIT's introductory computer science class and some small
clusters; but has been particularly popular for private machines whose
owners want to access Athena services easily (for example, AFS and
Kerberos should "just work") on an existing Debian/Ubuntu machine that
they are free to configure. Athena is planning to migrate to Debathena
later this year.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
> Anders Kaseorg and I created a system of CDBS modules (which we've
> tentatively packaged as the config-package-dev package) for creating
> Debian configuration packages. By configuration packages, we mean
> packages that configure an existing Debian system by applying
> dpkg-divert to configuration files. Our configuration package system
> makes the process of creating configuration packages efficient.
It's generally accepted wisdom that dpkg-divert doesn't work properly with
configuration files and isn't safe. I admit to have done something
similar in the past, but I have noticed odd things that didn't matter for
my particular use, but which meant that the support didn't work right.
That's likely to be an issue for a general package. Fixing dpkg-divert to
work correctly with configuration files (if possible) would probably be a
good idea.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
Russ Allbery
> We also ran into a few packages which will overwrite configuration files
> that they manage via debconf, overwriting our symlink every time the
> relevant package is upgraded. But I think that's a bug in those Debian
> packages, since the same problem would occur for any manual changes to
> those configuration files as well (I think in the cases I've seen it is
> a failure to check whether an upgrade is occuring when generating the
> configuration file in postinst).
Configuration files generated by debconf may not be manually changed
without running this risk, including by humans. Generally, this is
documented in the file. I have several of those in packages I maintain.
This is a pretty widely accepted way of dealing with configuration files,
and the right way to modify those files is with debconf pre-seeding rather
than by trying to overwrite the file, IMO.
I don't agree that this is a bug in the package in the general case,
although there may be packages that are more aggressive about overwriting
than need to be.
> What other problems have you experienced?
I've seen the diverted configuration file disappear, making it impossible
to undo the diversion, and never did track down why that happened. I
haven't run into any problems in cases where the diversion is never
dropped, though. (But renaming the package that manages the diversions is
something that dpkg-divert doesn't deal with at all well.)
We're using Puppet to handle this instead of Debian packages, and I'm much
happier with that solution. I wouldn't recommend doing what you're doing,
but of course you may have improved the tools sufficiently that it's a
good idea. :) Also, I know that most of the alternatives involve some
central management system, and there are a lot of use cases that don't
allow for that.
Tim Abbott
error-handling code that we found quite important for correctly recovering
from people hitting ^C in the middle of installation (see
<http://debathena/config-packages/code/config-package-dev-4.2/divert.sh.in>
for the code). Earlier iterations that did not do this were plagued with
problems whenever there were errors in installation.
We also ran into a few packages which will overwrite configuration files
that they manage via debconf, overwriting our symlink every time the
relevant package is upgraded. But I think that's a bug in those Debian
packages, since the same problem would occur for any manual changes to
those configuration files as well (I think in the cases I've seen it is a
failure to check whether an upgrade is occuring when generating the
configuration file in postinst).
What other problems have you experienced?
-Tim Abbott
Russ Allbery
> On Sun, 24 Feb 2008, Russ Allbery wrote:
>> Configuration files generated by debconf may not be manually changed
>> without running this risk, including by humans. Generally, this is
>> documented in the file. I have several of those in packages I
>> maintain. This is a pretty widely accepted way of dealing with
>> configuration files, and the right way to modify those files is with
>> debconf pre-seeding rather than by trying to overwrite the file, IMO.
> Many such files have configuration options other than those managed by
> debconf (/etc/krb5.conf would be a good example). For those files,
> pre-seeding debconf doesn't suffice.
And krb5.conf is not a configuration file managed in the way that you
describe for exactly that reason. It uses a conservative approach, only
installing a file based on debconf prompts if the file doesn't already
exist and otherwise trying to do automated updates of only the configured
data where applicable. (You can also make the file a symlink to disable
all automated modification.)
The ones that are overwritten completely that I'm aware of contain only
settings managed by debconf, or (as is the case for krb5-kdc and
krb5-admin-server) explicitly ask whether you want to manage the
configuration file through debconf or want to manage it manually so that
you can set more obscure options.
Tim Abbott
> Tim Abbott <tab...@MIT.EDU> writes:
>
>> We also ran into a few packages which will overwrite configuration files
>> that they manage via debconf, overwriting our symlink every time the
>> relevant package is upgraded. But I think that's a bug in those Debian
>> packages, since the same problem would occur for any manual changes to
>> those configuration files as well (I think in the cases I've seen it is
>> a failure to check whether an upgrade is occuring when generating the
>> configuration file in postinst).
>
> Configuration files generated by debconf may not be manually changed
> without running this risk, including by humans. Generally, this is
> documented in the file. I have several of those in packages I maintain.
> This is a pretty widely accepted way of dealing with configuration files,
> and the right way to modify those files is with debconf pre-seeding rather
> than by trying to overwrite the file, IMO.
Many such files have configuration options other than those managed by
debconf (/etc/krb5.conf would be a good example). For those files,
pre-seeding debconf doesn't suffice.
>> What other problems have you experienced?
>
> I've seen the diverted configuration file disappear, making it impossible
> to undo the diversion, and never did track down why that happened. I
> haven't run into any problems in cases where the diversion is never
> dropped, though. (But renaming the package that manages the diversions is
> something that dpkg-divert doesn't deal with at all well.)
Renaming a package requires no special effort in our system (the new
package and the old package will conflict because they divert the same
file, and the old package will remove the diversion on prerm because it
knows it is being removed, and then the new package will install the new
diversion on postinst).
-Tim Abbott
Frank Küster
> Anders Kaseorg and I created a system of CDBS modules (which we've
> tentatively packaged as the config-package-dev package) for creating
> Debian configuration packages. By configuration packages, we mean
> packages that configure an existing Debian system by applying dpkg-divert
> to configuration files. Our configuration package system makes the
> process of creating configuration packages efficient.
Uh, you can dpkg-divert conffiles, but not generally configuration files, since
many won't even be known to dpkg. I must admit I'm a bit sceptical about a
proposal on configuration, written by someone who lets this important
distinction slip by...
Later on you wrote something about symlinks, do you use them, too? Here's one
more complication: While I think that "preserve local changes" also includes the
symlink-or-not state of a file and thus must be respected by maintainer scripts,
I do not think that this is usually done. In particular, every configuration
file handling that was first written for sarge (where sed -i wouldn't work)
probably used tempfiles and mv, and I have not seen a single case where the
script checked whether it was dealing with a symlink.
[...]
> Since this package is adding a new feature to Debian itself, we think our
> system should be discussed before we submit an ITP bug. There are some
> changes to Debian that would enhance the effectiveness of this system,
> (such as having all packages include md5sums and making ucf interact well
> with dpkg-divert'ed configuration files), which should perhaps be
> discussed in this context as well.
It's not just ucf, it's also perfectly possible that a maintainer script edits a
configuration file, e.g. after debconf prompting, in order to take over user
changes from a related package's configuration file, or similar.
How do you deal with that?
Regards, Frank
Josselin Mouette
> The ones that are overwritten completely that I'm aware of contain only
> settings managed by debconf, or (as is the case for krb5-kdc and
> krb5-admin-server) explicitly ask whether you want to manage the
> configuration file through debconf or want to manage it manually so that
> you can set more obscure options.
This is really a bad idea, because you always end up modifying a
debconf-managed file by hand. The solution to this problem is to use ucf
instead.
--
.''`.
: :' : We are debian.org. Lower your prices, surrender your code.
`. `' We will add your hardware and software distinctiveness to
`- our own. Resistance is futile.
Russ Allbery
> Le dimanche 24 février 2008 à 19:46 -0800, Russ Allbery a écrit :
>> The ones that are overwritten completely that I'm aware of contain only
>> settings managed by debconf, or (as is the case for krb5-kdc and
>> krb5-admin-server) explicitly ask whether you want to manage the
>> configuration file through debconf or want to manage it manually so
>> that you can set more obscure options.
> This is really a bad idea, because you always end up modifying a
> debconf-managed file by hand. The solution to this problem is to use ucf
> instead.
Reading the ucf man page, I don't understand how it would help me for the
case of those two packages. But I may be suffering from insufficient
imagination.
The problem is this: both krb5-kdc and krb5-admin-server have /etc/default
files that control various aspects of the startup of the servers, such as
whether a krb524d is run and what level of Kerberos v4 support is enabled
in the KDC. All of those parameters are prompted for by debconf if you
select "manage my configuration with Debconf" and a corresponding
/etc/default file is written out. This lets people get the benefit of
translated prompts for the possible options.
However, people have also requested an escape hatch to pass arbitrary
options to the daemon without having to modify the init script, using the
semi-standard DAEMON_ARGS setting in the /etc/default file.
ucf, from its DESCRIPTION in its man page, seems to handle the case of
shipping a configuration file upstream that may also be locally modified,
but I don't see where it handles merging in the results of debconf
prompting. Am I just missing something?
The approach taken right now is that the user chooses whether the file is
debconf-managed or not debconf-managed, and a comment at the top of the
file tells the user to pick. Debconf-managed configuration seems to be
pretty common in Debian, although I know that some things that are common
still aren't good ideas and I'm happy to switch to something better.
(There are various bits of debconf cleanup in the krb5 packages that needs
to be done, so now's a good time to tell me how the whole thing *should*
work so that I can do it all at once.)
Josselin Mouette
> The problem is this: both krb5-kdc and krb5-admin-server have /etc/default
> files that control various aspects of the startup of the servers, such as
> whether a krb524d is run and what level of Kerberos v4 support is enabled
> in the KDC. All of those parameters are prompted for by debconf if you
> select "manage my configuration with Debconf" and a corresponding
> /etc/default file is written out. This lets people get the benefit of
> translated prompts for the possible options.
>
> However, people have also requested an escape hatch to pass arbitrary
> options to the daemon without having to modify the init script, using the
> semi-standard DAEMON_ARGS setting in the /etc/default file.
This is one of the standard ucf use cases.
> ucf, from its DESCRIPTION in its man page, seems to handle the case of
> shipping a configuration file upstream that may also be locally modified,
> but I don't see where it handles merging in the results of debconf
> prompting. Am I just missing something?
ucf can easily be fed with the debconf-generated file. When told to with
the proper option, it will even do a three-way merge of the proposed
(maintainer + debconf) changes to the locally modified file.
Russ Allbery
> On lun, 2008-02-25 at 10:23 -0800, Russ Allbery wrote:
>> ucf, from its DESCRIPTION in its man page, seems to handle the case of
>> shipping a configuration file upstream that may also be locally
>> modified, but I don't see where it handles merging in the results of
>> debconf prompting. Am I just missing something?
> ucf can easily be fed with the debconf-generated file. When told to with
> the proper option, it will even do a three-way merge of the proposed
> (maintainer + debconf) changes to the locally modified file.
Oh! Okay, that was the piece that I was missing. That's a better
solution than what I have now.
Thanks, I'll try to find time to look at this in the not-too-distant
future.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
Timothy G Abbott
> Uh, you can dpkg-divert conffiles, but not generally configuration files, since
> many won't even be known to dpkg. I must admit I'm a bit sceptical about a
> proposal on configuration, written by someone who lets this important
> distinction slip by...
No, I really meant configuration files. While our system certainly
applies to all conffiles, it also applies to various other classes of
files:
1) Those that are accidentally not marked as conffiles (e.g.
/etc/adduser.conf).
2) Those that are initially created via a system like debconf but that
preserve user changes, often because they support other configuration
options than those in debconf (e.g. /etc/krb5.conf or
/etc/ssh/sshd_config) [the dpkg-divert doesn't do anything here, but the
system still works].
3) Scripts that are not marked as conffiles but which cannot be configured
in any way other than by modifying the script.
I probably should have said this explicitly earlier, but our system is
currently only an 80% solution, because it cannot override any package's
configuration file handling system that does not preserve manual changes.
It turns out that these are fairly rare, and can be handled with some
annoyance (e.g. releasing a new version of our configuration package
whenever a new release of such a package comes out, so that the
configuration package wins).
I would like to see it become a 100% solution, since I think support for
easily creating configuration packages would be really valuable for a lot
of organizations using Debian and Ubuntu. The primary difficulties are
interacting nicely with systems like ucf and debconf; I don't think that
it is possible to achieve 100% without some changes to Debian itself.
Because of ucf's architecture, it would be easy to make ucf interact
nicely with our system (and our system should perhaps use ucf itself,
since it seems to be a strict improvement over the just marking things as
conffiles).
There are really two problems with debconf in our system. The first is
that debconf asks questions which our configuration package system will
override. Using 'DEBCONF_PRIORITY=critical apt-get install' limits them,
but some packages we configure prompt for information even with critical
DEBCONF_PRIORITY (is this a bug?). The second problem is that some
packages override any manual changes to the relevant configuration file.
Because the code to take action based on debconf selections in so
decentralized, this second problem is relatively hard.
For the second debconf problem, the easiest "solution" would be to add to
our system support for pre-seeding the debconf database for package X and
then running the configure script for package X. We have the tools to do
this, and it would work, but it would lose a property of our system that I
really like: support for uninstalling configuration packages.
I would prefer to come up with a solution that does not lose this feature.
I can think of several solutions preserving this feature that require
changing every package that blindly overwrites user configuration on
package upgrades to do some sort of check before doing so; I would love to
hear ideas for mechanisms for achieving our goals (see
<http://debathena.mit.edu/config-packages/#philosophy>) without having to
change all such packages.
> Later on you wrote something about symlinks, do you use them, too? Here's one
> more complication: While I think that "preserve local changes" also includes the
> symlink-or-not state of a file and thus must be respected by maintainer scripts,
> I do not think that this is usually done. In particular, every configuration
> file handling that was first written for sarge (where sed -i wouldn't work)
> probably used tempfiles and mv, and I have not seen a single case where the
> script checked whether it was dealing with a symlink.
To be precise, after installing a package that intends to configure
/etc/ldap/ldap.conf, the system will be in the following configuration:
$ ls -l /etc/ldap/ldap.conf* [irrelevant stat information removed]
/etc/ldap/ldap.conf -> ldap.conf.debathena
/etc/ldap/ldap.conf.debathena
/etc/ldap/ldap.conf.debathena-orig
$ /usr/sbin/dpkg-divert --list | grep ldap
diversion of /etc/ldap/ldap.conf to /etc/ldap/ldap.conf.debathena-orig by debathena-ldap-config
So, yes, our system uses both symlinks and dpkg-divert. Simply placing
files at /etc/ldap/ldap.conf (rather than using a symlink) doesn't work
because it would require running dpkg-divert at preinst rather than
postinst, which results in needing to pre-depend the packages whose
configuration one is changing, and lots of other problems.
> It's not just ucf, it's also perfectly possible that a maintainer script edits a
> configuration file, e.g. after debconf prompting, in order to take over user
> changes from a related package's configuration file, or similar.
>
> How do you deal with that?
Indeed, there's almost nothing that we can do about that. Fundamentally,
postinst scripts have arbitrary execution, and almost any system one
designs to handle configuration can be thwarted, at least temporarily, by
a package postinst script. I also haven't run across this in any package
I've cared about.
One idea that would deal with this problem (and solve all the other
problems we've discussed other than the unecessary prompting problem)
would be for dpkg to run postinst scripts in a special environment that
rewrites filenames according to the diversion database (an evil LD_PRELOAD
hack might work; chrooting into a special fuse filesystem certainly
would). However, this requires changes to dpkg that I doubt the dpkg
maintainers would accept (correct me if I'm wrong!).
-Tim Abbott
Frank Küster
>
> On Mon, 25 Feb 2008, Frank Küster wrote:
>
> > Uh, you can dpkg-divert conffiles, but not generally configuration files,
> > since many won't even be known to dpkg. I must admit I'm a bit sceptical
> > about a proposal on configuration, written by someone who lets this
> > important distinction slip by...
>
> No, I really meant configuration files. While our system certainly
> applies to all conffiles, it also applies to various other classes of
> files:
But in these cases you can't use dpkg-divert.
> 3) Scripts that are not marked as conffiles but which cannot be configured
> in any way other than by modifying the script.
If those scripts live below /etc, they definitely should be marked as conffiles.
If they live elsewhere, no package should edit them.
> I probably should have said this explicitly earlier, but our system is
> currently only an 80% solution, because it cannot override any package's
> configuration file handling system that does not preserve manual changes.
Such a package has a RC bug anyway.
> It turns out that these are fairly rare, and can be handled with some
> annoyance (e.g. releasing a new version of our configuration package
> whenever a new release of such a package comes out, so that the
> configuration package wins).
Of file a bug...
> So, yes, our system uses both symlinks and dpkg-divert.
Ah, I understand. I think here you have a much larger problem than with the
small number of RC-buggy packages that don't keep manual changes: Larger because
I fear there are more packages with such problems, because less people are aware
that it is a problem, and maybe even because there might be some debate whether
respecting a symlink state actually is required by policy.
> One idea [...]
> would be for dpkg to run postinst scripts in a special environment that
> rewrites filenames according to the diversion database
I would rather contemplate a new interface for configuration packages, something
like a hook which has to be executed by each postinst script and allows the
configuration package to bring its settings into effect. If you want to be able
to remove configuration packages, the original file needs to be preserved
somehow, but the current dpkg-divert is not sufficient for this:
Consider someone installing the configuration package in stable+0, then
upgrading to stable+1 and keeping it, and then upgrading to stable+2 and
subsequently removing the configuration package. Any special code in the
maintainer scripts which dealt with incompatible conf(iguration )file changes in
the stable-to-stable+1 upgrade is likely to have been removed by now, so the
old, incompatible, file from stable+0 will be used after the removal, and the
package is in a broken state.
Frans Pop
> There are really two problems with debconf in our system. The first is
> that debconf asks questions which our configuration package system will
> override. Using 'DEBCONF_PRIORITY=critical apt-get install' limits them,
> but some packages we configure prompt for information even with critical
> DEBCONF_PRIORITY (is this a bug?).
No. If you want to avoid all prompts you should use the noninteractive frontend.
Cheers,
FJP
Timothy G Abbott
[...]
>> 3) Scripts that are not marked as conffiles but which cannot be
>> configured in any way other than by modifying the script.
>
> If those scripts live below /etc, they definitely should be marked as
> conffiles. If they live elsewhere, no package should edit them.
I think it's totally reasonable for the MIT site configuration packages to
(e.g.) divert chsh to replace it with a wrapper that supports handling
both inherently local user accounts (like root or postfix) by calling
chsh.debathena-orig and network accounts (through some other mechanism).
While some might argue that instead MIT should create a modified version
of the passwd package to make this change, I think our solution is
substantially cleaner.
It's important to keep in mind is that while the system for creating
configuration packages should be part of Debian, the configuration
packages themselves are not distributed as part of Debian, but should
instead be considered as modifications to Debian. It makes sense for them
to be able to change some standard programs in a way that only makes sense
for that particular site, but doesn't break other uses of those programs,
as well as changing configuration file defaults.
[...]
>> One idea [...]
>> would be for dpkg to run postinst scripts in a special environment that
>> rewrites filenames according to the diversion database
>
> I would rather contemplate a new interface for configuration packages, something
> like a hook which has to be executed by each postinst script and allows the
> configuration package to bring its settings into effect. If you want to be able
Would deploying this new interface require modifying every Debian
package's postinst scripts to add support for this new hook? I'm not
enthusiastic about the deployment costs of that kind of sweeping change if
there are more localized changes to Debian that would work.
> to remove configuration packages, the original file needs to be preserved
> somehow, but the current dpkg-divert is not sufficient for this:
>
> Consider someone installing the configuration package in stable+0, then
> upgrading to stable+1 and keeping it, and then upgrading to stable+2 and
> subsequently removing the configuration package. Any special code in the
> maintainer scripts which dealt with incompatible conf(iguration )file changes in
> the stable-to-stable+1 upgrade is likely to have been removed by now, so the
> old, incompatible, file from stable+0 will be used after the removal, and the
> package is in a broken state.
That's certainly a potential problem with our current system.
The "filename rewriting" idea does not have this problem. During the
upgrades, the maintainer scripts for the package would update
/etc/ldap/ldap.conf.debathena-orig (rather than the running version), and
when the configuration package is uninstalled,
/etc/ldap/ldap.conf.debathena-orig will be moved back to
/etc/ldap/ldap.conf, leaving the package in the state it would have been
in had the configuration package never been installed in the first place.
The "filename rewriting" plan is really a natural extension to the current
dpkg-divert behavior; rather than just unpacking files in locations as
determined by the diversion database, all actions by the packaging system
would have their filenames transformed according to the diversion
database. I also can't think of anything that doing this would break --
its primary cost would be the added complexity in dpkg (and perhaps some
performance penalty, depending on implementation).
But it's also no silver bullet. There are packages that want to restart
their daemon after changing their configuration file, and it's unclear to
me how to prevent packages from restarting their daemon (or regerating
their database, etc.) in the "filename rewriting" environment.
-Tim Abbott
Ian Jackson
> applying dpkg-divert to configuration files.
Yikes.
Tim Abbott writes ("Re: Debian Configuration Packaging System"):
> I'll note that we wrap our dpkg-divert calls with a bunch of
> error-handling code that we found quite important for correctly recovering
> >from people hitting ^C in the middle of installation (see
> <http://debathena/config-packages/code/config-package-dev-4.2/divert.sh.in>
> for the code). Earlier iterations that did not do this were plagued with
> problems whenever there were errors in installation.
Yikes.
I think this is a bad idea.
Ian.
Timothy G Abbott
> Tim Abbott writes ("Re: Debian Configuration Packaging System"):
>> applying dpkg-divert to configuration files.
>
> Yikes.
At the moment, there are no choices for doing site configuration that are
general enough to change arbitrary configuration options and don't destroy
whatever configuration state the machine had before configuring it (as
well as suffering from a version of every bad interaction with existing
maintainer scripts that our system has).
All of the important problems with our dpkg-divert based configuration
package system that have been discussed in this forum are problems for any
configuration mechanism other than debconf preseeding. Debconf preseeding
is unacceptable for site configuration because it is insufficiently
general (often things need to be configured that are not set up as debconf
options).
Any system that is not based on debconf pre-seeding (or maintaining a fork
of all the relevant packages containing the relevant configuration files,
which is unmaintainable) will have all the problems with maintainer
scripts that our system has. Further, any configuration system that does
not use dpkg-divert will have to fight with dpkg over control of conffiles
(a problem that our system does not have).
It turns out that our system also works for almost all the other
configuration files that we've wanted to change (in particular, those
whose packages don't overwrite manual changes with answers from the
debconf database on package upgrades, something the Debian Policy Manual
forbids anyway).
So, anything attempting to achieve the functionality we want has to use
dpkg-divert, and I think the best way to remove the problems with dealing
with configuration files that are not conffiles should probably involve
making maintainer scripts that do overwrite existing configuration files
follow diversions when doing so (though one mechanism or another).
> Tim Abbott writes ("Re: Debian Configuration Packaging System"):
>> I'll note that we wrap our dpkg-divert calls with a bunch of
>> error-handling code that we found quite important for correctly recovering
>> from people hitting ^C in the middle of installation (see
>> <http://debathena.mit.edu/config-packages/code/config-package-dev-4.2/divert.sh.in>
>> for the code). Earlier iterations that did not do this were plagued with
>> problems whenever there were errors in installation.
>
> Yikes.
Those earlier iterations date to 2004 and were never deployed beyond a
testing environment. The particular shell code we're using now has been
basically unchanged since we wrote it back in Summer 2006, and has not
been a problem for us since.
-Tim Abbott
Ian Jackson
> All of the important problems with our dpkg-divert based configuration
> package system that have been discussed in this forum are problems for any
> configuration mechanism other than debconf preseeding. Debconf preseeding
> is unacceptable for site configuration because it is insufficiently
> general (often things need to be configured that are not set up as debconf
> options).
I think a better approach is just to have a package which writes stuff
over the configuration files without diverting them.
Such a package is putting itself in the position of the system
administrator, which is presumably what you intend. That's not
appropriate for a Debian package but for a site-specific configuration
setting system it's OK I think.
You do have to deal with conffile prompts and other kinds of things
that happen when software (package maintainer scripts, and dpkg) spot
you've modified the files. But if you do all of your installations in
a noninteractive mode (with a noninteractive debconf frontend and with
dpkg configured not to prompt about conffiles) you get predictable
behaviour which you can override as you like.
Ian.
Timothy G Abbott
> Timothy G Abbott writes ("Re: Debian Configuration Packaging System"):
>> All of the important problems with our dpkg-divert based configuration
>> package system that have been discussed in this forum are problems for any
>> configuration mechanism other than debconf preseeding. Debconf preseeding
>> is unacceptable for site configuration because it is insufficiently
>> general (often things need to be configured that are not set up as debconf
>> options).
>
> I think a better approach is just to have a package which writes stuff
> over the configuration files without diverting them.
>
> Such a package is putting itself in the position of the system
> administrator, which is presumably what you intend. That's not
> appropriate for a Debian package but for a site-specific configuration
> setting system it's OK I think.
If we were running a cluster of identical machines, such a system would
probably work. But we actually don't intend to be in the position of the
system administrator; we just want to provide defaults in a decentralized
environment with many different sysadmins. This is a harder setting, and
I think our users find it useful that they can run etch or sid or even
Ubuntu as needed to get their work done and benefit from the site
defaults.
Our current deployment is on a few hundred machines, primarily personal
machines owned by individuals who want to be able to access MIT services,
but who do their own system administration. We offer various metapackages
with different levels of login integration, and we support all Debian and
Ubuntu releases from sarge until present from a single set of source
packages, a valuable feature of the configuration package creator
interface of our system.
So, our goal is to provide our users with the same opportunities to
override our configuration defaults as they would have had if Debian had
been providing them instead. Using the Debian packaging system for this
configuration is a good way to achieve this.
> You do have to deal with conffile prompts and other kinds of things
> that happen when software (package maintainer scripts, and dpkg) spot
> you've modified the files. But if you do all of your installations in
> a noninteractive mode (with a noninteractive debconf frontend and with
> dpkg configured not to prompt about conffiles) you get predictable
> behaviour which you can override as you like.
We've found it pretty useful to be able to install our configuration on an
existing system.
-Tim Abbott
Marc Haber
wrote:
>Timothy G Abbott <tab...@MIT.EDU> writes:
>> Anders Kaseorg and I created a system of CDBS modules (which we've
>> tentatively packaged as the config-package-dev package) for creating
>> Debian configuration packages. By configuration packages, we mean
>> packages that configure an existing Debian system by applying
>> dpkg-divert to configuration files. Our configuration package system
>> makes the process of creating configuration packages efficient.
>
>It's generally accepted wisdom that dpkg-divert doesn't work properly with
>configuration files and isn't safe.
Which is actually a big pity since I have been looking for a method to
roll out _configuration_ to a lot of systems by means of the already
in-place, apt-based software distribution system.
I have once experimented with using ucf to force in new configuration,
but that has shown to have "interesting" quirks. A different method
would be to background a job which waits until no apt or dpkg jobs are
running and more and does the conffile change then, but that's a very
bad hack.
We need a documented way to do those things.
Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834
Ian Jackson
> override our configuration defaults as they would have had if Debian had
> been providing them instead. Using the Debian packaging system for this
> configuration is a good way to achieve this.
The Debian packaging system and configuration setups are not designed
to provide more than the one level of override for the contents of
system files.
Ian.
Timothy G Abbott
> Timothy G Abbott writes ("Re: Debian Configuration Packaging System"):
>> So, our goal is to provide our users with the same opportunities to
>> override our configuration defaults as they would have had if Debian had
>> been providing them instead. Using the Debian packaging system for this
>> configuration is a good way to achieve this.
>
> The Debian packaging system and configuration setups are not designed
> to provide more than the one level of override for the contents of
> system files.
Configuration packages made using our system distribute only conffiles.
End-users of the configuration packages can modify (e.g.)
/etc/ldap/ldap.conf by changing /etc/ldap/ldap.conf.debathena, which is a
normal Debian conffile (and is what users will change if they try to
modify /etc/ldap/ldap.conf using any editor).
While this may not have been something envisioned by the designers of the
Debian packaging system, our system's use of standard pieces of the Debian
packaging system certainly supports this.
-Tim Abbott
Steve Langasek
that I meant to follow up to.)
On Mon, Feb 25, 2008 at 10:43:55AM -0800, Russ Allbery wrote:
> Josselin Mouette <jo...@debian.org> writes:
> > On lun, 2008-02-25 at 10:23 -0800, Russ Allbery wrote:
> >> ucf, from its DESCRIPTION in its man page, seems to handle the case of
> >> shipping a configuration file upstream that may also be locally
> >> modified, but I don't see where it handles merging in the results of
> >> debconf prompting. Am I just missing something?
> > ucf can easily be fed with the debconf-generated file. When told to with
> > the proper option, it will even do a three-way merge of the proposed
> > (maintainer + debconf) changes to the locally modified file.
> Oh! Okay, that was the piece that I was missing. That's a better
> solution than what I have now.
> Thanks, I'll try to find time to look at this in the not-too-distant
> future.
I humbly offer up samba-common in unstable as an example of a package that
DTRT with ucf to manage a config file.
I wish the three-way merge support in ucf was better, but for all that, the
status quo is still pretty darn slick. :)
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slan...@ubuntu.com vor...@debian.org
Steve Greenland
> I humbly offer up samba-common in unstable as an example of a package that
> DTRT with ucf to manage a config file.
Is it just me, or does everyone get prompted twice about smb.conf on
samba/samba-common upgrades? (I've looked through the various install
and config scripts, but not closely enough to figure out why...)
Steve
--
Steve Greenland
The irony is that Bill Gates claims to be making a stable operating
system and Linus Torvalds claims to be trying to take over the
world. -- seen on the net
Mike Bird
> Is it just me, or does everyone get prompted twice about smb.conf on
> samba/samba-common upgrades? (I've looked through the various install
> and config scripts, but not closely enough to figure out why...)
It happened here too. I haven't figured out why.
--Mike Bird
Steve Langasek
> On 11-Apr-08, 16:07 (CDT), Steve Langasek <vor...@debian.org> wrote:
> > I humbly offer up samba-common in unstable as an example of a package that
> > DTRT with ucf to manage a config file.
> Is it just me, or does everyone get prompted twice about smb.conf on
> samba/samba-common upgrades? (I've looked through the various install
> and config scripts, but not closely enough to figure out why...)
On your first upgrade to the ucf-based samba-common, you may get two prompts
- first to reconcile the differences with your smb.conf which doesn't match
the known old template, and then to merge in the smb.conf from the current
package.
There's room for improvement on the first pass by doing a better job of
matching up the user's config with past smb.conf templates (which mostly
means doing some scary package archaeology), but the history of this config
file spans more than a decade, so there will definitely be users for whom we
can't help but prompt twice on the first upgrade.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slan...@ubuntu.com vor...@debian.org