Idempotent file editing
dd-b
all outgoing mail to a smarthost down the line?
I could have a script, check whether I'd edited sendmail.mc, edit it
if needed, and run make in /etc/mail. That doesn't somehow sound very
"puppety", if I'm getting any feel for how puppet really intends
things to work.
I could have my own sendmail.mc to replace the stock one, I suppose.
I've already done that with a couple of files that are only a few
lines or that really are completely customized, but sendmail.mc isn't
like that.
I could look for an alternate mail transport package that did exactly
that, and required just one name dropped into a configuration
somewhere, thus sidestepping the problem. (This might be better on a
sysadmin level, but I've still got the general file editing problem in
other places, so I need a solution.)
Mike Renfro
> I could have a script, check whether I'd edited sendmail.mc, edit it
> if needed, and run make in /etc/mail. That doesn't somehow sound very
> "puppety", if I'm getting any feel for how puppet really intends
> things to work.
http://www.cfwiki.org/cfwiki/index.php/Editfiles_Considered_Harmful may
be a good starting point for one point of view (i.e., that editing
config files on the client systems can be a problem for auditing and
other things). The alternative would be to have one or more sendmail.mc
files on the puppetmaster, edit them manually, and let puppet pull down
the new mc file, and then automatically run make afterwards. Something like:
file { "/etc/mail/sendmail.mc":
source => "puppet:///sendmail/sendmail.mc"
}
exec { "make":
cwd => "/etc/mail",
path => ["/bin", "/usr/bin" ],
refreshonly => true,
subscribe => File["/etc/mail/sendmail.mc"],
}
which is generally copied from refreshonly part of
http://reductivelabs.com/trac/puppet/wiki/TypeReference#exec
--
Mike Renfro / R&D Engineer, Center for Manufacturing Research,
931 372-3601 / Tennessee Technological University
dd-b
On Oct 2, 2:55 pm, Mike Renfro <ren...@tntech.edu> wrote:
> http://www.cfwiki.org/cfwiki/index.php/Editfiles_Considered_Harmfulmay
> config files on the client systems can be a problem for auditing and
> other things). The alternative would be to have one or more sendmail.mc
> files on the puppetmaster, edit them manually, and let puppet pull down
> the new mc file, and then automatically run make afterwards.
I believe in all the problems he describes happening; I've seen them
or things closely related, in various contexts.
Storing a whole file and pulling it down is the obvious alternative,
and I've done that, but I have misgivings about that approach, too.
When I'm doing too much of that, I feel kind of the same way he seems
to when he sees file editing going on in config scripts.
If (to go back to my original example) I keep my own copy of
sendmail.cm, and copy it onto each managed server, I'm creating a
Frankenstein -- all but one config file and all the executables from
the package, this one config file from my archive. And with the
package being kept current by "yum update" on a regular basis, I don't
know whether my file is going to work with the rest of the package or
not. If I'm editing the file instead of replacing it, there's still a
possibility that it breaks, and there are more interesting ways for it
to break (as mentioned in the cited article), but *most* of the time
even if the file changes somewhat in the package, my edit will apply
(this edit replaces one line) and the file will work. By making a
private copy, I've locked down a bunch of stuff that the package
maintainer *thinks* they are maintaining; not just the part I'm
actually altering.
I don't have the time in my budget to really carefully consider and
try out each possible package change before allowing it onto my
servers. That's why we use an enterprise linux distribution, is to
have automatic updates that cover security holes and still work with
each other. (Small site -- currently an entire *6* physical linux
servers on the premises.) And most of my time is budgeted as a
developer.
(The "editfiles" is a cfengine thing, right, not a puppet thing?)
Steven VanDevender
Here's how we did this, although it's probably a bit Red-Hat-specific,
and is designed to use only the message submission agent functionality
in Sendmail so that all the client hosts don't have to run their own
SMTP daemons.
class localmail {
# ensure sendmail daemon is not started and is not running
service { "sendmail":
enable => false,
ensure => stopped,
}
# need sendmail-cf to build submit.cf from submit.mc
package { "sendmail-cf":
ensure => present,
}
file { "/etc/mail/submit.mc":
mode => 444,
owner => root,
group => root,
source => "puppet:///localmail/submit.mc",
}
# build submit.cf from submit.mc
exec { "/usr/bin/make -C /etc/mail submit.cf":
require => Package["sendmail-cf"],
subscribe => [ File["/etc/mail/submit.mc"], Package["sendmail-cf"] ],
refreshonly => true,
}
# add cron job to run client queue periodically
cron { clientmqueue:
user => root,
command => "/usr/sbin/sendmail -Ac -q",
minute => 0,
}
}
Then the submit.mc contains a line like
FEATURE(`msp', `smarthost.example.com.')dnl
instead of the usual line that forwards to 127.0.0.1. Being able to
edit that line into whatever submit.mc is there might be nice, although
it turns out the sendmail submit.mc/submit.cf doesn't really change much
over time so replacing the whole file isn't likely to cause problems.
The Puppet documentation site has a recipe for doing simple line editing
in files, but it appears to require some non-trivial hacking. I also
have noticed that the "mount" resource type edits lines into /etc/fstab,
but haven't figured out how it does so internally and whether that
mechanism might be used elsewhere.
Brian Mathis
Another reason that people use Enterprise Linux distributions is
because they don't do silly things like overwrite config files that
you've already edited every time you update the package. No one would
stand for that. Also, they don't change the packages around so you
suddenly have a totally new package version with a completely new
config file scheme.
At some point you have to have config files that tells the system how
to act, and you have to change them. Any file that you didn't create
completely by yourself at some point came out of the package system
somewhere. You would just have to edit them and the package system
would have to know how to handle it.
Based on all the evidence and discussions about it, I think you should
forget about editing files and instead keep copies as recommended.
There are so many people doing it successfully, the market has spoken.
Thomas Bellman
> Another reason that people use Enterprise Linux distributions is
> because they don't do silly things like overwrite config files that
> you've already edited every time you update the package. No one would
> stand for that. Also, they don't change the packages around so you
> suddenly have a totally new package version with a completely new
> config file scheme.
But they can still introduce work-arounds for security vulnerabilities
in new config files, that you must get into your config files to take
advantage of. I vaguely remember some vulnerability found in sshd, which
you could mitigate by turning on or off some feature in sshd_config, and
RedHat pushed out RPM:s with that feature changed from its earlier value.
If you hadn't changed your sshd_config file locally, it got updated,
otherwise you needed to check for .rpmnew files to find what they had
changed (or track the security advisory to know it anyway).
> At some point you have to have config files that tells the system how
> to act, and you have to change them. Any file that you didn't create
> completely by yourself at some point came out of the package system
> somewhere. You would just have to edit them and the package system
> would have to know how to handle it.
>
> Based on all the evidence and discussions about it, I think you should
> forget about editing files and instead keep copies as recommended.
> There are so many people doing it successfully, the market has spoken.
There are also a lot of us who think that mode of operation sucks.
If you keep entire files in your repository, you will never take advantage
of improvements that your vendor gives you. You can't see the difference
between parameters that you actually care about, and parameters that you
don't care about, and trust the OS vendor to set to reasonable values.
In /etc/sysctl.conf in CentOS 5.2, there are 10 parameters set. Some of
those 10 I care about (ip_forward and rp_filter), and set explicitly. Some
I have no idea what a reasonable value is (e.g, kernel.msgmax), and I don't
want to keep track of them. Other config files may have parameters that
matters a lot to other parts of the OS distribution, but what the correct
value should be varies between, e.g, CentOS 4, CentOS 5, Fedora 8, Fedora 10
or Debian Etch.
I have also used editfiles (in Cfengine) to make patches to startup scripts,
to work around bugs or limitations in them.
I edit /boot/grub/grub.conf to remove the "rhgb" and "quiet" flags on the
kernel lines, to remove the "hiddenmenu" directive, and to change the
timeout value. Keeping that file as an entire file in my repository would
be disastrous!
Yes, there are advantages to copying entire files. But there are also some
large *dis*advantages. It varies from config file to config file which mode
works best (and of course from site to site), though.
The lack of an "editfiles" type in Puppet was to horrible to stand, so I
wrote a few custom types to do some useful file editing. You can find them
at <http://groups.google.com/group/puppet-users/msg/203da69a6994d8e1>.
Be warned, however, that I have never tried them with puppetd/puppetmasterd,
only with standalone puppet.
Many of the problems mentioned in the "Editfiles Considered Harmful" article
stems from the Cfengine editfiles language. In Puppet you can do better.
Not the least because you can create defines saying more specifically what
you are doing; for example, I have defines named 'portage_makeconf' for
setting options in /etc/make.conf (for Portage in Gentoo Linux),
'portage_useflags' to set USE flags for packages, gentoo_conf manipulating
options in /etc/conf.d/*, 'rh_sysconfig to set options in /etc/sysconfig/*
on RedHat/Fedora, 'sysctl' to define sysctl settings in /etc/sysctl.conf,
and 'exportfs' to edit /etc/exports. And of course, we already have types
like 'host', 'cron', 'mailalias', 'sshkey', 'yumrepo', and an entire bunch
of 'nagios_*' types in stock Puppet. And do you know what -- they edit files!
/Thomas Bellman
Peter Meier
> The lack of an "editfiles" type in Puppet was to horrible to stand, so I
> wrote a few custom types to do some useful file editing. You can find them
> at <http://groups.google.com/group/puppet-users/msg/203da69a6994d8e1>.
> Be warned, however, that I have never tried them with puppetd/puppetmasterd,
> only with standalone puppet.
I think that the future way to go regarding this problem is to use
augeas with puppet, which seems to be the right tool for it.
greets pete