Puppet best practices: location specific configuration...

[Daniel Pittman]

G'day.

We are using Puppet to manage hosts on our network at work, and one
aspect that is causing me some cognitive discomfort is the question:

How to best manage location-specific configuration

We have three network, with different topologies, which communicate
between themselves: one at a data center, one at our head office, and a
third that has machines physically in the data center but logically able
to access only the Fitzroy network, no Internet at all.[1]

This poses some questions on how to best manage configuration based on
location, since we have similar-but-not-identical requirements on the
different networks:

A good example of this is NTP:

In head office and the data center we want a single, local machine to
fetch NTP information from the outside world, and then all other
machines to fetch time from that.

In the third, limited, network we want one local machine to fetch time
from Fitzroy, and then all other machines on the local network to fetch
time from it.

So, we have these machine configurations:

NTP master: fetch from an upstream server, serve local subnet
NTP client: fetch from local server, serve no one

We need, for each host, to configure it as either an NTP master or
client, talking to the appropriate servers — and the upstream server
varies based on location.

I can see several solutions, none of which really appeal:

1. On every node declare the NTP server(s) to use, and use a template in
the NTP manifest to build ntp.conf appropriately.

2. In each location set a variable ($ntp_server) to the correct hostname
of the upstream NTP server, and use that together with an if
statement matching on fqdn to determine if this is an NTP server or
not.

3. Use a 'declare' rather than a class and variable, and pass the NTP
server name(s) as an argument, but otherwise like 2.

What I would really like is for the NTP manifest to be sufficiently
smart that it can work this all out itself.

Oh, and not to have to do something like this:

node default { include <location> } # <location> is one of -1, -2, etc
class <location-1> { $foo = 'bar'; include ntp }
class <location-2> { $foo = 'baz'; include ntp }

It would be much nicer, and cleaner, if I could simply include NTP at
the top level, after the location include, and have things "just work"
for the local network.

So, any suggests on the best practice for managing this sort of variable
configuration?

Regards,
Daniel

Footnotes:
[1] Contractual obligation.

[R.I.Pienaar]

Hello,

> We are using Puppet to manage hosts on our network at work, and one
> aspect that is causing me some cognitive discomfort is the question:
>
> How to best manage location-specific configuration

The way I generally deal with this is with a parser function[1] that look up data elsewhere, outside of puppet, I can set up my environment via site.pp as follows:

$extlookup_datadir = "/etc/puppet/manifests/extdata"
$extlookup_precedence = ["%{fqdn}", "location_%{location}", "domain_%{domain}", "common"]

And in my manifests I have something like:

$nameserver = extlookup("resolver_nameservers")
$search = extlookup("resolver_search")

What this does it check if I have these variables set for ${fqdn} if its not there then for location_${location} and so forth, finally falling back to a one-size-fits-all answer in the common data source, and if its not there then its a parser error and the manifest wont run.

This effectively replaces:

case $location {
"londonixn": {
$nameserver = "1.1.1.1"
$search = "foo.com"
}
"hetznerde": {
}
}

style constructs with simple external data sources. I have machines in around 10 locations now so the whole case statement thing was driving me mad so it's massively simplified my life. Currently I use CSV files for the data sources, of course you can use databases or whatever and this whole thing lends it self great to be a web UI.

[1] http://reductivelabs.com/trac/puppet/wiki/WritingYourOwnFunctions

--
R.I.Pienaar

[Macno]

Hi,

>     How to best manage location-specific configuration

I generally approach this issue with node's inheritance, managing
variables at different levels, according to custom needs..
An example is better than many words:

node basenode {
$puppet_server = "10.42.0.10"
$local_network = "10.42.0.0/24"

}

node devel inherits basenode {
$local_network = "192.168.0.0/24"
$ntp_server = "192.168.0.2"
$zone = "devel"
}

node prod inherits basenode {
$local_network = "10.0.0.0/24"
$ntp_server = "10.0.0.14"
$puppet_server = "10.0.0.10"
$zone = "prod"
}

node 'delta.lab42.it' inherits devel {
include general::webserver
}

node 'ntp.example42.com' inherits prod {
$ntp_server = "192.43.244.18"
include general::ntpserver
}

So,
you can define variables (used then in templates or wherever
necessary) at a top level (basenode) and redefine them at lower levels
(prod, test... generally these are intended as separated networks,
that may have different settings for services ar dsn, ntp and
whatever).
You can also redefine some variables at the node level, for managing
specific cases.
The definition of a $zone variable, can be useful to manage broader
differencies (for example serving totally different files according to
the zone, or including only specific classes

class general {
include sysctl
include puppet
include nrpe
include snmpd
include ntp
include hosts
include cron
include resolver
include syslog

case $zone {
prod: { include hardening }
default: { }
}
}

class general::webserver inherits general {
$role = "webserver"
include apache
}

Just as a side note, I define $role as the general role of a system
(webserver, loadbalancer, dbserver, mailserver) and use the $role
variable in classes to eventually differentiate configuration on the
role od the system.
So, for each node, you have 2 main characterizations:
- The zone (network) where it stays
- The role (functions) it has


Regards
Al

[Joshua Anderson]

Hi, Daniel.

So, we have these machine configurations:

 NTP master: fetch from an upstream server, serve local subnet
 NTP client: fetch from local server, serve no one

We need, for each host, to configure it as either an NTP master or
client, talking to the appropriate servers — and the upstream server
varies based on location.

Have you considered using a custom fact to determine each server's location? All the fact would have to do is figure out which subnet a server is on and compare that to a list of subnets for each network. The value of the fact could be your name for the network. (E.g., "fitzroy".)

If your custom fact was called "networkname", your manifests might look something like this:

class ntp {

  file { "/etc/ntp.conf":

    source => [ "puppet:///ntp/ntp.conf.${hostname}", "puppet:///ntp/ntp.conf.${networkname}" ]

  }

  service { "ntp":

    ensure => running,

    enable => true,

    subscribe => File["/etc/ntp.conf"]

  }

}

Each ntp master would have its own copy of ntp.conf, named ntp.conf.<master hostname>. Each client would use a copy of ntp.conf customized for the network it resides on, named ntp.conf.<network name>. (Giving a list of sources to the "source" parameter is like specifying a search path; the first source that exists will be used.)

This is much simpler than trying to declare which network every single host is on.

-Josh