external_nodes setting with Environments

[Douglas Garstang]

So... in puppet.conf on the puppet master, there's the external_nodes setting, which defines the location of the external nodes script. If I am using multiple environments, I guess I have to have ONE set of external nodes since external_nodes in puppet.conf is set ONCE, right?

Ie, I can't have:

/etc/puppet/env/env_1/manifests/extdata/nodes

/etc/puppet/env/env_2/manifests/extdata/nodes

.... I have to have instead:

/etc/puppet/manifests/extdata/nodes

.... right? I was kind of hoping I could have one set of external nodes per environment.

Doug.

[Alan Barrett]

On Wed, 23 Feb 2011, Douglas Garstang wrote:
> So... in puppet.conf on the puppet master, there's the external_nodes
> setting, which defines the location of the external nodes script. If
> I am using multiple environments, I guess I have to have ONE set of
> external nodes since external_nodes in puppet.conf is set ONCE, right?

If you are using an external_nodes script, then you have to have exactly
one such script. The script can be as complex as you like, and it
can certainly do different things for different environments (once it
figures out which environment a node belongs to).

I don't know what you mean by "set of external nodes"; if you are
referring to some sort of data that is used by the external_nodes
script, then obviously it depends on how the script is written.

--apb (Alan Barrett)

[Thomas Bellman]

Douglas Garstang wrote:

> So... in puppet.conf on the puppet master, there's the external_nodes
> setting, which defines the location of the external nodes script. If I
> am using multiple environments, I guess I have to have ONE set of
> external nodes since external_nodes in puppet.conf is set ONCE, right?

That is correct. If you want different node classifier scripts
per environment, you have to code around it by having a "wrapper"
classifier that just looks at the requested environment and then
executes the environment-specific classifier.

I think what you want is perfectly reasonable. Indeed, I have
a hard time understanding why anyone *wouldn't* want it, since
otherwise you can't test a new version of the classifier in a
test environment without risking breaking production. But, alas,
that is not to be.

(I believe the only way for the global classifier to find out
which environment the client requests, is to read and parse the
/var/lib/puppet/yaml/facts/HOSTNME.yaml file. But I'm not sure.
I don't use an external node classifier myself, and I'll do my
best to not need it ever.)

/Bellman

[Dave Augustus]

Doug,

There are lots of options on the storage of external node information.
We are using LDAP for this. With LDAP, there is an attribute that you
have to set: "environment". I am sure the same is possible, no matter
what your external node storage method is. The benefit of external
storage is that this allows for separation of critical details from
manifests, like passwords, that you do not want in version control.

But the end result is the same. You have to use the variable,
$environment, for those manifests that you apply to all your nodes.
Then for each environment, you do the following:
Partition your manifests according to your environment. Here is how I
did that:

I have added the following in my puppetmaster.conf file:

[production]
manifest = $vardir/env/production/manifests/site.pp
modulepath = $vardir/env/production/modules:$vardir/env/common/modules

[test]
manifest = $vardir/env/test/manifests/site.pp
modulepath = $vardir/env/test/modules:$vardir/env/common/modules

[development]
manifest = $vardir/env/development/manifests/site.pp
modulepath = $vardir/env/development/modules:$vardir/env/common/modules

On each client, in puppet.conf:

[main]
environment = test

The end result is 3 sets of manifests and modules. I then can move a
module between the environments as needed.

Thanks,
Dave

[Thomas Bellman]

On 2011-02-25 13:27, Dave Augustus wrote:

> I have added the following in my puppetmaster.conf file:
>
> [production]
> manifest = $vardir/env/production/manifests/site.pp
> modulepath = $vardir/env/production/modules:$vardir/env/common/modules
> [test]
> manifest = $vardir/env/test/manifests/site.pp
> modulepath = $vardir/env/test/modules:$vardir/env/common/modules
> [development]
> manifest = $vardir/env/development/manifests/site.pp
> modulepath = $vardir/env/development/modules:$vardir/env/common/modules
>
> On each client, in puppet.conf:
>
> [main]
> environment = test
>
> The end result is 3 sets of manifests and modules. I then can move a
> module between the environments as needed.

But you can't have different versions of the external node classifier
script in your environments. You can't try out a new version of that
in development, while still having the tried and trusted version
running in production.

Unless you go for a wrapper script that just digs out the environment
from /var/lib/puppet/yaml/facts/NODENAME.yaml and execs the script
that belongs to that environment. And except for the part of digging
through a YAML file, it is a pretty easy thing to do, but it is a tiny
bit ugly.

/Bellman

[Douglas Garstang]

Dave, we're sticking with text files for external nodes because revision control is important to us, and I'm quite surprised why it's not important for others as well.

Here's another question.... where is the best place to put the external nodes if they are files? Should they be above the environments (ie /etc/puppet/manifests/extdata/nodes), or should they be put INTO the environments (ie /etc/puppet/env/env1/manifests/extdata/nodes). Which one makes more sense?

Doug.

 

[Vincent Bernat]

I am digging out this old topic. Has anything changed? I have tried the above method but unfortunately, /var/lib/puppet/yaml/facts/NODENAME.yaml is only updated after classification. On first run, there is no such file. There is nothing udpated in /var/lib/puppet on first run that would get us the right environment. The classifier is run without any environment variable.

Since 2011, is there anything new for this?