external node classifier my setup

[Alex Samad]

Hi

Might have this wrong. But..

I have puppet install - got r10k and will be using hiera

I wanted to have 1 script / programatic way of classifying scripts into environments. I wanted to be able to look at certname and maybe ip address.

So I found this 

https://docs.puppet.com/guides/external_nodes.html

I have added this

[master]
  node_terminus = exec
  external_nodes = /usr/local/bin/puppet_node_classifier

to my /etc/puppetlabs/puppet/puppet.conf at the bottom

for this /usr/local/bin/puppet_node_classifier i have 

#!/bin/bash

#

# puppet external node classifier

# https://docs.puppet.com/guides/external_nodes.html

#

echo "$*" > /tmp/a1

set -e

HSTNAME="$1"

if [ -z "$HSTNAME" ]

then

        # can't be called with no arguments

        exit 1

fi

case "$HSTNAME" in

        testing.xyz.com)

                echo "---"

                #echo "classes:"

                #echo " - common"

                #echo "parameters:"

                #echo "environment: production"

                echo "environment: alex"

                ;;

        *)

                exit 1

                ;;

esac

echo

exit 0

when i go to box testing.xyz.com  which is the box I have installed puppet on and try this 

puppet  config print environment

I still get production not alex 

what am i doing wrong ?

How do I try and debug this (where do i look)?

Is this the right way to do it. our should I be adding nodes to config files (which ones?) in the environments or one master file in the global sites.pp (is this the file and if so where does it exist )

Thanks

Alex - slightly confused

[Alex Samad]

:)))

seems like after writing this I found my answer

I used 

puppet agent --test --verbose

shows me that it is classified as environment alex. thats good.

but

puppet  config print environment 

still show production? so I am guessing the above just looks at the puppet config files and as I haven't set environment it defaults to production !

So the question is, is this the best way to do it ?

Alex

[jcbollinger]

On Wednesday, June 22, 2016 at 2:21:27 AM UTC-5, Alex Samad wrote:

:)))

seems like after writing this I found my answer

I used 

puppet agent --test --verbose

shows me that it is classified as environment alex. thats good.

but

puppet  config print environment 

still show production? so I am guessing the above just looks at the puppet config files and as I haven't set environment it defaults to production !

So the question is, is this the best way to do it ?

The command ...

puppet config print environment

... indeed does print a value derived from the local Puppet configuration file.  The default for this value is 'production'.  Whatever value the command prints is the value that will be used for the node's environment, provided that the node-specified environment is not overridden by an external node classifier running on the master.  This allows nodes to request specific environments, while still affording the master the final say.  No matter whether the setting takes a default or explicit value in the node's local configuration (which 'puppet config print environment' will print), that value must be taken as provisional at best.

If your nodes care deeply about which Puppet environment they are assigned to, then you are doing something wrong.  If you just want to check, however, then the method you discovered, relying on verbose output from the agent, seems entirely reasonable.  You could perhaps tweak that by adding a --tags option that filters out all resources, so that you get node information without applying anything.  You cannot get a reliably correct answer without consulting the master, because the master has the final word on the matter.


John

[Alex Samad]

Hi

So I am a bit of a newbie. My assumption was to setup using a master
puppet server. But I wanted to make sure that environment was handled
by the master puppet - I have control over that and I might not be
able to exclude control over the managed box from other users (dam
developers !).

I wanted some way to test what I was doing was correct.

"
If your nodes care deeply about which Puppet environment they are
assigned to, then you are doing something wrong.
"

so I am planning on having atleast a production, sim , inf, non prod
and a dev environment.

I would presume a box would want to know which environment they are
in, because in prod they might be on a certain rpm / module or
certain config - lets say for example MOTD.

But i might be wrong ?

My thought had been to align production environment with production
server, infra with infra servers and non prod non infra in the non
prod environment.

Thanks
Alex

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/mZBLZQKZ0xM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/c9d7b296-ff84-4887-9707-5fe4d647fde7%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Alex Samad]

Hi

sorry bit more.

puppet config print environment

So from what I understand you saying this show the environment from
the agent side no talking to the master puppet !

so a puppet agent --noop --test would be a way of looking. is there
no other way of querying what the master things of a node

I tried
puppet catalog find <certname>

and it still shows me the wrong environment

if I can add to this. How in puppet can i set the local environment.
So say box A in my enc I say its in environment xyz, so when puppet
agent runs it set the puppet environment on that box to environment
xyz ?

A

[jcbollinger]

On Thursday, June 23, 2016 at 1:30:37 AM UTC-5, Alex Samad wrote:

Hi

So I am a bit of a newbie.  My assumption was to setup using a master
puppet server. But I wanted to make sure that environment was handled
by the master puppet - I have control over that and I might not be
able to exclude control over the managed box from other users (dam
developers !).

I'm inclined to agree that central control is to be preferred.  Do be aware, however, that control over node environment is mostly a management feature, not a security feature.  Your master can control what resources it records in nodes' catalogs, but those nodes' admins can disable Puppet, make it run in --noop mode, make it present false facts to the master, and many other things.  Do not grant privileges to people whom you do not trust, and do not trust anyone any more than you need to do.

 

I wanted some way to test what I was doing was correct.

And you found one.

 

"
If your nodes care deeply about which Puppet environment they are
assigned to, then you are doing something wrong.
"

so I am planning on having atleast a production, sim , inf, non prod
and a dev environment.

I would presume a box would want to know which environment they are
in, because in prod they might be on  a certain rpm / module or
certain config - lets say for example MOTD.

But i might be wrong ?

In the first place, I recommend not using multiple Puppet environments unless you have a Puppet-related reason for doing so.  The prime reason in this category would be that you want to allow for use of different versions of the same Puppet modules to be used with one group of nodes than with another.  When no such reason applies, environments do not provide a benefit commensurate with the extra complication and work they involve.

In the second place, yes, you're wrong.  The Puppet environment to which a node is assigned affects the details of the catalogs built for it, which in turn affects those nodes' configurations.  The master makes decisions based on node environment, but nodes need not and should not care why they are configured as they are.  For example, nodes do not need to know or care about the meaning of the contents of their MOTD; they just need to present the text -- whatever it is -- to users when they log in and when they ask for it.  Likewise, they do not need to know why they are configured to access a particular database server, why they have the particular vhosts configured that they do, why they have the particular users and passwords they have, why they mount the particular remote file systems they mount, etc..

 

My thought had been to align production environment with production
server, infra with infra servers and non prod non infra in the non
prod environment.

Even if you ignore my advice and do that, what I'm saying is that you should not identify Puppet's sense of "environment" with any external concept going by the same name.  I maintain that nodes probably don't need to be explicitly aware of the label of their operational environment, either, but especially if you're exerting central control over Puppet environments, there is no reason for nodes to care how Puppet labels those environments.


John

[Alex Samad]

I agree but how do you make them difference for different nodes, if I
specify a group of nodes that have a specific MOTD... the nodes don't
care but ( i was using it as an example). maybe a better one would be
say smtp setup - all nodes have it, non prod must point to the non
prod smtp server.

> for it. Likewise, they do not need to know why they are configured to
> access a particular database server, why they have the particular vhosts
> configured that they do, why they have the particular users and passwords
> they have, why they mount the particular remote file systems they mount,
> etc..
>

Sorry our argument seems counter intuitive. Maybe I am miss understanding.

For example I have had a lot of issue with winbind. (centos 6.x). So
my thought is

production environment - has all the prod nodes.
It has a specific version of winbind, might be old but it works

My other environments have different newer versions of winbind.


if you can explain how I can do that with 1 environment . happy to
learn. I haven't done a puppet setup before - which is why i'm asking
and questioning.

>
>>
>> My thought had been to align production environment with production
>> server, infra with infra servers and non prod non infra in the non
>> prod environment.
>
>
>
> Even if you ignore my advice and do that, what I'm saying is that you should
> not identify Puppet's sense of "environment" with any external concept going
> by the same name. I maintain that nodes probably don't need to be
> explicitly aware of the label of their operational environment, either, but
> especially if you're exerting central control over Puppet environments,
> there is no reason for nodes to care how Puppet labels those environments.

so environments should only really align with puppets production code
and none prod code. and by non prod code you are talking only about
puppet module code ?

>
>
> John
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/mZBLZQKZ0xM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/df3051e0-c516-4e17-8835-9ebabb82ae5e%40googlegroups.com.

[Rob Nelson]

The values they receive for a particular module parameter does not need to have anything to do with their environment. That can come from hiera. 

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJ%2BQ1PXEBMSmqdeshY9N9Q%3DeQGhyD7g3cBSx3dt8nFA8ENzajA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson
rnel...@gmail.com

[R.I.Pienaar]

>> I agree but how do you make them difference for different nodes, if I
>> specify a group of nodes that have a specific MOTD... the nodes don't
>> care but ( i was using it as an example). maybe a better one would be
>> say smtp setup - all nodes have it, non prod must point to the non
>> prod smtp server.

Environments is a very big hammer for whats basically a tiny nail you're describing.

Suggest you look to hiera instead for a solution.

[jcbollinger]

On Thursday, June 23, 2016 at 6:11:11 PM UTC-5, Alex Samad wrote:

On 24 June 2016 at 00:16, jcbollinger <John.Bo...@stjude.org> wrote:

[...]
 

> In the first place, I recommend not using multiple Puppet environments
> unless you have a Puppet-related reason for doing so.  The prime reason in
> this category would be that you want to allow for use of different versions
> of the same Puppet modules to be used with one group of nodes than with
> another.  When no such reason applies, environments do not provide a benefit
> commensurate with the extra complication and work they involve.
>
> In the second place, yes, you're wrong.  The Puppet environment to which a
> node is assigned affects the details of the catalogs built for it, which in
> turn affects those nodes' configurations.  The master makes decisions based
> on node environment, but nodes need not and should not care why they are
> configured as they are.  For example, nodes do not need to know or care
> about the meaning of the contents of their MOTD; they just need to present
> the text -- whatever it is -- to users when they log in and when they ask

I agree but how do you make them difference for different nodes, if I
specify a group of nodes that have a specific MOTD... the nodes don't
care but ( i was using it as an example). maybe a better one would be
say smtp setup - all nodes have it, non prod must point to the non
prod smtp server.

How is configuring which SMTP server a node uses fundamentally different from configuring a node's MOTD?  In either case you're just plugging the right data into the right hole.  The holes are even similar in shape: part or all of a particular text file in a known location (or on Windows, the holes are sometimes registry entries).

You said earlier that your master will assign nodes to environments, and at that time you were identifying Puppet environments with operational environments.  It follows that you anticipate that your master will have enough information to identify nodes' intended operational environments.  How the master should translate that information into node configurations is an entirely different question, and largely orthogonal to whether Puppet environments are associated with operational environments.  Surely you don't suppose that every machine in the same operational environment will be configured identically to every other, so even if you do match Puppet environments to operational environments, that does not in itself address questions about how to assign configurations to nodes -- or, in Puppet speak, how to classify nodes.

In fact, Puppet has several mechanisms for that, one of them being the one that you already plan on using to assign nodes to environments: the external node classifier (ENC).  Puppet also makes heavy use of external data for configuring nodes, generally accessed via its Hiera hierarchical data subsystem.  Hiera data can be used to directly classify nodes (hiera_include()), or to supplement classification performed primarily elsewhere (automated data binding, hiera(), hiera_hash(), hiera_array()).  I prefer to put most of the load on Hiera, myself, but the reasons for that would be an entirely separate discussion.

Furthermore, how you structure your manifest set plays heavily into how easily this all works out.  If you are not already aware of Craig Dunn's "Roles and Profiles" pattern, then you should familiarize yourself with it.  You are not obligated to use it, but I think it synergizes with your objectives, and it has been applied with great success in a lot of places.

 

> for it.  Likewise, they do not need to know why they are configured to
> access a particular database server, why they have the particular vhosts
> configured that they do, why they have the particular users and passwords
> they have, why they mount the particular remote file systems they mount,
> etc..
>


Sorry our argument seems counter intuitive.  Maybe I am miss understanding.

For example I have had a lot of issue with winbind. (centos 6.x).  So
my thought is

production environment - has all the prod nodes.
It has a specific version of winbind, might be old but it works

My other environments have different newer versions of winbind.


if you can explain how I can do that with 1 environment . happy to
learn. I haven't done a puppet setup before - which is why i'm asking
and questioning.

In the first place, this still has nothing to do with nodes being mindful of which Puppet environment they are assigned to, nor even of which operational environment they are assigned to.  Nodes will use whichever winbind (for example) is installed on them, regardless of which environments you label them with.  The nodes themselves don't much care what you call them -- they simply operate according to the way you configure them.

In the second place, if the master has enough information to determine nodes' intended operational environments, and it knows how nodes in those operational environments should be configured, then it can configure those nodes appropriately.  If not, then not.  Puppet environments are one vehicle for expressing such knowledge, but using them that way for your particular purposes would be clumsy, would take more work in the long run, and would interfere with using Puppet environments for other purposes to which they are better suited and in which they confer much greater advantage.

I already mentioned external node classifiers and Hiera -- these are Puppet's primary mechanisms for matching configuration details to nodes.  You can also use node blocks in conjunction with one or both of those, though these days the practical role of node blocks seems a bit diminished.  That still has no essential bearing on either Puppet environments or operational environments.  Even if you planned on using node blocks as your sole means of node classification, that would not require you to use Puppet environments to achieve your objectives.

 

>>
>> My thought had been to align production environment with production
>> server, infra with infra servers and non prod non infra in the non
>> prod environment.
>
>
>
> Even if you ignore my advice and do that, what I'm saying is that you should
> not identify Puppet's sense of "environment" with any external concept going
> by the same name.  I maintain that nodes probably don't need to be
> explicitly aware of the label of their operational environment, either, but
> especially if you're exerting central control over Puppet environments,
> there is no reason for nodes to care how Puppet labels those environments.

so environments should only really align with puppets production code
and none prod code. and by non prod code you are talking only about
puppet module code ?

Puppet environments are highly appropriate for change management in your Puppet infrastructure itself, if that's what you're saying.  There is no good reason for coupling that with change management for your product / service pipeline -- you have the substantially the same uptime and service availability requirements for your dev and QA machines that you do for your prod machines; it's the applications running on top where you can afford instability.  Do you want all your developers stalled in the event that upgrading the version of a Puppet module on the master has an unanticipated negative effect?

But that's not the only reasonable use of environments.  For example, some organizations maintain a central Puppet master, but afford different divisions control over their own configurations.  This can be done by assigning a different environment to each division, and allowing that division to control the contents of that environment.  Environment isolation is pretty firm in the latest versions of Puppet (slightly less so in earlier versions) so that works fairly well.  There are other applications.

Moreover, no, I am not talking about only Puppet modules.  Environments do have -- or can have -- their own module sets, but they can also have their own ENCs, their own data, and their own site manifests.  There are good applications for environments that use all of those things; what you described just isn't one of them.  If you're looking for a rule of thumb, though, then it might be reasonable to use the need (or the potential need) for a different module set as a criterion for judging whether establishing a separate Puppet environment is a good solution to a given problem.


John

[Alex Samad]

The point i was trying to make was not the how. But that a group of
nodes will have 1 config and another a different config. It seems
like environments would be the way to group that.

>
> You said earlier that your master will assign nodes to environments, and at
> that time you were identifying Puppet environments with operational
> environments. It follows that you anticipate that your master will have
> enough information to identify nodes' intended operational environments.

I believe in my environment I can do that by hostname as a general rule.

> How the master should translate that information into node configurations is
> an entirely different question, and largely orthogonal to whether Puppet
> environments are associated with operational environments. Surely you don't
> suppose that every machine in the same operational environment will be
> configured identically to every other, so even if you do match Puppet
> environments to operational environments, that does not in itself address
> questions about how to assign configurations to nodes -- or, in Puppet
> speak, how to classify nodes.

Why can't I expect that. if I expect puppet to look after things like
MOTD, SSHD config, smtp config, firewall config. users. SOE directory
setup. Why can't I expect them to be the same. I understand that ip
address and name will be different.

>
> In fact, Puppet has several mechanisms for that, one of them being the one
> that you already plan on using to assign nodes to environments: the external
> node classifier (ENC). Puppet also makes heavy use of external data for
> configuring nodes, generally accessed via its Hiera hierarchical data
> subsystem. Hiera data can be used to directly classify nodes
> (hiera_include()), or to supplement classification performed primarily
> elsewhere (automated data binding, hiera(), hiera_hash(), hiera_array()). I
> prefer to put most of the load on Hiera, myself, but the reasons for that
> would be an entirely separate discussion.

I plan to use hiera - still getting my head around that.

>
> Furthermore, how you structure your manifest set plays heavily into how
> easily this all works out. If you are not already aware of Craig Dunn's
> "Roles and Profiles" pattern, then you should familiarize yourself with it.
> You are not obligated to use it, but I think it synergizes with your
> objectives, and it has been applied with great success in a lot of places.

I will definitely be looking at this. I had already planned on using
profiles and roles - got that from the sydney user group. I have most
of mine defined I believe. Now to work our how to implement

>
>
>>
>> > for it. Likewise, they do not need to know why they are configured to
>> > access a particular database server, why they have the particular vhosts
>> > configured that they do, why they have the particular users and
>> > passwords
>> > they have, why they mount the particular remote file systems they mount,
>> > etc..
>> >
>>
>>
>> Sorry our argument seems counter intuitive. Maybe I am miss
>> understanding.
>>
>> For example I have had a lot of issue with winbind. (centos 6.x). So
>> my thought is
>>
>> production environment - has all the prod nodes.
>> It has a specific version of winbind, might be old but it works
>>
>> My other environments have different newer versions of winbind.
>>
>>
>> if you can explain how I can do that with 1 environment . happy to
>> learn. I haven't done a puppet setup before - which is why i'm asking
>> and questioning.
>
>
>
> In the first place, this still has nothing to do with nodes being mindful of
> which Puppet environment they are assigned to, nor even of which operational
> environment they are assigned to. Nodes will use whichever winbind (for
> example) is installed on them, regardless of which environments you label
> them with. The nodes themselves don't much care what you call them -- they
> simply operate according to the way you configure them.

But ... this is what i want to use puppet for. winbind is a key
package that I want to manage - 99% of the other packages I don't care
about . Also something like java i want to manage.

when i say want to - I would like to. if i want to roll out a new test
version of winbind I had presume I would just change the package
modules for winbind to a newer version and it would roll out for that
environment - I would plan to leave it there for a week or month or
two to test. then once satisfied move than onto production.

I do this for winbind because its critical for our environment. same
with tzdata - but no so long testing times.
and kernel-image ..

>
> In the second place, if the master has enough information to determine
> nodes' intended operational environments, and it knows how nodes in those
> operational environments should be configured, then it can configure those
> nodes appropriately. If not, then not. Puppet environments are one vehicle

would i then need to have lots of if statements or ???

> for expressing such knowledge, but using them that way for your particular
> purposes would be clumsy, would take more work in the long run, and would
> interfere with using Puppet environments for other purposes to which they
> are better suited and in which they confer much greater advantage.
>
> I already mentioned external node classifiers and Hiera -- these are
> Puppet's primary mechanisms for matching configuration details to nodes.
> You can also use node blocks in conjunction with one or both of those,
> though these days the practical role of node blocks seems a bit diminished.
> That still has no essential bearing on either Puppet environments or
> operational environments. Even if you planned on using node blocks as your
> sole means of node classification, that would not require you to use Puppet
> environments to achieve your objectives.

Okay so I haven't run/installed/managed a puppet install. so i'm not
arguing I'm trying to understand and its still not that clear.


But I have asked this before and still haven't really had an answer
that fits in

Forgetting about the weather it MOTD, smtp or .. Some of my
environment is going to be different to other parts of it. from
something simple like a MOTD file or a smtp config or a rpm package or
even a java version.

These sort of things atleast to me sounds like they should be
separated by environments. If my prod / inf environments are running
java 7 and I am slowly rolling out java 8. I would want my none prod
to have java8.


How do I do that if i have one environment ... I am guessing with
hiera.. I am guessing I have to go in there and group the nodes into
groups and then do actions based on that groupings ... Sounds like
what environments would be good for. not saying one environment for
each difference

thanks .. if you can show me how to work my problem i outlined above
with 1 environment then I can get my head around it.

I will check out that blog.. and maybe after I have struggled with it
for a while, come to realise one is doing able!.

What is the down side to having multiple environments - why have R10k + git

>
>
> John
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/mZBLZQKZ0xM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/c327cef8-5dd5-448a-97cc-4c743db4b8a7%40googlegroups.com.

[Rob Nelson]

On Friday, June 24, 2016, Alex Samad <al...@samad.com.au> wrote:

The point i was trying to make was not the how. But that a group of
nodes will have 1 config and another a different config.  It seems
like environments would be the way to group that.

Alex,

Environment is a very overloaded term. In this case, 'environment' means 'a discrete set of puppet code and data.' It has no direct relation to 'environment' meaning 'an operational status of a set of services' (or however one might deceive the concepts of 'production' vs 'development' vs …). It is important not to confuse those two, or any other use of the term environment.

The reason there is no direct relationship between those two versions of the term is because a node that is in the operational status of production can switch between the production environment and any other environment of your puppet code without affecting its operational status (whether it should is a very different question!). You may also have two nodes that are both production nodes that receive different values for e.g. their mail relay.

This data is better separated via hiera, where you can say "all nodes in the Sydney datacenter use mail relay A, and all those in Perth use mail relay B." They don't have to be in separate puppet environments for that to occur.

Environments are more often used in your code development workflow, where environments map to your version control branches. You branch off of development to a branch called feature_a. None of your actual nodes check in against either of these environments, but your CI pipeline and your developers test against these with vagrant or similar. You maybe even take a canary node that normally uses the puppet environment production and have it check in against the feature_a environment instead.  When the feature works properly, you merge feature_a->development and delete the branch feature_a, which deletes the environment feature_a.

You repeat this for a few features, and then you merge from development->production. Now the code that was tested in CI is being used by the operational production nodes, which are using the operational environment as well. Then you go back to making more feature branches and repeat.

Along the way, a given node may check into one puppet environment 99% of the time, but nothing prevents the node from using a different puppet environment, for testing or urgent fixes or for any other reason. The puppet environment can be pretty fluid without changing the data and configuration a given node receives much if at all.

I hope that's not too confusing. You did say you haven't managed a puppet install yet, and I think if you set something up in vagrant or a lab, some of the concepts would become clear a lot quicker than when we discuss them in the abstract.

--

Rob Nelson
rnel...@gmail.com

[Alex Samad]

Hi


Okay so what i am getting from this is.

have 2 maybe 3 environments
production - production puppet code
testing - testing modules and
dev - another testing / developer


Inside production I can use a "ENC" ? to force nodes into groups
production
sim
inf

inside there I can allocate "environment stuff"


Again I am unsure how I implement what I am trying to do. I will pick
winbind - just to highlight the difference not the package etc etc.


My plan was to create a profile lets say "alex winbind" it would have
all of my config info for the module.

this profile and others would be pulled together into roles and the
roles assigned to nodes.

So in my puppet production environment I would have 1 module called
alex winbind and because its all in 1 puppet environment I then have
to have lots of if statements (or case statement) inside it that
basically says

if node is in prod use this module
if node is in sim use this module
if node is in inf use this module


is that right ????
A

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/mZBLZQKZ0xM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/CAC76iT_ZScBbqvtOJ7xdkwr8--JspDnYBgKGjJ-%3DvdDp-tAcJQ%40mail.gmail.com.

[jcbollinger]

Alex,

I will be brief(ish) this time, since wordiness seems not to be working.  I've been around Puppet for a fairly long time, and I think I know what I'm talking about, but by all means do evaluate my claims for yourself.

On Friday, June 24, 2016 at 9:20:58 PM UTC-5, Alex Samad wrote:

 

The point i was trying to make was not the how. But that a group of
nodes will have 1 config and another a different config.  It seems
like environments would be the way to group that.

Your ENC does the grouping.  Environments are one way to organize each group's manifests and data, but multiple people are advising you against using them that way.

> Surely you don't
> suppose that every machine in the same operational environment will be
> configured identically to every other, so even if you do match Puppet
> environments to operational environments, that does not in itself address
> questions about how to assign configurations to nodes -- or, in Puppet
> speak, how to classify nodes.
Why can't I expect that. if I expect puppet to look after things like
MOTD, SSHD config, smtp config, firewall config. users. SOE directory
setup. Why can't I expect them to be the same. I understand that ip
address and name will be different.

Evidently you do suppose.  Wow.

Most people have machines of different kinds under management -- web servers, database servers, workstations, etc..  These normally have some commonalities in their configurations -- maybe many commonalities -- but their overall configurations are not identical.  If indeed no machine in any of your groups is distinguishable from any other in the same group except by its network identifiers (hostname / IP / MAC) then Puppet environments are even less appropriate for you than I thought.  In that case, the key thing you should do is define one class per group, and have your ENC assign the appropriate one of those classes to each machine.

 

> In the first place, this still has nothing to do with nodes being mindful of
> which Puppet environment they are assigned to, nor even of which operational
> environment they are assigned to.  Nodes will use whichever winbind (for
> example) is installed on them, regardless of which environments you label
> them with.  The nodes themselves don't much care what you call them -- they
> simply operate according to the way you configure them.

But ... this is what i want to use puppet for.

You're missing my point.  Which winbind (as an example) is installed on a given node can absolutely be managed by Puppet.  The point is that the nodes themselves don't need to know which Puppet environment -- or even which operating environment -- they are assigned to; they just use whichever winbind they currently have (as managed by Puppet).

 

when i say want to - I would like to. if i want to roll out a new test
version of winbind I had presume I would just change the package
modules for winbind to a newer version and it would roll out for that
environment - I would plan to leave it there for a week or month or
two to test. then once satisfied move than onto production.

And you can do that with Puppet.  You just shouldn't use Puppet environments as your mechanism for mapping which nodes get which package versions.

 

>
> In the second place, if the master has enough information to determine
> nodes' intended operational environments, and it knows how nodes in those
> operational environments should be configured, then it can configure those
> nodes appropriately.  If not, then not.  Puppet environments are one vehicle

would i then need to have lots of if statements or ???

No.  It's a question of defining and assigning appropriate classes, and of associating the correct data with each node.  I cannot overemphasize the importance of data -- this is everything from which versions or which packages are supposed to be installed, to which users, to which servers to rely on, to the contents of the MOTD.  One of the most fundamental lessons of the last seven or so years of Puppet has been that externalizing data of almost all types is a tremendous win.

John

[Alex Samad]

On 27 June 2016 at 23:33, jcbollinger <John.Bo...@stjude.org> wrote:
>
> Alex,
>
> I will be brief(ish) this time, since wordiness seems not to be working.
> I've been around Puppet for a fairly long time, and I think I know what I'm
> talking about, but by all means do evaluate my claims for yourself.

I have come to the list to get the input from people like yourself
with experience. I am in the learning stage. So I am taking in stuff.
and maybe asking silly questions.

But what I like to hear is why, not just being told what to do.

I have heard environments is not the way to go, but not why what is
the down side. I have also heard from other people who use it a lot
that they use a lot of environments.

I've have decided to give it a go under 2 envornments and try and map
my machines in some other grouping under production. I presume I can
always go back.

>
> On Friday, June 24, 2016 at 9:20:58 PM UTC-5, Alex Samad wrote:
>
>>
>> The point i was trying to make was not the how. But that a group of
>> nodes will have 1 config and another a different config. It seems
>> like environments would be the way to group that.
>
>
>
> Your ENC does the grouping. Environments are one way to organize each
> group's manifests and data, but multiple people are advising you against
> using them that way.

Okay - what is the down side ... ? Note I am going to try the single
environment setup

>
>> > Surely you don't
>> > suppose that every machine in the same operational environment will be
>> > configured identically to every other, so even if you do match Puppet
>> > environments to operational environments, that does not in itself
>> > address
>> > questions about how to assign configurations to nodes -- or, in Puppet
>> > speak, how to classify nodes.
>> Why can't I expect that. if I expect puppet to look after things like
>> MOTD, SSHD config, smtp config, firewall config. users. SOE directory
>> setup. Why can't I expect them to be the same. I understand that ip
>> address and name will be different.
>
>
>
> Evidently you do suppose. Wow.
>
> Most people have machines of different kinds under management -- web
> servers, database servers, workstations, etc.. These normally have some
> commonalities in their configurations -- maybe many commonalities -- but

There are going to be a lot of commonalities (i believe), standard
patching, standard security, standard ... these are typically company
wide.

But my current environment small number of nodes. we have app, web,
rp, mail, proxy servers, but they have quite a lot in common.

> their overall configurations are not identical. If indeed no machine in any
> of your groups is distinguishable from any other in the same group except by
> its network identifiers (hostname / IP / MAC) then Puppet environments are
> even less appropriate for you than I thought. In that case, the key thing
> you should do is define one class per group, and have your ENC assign the
> appropriate one of those classes to each machine.

Q) why classes - i thought the way to go was roles / profiles.

I have defined a who bundle of profiles - like for example one for ssh
- have company standard setup for ssh, and other packages that are
part of the SOE.

Then I will have roles which will be a collection of profiles. those
roles will be things like web server, app server, proxy server etc
etc.

? are roles and profiles made up of classes, are classes the
fundemental building block in puppet and the term role / profile is
just a type of class ?

>
>
>>
>> > In the first place, this still has nothing to do with nodes being
>> > mindful of
>> > which Puppet environment they are assigned to, nor even of which
>> > operational
>> > environment they are assigned to. Nodes will use whichever winbind (for
>> > example) is installed on them, regardless of which environments you
>> > label
>> > them with. The nodes themselves don't much care what you call them --
>> > they
>> > simply operate according to the way you configure them.
>>
>> But ... this is what i want to use puppet for.
>
>
>
> You're missing my point. Which winbind (as an example) is installed on a
> given node can absolutely be managed by Puppet. The point is that the nodes
> themselves don't need to know which Puppet environment -- or even which
> operating environment -- they are assigned to; they just use whichever
> winbind they currently have (as managed by Puppet).

Yes I understand that, but how (again try and explain how not that I shouldn't).


So I have 1 environment production
I have my prod server in the prod grouping ???
I have my standard profile MY Winbind. its linked to a specific version.

How do I now test a new version. I presume in my "MY WInbind" class I
need some statements to say if in prod then this version if in test
this version. Is this the way to do it ?

>
>
>>
>> when i say want to - I would like to. if i want to roll out a new test
>> version of winbind I had presume I would just change the package
>> modules for winbind to a newer version and it would roll out for that
>> environment - I would plan to leave it there for a week or month or
>> two to test. then once satisfied move than onto production.
>
>
>
> And you can do that with Puppet. You just shouldn't use Puppet environments
> as your mechanism for mapping which nodes get which package versions.

Again why ? Too much overhead too much???? Why have R10K if you are
meant to have lots of environments

>
>
>>
>> >
>> > In the second place, if the master has enough information to determine
>> > nodes' intended operational environments, and it knows how nodes in
>> > those
>> > operational environments should be configured, then it can configure
>> > those
>> > nodes appropriately. If not, then not. Puppet environments are one
>> > vehicle
>>
>> would i then need to have lots of if statements or ???
>
>
>
> No. It's a question of defining and assigning appropriate classes, and of
> associating the correct data with each node. I cannot overemphasize the
> importance of data -- this is everything from which versions or which

This data - maybe this is what I am missing from the picture

http://www.craigdunn.org/2012/05/239/ somebody pointed to this blog
covering profiles/roles.

Looking a the diagram in the bottom showing resource -> modules ->
profiles -> roles -> node
it links in the data -> hiera which inserts into profiles and modules.



Maybe this is what I need to understand better ..

> packages are supposed to be installed, to which users, to which servers to
> rely on, to the contents of the MOTD. One of the most fundamental lessons
> of the last seven or so years of Puppet has been that externalizing data of
> almost all types is a tremendous win.
>
>
> John
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/mZBLZQKZ0xM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/3d5d1615-0506-4373-a507-ca586cb3a397%40googlegroups.com.

[jcbollinger]

On Monday, June 27, 2016 at 6:42:58 PM UTC-5, Alex Samad wrote:

On 27 June 2016 at 23:33, jcbollinger <John.Bo...@stjude.org> wrote:
>
> Alex,
>
> I will be brief(ish) this time, since wordiness seems not to be working.
> I've been around Puppet for a fairly long time, and I think I know what I'm
> talking about, but by all means do evaluate my claims for yourself.

I have come to the list to get the input from people like yourself
with experience. I am in the learning stage. So I am taking in stuff.
and maybe asking silly questions.

But what I like to hear is why, not just being told what to do.

I have heard environments is not the way to go, but not why what is
the down side.  I have also heard from other people who use it a lot
that they use a lot of environments.

We have given you several reasons why.  Examples include,

• "[for your purposes] environments do not provide a benefit commensurate with the extra complication and work they involve."
  

• "Environments is a very big hammer for whats basically a tiny nail you're describing."

• "Puppet environments are highly appropriate for change management in your Puppet infrastructure itself [, but ...] There is no good reason for coupling that with change management for your product / service pipeline."
• "The reason there is no direct relationship between ['operating environments' and 'puppet environments'] is because a node that is in the operational status of production [for example] can switch between the [Puppet] production environment and any other environment of your puppet code without affecting its operational status."
• "This data is better separated via hiera."

We have also talked a bit about the uses to which environments are better suited; for the most part, using them for the purposes you propose interferes with using them (later) for those more apt purposes.

I'm sorry if these seem to be higher-level claims than you would like to hear, but I'm uncertain how to answer in any deeper detail.

 

I've have decided to give it a go under 2 envornments and try and map
my machines in some other grouping under production.  I presume I can
always go back.

As I said from the beginning, your machines themselves do not (or should not) care to which Puppet environment they are assigned if you are performing those assignments centrally.  So yes, you can go back and forth among different approaches on the master's side.

 

> their overall configurations are not identical.  If indeed no machine in any
> of your groups is distinguishable from any other in the same group except by
> its network identifiers (hostname / IP / MAC) then Puppet environments are
> even less appropriate for you than I thought.  In that case, the key thing
> you should do is define one class per group, and have your ENC assign the
> appropriate one of those classes to each machine.

Q) why classes - i thought the way to go was roles / profiles.

Roles and profiles are implemented via classes.  What you seemed to have said suggested that even roles and profiles was more complex than you really needed.  Nevertheless, the one class per machine group maps directly to one role per group, and can be implemented that way.  There is no essential inconsistency here.  Your subsequent comments make me think that your real requirements are at least a bit more complex, as indeed I had initially supposed when I first suggested roles and profiles.

So I have 1 environment production
I have my prod server in the prod grouping ???
I have my standard profile MY Winbind. its linked to a specific version.

We already covered this.  The conventional ways to approach problems of that sort are via hiera or via your ENC.  There are many ways to implement the details.
 

How do I now test a new version.  I presume in my "MY WInbind" class I
need some statements to say if in prod then this version if in test
this version. Is this the way to do it ?

Supposing that the classes you are applying to your nodes are built to accommodate it (mainly that they rely on external data), you do it by binding different data to some nodes than you do to others.  Again, details vary.

John

[Alex Samad]

Hi

Sorry not trying to argue but just like to get my view point in

On 29 June 2016 at 08:30, jcbollinger <John.Bo...@stjude.org> wrote:
>
>
> On Monday, June 27, 2016 at 6:42:58 PM UTC-5, Alex Samad wrote:
>>
>> On 27 June 2016 at 23:33, jcbollinger <John.Bo...@stjude.org> wrote:
>> >
>> > Alex,
>> >
>> > I will be brief(ish) this time, since wordiness seems not to be working.
>> > I've been around Puppet for a fairly long time, and I think I know what
>> > I'm
>> > talking about, but by all means do evaluate my claims for yourself.
>>
>> I have come to the list to get the input from people like yourself
>> with experience. I am in the learning stage. So I am taking in stuff.
>> and maybe asking silly questions.
>>
>> But what I like to hear is why, not just being told what to do.
>>
>> I have heard environments is not the way to go, but not why what is
>> the down side. I have also heard from other people who use it a lot
>> that they use a lot of environments.
>>
>
>
> We have given you several reasons why. Examples include,
>
> "[for your purposes] environments do not provide a benefit commensurate with
> the extra complication and work they involve."

So isn't R10K meant to handle this - a lit bit of complexity doesn't
seem like a big problem

> "Environments is a very big hammer for whats basically a tiny nail you're
> describing."


> "Puppet environments are highly appropriate for change management in your
> Puppet infrastructure itself [, but ...] There is no good reason for
> coupling that with change management for your product / service pipeline."


> "The reason there is no direct relationship between ['operating
> environments' and 'puppet environments'] is because a node that is in the
> operational status of production [for example] can switch between the
> [Puppet] production environment and any other environment of your puppet
> code without affecting its operational status."
> "This data is better separated via hiera."

There is nothing there that really says why not to use lots of
environments - except some slight comlexity.

having said that I am proceeding with the single environment and will
be able to better comment once I have played with it for a while

>
> We have also talked a bit about the uses to which environments are better
> suited; for the most part, using them for the purposes you propose
> interferes with using them (later) for those more apt purposes.
>
> I'm sorry if these seem to be higher-level claims than you would like to
> hear, but I'm uncertain how to answer in any deeper detail.

thanks for the input much appreciated

Hmm not very encouraging for siding one way or the other. But I really
need to get something up and play with it to be better informed

thanks

>
>
> John
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Puppet Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/puppet-users/mZBLZQKZ0xM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/531d464d-8edd-45f5-ab43-0e7818f2b1b1%40googlegroups.com.