How to group systems using Puppet and hiera?

[Alex Scoble]

Hi All,

We are currently transitioning from using the Puppet Enterprise (PE) ENC to using hiera. Howevver, one piece of functionality that we lack with hiera is the ability to group systems together based on function. We can only currently group the systems based on available facts and are currently using environment, osfamily and domain in our hierarchy, however it would be extremely useful to also have a "group" or "role" category in our hiera hierarchy.

As I see it we have a few options and hopefully, people here can add better options that they are using.

The first is to use groups within the PE ENC which can apparently be used as a variable within the hiera hierarchy. This would be an easy solution, but not a very scalable one as putting systems into groups using the PE console is very slow and I haven't seen a documented way to manipulate the PE console data using a command line tool.

Second is to create a custom fact named group, but this requires that each system has a yaml file dropped in the correct place and I don't see a good way to automate creation of these files using PE, which would make it scalable. One way to do it would be to create a hiera yaml for each system with group parameter set, but the whole point of this exercise is to avoid creating hiera yamls for each system in our environment. I don't see having 1000+ hiera yamls as very scalable or manageable.

Third, of course, is to just manage this with individual hiera yamls for each system where classes required for that system's functions, but can't be defined higher up in the hierarchy, would be called out, but as I said before, this option isn't very scalable or manageable.

If you are using Puppet and hiera, how are you managing groups of systems (web servers, Nessus servers, mail servers, etc.)?

Thanks,

Alex

[Atom Powers]

My environment is pretty small, so far, so I can get away with having
host specific yaml files. I wrote an ENC that reads the host yaml
files for role/group information because I plan to eventually move
that data into a CMDB of some kind.

At $previous_job I used LDAP as the CMDB and an ENC script to pull the
host role data out of the directory service.

Basically, any database or application you use to store role
information and can be accessed via a script from the puppetmaster
should work for you. The hard part is finding one that you like.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/234ab5f3-bb1e-43a8-b365-bd110e5052da%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--

[Ramin K]

We add a role fact as part of the provisioning process when Puppet
kicks off for the first time. The equivalent of doing the following from
the command line.

FACTER_role=$role puppet agent --server $server --environment $env

::role is written to a file which in future runs populates ::role which
closes the loop.

Ramin

[Martin Alfke]

Hi Alex,

On 17 Feb 2014, at 22:05, Alex Scoble <bloggi...@gmail.com> wrote:

> Hi All,
>
> We are currently transitioning from using the Puppet Enterprise (PE) ENC to using hiera. Howevver, one piece of functionality that we lack with hiera is the ability to group systems together based on function. We can only currently group the systems based on available facts and are currently using environment, osfamily and domain in our hierarchy, however it would be extremely useful to also have a "group" or "role" category in our hiera hierarchy.
>
> As I see it we have a few options and hopefully, people here can add better options that they are using.

We use environment per server class.
This allows us to use a default node definition only.
Data are completely put into hiera and we use the %environment variable also in hiera for data separation between server groups.

hth,

Martin

[Grant Street]

I am looking to implement puppet coming from cfe2
I am also finding it hard to make the connection between nodes and groups using hiera. I would like a solution where I can define host groups etc on the puppet master rather than relying on the client to have a file(s) put in particular locations.
It would be good to remove the node configuration from the puppet DSL but I am struggling to put this together.
Any help ?

[Atom Powers]

Grant,

You probably want to use an ENC, which will allow you to define any fact from a specified datasource.

In a simple implementation I use an ENC to read the node's hiera file and output a few facts, such as a node's role/group, which are then used by hiera to define additional information for Puppet.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/0788d35c-b315-4ae9-aca5-8018427abe87%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Grant Street]

Hey
Thanks for the reply

I want the ENC to pass on variables or the like so that these can be used by hiera to apply roles/profiles etc.

eg
ENC returns a nodes software role eg role: desktop_software
Then hiera can load the desktop_software.yaml file to apply that software role as part of it's normal hierarchy.

Is that possible? What would it envolve?

Grant 

[Atom Powers]

The ENC can pass on any variable you wish to define and they can be used by Hiera.

I haven't got around to implementing a true CMDB yet so I use a very simple ruby script as my ENC. It reads the hieradata/nodes/fqdn.yaml file for a "role" value and returns that in the output to puppet. My hiera.yaml hierarchy includes "hieradata/roles/%role"

The output of the ENC is very specific, if you can't find the information you need to get it to work I'll find out if I can share my ENC script with you off-list.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/e811237c-edfc-4eb3-b19a-af4f6e182f8b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Grant Street]

That would be great. My first attempt didn't seem to work. Even some example output defining the roles etc, would be great from the script. I must be doing something wrong.
Cheers

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAF-H%3DO%3DfKbZcWyxwxSQwPX_YdwUi%2BuT9RLrUchOK0x2CuaA8ug%40mail.gmail.com.

[Pete Brown]

I have taken to using hiera as my ENC these days.

My site.pp is basically
hiera_Include(role)
And each nide as a role variable set in my certname.yaml file.
Hiera basically does the same as most ENCs anyway and it's nice having roles and node data in one location.

On 25 Jul 2014 14:33, "Atom Powers" <atom....@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAF-H%3DO%3DfKbZcWyxwxSQwPX_YdwUi%2BuT9RLrUchOK0x2CuaA8ug%40mail.gmail.com.

[Pete Brown]

On 27 Jul 2014 16:42, "Pete Brown" <rendh...@gmail.com> wrote:
>
> I have taken to using hiera as my ENC these days.
>
> My site.pp is basically
> hiera_Include(role)
> And each nide as a role variable set in my certname.yaml file.

Dammit.
Each node has a role variable.

[Grant Street]

I think my problem was I was trying to set the role variable under a "parameters:" clause in my group yaml file... errrr
I'm also trying your method, Pete, but I'm hitting a snag.

I have certname.yaml file that contains
---
role: modeler

and a file modeler.yaml that contains
---
classes:
 - my_class

But I am getting an error saying
Error: Could not find class modeler for .....

If I set the certname.yaml to
---
parameters:
  role: modeler

I get
Error: Could not find data item role in any Hiera data file and no default supplied at .....

Am I doing something wrong?

[Pete Brown]

On 29 July 2014 17:24, Grant Street <grantf...@gmail.com> wrote:
> I think my problem was I was trying to set the role variable under a
> "parameters:" clause in my group yaml file... errrr
> I'm also trying your method, Pete, but I'm hitting a snag.
>
> I have certname.yaml file that contains

Sorry I composed that email on my phone.
I meant certname to be the variable $::certname.
Which is usually the $::fqdn of your node.

So if the $::fqdn is www.example.com the file should be named
www.example.com.yaml

> ---
> role: modeler
>
> and a file modeler.yaml that contains
> ---
> classes:
> - my_class
>
> But I am getting an error saying
> Error: Could not find class modeler for .....
>
> If I set the certname.yaml to
> ---
> parameters:
> role: modeler
>
> I get
> Error: Could not find data item role in any Hiera data file and no default
> supplied at .....
>
> Am I doing something wrong?
>
>
> On Sunday, July 27, 2014 4:46:45 PM UTC+10, Pete wrote:
>>
>>
>> On 27 Jul 2014 16:42, "Pete Brown" <rendh...@gmail.com> wrote:
>> >
>> > I have taken to using hiera as my ENC these days.
>> >
>> > My site.pp is basically
>> > hiera_Include(role)
>> > And each nide as a role variable set in my certname.yaml file.
>>
>> Dammit.
>> Each node has a role variable.
>>
>>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/c4f24ba9-ecdf-4707-b24e-c61e28473c99%40googlegroups.com.