Roles/profiles and hiera

[Frederiko Costa]

Hi everyone,

Do you guys know any article/doc talking about the use of roles/profiles approach with hiera?

I'm particularly interested in how to organize the manifests when having multiple data centers, parametized classes and wants to use hiera. 

Being even more specific, how to organize the code using the Craig's article (http://www.craigdunn.org/2012/05/239/) and use hiera to  provide node specific data.

thank you,

-fred

[Ramin K]

Couple of links on the subject that I like.

Craig Dunn at Puppet Camp Feb 2013 which is a good addendum to his
original articles, http://www.slideshare.net/PuppetLabs/roles-talk

Carla Souza's Puppet Conf talk on managing Hiera values. IMO this will
become a very influential presentation over the next year as generally
available tooling catches up to the ideas presented. I'm surprised there
hasn't been more discussion about it.
http://carlasouza.com/puppetconf13/#/slide1

Hunner's github repo for his Role/Profile session at Puppet Conf.
https://github.com/hunner/roles_and_profiles

My example of using role/profile. I skipped over most of the design and
philosophy which Craig covered quite well and dove straight into what it
might looks like with a complicated set of data in a real world
application.
https://ask.puppetlabs.com/question/1655/an-end-to-end-roleprofile-example-using-hiera/

Ramin

[Frederiko Costa]

Excellent.. thanks! 

And now sorry for the long email... hopefully I'm clear enough.

I'd also to expose one example that I have here in my company. I'm not too confident of how we setup roles and profiles, specially when it comes to add hiera into the game.

Say we have a module called zabbix20::agent. The configuration file will be generated using erb templated with data coming from parametized classes. So far, it looks good. Data separation, modules look portable, etc.

As far as I understood going through the article is that you define the technology stack in the profile, and the role as collection of profiles. Well, in that case I'd say I would have something similar to this:

class profile::zabbix20::server {

  class { '::zabbix20::server' :

      bind_ip => "1.2.2.3",

        ...

 }

}

and then it would probably go to a base profile (profile::base) and inherited by a base role. That fits perfectly with single site scenario. Say you now have multiple data centers with different zabbix servers on each. The way I understood ...

class profile::zabbix20::server::dc1 {

  class { '::zabbix20::server' :

      bind_ip => "1.2.2.3",

        ...

 }

}

 

class profile::zabbix20::server::dc2 {

  class { '::zabbix20::server' :

      bind_ip => "1.2.3.4",

        ...

 }

 include httpd

 ...

}

then the roles:

class role::zabbix20::server::dc1 {

  include profile::zabbix20::server::dc1

}

and the nodes ...

node 'a.b.c.d' { include   include profile::zabbix20::server::dc1 }

node 'x.y.z.w' { include   include profile::zabbix20::server::dc2 }

That being said ... How would I actually add hiera into the game? I don't a straightforward way to use hiera and benefit the data separation. I have to include the business logic in the profile. How would I actually do that using hiera? Can't see a direct way.

The other discussion I had with my co-worker is ... they actually created two modules: roles and profiles. If I want to change, I have to actually change the modules. Isn't it desirable to have these out of the modulespath? I don't see why we have to do this way if are trying to abstract things and avoiding touching modules whenever we can.

Thanks in advance.

[Chad Huneycutt]

LIke this:

class profile::zabbix20::server (
bind_ip,
...
) {
class { '::zabbix20::server':
bind_ip => $bind_ip,
...
}
}

Then your hieradata would set

in a.b.c.d.yaml:
profile::zabbix20::server::bind_ip: 1.2.2.3

in x.y.z.w.yaml:
profile::zabbix20::server::bind_ip: 1.2.3.4

That way you have a single profile for all datacenters.

- Chad

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users...@googlegroups.com.
> To post to this group, send email to puppet...@googlegroups.com.
> Visit this group at http://groups.google.com/group/puppet-users.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Chad M. Huneycutt

[Frederiko Costa]

Thanks Chad ...

I understand it from the syntax point of view, but my point is more of a conceptual question in how to apply Craig's concepts using hiera with parasitized classes ... in that case, the node definition using hiera declares one (and only one role). The profile would define the technology stack with all the modules.

That's what Craig's article suggests (he doesn't mention hiera in the article). A role could specify 1+ profiles. So far, great. 

The thing is, in the node definition, using hiera, I'd break what they're proposing, because I'm actually not totally separating and externalizing the data. The yaml file would be (the way I see). 

x.y.z.w.yaml:

  role::zabbix20::server::dc2

a.b.c.d.yaml:

  role::zabbix20::server::dc1

Now, what I expected to have, for example, is the bind_ip parameter passed here in hiera, not in the profile definition.

I hope I'm being clear ... :-)

-frederiko

[Ramin K]

Without hiera you have all those extra classes you posted below
including this very specific one. I think your classes are too
complicated to begin with regardless of where the data is, but the lack
of data separation probably sent you down that path.

class role::zabbix20::server::dc1 {
include profile::zabbix20::server::dc1
}

With Hiera this is enough

include role::zabbix

You're now thinking, "How do I specify this machine is a server, in dc1,
and installing zabbix 2?" To answer we will start over and try to build
what you have with Hiera. This is a little hard to deal with in email,
but I think you'll get the idea.

node zabbixserver {
# this is a top level role for the zabbix server
include role::zabbix
}

class role::zabbix {
include profile::base
include profile::zabbixserver
}

class profile::zabbixserver {
include profile::apache
include zabbix
}

class zabbix(
$bind_ip = 127.0.0.1,
$version = present,
) {
blah blah
}

This Hiera config assumes you have a dc and role fact of some sort. This
may or may not be easy in your environment.

hiera.yaml
---
:hierarchy:
- %{fqdn}
- %{dc}
- %{role}
- %{environment}
- common

This is our role data
./hieradata/common.yaml
---
zabbix::version: '2.0'
zabbix::bind_ip: '1.1.1.1'


These are the dc bits of data
./hieradata/dc1.yaml
---
zabbix::bind_ip: '1.2.3.1'

./hieradata/dc1.yaml
---
zabbix::bind_ip: '1.4.5.1'


Your parametrized class will autolookup matching parameters from Hiera
in Puppet 3.x or you can specify them manually. Machines in dc1 get
1.2.3.1, dc2 gets 1.4.5.1, and any machine gets 1.1.1.1.

In summary we move data into Hiera and replace module::someclass::dc
with a simple module that does a lookup to a data file based on facts
that have classified the node.

Ramin

> <http://www.craigdunn.org/2012/05/239/>) and use hiera to

> > provide node specific data.
> >
> > thank you,
> > -fred
>
> Couple of links on the subject that I like.
>
> Craig Dunn at Puppet Camp Feb 2013 which is a good addendum to his
> original articles, http://www.slideshare.net/PuppetLabs/roles-talk
> <http://www.slideshare.net/PuppetLabs/roles-talk>
>
> Carla Souza's Puppet Conf talk on managing Hiera values. IMO this will
> become a very influential presentation over the next year as generally
> available tooling catches up to the ideas presented. I'm surprised
> there
> hasn't been more discussion about it.
> http://carlasouza.com/puppetconf13/#/slide1
> <http://carlasouza.com/puppetconf13/#/slide1>
>
> Hunner's github repo for his Role/Profile session at Puppet Conf.
> https://github.com/hunner/roles_and_profiles
> <https://github.com/hunner/roles_and_profiles>
>
> My example of using role/profile. I skipped over most of the design and
> philosophy which Craig covered quite well and dove straight into
> what it
> might looks like with a complicated set of data in a real world
> application.
> https://ask.puppetlabs.com/question/1655/an-end-to-end-roleprofile-example-using-hiera/

[Frederiko Costa]

Hi,

Ramin - you are correct. I got down to this path when I decided to follow this pattern. It turns out that it became overkill. 

In regards to hiera, that's also exactly what I was looking at. I knew I'd be nuts if I had to differentiate for each DC. Not clever whatsoever. I thought about facts and I actually wrote one to tell which DC the host is at. That would make my life easier, I thought. All I need is to make sure hiera uses it.

I will implement following you suggestion. That's the path I was looking for. The only thing I might not need it is the role fact, but I wanted to get my hands dirty first and make sure I don't need it.

thanks!

-fred

-frederiko

an email to puppet-users+unsubscribe@googlegroups.com.

To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.

[Mike Lehner]

I can't begin to describe how helpful it is to read this. I also started down the same path using a profile::base. A slightly different setup where all other profiles inherited from ::profile::base. Luckily, shortly after going down this path I ran into a major roadblock (better now than 9 months in) when I tried to override a value from profile::base in another profile. Can't really be done. Puppet Support also pointed me in a similar direction as Ramin using custom facts.

I really wish there was more guidance on this from Puppetlabs. When reading through learning sites and documentation, the first thing you read is that Roles/Profiles are the be all end all. Then you find out you actually don't want ANY data in code. So you have to go rewrite your all your profiles you already wrote, to work with hiera. In my opinion, the lack of guidance in this area is the single biggest barrier to deploying Puppet. Actually writing the code is fairly easy. The issue is finding the best place to organize the code. Most of the examples from Puppetlabs (or the blogs written by puppet employees before they were) are over-simplified and tend to lead down a path that's a dead-end. I'm not a professional developer, I'm a sysadmin simply trying to puppetize my infrastructure. There is a lot of work to be done in clearing this up for those of us who don't develop for a living. In my honest opinion.

[Ramin K]

Thanks and I'm glad it helped point you in the right direction. Here
are two more (from me at least) to add to your collection on
role/profile and Hiera. And yes I feel your oversimplified example pain.
It's very unhelpful though Puppet is better than most projects.

https://ask.puppetlabs.com/question/5235/what-goes-in-the-profile-part-of-roleprofile/
https://ask.puppetlabs.com/question/3146/how-to-build-a-proper-hiera-hierarchy/

What goes in profile? is the basis of my proposed Puppet Conf talk.
With any luck that will be accepted and force me to spend the 40+ hours
getting it out of my head and into a coherent presentation. Now I'm
thinking I should have called it "If you're not embarrassed of your
profile:: manifests you're doing it wrong!"

I agree that documentation is lacking, but there are some mitigating
circumstances. Parameterized classes, Hiera, role/profile, and automatic
lookup of parameters in Hiera only recently arrived on the scene.
Without all parts and some community consensus the way forward was still
unclear in my opinion. And I say that as someone who went down every
blind alley from 0.24 on. As always good ideas usually appear obvious in
hindsight.
On the flip side I was surprised with just how little reference to
role/profile there was on *.puppetlabs.com. You might consider opening a
Documentation bug to create a Role/Profile doc or at least collection of
the 20-30 external links that most people refer to in one place.

Ramin

> <http://www.craigdunn.org/2012/05/239/>) and use hiera to provide

> node specific data.
>
> thank you,
> -fred
>

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/ca33d5be-5f90-4c90-afd1-1dd154699516%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/ca33d5be-5f90-4c90-afd1-1dd154699516%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.