Thoughts on roles/profiles class paradigm

[Wolf Noble]

Hi Gang,

My colleagues and I are contemplating refactoring our modules to take advantage of the "roles/profiles" paradigm suggested by Craig Dunn in his blog post found here:
http://www.craigdunn.org/2012/05/239/

Before we jump feet-first into adopting this paradigm, I thought it a good idea to reach out and see what everyone else thinks about this.

We're currently struggling with some problems that this paradigm seems to solve quite nicely; but I'm not convinced it's the best abstraction / organization paradigm out there.

If you've not read it, I think It's worth the few minutes it takes to read. Thanks Craig, for writing it.




________________________________

This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/legal/email_disclaimer/ for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you.

[Gary Larizza]

The professional services team at Puppet Labs has been using that with customers, and many customers have been happy with what that affords you.  You effectively create a series of abstraction layers so that more junior sysadmins can look at a 'Role' and quickly see what's on the node without wading through a bunch of code.  This also allows more senior sysadmins to trace the code back down through 'Profiles' and ultimately to the modules themselves to track down errors.  It's a good pattern that lets everybody in charge of managing nodes be able to benefit from Puppet without having to get their hands TOO dirty (if that makes sense).  

Adrien Thebo also posted a sysadmin advent post similar here --> http://sysadvent.blogspot.com/2012/12/day-13-configuration-management-as-legos.html

Does this help?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--

Gary Larizza

Professional Services Engineer

[Roman Shaposhnik]

Hi!

On Wed, Jan 9, 2013 at 1:34 PM, Wolf Noble <wno...@datapipe.com> wrote:

> Hi Gang,
>
> My colleagues and I are contemplating refactoring our modules to take advantage of the "roles/profiles" paradigm suggested by Craig Dunn in his blog post found here:
> http://www.craigdunn.org/2012/05/239/

I think I've seen this one before and got curious about it as well.
It seems that Craig is advocating 1-1 mapping between nodes
and roles and that makes me think of the 'roles' as a sort of
poor man's ENC. As such, I'd be very curious to hear what
kind of issues do you think it will help you solve.

Now, having 'profiles' as the place to handle inter-module
dependencies seems like a pretty good idea.

Thanks,
Roman.

[Craig Dunn]

On 09/01/2013 13:56, Roman Shaposhnik wrote:
> I think I've seen this one before and got curious about it as well. It
> seems that Craig is advocating 1-1 mapping between nodes and roles and
> that makes me think of the 'roles' as a sort of poor man's ENC. As
> such, I'd be very curious to hear what kind of issues do you think it
> will help you solve. Now, having 'profiles' as the place to handle
> inter-module dependencies seems like a pretty good idea. Thanks, Roman.

The point was not a 1-1 mapping between nodes and roles (although that
was mentioned), the key point I was trying to make is to add layers of
classes to provide abstraction between your node definition (whether
thats in an ENC or site.pp) and the components that get pulled in.

If my post is tl;dr I'll summarise it with;

I have two modules called 'roles' and 'profiles', and other modules
we'll just refer to as 'component modules' and nodes have role classes
applied to them, which include profiles, which include component modules

* A role contains business logic
* A profile defines the logical software stack that defines what
components are needed
* The component modules are the building blocks that manage resources
(eg: ssh, mysql, apache...)

Theres probably a few ways of achieving the same thing - but they key
here is abstracting the components from the nodes.

Craig


--
Craig Dunn
Professional Services
Puppet Labs Inc.
http://www.puppetlabs.com

[Schofield]

I was in a position to build a puppet environment from scratch.  After lots of studying and trying different suggestions the profile/role paradigm is proving flexible enough to meet any challenge we have faced thus far.  We apply a role to nodes that should be identical (typically clusters of nodes) and the same role gets applied to dev/test/prod clusters.  This solved two problems.  1)  It works nicely for integrating with the release process where clusters are tested as a platform.  We can make changes to applications, middleware and the OS in the role.  Dashboard reports then show what will change on  dev/test/prod.  The changes are then applied following the schedule of the release process.  2) Applying only roles to nodes keeps the dashboard clean if using it as an ENC.  The only classes found on the dashboard are roles.  No one has to guess what a server is used for because the single applied role to the cluster makes it obvious.  In all honesty though we do apply roles to node groups.   So if a node finds itself in multiple groups it will have multiple roles.  But the roles are always mutually exclusive or the underlying classes are written to handle the potential for resource conflicts. 

The final big advantage I see is having terminology and a proven pattern to communicate ideas between team members.  Once we settled on the profile/role pattern we quit going in circles on how to implement and solutions to  perceived problems became clear and in some cases solved themselves.

All and all very satisfied with the pattern so far.

Those are my thoughts.  Hope they are helpful.

[Roman Shaposhnik]

On Wed, Jan 9, 2013 at 2:08 PM, Craig Dunn <cr...@puppetlabs.com> wrote:
> The point was not a 1-1 mapping between nodes and roles (although that was
> mentioned), the key point I was trying to make is to add layers of classes
> to provide abstraction between your node definition (whether thats in an ENC
> or site.pp) and the components that get pulled in.

So what's the advantage of managing roles explicitly in Puppet as opposed
to managing them in an ENC of some kind? I get the re-usability point (you
can swap out ENC, use node definitions, etc). Is that all?

> I have two modules called 'roles' and 'profiles',

And as I mentioned I find profiles to be quite useful. Its the
role concept that I'm not necessarily

> * A role contains business logic

Could you give an example here? What else are roles, but a flat collection
of profiles?

Thanks,
Roman.

[Brian Lalor]

On Jan 9, 2013, at 4:34 PM, Wolf Noble <wno...@datapipe.com> wrote:

> My colleagues and I are contemplating refactoring our modules to take advantage of the "roles/profiles" paradigm suggested by Craig Dunn in his blog post found here:
> http://www.craigdunn.org/2012/05/239/
>
> Before we jump feet-first into adopting this paradigm, I thought it a good idea to reach out and see what everyone else thinks about this.

This looks very similar to Jordan Sissel's "pure fact-driven" nodeless puppet design[1], something I'm looking to adopt for an upcoming project. The idea makes a lot of sense to me.

[1]: http://www.semicomplete.com/blog/geekery/puppet-nodeless-configuration

[Craig Dunn]

On 09/01/2013 20:03, Roman Shaposhnik wrote:
>> * A role contains business logic
> Could you give an example here? What else are roles, but a flat collection
> of profiles?
>

Essentially yes, thats all they are and while I generally opt to make
them classes you could do this from within an ENC instead. The
implementation is not as important as the concept, so if you want to
replace the roles function with groups of profiles in an ENC you'll
still get the same benefits.

One reason I keep roles in code is it allows me to be a bit more
flexible. For example I can use inheritance to minimize duplication
roles::appserver::foo roles::appserver::bar....etc inheriting from
roles::appserver where I might have stuff like tomcat.

[jim]

Hello Craig,

I was wondering if someone could please help or explain the best approach to setting up puppet as our first requirement is to support multi-tenant within our company  what I mean by this is we have different teams supporting different O/S or the same O/S but different configurations, and will probably call the same modules but can't touch the others configurations etc

Team A - Windows O/S Config A

Team B - Linux O/S Config B

Department C - Windows O/S Configs C & D

From what I've been reading there seems multiple ways of doing this, some are being phased out, and some are aren't that clear.

The easiest option would be to add different manifests for different groups / teams within site.pp, but if I make changes to the sub-manifests, i'd need to "touch" the site.pp file for changes to kick in, which could also effect the other teams / group changes and cause outages ?

Can some please recommend the best approach to multi-tenant

Regards

James

[Nathan Nobbe]

I read Craig's article numerous times and have recently published an article on my thoughts. To summarize, I feel the big lesson from the article is composition is needed to define reusable grouped module declarations (aka roles). Whether or not you like the notion of the 2-layered approach (roles and profiles) is something else and a bit extra IMO.

That said I'm only managing tens of servers rather than hundreds or thousands so far, but for me one layer to represent 'roles' has worked great.

What it amounts to for me is a simple guideline - leverage inheritance (or the hiera hierarchy) as much as possible and introduce composition on a need-to basis. Composition is necessary though, unless all your systems are identical.

Thanks again Craig for the article. Neat to find you on the google group!

-nathan

[Ramin K]

On 1/22/2014 7:35 PM, Nathan Nobbe wrote:
>
> I read Craig's article numerous times and have recently published an

> article <http://quickshiftin.com/blog/2014/01/composition-in-puppet/> on

> my thoughts. To summarize, I feel the big lesson from the article is
> composition is needed to define reusable grouped module declarations

> (aka /roles/). Whether or not you like the notion of the 2-layered
> approach (roles /and/ profiles) is something else and a bit extra IMO.

>
> That said I'm only managing tens of servers rather than hundreds or
> thousands so far, but for me one layer to represent 'roles' has worked
> great.
>
> What it amounts to for me is a simple guideline - leverage inheritance
> (or the hiera hierarchy) as much as possible and introduce composition
> on a need-to basis. Composition is necessary though, unless all your
> systems are identical.
>
> Thanks again Craig for the article. Neat to find you on the google group!
>
> -nathan

If I had the 20-30 hours to write a Puppet Conf presentation it would
be called "The profile is the most important part of role/profile." :-)

In a simple system with a webserver and database, profiles don't appear
to add much. However in a complex system where an Apache server could be
a proxy, app server, ssl terminator, or other function the added layer
is very necessary. In my system I have 25+ roles half of which use
profile::apache to get vastly different configs. In each case
profile::apache provides the entry point for the data Hiera provides
based on Role.

Profile classes are where you get to be opinionated about your config.
In my sample profile::apache class below my Apache module can remain
generic and shareable while profile::apache pulls in things like
collectd, logstash, etc that are specific to how *I* think any server
with Apache should be installed.

You might also check out Craig's later presentation on role profile
which provides a clearer picture than his earlier blog post.
http://www.slideshare.net/PuppetLabs/roles-talk

Ramin

class profile::apache {

include ::apache
include profile::logstash
include profile::sslcerts

collectd::plugin { 'apache': }
logrotate::simple { 'apache':}

$mymods = hiera('apache::a2mods', {})
create_resources('apache::a2mod', $mymods)

$myvhosts = hiera('apache::vhosts', {})
create_resources('apache::vhost', $myvhosts)

Sslcerts::Cert<||> -> Class['apache::service']
}