Roles / profile pattern , inquire on how you handle some specific situations

[JuanBrein]

I've been using puppet on different companies and implementing the roles / profile pattern on some of them.

In theory the patter works very well but in practice I usually face challenges that I sort out implementing my own designs / solutions. I would like to know how you guys deal with that in case you do.

Say you have a typical LAMP stack and you have to deploy a web app so my classes would look something like this (super simplified version):

Modules:

class apache { //puppetlabs class }
class mysql { //puppetlabs class }
etc./.

Profile:

class profile::webapp {

  class 'apache'
  class 'mysql'

  $name = hiera('webapp::name')
  apache::vhost {$webapp::name:}

}

Roles:

class role::prod_web {
  include 'base'
  include 'profile::webapp'
}

Now some of the questions I face:

1- Say thate for whatever reason the profile::webap requires a specific package... ie php-apc that is not covered by the apache module. The roles / profile states that you should always reference modules. Would you guys create a new class just to include a resource? What I usually end up doing is to add that package into the profile for the sake of simplicity.

2- Sometimes modules from puppetlabs or other contributors lacks of some functionality. Say for example you need to deploy a file under /etc/sysconfig. I wouldn't place that file under the profile class as that is used for multiple profiles definitions. However creating a new module for just a single file seams like too much of an overhead. What I usually do is I split up the profile module into multiple profile modules and use the repo -> install -> config -> service pattern. That allows me to create a file / template where to place my specific resources for that profile and still consume data from hiera to customize the behaviour.

3- The problem with point 2 is that you might end up with too many profile classes and some of them might include a simple reference to a module. That is not much of a problem to me as I prefer to have my files attached to the right profile module rather than having multiple files on a single profile module... or multiple modules with just a couple of files.


Cheers!

Juan Breinlinger

[Ramin K]

On 2/9/2014 4:47 AM, JuanBrein wrote:
>
>
> I've been using puppet on different companies and implementing the roles
> / profile pattern on some of them.
>
> In theory the patter works very well but in practice I usually face
> challenges that I sort out implementing my own designs / solutions. I
> would like to know how you guys deal with that in case you do.
>

> **Say you have a typical LAMP stack and you have to deploy a web app so

> my classes would look something like this (super simplified version):
>

> *Modules:*

>
> class apache { //puppetlabs class }
> class mysql { //puppetlabs class }
> etc./.
>

> *Profile*:

>
> class profile::webapp {
>
> class 'apache'
> class 'mysql'
>
> $name = hiera('webapp::name')
> apache::vhost {$webapp::name:}
>
> }
>

> *Roles:*

1. profiles::php with create_resources around a Package resource that
pulls in php-apc, php-mcrypt, php-gd, and all the other usual suspects
based on Hiera data. When was the last time anyone needed just one PHP
module? Also not a terrible place to set apc.ini and other config files.

2. profile::myrole and yeah I add the resource directly particularly if
it'll never ever conflict with another module. Also a good place to pull
in very simple modules. I'm not a fan of breaking things up into more
specific subclasses within a profile::class.

3. See #2

I recently took a crack at writing some examples of profile uses as
well as philosophizing on good profile classes. Probably needs another
hour of editing, but might be helpful in its current state.
https://ask.puppetlabs.com/question/5235/what-goes-in-the-profile-part-of-roleprofile/

Ramin

[JuanBrein]

THanks and great post by the way!

I think we are pretty much on the same thinking behind. You don't add the "package"  resource directly but using create_resources from hiera is almost the same thing. THe only difference is that your way is more flexible as you can add / remove packages just changing data and not code. But if you know beforehand what are the requires and you think they'll be static in the long term I prefer that to be on the code side so my hiera data looks small compact relevant and tidy.

My problem is with the file resources and templates. if if you have a decent amount of different applications you'll end up with a super profile class. It'll contain all different type of files and templates and too many sub profile modules. Some companies have more than 200 different applications type with an average of 2 to 4 config files to be deployed by app. I know some of them could be moved to "rpms" but is normal to have at least 1 config file managed by templates. DO you think it is good to have a profile class with say 300 400 files from different applications?

That's where I prefer to use a different pattern and that is one profile class per application: ie:

profile_webapp
profile_alpha_app
profile_gamma_app
etc...

And sometimes when needed use the repo->config->install->service pattern.

Do you see any cons on that approach?

Thanks!

Juan

1. profiles::php with create_resources around a Package resource that
pulls in php-apc, php-mcrypt, php-gd, and all the other usual suspects
based on Hiera data. When was the last time anyone needed just one PHP
module? Also not a terrible place to set apc.ini and other config files.

2. profile::myrole and yeah I add the resource directly particularly if
it'll never ever conflict with another module. Also a good place to pull
in very simple modules. I'm not a fan of breaking things up into more
specific subclasses within a profile::class.

[Ramin K]

On 2/9/2014 11:37 PM, JuanBrein wrote:
> THanks and great post by the way!
>
> I think we are pretty much on the same thinking behind. You don't add
> the "package" resource directly but using create_resources from hiera
> is almost the same thing. THe only difference is that your way is more
> flexible as you can add / remove packages just changing data and not
> code. But if you know beforehand what are the requires and you think
> they'll be static in the long term I prefer that to be on the code side
> so my hiera data looks small compact relevant and tidy.

You'll have better luck if you data is large and your code is small and
tidy. :-) There are cases where adding a Package or File resource
without any lookup or generalization is the right choice. In cases like
your PHP module example where you know you'll need more than one and
probably 10-15 which will need updating as the webapp increases in
functionality I'd go with a data driven solution.

> My problem is with the file resources and templates. if if you have a
> decent amount of different applications you'll end up with a super
> profile class. It'll contain all different type of files and templates
> and too many sub profile modules. Some companies have more than 200
> different applications type with an average of 2 to 4 config files to be
> deployed by app. I know some of them could be moved to "rpms" but is
> normal to have at least 1 config file managed by templates. DO you think
> it is good to have a profile class with say 300 400 files from different
> applications?

I'm not sure I understand the problem as you describe it. Each
application should or likely runs on its own server, vm, container, or
whatever. That's going to limit the actual number of profiles applied to
that node to a reasonable amount. In my system the most complex role or
hostgroup has 18 profiles which apply 46 modules and manages 332 File
resources of actual config (no large dir sync nonsense). That looks
reasonably complex to me unless you're building some sort of junk drawer
monstrosity of a multifunction server.

> That's where I prefer to use a different pattern and that is one profile
> class per application: ie:
>
> profile_webapp
> profile_alpha_app
> profile_gamma_app
> etc...
>
> And sometimes when needed use the repo->config->install->service pattern.
>
> Do you see any cons on that approach?
>
> Thanks!
> Juan

Without seeing a real example of what you're doing it sounds like most
of your code should be in a module that is then included by a profile. I
can't think of any reasons to be declaring a Service in a profile class.
Enabling it, yes. Adding additional config, yes. Declaring, no.

Taking the example of Apache yet again, your Apache module should
install Apache, minimally configure it, and start it if so set in your
code or data. That's it. No modules, no vhosts, or anything beyond a
minimal config by default. Because it does so little you can include it
anywhere and add the additional site specific config on top. Because it
does so little you can share it without someone needing to immediately
rip your system's idiosyncrasies out of it.

Ramin

> <https://ask.puppetlabs.com/question/5235/what-goes-in-the-profile-part-of-roleprofile/>
>
>
> Ramin
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/b8ad138c-e10e-454e-8151-3239ce1e37b1%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[JuanBrein]

Interesting... All your comments are very valuable and I definitely see some differences

I'll test some of your suggestions and see how it goes.

Thanks for taking the time to answer!

Cheers!

Juan