Using foo::params, inheritance, and parameterized classes simultaneously?

[Justin Ellison]

I asked in IRC today, and no one could tell me the benefit -- maybe
puppetlabs or example42 can answer my question.

I've seen a handful of pretty bleeding-edge modules combining the use
of foo::parms, inheritance, and parameterized classes. Here's two
examples:

https://github.com/puppetlabs/puppetlabs-mysql/blob/master/manifests/server.pp
https://github.com/example42/puppet-openssh/blob/master/manifests/init.pp

Why use inheritance instead of an 'include foo::params'? Why use
params at all if the class is parameterized?

Thanks,

Justin

[Gary Larizza]

The inheritance allows you to reference the foo::params::bar variable inside the parameter section of the class definition (the part at the top where there is '$my_class = $openssh::params::my_class,'.

Why use parameters?  It's a way to set a 'sane default' to the parameters in the class back to the value that's in the foo::params class.  If you want to override what's set in foo::params, just pass it in the class declaration.  If you want to use the default value in foo::params, just declare the class by doing 'include <class>' and NOT pass any parameters.

Does this help?

Thanks,

Justin

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--

Gary Larizza

Professional Services Engineer

Puppet Labs

[Alessandro Franceschi]

Gary has already given a good reply, let me just add some of the motivations that have driven this approach in my new modules (example42):

- I want all the variables used in a Puppet class to be visible to the system (in my case Puppi, so that I can do interesting things with them): Puppet know so much about how a system is configured that I find it somehow reductive to use all this knowledge "just" to configure things :-)

- Parametrized class are cool, but they can be a PITA to repeat every time you use them some common parameters (in my case if and how to monitor the class' resources, if to firewall it or activate puppi for it and so on)

- In params.pp you can set default values and manage the differences among different operating system (which is something that the user should not care about) . I present these module internal variables  (such as packagename, processname, servicename and so on) as class parameters because I need them for my puppi things (look at what is done when puppi is set to true in https://github.com/example42/puppet-openssh/blob/master/manifests/init.pp  )

- Parametrized classes are cool but you can't (still) use them in the common ENCs around (PuppetDashboard, the Foreman). In Puppet AFAIK, extenarl nodes can have just some variables set and classes included. That's why I wanted to preserve the "old" style set variables+include class approach.

Actually in my new modules you can mix global variables and class parameters seamlessly.

All this approach requires some verbose extra code in params.pp but for me is worth the effort.

Also Dan Bode has suggested to create a custom function that makes the same lookups for each parameter (params class, custom classes, top scope variables and hiera lookups) that would save a lot of extra code in params.pp . So there might be some further development in this area.

Hope to have given reasonable answers ;-)

al 

[Nigel Kersten]

What would it take for you to just have one source of truth for data Alessandro?

e.g. what would we have to change about Puppet for us to simply use Hiera ? 

 

We're looking to integrate Hiera into Puppet for Telly, and in general moving data outside of manifests has a lot of benefits, but I'm interested to know whether you're of a similar mind with regard to the data you're considering.

Hope to have given reasonable answers ;-)

al 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/VaO0zq3x_OsJ.

To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--
Nigel Kersten
Product Manager, Puppet Labs

[Justin Ellison]

I see now - it gives you a way to have intelligence in the selection
of the default value, yet allow the declaration to ultimately override
that intelligence if it chooses to do so.

Thanks!

Justin

On Jan 13, 4:17 pm, Gary Larizza <g...@puppetlabs.com> wrote:
> On Fri, Jan 13, 2012 at 2:02 PM, Justin Ellison <jus...@techadvise.com>wrote:
>
> > I asked in IRC today, and no one could tell me the benefit -- maybe
> > puppetlabs or example42 can answer my question.
>
> > I've seen a handful of pretty bleeding-edge modules combining the use
> > of foo::parms, inheritance, and parameterized classes.  Here's two
> > examples:
>

> >https://github.com/puppetlabs/puppetlabs-mysql/blob/master/manifests/...
> >https://github.com/example42/puppet-openssh/blob/master/manifests/ini...

[Alessandro Franceschi]

On Sunday, January 15, 2012 12:41:50 AM UTC+1, Nigel Kersten wrote:

What would it take for you to just have one source of truth for data Alessandro?

e.g. what would we have to change about Puppet for us to simply use Hiera ? 

 

We're looking to integrate Hiera into Puppet for Telly, and in general moving data outside of manifests has a lot of benefits, but I'm interested to know whether you're of a similar mind with regard to the data you're considering.

Nigel,

first of all do consider that I don't exclude Hiera's use in this approach.

It's a choice, you can use Hiera  to assign values to top scope variables or directly when you call parametrized classes, actually where and how you give values to variables is not forced by the module.

The reason why I didn't place  hiera lookups directly in the module is that these "nextgen" modules are supposed to work for Puppet > 2.6 and not everybody might want to use it or have integrated Hiera in their setup, but if the same modules can provide out of the box whatever is needed to work seamlessly, I've no problems with that. (see third point).

Second point is that some variables, which are typically module's internal variables (names, paths and so on), should be IMHO provided by the module itself (the typical stuff for params class), to give some extra value (cross operating system support) to the module and preserve the user from providing this kind of common and preset data.

I do present also these variables as class parameters because I need a quick way to pour all the modules variables (both "internal like these and the ones the module's users is likely to provide) to some place on the target system (for the puppi stuff, but actually once you have them in a yaml file you can use them in other ways).  To do this I use Ken's neat get_class_vars () function.

At the same time there are some rare cases when a user may need to supply these variables (for example paths and names for a custom package), so it's not bad to have them as parameters available in the class.

Third point is that actually I would love to use Hiera directly in the module (or better something that fallbacks to Hiera... see below) if there would be a backend that permits something like what Dan has suggested me (sorry for public quoting Dan, but I think your suggestions deserves wider attention):

---QUOTE----

Instead of having to code all of the logic for how to perform the lookups for all of the data in the params namespace(this is a lot of code!), I would recommend using hiera (or another function) and coding the class inclusion, and data lookup precedence in a custom backend.

I have a feeling that if you do a good enough job on the backend, it could just become the default puppet hiera backend.

I imagine it working something like this:

when looking up a variable $var

class foo::bar (

  $var = als_hiera($var)

) {

}

It does the followins:

   - if the parameter was set in the class declaration, that always wins (this is just how Puppet works)

   - if an explicit class parameter was not set, the hiera function is called, and does the following:

1. Look for a variable called $::foo_var ($::${module_name}_var)

2. Look for a variable called $::var

I like this capability of being able to set common behaviors across modules with a single topscope variable, I do think this needs to be configurable somehow:

  $var = als_hiera($var, {'topscope_lookup' => false})

since this pattern may not be desirable for certain parameters

3. If the class params::foo (::params::${module_name} exists, include it and look for the variable:

  ::params::foo::var

4. if the class ::foo::params (:$module_name::params) exists, include it and lookup the value of the variable:

::foo::params::var

--- QUOTE ----

To this I might add that it would be nice to have the option to set a default value if none is provided in the above sources:

var = als_hiera($var, {'default' => "mydefault"})

So, to reply to your question, Nigel, something like a hiera backend or a Puppet function that does what is written before in any Puppet setup would be what  I need.

The ideal, maybe, would be a simple function that does the above and then if Hiera is in place, uses Hiera without requiring it: I really would like to reach a scenario where a person picks a single module and eventually the dependent modules (stdlib and so on), and can use it on his Puppetmaster without the need to introduce Hiera on it, if it's not already there.

This, at least, until most of the people will have a  PuppetMaster with integrated Hiera ... till that moment I would like to have modules usable in the widest range of Puppet setups possible.

A similar function would be more than enough for me to throw happily away 80% of the code that is in https://github.com/example42/puppet-openssh/blob/master/manifests/params.pp 

All the best

al

[Walter Heck]

I would be very happy to see a seamless/lazy/magic approach to using
variables. Something similar to what Al quoted seems nice: I'd like to
be able to have the option to feed values to variables in different
places (of which one will take precedence over the other, and you
preferrably add a value only in one place) and then use that variable
in the module itself. That would make reuse of modules by others so
much nicer as they wouldn't be tied to using whatever your environment
looks like that you developed the modules for.

I guess it would look much like facter: it just provides values, but
hides where it got the values for those variables from. As Nigel
pointed out, data needs to be moved to outside manifests, and hiera
seems like a good tool for it.

Walter

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/puppet-users/-/l1YoNqgCGqsJ.

>
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.

--
Walter Heck

--
follow @walterheck on twitter to see what I'm up to!
--
Check out my new startup: Server Monitoring as a Service @ http://tribily.com
Follow @tribily on Twitter and/or 'Like' our Facebook page at
http://www.facebook.com/tribily

[R.I.Pienaar]

----- Original Message -----
> I would be very happy to see a seamless/lazy/magic approach to using
> variables. Something similar to what Al quoted seems nice: I'd like
> to
> be able to have the option to feed values to variables in different
> places (of which one will take precedence over the other, and you
> preferrably add a value only in one place) and then use that variable
> in the module itself. That would make reuse of modules by others so
> much nicer as they wouldn't be tied to using whatever your
> environment
> looks like that you developed the modules for.
>
> I guess it would look much like facter: it just provides values, but
> hides where it got the values for those variables from. As Nigel
> pointed out, data needs to be moved to outside manifests, and hiera
> seems like a good tool for it.

sounds like you want to use hiera

http://www.devco.net/archives/2011/06/06/puppet_backend_for_hiera.php

[Nigel Kersten]

On Sun, Jan 15, 2012 at 12:42 AM, Alessandro Franceschi <a...@lab42.it> wrote:

So, to reply to your question, Nigel, something like a hiera backend or a Puppet function that does what is written before in any Puppet setup would be what  I need.

The ideal, maybe, would be a simple function that does the above and then if Hiera is in place, uses Hiera without requiring it: I really would like to reach a scenario where a person picks a single module and eventually the dependent modules (stdlib and so on), and can use it on his Puppetmaster without the need to introduce Hiera on it, if it's not already there.

This, at least, until most of the people will have a  PuppetMaster with integrated Hiera ... till that moment I would like to have modules usable in the widest range of Puppet setups possible.

So here's the rough idea we have in place.

Parameterized class lookups are automatically consulted in a Hiera backend.

The lookup order would be as follows:

1. directly supplied value in the declaration of the class:

  class { "foo": param => "value" }

2. Hiera backend

3. Default value supplied in the definition of the class:

  class foo ($param = "value") { ... } 

This would happen automatically, without requiring the use of a hiera() like function in the class definition.

It's clear that we need to have a way for module authors to specify values in the definition of the class.

Is this satisfied well by having the values in the manifests themselves as in (3) above?

Or should we be encouraging authors to put a Hiera-style backend inside their modules and have that be consulted for default values?

I'm not particularly fond of the work we're making people do around polluting the global namespace with class-specific data. 

That's something I think we should probably remove entirely, but open to suggestions.

The primary goal here is to finally enable seamless module reuse. We've made people jump through far too many hoops in the past.

[Walter Heck]

On Sun, Jan 15, 2012 at 16:48, R.I.Pienaar <r...@devco.net> wrote:
>
>
> sounds like you want to use hiera
>
> http://www.devco.net/archives/2011/06/06/puppet_backend_for_hiera.php

Ah, I didn't see before that it actually gets it's data from mutliple
backends. That's pretty sweet!

[Alessandro Franceschi]

On Sunday, January 15, 2012 9:30:02 PM UTC+1, Nigel Kersten wrote:

On Sun, Jan 15, 2012 at 12:42 AM, Alessandro Franceschi <a...@lab42.it> wrote:

So, to reply to your question, Nigel, something like a hiera backend or a Puppet function that does what is written before in any Puppet setup would be what  I need.

The ideal, maybe, would be a simple function that does the above and then if Hiera is in place, uses Hiera without requiring it: I really would like to reach a scenario where a person picks a single module and eventually the dependent modules (stdlib and so on), and can use it on his Puppetmaster without the need to introduce Hiera on it, if it's not already there.

This, at least, until most of the people will have a  PuppetMaster with integrated Hiera ... till that moment I would like to have modules usable in the widest range of Puppet setups possible.

So here's the rough idea we have in place.

Let me contextualize: What you suggest here is what may be introduced in 2.8 , when Hiera will be integrated into Puppet?

 

Parameterized class lookups are automatically consulted in a Hiera backend.

The lookup order would be as follows:

1. directly supplied value in the declaration of the class:

  class { "foo": param => "value" }

2. Hiera backend

3. Default value supplied in the definition of the class:

  class foo ($param = "value") { ... } 

This would happen automatically, without requiring the use of a hiera() like function in the class definition.

+1  

I like the fact that this is transparent... if you have an Hiera backend then it's used, if not (or if you use an older Puppet version), normal behavior is followed.

It's clear that we need to have a way for module authors to specify values in the definition of the class.

Is this satisfied well by having the values in the manifests themselves as in (3) above?

Well a simple  class foo ($param = "value") { ... } might not be enough, some logic (for example a different value for different operating systems) to give defaults values might be needed (back to params.pp ...)

 

Or should we be encouraging authors to put a Hiera-style backend inside their modules and have that be consulted for default values?

For me is saner to leave to puppet dsl the management of the values to attribute to these internal variables. 

It's how we always have done it and, for me, it works.

 

I'm not particularly fond of the work we're making people do around polluting the global namespace with class-specific data. 

That's something I think we should probably remove entirely, but open to suggestions.

-1 if I understood well

If you mean that you would remove the possibility to define top scope variables with custom names (ie things like ::monitor = true but also ::monitor_openssh = true) please let me know when and how this could happen.

As a user I'm not so fond of radical changes in the Puppet code that make unusable manifests made for earlier Puppet versions (hint: dynamic variables scoping no more possible in 2.8: I understand that they were a source of various problems, but that was just for users misuse due eventually to not clear enough directives/documentation/best practices).

I understand that this is done for a "better future" and that retrocompatibility should not be a dogma, but... well... when I write some Puppet code I hope it's going to work for some years...

my2c

al

[Nigel Kersten]

On Mon, Jan 16, 2012 at 7:23 AM, Alessandro Franceschi <a...@lab42.it> wrote:

On Sunday, January 15, 2012 9:30:02 PM UTC+1, Nigel Kersten wrote:

So here's the rough idea we have in place.

Let me contextualize: What you suggest here is what may be introduced in 2.8 , when Hiera will be integrated into Puppet?

Yes, but "Telly", not "2.8".

We may use Telly to fully adopt Semantic Versioning ( http://semver.org ) and take this opportunity to reset to "3.0.0" instead of "2.8".

 

 

Parameterized class lookups are automatically consulted in a Hiera backend.

The lookup order would be as follows:

1. directly supplied value in the declaration of the class:

  class { "foo": param => "value" }

2. Hiera backend

3. Default value supplied in the definition of the class:

  class foo ($param = "value") { ... } 

This would happen automatically, without requiring the use of a hiera() like function in the class definition.

+1  

I like the fact that this is transparent... if you have an Hiera backend then it's used, if not (or if you use an older Puppet version), normal behavior is followed.

Yep.

 

It's clear that we need to have a way for module authors to specify values in the definition of the class.

Is this satisfied well by having the values in the manifests themselves as in (3) above?

Well a simple  class foo ($param = "value") { ... } might not be enough, some logic (for example a different value for different operating systems) to give defaults values might be needed (back to params.pp ...)

 

Or should we be encouraging authors to put a Hiera-style backend inside their modules and have that be consulted for default values?

For me is saner to leave to puppet dsl the management of the values to attribute to these internal variables. 

It's how we always have done it and, for me, it works.

But this is really because there has been no other option right? It's never really been easy for module authors to ship default data values inside modules, but not in manifests.

I agree it works, but I'd like us to think about whether we can improve it.

 

 

I'm not particularly fond of the work we're making people do around polluting the global namespace with class-specific data. 

That's something I think we should probably remove entirely, but open to suggestions.

-1 if I understood well

If you mean that you would remove the possibility to define top scope variables with custom names (ie things like ::monitor = true but also ::monitor_openssh = true) please let me know when and how this could happen.

No no no.

I more meant that we've had a whole lot of clustered behavior that has meant we've forced people to use global namespace variables in say ENCs to pass down to modules.

Users and authors shouldn't *have* to do that, and I'd like to make sure we're giving people better options than always having to put everything in top scope.

 

As a user I'm not so fond of radical changes in the Puppet code that make unusable manifests made for earlier Puppet versions (hint: dynamic variables scoping no more possible in 2.8: I understand that they were a source of various problems, but that was just for users misuse due eventually to not clear enough directives/documentation/best practices).

I understand that this is done for a "better future" and that retrocompatibility should not be a dogma, but... well... when I write some Puppet code I hope it's going to work for some years...

I'd argue that the dynamic variable scoping resulted in quite fundamental problems, but yes, there's totally a balancing act here between progress and compatibility.

[Ryan Bowlby]

Thanks for the great replies Nigel. As a person currently rolling out
Puppet into production I'm stricken with doubt about how to best
represent variables in a module. It would seem to me that I need to:

1. set default values in params.pp (based on facts, yadda yadda)
2. allow those values to be overridden by global namespace variables
(for enc support)
3. further allow parametrized class declarations to take precedence
over 1 and 2

Can you point me to a module that does this? Alternatively, can you
make a recommendation for module best practices going forward.

Thanks,
Ryan Bowlby

[Nigel Kersten]

On Tue, Jan 17, 2012 at 9:33 PM, Ryan Bowlby <rbow...@gmail.com> wrote:

Thanks for the great replies Nigel. As a person currently rolling out
Puppet into production I'm stricken with doubt about how to best
represent variables in a module. It would seem to me that I need to:

1. set default values in params.pp (based on facts, yadda yadda)
2. allow those values to be overridden by global namespace variables
(for enc support)
3. further allow parametrized class declarations to take precedence
over 1 and 2

Can you point me to a module that does this? Alternatively, can you
make a recommendation for module best practices going forward.

Have you looked at Hiera yet Ryan? An awful lot of this is baked into it, and you may find you don't need to set these at the ENC level.

 

[Ryan Bowlby]

Looking into it now, thanks. Related question, I see a lot of modules
where an optional package is available as another class, example

include apache::ssl

would install apache and mod_ssl. With parametrized classes is it now
better to make this a parameter?

node blahblah {
class { 'apache':
ssl_package => true,
}
}

I'm just trying to do things "right" the first time.

On Jan 17, 9:59 pm, Nigel Kersten <ni...@puppetlabs.com> wrote:

[jcbollinger]

On Jan 18, 1:32 am, Ryan Bowlby <rbowlb...@gmail.com> wrote:
> Looking into it now, thanks. Related question, I see a lot of modules
> where an optional package is available as another class, example
>
> include apache::ssl
>
> would install apache and mod_ssl. With parametrized classes is it now
> better to make this a parameter?

With parameterized classes, it is usually better to avoid
parameterized classes. Especially so now that hiera being
incorporated into the core. I can't think of much that parameterized
classes gain you that hiera cannot provide, and use of hiera does not
incur the costs attending use of parameterized classes.

The deeper question, however, is whether it is better to use classes
or conditionals (regardless of data source) to determine which
optional features to manage on a node. It is better to use classes.
Doing so keeps individual classes cleaner and more maintainable, and
it is far more flexible. It also plays more nicely with your SCM
system (you *are* keeping your manifests in an SCM system, right?)
because the nature and scope of each change is more clearly reflected
by which files are affected.


John

[Justin Ellison]

I was looking through the Puppet style guide for something unrelated,
and noticed that it explicitly discourages doing things in this exact
manner:
http://docs.puppetlabs.com/guides/style_guide.html -> Section 11.9.

Does the style guide need updated then?

Justin

On Jan 13, 4:17 pm, Gary Larizza <g...@puppetlabs.com> wrote:

> On Fri, Jan 13, 2012 at 2:02 PM, Justin Ellison <jus...@techadvise.com>wrote:
>
> > I asked in IRC today, and no one could tell me the benefit -- maybe
> > puppetlabs or example42 can answer my question.
>
> > I've seen a handful of pretty bleeding-edge modules combining the use
> > of foo::parms, inheritance, and parameterized classes.  Here's two
> > examples:
>

> >https://github.com/puppetlabs/puppetlabs-mysql/blob/master/manifests/...
> >https://github.com/example42/puppet-openssh/blob/master/manifests/ini...