Re: Profiles, Hiera, and create_resources('class','...')

[jcbollinger]

On Friday, January 11, 2013 2:14:46 PM UTC-6, Eric Sakowski wrote:

Hi all,

We've recently started exploring the role / profile / component module described by Craig Dunn in his blog here:
http://www.craigdunn.org/2012/05/239/ and discussed on the list the other day.  As I was implementing this for
a profile using the apache module, I realized that I could make another refinement to our approach by using
create_resources('class','<module name>') to pull our hiera data into the apache class and override any defaults needed. 
It looks good to me but there are some concerns that it will come back and bite us in ways we don't expect
later on, when updating to Puppet 3.0.

Good news: it's unlikely that this will bite you later on.
Bad news: that's because it will bite you now.

Classes are not resources, and the create_resources() function does not work with them.


John

[R.I.Pienaar]

sure it does - just like any other resources.

class x($msg) { notify{$msg: } }

create_resources("class", {"x" => {"msg" => "hello world"}})

will produce:

Notice: hello world
Notice: /Stage[main]/X/Notify[hello world]/message: defined 'message' as 'hello world'

[jcbollinger]

My apologies, then.  That did not always work.  The issue has come up here, I think within the past year.  Do you know when it changed?


John

[R.I.Pienaar]

No I am not sure - I dont tend to use create_resources as I consider it about
as bad as eval() :)

I just know I recently went spelunking into that code while doing some prototype
and noticed it worked

[GRANIER Bernard (MORPHO)]

Due to internet policy of my company, I can not access to gepetto install site thru eclipse.

Is it possible to download gepetto plugin as a zip ? I mean not a full eclipse but just the plugin.

Cordialement,

Bernard Granier
CE Plateforme Système
bernard...@morpho.com
01 58 11 32 51


#
" This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system."
#

[jcbollinger]

On Friday, January 11, 2013 2:14:46 PM UTC-6, Eric Sakowski wrote:

Hi all,

We've recently started exploring the role / profile / component module described by Craig Dunn in his blog here:
http://www.craigdunn.org/2012/05/239/ and discussed on the list the other day.  As I was implementing this for
a profile using the apache module, I realized that I could make another refinement to our approach by using
create_resources('class','<module name>') to pull our hiera data into the apache class and override any defaults needed. 
It looks good to me but there are some concerns that it will come back and bite us in ways we don't expect

later on, when updating to Puppet 3.0.  The benefit right away is that it saves us from having to even touch
puppet forge modules to 'hierafy' them, and it is much less code to write and maintain, simply by structuring
class param hiera data as a hash of key-value pairs instead of flat key-value pairs in the yaml.  Any guidance
or warnings or comments in general would be most helpful to us in deciding how to go forward.

Accepting R.I. that the approach won't break outright, I'll take up the question of subtle bite-us-later issues, plus some other issues.

First, do note that the second argument to create_resources() is not expected to be a plain resource name: it is expected to be a hash, whose keys are names of resources to create and whose values are hashes of resource parameter mappings.  (Perhaps you already understand that and were merely shortcutting.)  Perhaps it would work with scalars as shown, but it is better to use the 'include' function if you do not intend to declare class parameters.

Second, be aware that in Puppet 3, you already don't need to do anything special in your manifests to hierify the parameters of parametrized classes.  Puppet 3 will automatically consult hiera for class parameter values, using keys based on the class and parameter names.  This is one of Puppet 3's best advances.

Although parametrized classes work fine in Puppet 3 with hiera-based parameter lookup, the same is not true of classes declared with local parameter customizations via parametrized-class syntax (in any Puppet version since parametrized classes were introduced).  The problems with such declarations must also apply if you use the create_resources() function to set class parameters.

The most important problem there that classes may be declared any number of times -- and there are good reasons for multiple declarations of the same class -- but only the first declaration parsed may declare explicit parameters.  Furthermore, Puppet's parse order can be difficult to predict (indeed, controlling parse order is one of the most important reasons for multiple class declarations) so it may be tricky to be confident that any particular declaration will be parsed first.

Even if your create_resources() appears at top scope or in a node block, you could still be bitten by the multiple-declaration issue if one of the modules you declare that way itself declares one of the others that you declare later.  Worse (in this regard), if you declare multiple classes in the same declare_resources() call, then you absolutely cannot predict the order in which those classes will be declared.

If, on the other hand, you use create_resources() to declare one class at a time, then you gain little.  Plain 'include' combined with Puppet 3's parameter autolookup would serve you better for that purpose.


John

[jcbollinger]

On Monday, January 14, 2013 9:49:33 AM UTC-6, R.I. Pienaar wrote:

No I am not sure - I dont tend to use create_resources as I consider it about
as bad as eval() :)

I just know I recently went spelunking into that code while doing some prototype
and noticed it worked

Bottom line, then: it will bite the OP either now or later, or maybe even both.


John

[Henrik Lindberg]

On 2013-14-01 16:58, GRANIER Bernard (MORPHO) wrote:
> Due to internet policy of my company, I can not access to gepetto install site thru eclipse.
>
> Is it possible to download gepetto plugin as a zip ? I mean not a full eclipse but just the plugin.
>

It is not available to the public, please contact me directly and I will
help you out. (Email or on #geppetto on IRC)

Supposedly it should also be possible to install into an Eclipse IDE
from the pacakged Geppetto by refering to its location in Eclipse - but
have not tried this.

Longer term we will make the repository itself available. We intended to
put it up on github download, but now that service has been deprecated
and we are looking for a new download home.


Regards
- henrik

[GRANIER Bernard (MORPHO)]

My mail is bernard...@morpho.com

Cordialement,

Bernard Granier
CE Plateforme Système
bernard...@morpho.com
01 58 11 32 51

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Eric Sakowski]

On Monday, January 14, 2013 11:17:47 AM UTC-5, jcbollinger wrote:

On Friday, January 11, 2013 2:14:46 PM UTC-6, Eric Sakowski wrote:

Hi all,

We've recently started exploring the role / profile / component module described by Craig Dunn in his blog here:
http://www.craigdunn.org/2012/05/239/ and discussed on the list the other day.  As I was implementing this for
a profile using the apache module, I realized that I could make another refinement to our approach by using
create_resources('class','<module name>') to pull our hiera data into the apache class and override any defaults needed. 
It looks good to me but there are some concerns that it will come back and bite us in ways we don't expect
later on, when updating to Puppet 3.0.  The benefit right away is that it saves us from having to even touch
puppet forge modules to 'hierafy' them, and it is much less code to write and maintain, simply by structuring
class param hiera data as a hash of key-value pairs instead of flat key-value pairs in the yaml.  Any guidance
or warnings or comments in general would be most helpful to us in deciding how to go forward.

Accepting R.I. that the approach won't break outright, I'll take up the question of subtle bite-us-later issues, plus some other issues.

 

Thanks John for the reply.  It definitely works.  The code I posted was copied from a working setup, not pseudo-code.
 

First, do note that the second argument to create_resources() is not expected to be a plain resource name: it is expected to be a hash, whose keys are names of resources to create and whose values are hashes of resource parameter mappings.  (Perhaps you already understand that and were merely shortcutting.)  Perhaps it would work with scalars as shown, but it is better to use the 'include' function if you do not intend to declare class parameters.

 
Understood.  apache_params is a hash containing key apache w/ value <hash of apache values from hiera>
 

Second, be aware that in Puppet 3, you already don't need to do anything special in your manifests to hierify the parameters of parametrized classes.  Puppet 3 will automatically consult hiera for class parameter values, using keys based on the class and parameter names.  This is one of Puppet 3's best advances.

OK, so given that I am a user of the apache module, and it is parameterized already when I install it, for Puppet 3.0 then I could simply drop create_resources call and Puppet 3 will automatically look it up in hiera? 
 

Although parametrized classes work fine in Puppet 3 with hiera-based parameter lookup, the same is not true of classes declared with local parameter customizations via parametrized-class syntax (in any Puppet version since parametrized classes were introduced).  The problems with such declarations must also apply if you use the create_resources() function to set class parameters.

The most important problem there that classes may be declared any number of times -- and there are good reasons for multiple declarations of the same class -- but only the first declaration parsed may declare explicit parameters.  Furthermore, Puppet's parse order can be difficult to predict (indeed, controlling parse order is one of the most important reasons for multiple class declarations) so it may be tricky to be confident that any particular declaration will be parsed first.

OK.  Sounds like having my defaults come from the apache modules parameter declarations may be a problem in Puppet 3.0.  I am installing a Puppet 3 master to try and test this.  Any suggestions on how best to set up the error condition you describe?  I have run across some error messages from puppet rdoc finding multiple class declarations.  I'd like to better understand what you're talking about here -- I'll go google but any links would be appreciated!

Even if your create_resources() appears at top scope or in a node block, you could still be bitten by the multiple-declaration issue if one of the modules you declare that way itself declares one of the others that you declare later.  Worse (in this regard), if you declare multiple classes in the same declare_resources() call, then you absolutely cannot predict the order in which those classes will be declared.

If, on the other hand, you use create_resources() to declare one class at a time, then you gain little.  Plain 'include' combined with Puppet 3's parameter autolookup would serve you better for that purpose.

That sounds great and probably what I will do when we eventually upgrade to Puppet 3.  Basically just drop the calls to create_resources  and simply include.

John

Thanks again,
Eric
 

[jcbollinger]

On Monday, January 14, 2013 11:00:42 AM UTC-6, Eric Sakowski wrote:

[...]

OK, so given that I am a user of the apache module, and it is parameterized already when I install it, for Puppet 3.0 then I could simply drop create_resources call and Puppet 3 will automatically look it up in hiera? 

That is correct.  Just use this:

include 'apache'

In addition, you will need to structure your hiera data slightly differently.  Instead of collecting the class parameters in a hash, they need to be (namespaced) top-level data:

apache::default_mods :         true
apache::default_vhost :        false
apache::default_ssl_vhost :    false
apache::service_enable :       true
apache::serveradmin :          'root@localhost'
apache::sendfile :             false
apache::error_pages :          false

Note that the namespace prefix is just part of the keys as far as hiera is concerned; the magic happens on the Puppet side.

 

[...]

OK.  Sounds like having my defaults come from the apache modules parameter declarations may be a problem in Puppet 3.0.

Not at all.  Puppet will fall back to the default (if any) in the class definition if hiera does not provide a value for a given parameter.  The precedence is:  explicit parameters in your class declaration (do not use) > parameters read from hiera > parameters in the class definition > undef.  Puppet uses the first one found, on a parameter-by-parameter basis.

 

  I am installing a Puppet 3 master to try and test this.  Any suggestions on how best to set up the error condition you describe?  I have run across some error messages from puppet rdoc finding multiple class declarations.  I'd like to better understand what you're talking about here -- I'll go google but any links would be appreciated!

A simple, but very synthetic test case would be:

node test {
  include 'apache'
  class { 'apache': service_enable => true }
}

where you test with a node 'test'.  Replace the "class ..." line with your create_resources() call to test that flavor.  Note also that order matters: if you swap the "include" and "class" lines then it should work.  On the other hand, it should not work if you change the "include" line to a copy of the "class" line: multiple parametrized-style declarations of the same classes are not forbidden even when the parameter lists match (except maybe where no parameters are explicitly declared at all).

 

That sounds great and probably what I will do when we eventually upgrade to Puppet 3.  Basically just drop the calls to create_resources  and simply include.

Yes, that is precisely my recommendation.


Best,

John

[jcbollinger]

On Tuesday, January 15, 2013 9:44:00 AM UTC-6, jcbollinger wrote:

[...] multiple parametrized-style declarations of the same classes are not forbidden even when the parameter lists match

Clarification: there is an extraneous "not" in there.  It should have been "multiple parametrized-style declarations of the same classes are forbidden even when the parameter lists match."


John

[Stephen Price]

On Monday, January 14, 2013 7:49:33 AM UTC-8, R.I. Pienaar wrote:

No I am not sure - I dont tend to use create_resources as I consider it about
as bad as eval() :)

I just know I recently went spelunking into that code while doing some prototype
and noticed it worked

Sorry to derail this discussion, but R.I., what is it that you find bad about create_resources? I've found it invaluable when creating resources from definitions. 

For example, I have an apache::conf definition for a config file, and I create a hash for each config file I want created on the node. Then just use "create_resources(apache::conf, $apache_conf_hash)" in the module's init.pp. 

How would you do it otherwise? Documentation on Hiera is pretty lacking right now, and it's the best method I've found for this sort of thing.