Re: Apply multiple defines in sequence

[Erwin Bogaard]

In retrospect: a better subject would have been 'Apply multiple defines to one resource'.
But as far as I can see, there is no way to change a subject

On Thursday, November 1, 2012 3:27:10 PM UTC+1, Erwin Bogaard wrote:

I'm having trouble getting the following scenario to work:
- I have two defines, for example 'sugar' and 'wordpress'.
- Some nodes need to have both applied, others only sugar.
- Because you can't use the same resource twice (not that I want to), I can't figure out how to apply two separate defines to one resource

At  the moment I use the following workaround: I have two defines, one with only sugar and one with both sugar and wordpress. As I have to change both files if I change something for Sugar, this really isn't optimal.
Anyone a suggestion?

[Erwin Bogaard]

As there is no response, maybe some extra info:
For classes I use 'inherits' to apply multiple classes to one resource. For defines I can't find a similar technique. Is there one?

[jcbollinger]

On Friday, November 2, 2012 7:14:14 AM UTC-5, Erwin Bogaard wrote:

As there is no response, maybe some extra info:
For classes I use 'inherits' to apply multiple classes to one resource. For defines I can't find a similar technique. Is there one?

Perhaps there are no responses because your question is difficult to understand.  I don't think you are using Puppet jargon in the conventional way.

In particular, you do not apply defines to resources, though you can apply instances of defined types (which instances are resources) to nodes.  Likewise, you certainly do not apply classes to resources (though classes can and normally do declare resources); rather, you apply classes to nodes.

Furthermore, there is no general issue with applying multiple distinct instances of the same or different defined types to any given node.  You can, however, cause yourself trouble in that area with the way you write the defines.  To help you troubleshoot such an issue we would need to see some details of the definitions.
 
Finally, do not use class inheritance to combine classes.  Instead, use the 'include' function, which is expressly for that purpose.  Be aware that 'include' expresses a logical inclusion, not a textual one (i.e. it is not analogous to the C preprocessor's #include directive), therefore it does not cause multiple definition of the named class.  Example:

class mymodule::class1 {
  # resource declarations ...
}

class mymodule::class2 {
  include 'mymodule::class1'
  # other resource declarations ...
}


John

[Erwin Bogaard]

Thanks for your reply. Let me clarify things a little more.
I use the following class to apply to a certain node:

class sugar::instances {

    sugar::definitions_sugar {
    # SugarCRM - ECM2
    'node1':
      mysql_dbname => 'dbname1',
      mysql_pwd    => 'password1';

    sugar::definitions_sugar_wp {
    'node2':
      sugar_admin  => 'text1',
      sugar_pwd    => 'password2',
      mysql_dbname => 'dbname2',
      mysql_pwd    => 'password3';

As I wrote earlier: there are some nodes I like to apply just the sugar-defines to, and there are nodes that I like to apply the sugar-defines and the wordpress-defines to.
As you can see, some of the input (mysql_dbname/pwd) is the same for both defines. So I would really like to reuse this. As I don't understand how to apply multiple defines to one node, I use a workaround where I add the information for sugar to both defines. As this isn't really a good idea and creates extra work and a source for problems and inconsistency, I would really like to have one define for sugar and one for wordpress. In that situation I would like to apply only the sugar-define to node1 and both defines to node2, when only declaring node2 once.

I hope this makes things clearer, but as you said, maybe I'm using the wrong words (or jargon) for the right things.

[jcbollinger]

On Friday, November 2, 2012 2:54:11 PM UTC-5, Erwin Bogaard wrote:

Thanks for your reply. Let me clarify things a little more.
I use the following class to apply to a certain node:

class sugar::instances {

    sugar::definitions_sugar {
    # SugarCRM - ECM2
    'node1':
      mysql_dbname => 'dbname1',
      mysql_pwd    => 'password1';

    sugar::definitions_sugar_wp {
    'node2':
      sugar_admin  => 'text1',
      sugar_pwd    => 'password2',
      mysql_dbname => 'dbname2',
      mysql_pwd    => 'password3';

As I wrote earlier: there are some nodes I like to apply just the sugar-defines to, and there are nodes that I like to apply the sugar-defines and the wordpress-defines to.
As you can see, some of the input (mysql_dbname/pwd) is the same for both defines. So I would really like to reuse this. As I don't understand how to apply multiple defines to one node, I use a workaround where I add the information for sugar to both defines. As this isn't really a good idea and creates extra work and a source for problems and inconsistency, I would really like to have one define for sugar and one for wordpress. In that situation I would like to apply only the sugar-define to node1 and both defines to node2, when only declaring node2 once.

I hope this makes things clearer, but as you said, maybe I'm using the wrong words (or jargon) for the right things.

Well, you haven't given any bits that are likely to be causing trouble, so I can offer only general advice.  The main issues to watch out for with applying multiple instances of the same defined type to one node are

1. The instances must have different titles
2. The instances must not declare duplicate resources (as each other or as anything else)
   

As long as you observe those requirements, there is nothing special to what you say you want to do.  There are a variations on how you can express it in Puppet DSL, but they are all equivalent to this:

sugar::definitions_sugar {
    # SugarCRM - ECM2
    'node1':
      mysql_dbname => 'dbname1',
      mysql_pwd    => 'password1';
} 

sugar::definitions_sugar {
    # SugarCRM - ECM2, node2
    'node2':

      mysql_dbname => 'dbname1',
      mysql_pwd    => 'password1';
} 

If something like that is giving you trouble, then it probably arises from a violation of my point (2) above.  That can arise if their definitions declare resources that aren't specific to individual instances.  For example,

define sugar::definitions_sugar (
    $mysql_dbname,
    $mysql_pwd) {
  mysql::database { "$mysql_dbname":
    password => "$mysql_pwd"
  }
  # other declarations ...
}

If you want multiple Sugar instances to be able to share the same DB, then no particular instance owns that DB, hence none of them should declare it.  Instead, the DB should be declared separately, in a class, which each instance can then 'include':

class sugar::database {
  mysql::database { "$sugar::params::mysql_dbname":
    password => "$sugar::params::mysql_pwd"
  }
  # other declarations ...
}

define sugar::definitions_sugar {
  include 'sugar::database'
  # other declarations ...
}


John

[Erwin Bogaard]

Thanks again for you reply, but it seems like you don't fully understand what I'm having problems with. So I'll try to clarify it a little more:
1. The current way of using two defines is working flawlessly. So I (at least partly) understand the concepts surrounding those.
2. Because I have two types of machines: some with just sugar and some with sugar and wordpress, I now use two defines that overlap in part (define1 contains all kinds of info about creating sugar db + unpacking tar, etc, while define2 contains all the sugar info of define1 + stuff about creating a wordpress db + unpacking wp tar, etc), this means editing two files when I change something in the sugar define. As this can lead to differing configurations because of editing errors (and always twice the work), I would like to be able to split the defines up, so I can call define1 (sugar) on all machines and define1 (sugar) and sefine 2 (wordpress) on the other machines.
3. Some of the variables are shared, for example the mysqld_pwd is used twice, and I add a different suffix for sugar and wordpress to get two databases. For the httpd-configuration, I specify a different template, which is easy to to with defines. So all instances have unique resources, hence the choice for defines, not classes.

Does this help you help me?

[jcbollinger]

On Monday, November 5, 2012 3:48:58 AM UTC-6, Erwin Bogaard wrote:

Thanks again for you reply, but it seems like you don't fully understand what I'm having problems with. So I'll try to clarify it a little more:
1. The current way of using two defines is working flawlessly. So I (at least partly) understand the concepts surrounding those.
2. Because I have two types of machines: some with just sugar and some with sugar and wordpress, I now use two defines that overlap in part (define1 contains all kinds of info about creating sugar db + unpacking tar, etc, while define2 contains all the sugar info of define1 + stuff about creating a wordpress db + unpacking wp tar, etc), this means editing two files when I change something in the sugar define. As this can lead to differing configurations because of editing errors (and always twice the work), I would like to be able to split the defines up, so I can call define1 (sugar) on all machines and define1 (sugar) and sefine 2 (wordpress) on the other machines.
3. Some of the variables are shared, for example the mysqld_pwd is used twice, and I add a different suffix for sugar and wordpress to get two databases. For the httpd-configuration, I specify a different template, which is easy to to with defines. So all instances have unique resources, hence the choice for defines, not classes.

Does this help you help me?

No, not really.  You have been relatively clear about what you are trying to accomplish, but I don't understand what is preventing you from accomplishing it.  Perhaps that means you're stumbling over something that seems trivial to me.  For example, if the real question is how to share data between two or more defined types, then you have at least three choices:

1. Define the data in a class, and have each definition reference the class variables instead of taking that data in the form of parameters.
2. Externalize the data (e.g. into an Hiera data store), and have each definition reference the needed values by the same fixed key.
3. Record the data in global variables, and have each definition reference the global values.  (I wouldn't recommend this one except as a temporary hack.)

If that doesn't help then perhaps you should try reducing the problem to the simplest possible example that captures the issue.  Often such an exercise will itself help you work out the problem, but if it doesn't then we can be a lot more helpful to you with actual (simple) manifests to critique.


John

[Erwin Bogaard]

As I'm in over my head, let's supply you with (part of) the manifests in question:

The define:

define sugar::definitions_sug_wp (
      $template       = 'sugar/etc/httpd/conf.d/sugar6x.conf.erb',
      $client_domain  = "$title",
      $mysql_rootpwd  = "$mysql_password",
      $mysql_dbname,
      $mysql_pwd,
      $sugar_admin,
      $sugar_pwd,

    ) {

# This is for example to create the httpd.conf and the sugar-folder. So this has to be in both. As you can see, I parametrized the httpd-conf, so I can specify it in my class. Also, the  ${client_domain}-variable is used throughout this define.
    file {
    "/etc/httpd/conf.d/sug-${client_domain}.conf":
      content => template($template),
      owner   => 'root',
      group   => 'root',
      mode    => '0644',
      notify  => Service['httpd'];
    "/var/log/sugar/${client_domain}":
      ensure  => 'directory';
   }


# The following is specific to the WordPress-installation and doesn't need to be applied to every machine. So this part isn't in the define 'define sugar::definitions_sug'.
    file {
    # Configuratie van publieke html
    "/var/www/html/${client_domain}":
      owner   => 'apache',
      group   => 'apache',
      mode    => '0744',
      ensure  => 'directory';
    # Configuratiefile WordPress
    "/var/www/html/${client_domain}/wp-config.php":
      owner   => 'apache',
      group   => 'apache',
      mode    => '0744',
      require => File["/var/www/html/${client_domain}"],
      content => template('sugar/wordpress/wp-config.php.erb');

I call both defines ('sugar::definitions_sug_wp' and 'define sugar::definitions_sug') in the following class:
class sugar::instances {

    sugar::definitions_sug {

    # SugarCRM - ECM2
    'node1':
      mysql_dbname => 'dbname1',
      mysql_pwd    => 'password1';

    sugar::definitions_sug_wp {

    'node2':
      sugar_admin  => 'text1',
      sugar_pwd    => 'password2',
      mysql_dbname => 'dbname2',
      mysql_pwd    => 'password3';

I include this class on one node to get several sugar-only vhosts and several sugar+wordpress-nodes on that node.

Hope this helps you explain things to me!

[Erwin Bogaard]

After my previous post, I did some more thinking and reasearch. What do you think about:
- using an if/else-conditional. Would that work?
- using virtual resources? would that work?

[Tim Mooney]

In regard to: [Puppet Users] Re: Apply multiple defines in sequence, Erwin...:

> Thanks again for you reply, but it seems like you don't fully understand
> what I'm having problems with. So I'll try to clarify it a little more:
> 1. The current way of using two defines is working flawlessly. So I (at
> least partly) understand the concepts surrounding those.
> 2. Because I have two types of machines: some with just sugar and some with
> sugar and wordpress, I now use two defines that overlap in part (define1
> contains all kinds of info about creating sugar db + unpacking tar, etc,
> while define2 contains all the sugar info of define1 + stuff about creating
> a wordpress db + unpacking wp tar, etc), this means editing two files when
> I change something in the sugar define.

So have one additional parameter to your define,

sugar_with_wordpress => 'no'

and then in the body of your define, after you've done all the necessary
setup for sugar, that is always present, have

if $sugar_with_wordpress == 'yes' {
# wordpress-specific stuff here
}

I use yes/no strings instead of boolean true/false because you'll
eventually want to have those settings come from hiera, and as things
currently stand booleans from hiera are just a trap for the unwary.

I would actually do the wordpress stuff as a separate class, which has
its own wordpress::instance define, and then call that define from within
your sugar::instance define.

Tim
--
Tim Mooney Tim.M...@ndsu.edu
Enterprise Computing & Infrastructure 701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164

[jcbollinger]

You're still making me guess about what the actual problem is.  Have you actually tried anything, or is this all hypothetical?  If you tried and failed, then what error messages were produced?

Bits and pieces of your manifests are not particularly helpful.  I'm looking for something manageable that provides a complete example of the problem.  Ideally something as stripped-down as possible.  If you are totally new to Puppet then you might consider playing with a simplified model before moving on to a full-blown solution.

My best guess at this point is that the problem -- if there is one -- concerns resources that both your definitions declare, such that you get duplicate resource declarations if you try to instantiate some combinations of those definitions.  I suggested such a problem a couple of posts ago, however, and you more or less ignored it.

If duplicate resource declarations are indeed the problem then the solution is simple: don't do that!  Every managed resource should have exactly one owner -- generally either a class or defined-type instance.  Resources should be declared only by their owners.  If you stick to that -- remembering that defined-type instances are themselves resources -- then you will not have problems with duplicate resource declarations.

So, what obstacles must you overcome to (re-)structure your manifests so that each resource has exactly one owner?  (Not a rhetorical question.)  Virtual resources and conditionals are some of the tools at your disposal for overcoming such obstacles, but it's all about how you use them.  In particular, just making a declaration virtual does not inherently prevent duplicate declaration issues.

Tim's suggestion of combining the definitions and using a parameter to direct whether to include WordPress components would be one possible solution.  There are several others.  Which would be best depends on the details of the problem.


John

[Erwin Bogaard]

John and Tim, thanks for your replies and patience.
In the end, the solution of Tim seems to work great! That was what I was looking for: a relatively straight solution to a relatively straight problem (at least, that was what I thought to improve the previous situation).
As a novice to intermediate Puppet user, I've again learned a lot about writing some nice manifests and getting things done the way I want to.