Resolving duplicated resources

[Adam Cohen]

So I've run into a bit of a problem that has bitten many others in the past: I'm using module A and module B which both require package C, and I'm ending up with a duplicated resource definition for the mod_ssl package.  In my case, the modules are puppet-rvm and puppetlabs-apache which both require mod_ssl.

The problem stems from here:

in puppet-rvm:

puppet-rvm/manifests/passenger/apache/centos/pre.pp

class rvm::passenger::apache::centos::pre {
  # Dependencies
  if ! defined(Package['httpd'])       { package { 'httpd':       ensure => present } }
  if ! defined(Package['httpd-devel']) { package { 'httpd-devel': ensure => present } }
  if ! defined(Package['mod_ssl'])     { package { 'mod_ssl':     ensure => present } }
}

and in puppetlabs-apache:

puppetlabs-apache/manifests/params.pp:63

$mod_packages = {
  'proxy_html' => 'mod_proxy_html',
  'python' => 'mod_python',
  'shibboleth' => 'shibboleth',
  'ssl' => 'mod_ssl',
  'wsgi' => 'mod_wsgi',
  'dav_svn' => 'mod_dav_svn',
   'xsendfile' => 'mod_xsendfile',
}

puppetlabs-apache/modules/apache/manifests/mod.pp:35

package { $mod_packages: ensure => present,
     require => Package['httpd'],
      before => File["${mod_dir}/${mod}.load"],
}

So the rvm module attempts to resolve a conflict by using "! defined(Package['mod_ssl'})", while the apache module just defines an array of packages and then requires them all.  This causes a failure if the apache module is included after the rvm module, since the apache module doesn't check to see if any of the packages are already defined.

It seems there are a few ways to remedy this, such as defining a new class for the mod_ssl package and including that class in both modules.  However, I'd like to figure out if it's possible to rectify this situation without modifying either module.  I've tried in vain to use many different permutations of require and -> for resource ordering to ensure that the puppetlabs-apache module gets loaded first, but I just can't manage to get it to work. it seems no matter what I do, the rvm module is loaded first, which causes the puppetlabs-apache module to fail.

So can anyone tell me how I can get around this problem? I'd very much like to modify the puppetlabs-apache module, but since they use an array to define the required mod packages, it makes it a little tricky to check if each one is defined before using it, since as far as I know it's not very straight forward to iterate over an array in the puppet DSL.  Thanks for any suggestions!

Adam

[Paul Tötterman]

Hi,

It seems there are a few ways to remedy this, such as defining a new class for the mod_ssl package and including that class in both modules.  However, I'd like to figure out if it's possible to rectify this situation without modifying either module.  I've tried in vain to use many different permutations of require and -> for resource ordering to ensure that the puppetlabs-apache module gets loaded first, but I just can't manage to get it to work. it seems no matter what I do, the rvm module is loaded first, which causes the puppetlabs-apache module to fail.

So can anyone tell me how I can get around this problem? I'd very much like to modify the puppetlabs-apache module, but since they use an array to define the required mod packages, it makes it a little tricky to check if each one is defined before using it, since as far as I know it's not very straight forward to iterate over an array in the puppet DSL.  Thanks for any suggestions!

https://forge.puppetlabs.com/puppetlabs/stdlib -- check out ensure_packages().

Cheers,

Paul

[Adam Cohen]

Thanks for the response Paul, I took a look at ensure_packages and it seems useful, but it of course has the same drawback as using defined, in that it'll only work if all modules use it.

I've reduced my problem to the following. I created two modules, module1 and module2 as follows:

modules/module1/manifests/init.pp:

define module1::before { package { 'mod_ssl': ensure => present } }

modules/module2/manifests/init.pp:

define module2::after { if ! defined(Package['mod_ssl']) { package { 'mod_ssl': ensure => present } } }

and in my main manifest, I have the following which works:

module1::before {'test': }
module2::after {'test': }

but if I change around the order, I receive an error:

module2::after {'test': }
module1::before {'test': }

Error: Duplicate declaration: Package[mod_ssl] is already declared in file /tmp/vagrant-puppet/modules-0/mod2/manifests/init.pp at line 7; cannot redeclare on node vagrant

of course this happens because module 2 performs a check to only require the mod_ssl package if it isn't defined, but module 1 has no such check, so if module 1 is included after module 2, it ends up duplicating the mod_ssl resource.

I've tried resolving this by using the require directive, but it doesn't seem to work:

module2::after {'test': require => Module1::Before['test'] }
module1::before {'test': }

Can anyone tell me how I can fix this, and have the code in module1 included before module2, regardless of the order they appear in the file?  I thought that's what the require directive would enforce, but I must be using it incorrectly, since it doesn't seem to work for me in this instance.  if I can solve this small issue, I can fix my larger problem without needing to modify the code in either module, which is what I'd prefer.  Thanks for any help

Adam

[jcbollinger]

On Monday, July 22, 2013 1:29:43 AM UTC-5, Adam C wrote:

Thanks for the response Paul, I took a look at ensure_packages and it seems useful, but it of course has the same drawback as using defined, in that it'll only work if all modules use it.

I've reduced my problem to the following. I created two modules, module1 and module2 as follows:

modules/module1/manifests/init.pp:

define module1::before { package { 'mod_ssl': ensure => present } }

modules/module2/manifests/init.pp:

define module2::after { if ! defined(Package['mod_ssl']) { package { 'mod_ssl': ensure => present } } }

and in my main manifest, I have the following which works:

module1::before {'test': }
module2::after {'test': }

but if I change around the order, I receive an error:

module2::after {'test': }
module1::before {'test': }

Error: Duplicate declaration: Package[mod_ssl] is already declared in file /tmp/vagrant-puppet/modules-0/mod2/manifests/init.pp at line 7; cannot redeclare on node vagrant

of course this happens because module 2 performs a check to only require the mod_ssl package if it isn't defined, but module 1 has no such check, so if module 1 is included after module 2, it ends up duplicating the mod_ssl resource.

I've tried resolving this by using the require directive, but it doesn't seem to work:

module2::after {'test': require => Module1::Before['test'] }
module1::before {'test': }

Can anyone tell me how I can fix this, and have the code in module1 included before module2, regardless of the order they appear in the file?  I thought that's what the require directive would enforce, but I must be using it incorrectly, since it doesn't seem to work for me in this instance.  if I can solve this small issue, I can fix my larger problem without needing to modify the code in either module, which is what I'd prefer.  Thanks for any help

There is no solution to the problem as you have framed it.  Resource declarations, including for resources of defined types, are parsed in the order in which they are encountered in Puppet's linear traversal of the manifest file in which they appear.  Similarly, classes are 'include'd in the order they appear in the file, modulo a few caveats related to classes that are declared in multiple files or via an ENC.  The 'require' metaparameter and its compatriots affect the relative order in which those resources and classes are applied to the client, which is an entirely separate question.

If you want one resource module1::before to be parsed before resource module2::after when they are both declared by the same class, then the declaration of the former must precede the declaration of the latter.  Inasmuch as that is easy to accomplish, it it not the usual problem in this area.  Instead, the problem is usually about controlling the relative order in which declarations appearing in different classes are parsed, which ultimately comes down to controlling the order in which classes are parsed.

A requirement of that kind is often described as a "parse-order dependency", and it constitutes a serious problem in your manifests.  You will find a lot of discussion of parse-order issues in the archives of this group.  The best solution would be to solve the dependency, and the best way to do that is to factor the duplicate resource declaration out into a separate class, which each dependent then declares via an 'include' statement.  The class may be broader than just one resource, as seems appropriate to you.  Example:

class ssl {

  package { 'mod_ssl': ensure => present }
}

define module1::foo {
  include 'ssl'
  # other stuff ...
}

define module2::bar {
  include 'ssl'
  # other stuff ...
}

This solves the problem because classes are singletons, so declaring the same class multiple times is safe and idempotent(*).

(*) There is a caveat, however.  Binding data or metaparameters to a class via a parameterized-style class declaration (e.g. class { 'ssl': stage => 'main' }, or even class { 'httpd': }) is not idempotent.  Such a declaration of a given class must be the first declaration of that class parsed, thus making it another parse-order dependency.  In Puppet 3 you can use hiera data bindings to avoid that issue.  You can also use direct hiera calls in the class body instead of class parameters.


John

[Adam Cohen]

Thanks for the great response John, you've cleared up my confusion. I guess for now, I'll just have to live with a parse-order-dependency in my manifest, until I can get the time to fix the modules in question.  At least I know now that it can't be solved cleanly without modifying the actual modules.  Thanks again,

Adam