2 package resources for the same package installation in two differents modules caused errors

[Christophe L]

Hello,

We had the following situation where including two classes that were
ensuring the installation of the same package "php5-imagick" and it
was causing an error indicating that two ressources of the same name
are forbidden (sorry, I don't have the exact message error since we
fixed quickly the error).

Since we are newbies in puppet, we would like to understand what this
error meant:
does it mean that all the package resource ensuring the installation
of a package need to have a unique resource name ?
Or maybe have we done something wrong in our usage of puppet ?

Below, the code causing the error and how we fixed it.

Thanks in advance for your feebacks.

Best regards,
Christophe

Code causing the error:

/etc/puppet/modules/apache/manifests/init.pp

class apache::install {
...
package { [ "php5", "php5-cli", "php5-gd", "php5-imagick", "php5-
mysql", "phpmyadmin", "mysql-client" ]:
ensure => installed,
}
...
}

/etc/puppet/modules/cms/manifests/init.pp

class cms::installpackage {
...
package { 'php5-imagick':
name => 'php5-imagick',
ensure => 'installed',
}
...
}

In order to fix the error, we prefixed the resource in
cms::installpackage class with cms_

package { 'cms_php5-imagick':
name => 'php5-imagick',
ensure => 'installed',
}

[Miguel Di Ciurcio Filho]

On Tue, Apr 3, 2012 at 3:33 PM, Christophe L <cl.subs...@gmail.com> wrote:
> Hello,
>
> We had the following situation where including two classes that were
> ensuring the installation of the same package "php5-imagick" and it
> was causing an error indicating that two ressources of the same name
> are forbidden (sorry, I don't have the exact message error since we
> fixed quickly the error).
>

By the nature of the Puppet's language, resources must have a unique
title and must have only one definition.

Quoting http://docs.puppetlabs.com/guides/language_guide.html#resources:

"The field before the colon is the resource’s title, which must be
unique and can be used to refer to the resource in other parts of the
Puppet configuration."

> Code causing the error:
>
> /etc/puppet/modules/apache/manifests/init.pp
>
> class apache::install {
>  ...
>  package { [ "php5", "php5-cli", "php5-gd", "php5-imagick", "php5-
> mysql", "phpmyadmin", "mysql-client" ]:
>        ensure => installed,
>  }

The line above is just a shortcut to this something like this:

package {"php5": .... }
package {"php5-cli": .... }
package {"php5-imagick": .... }

So when Puppet compiles all your manifests, there can be only one
php5-imagick, in your case.

[Christophe L]

Hello,

Thanks for your answer.

I understand the restriction about resource name uniqueness, but I
don't understand the practical usage of it on the package resources.

Let's say there is two modules totally different, written by two
different developpers, but both depending on the same debian
package :
the two developper wills use the simplified syntax of the package
resource such package { "MODULE_NAME" : ensure => installed }, like we
can see it on usual puppet sample on internet.

Then, when an administrator will include the two modules, it will fail
because the same package is defined twice, even if the description of
the resource is exactly the same.

It doesn't sound as a normal behavior to me, unless if a best practice
is to never use the simplified syntax for package resource ?

Or is there an other best practice for avoiding this situation ?

Thanks in advance for your answers.

Best regards.
Christophe


On 4 avr, 00:15, Miguel Di Ciurcio Filho <miguel.fi...@gmail.com>
wrote:

> On Tue, Apr 3, 2012 at 3:33 PM, Christophe L <cl.subscript...@gmail.com> wrote:
> > Hello,
>
> > We had the following situation where including two classes that were
> > ensuring the installation of the same package "php5-imagick" and it
> > was causing an error indicating that two ressources of the same name
> > are forbidden (sorry, I don't have the exact message error since we
> > fixed quickly the error).
>
> By the nature of the Puppet's language, resources must have a unique
> title and must have only one definition.
>

[Andrew Stangl]

Hi,

The best way to overcome the problem you're experiencing, is to use virtual resources 

http://docs.puppetlabs.com/guides/virtual_resources.html

So, for example, you would have a class that declares the resources:

class php_v_packages {

    @package { [ "php5", "php5-cli", "php5-gd", "php5-imagick", "php5- 

        mysql", "phpmyadmin", "mysql-client" ]: 
        ensure => installed, 
    } 

}

and when you need to use those virtual packages, include that class, and realize the resources:

class apache::install { 

    include php_v_packages

    realize Package [ [ php5"], [php5-cli], [php5-gd], [php5-imagick]]

}

class cms::installpackage { 

    include php_v_packages

    realize Package [php5-imagick]

}

That should do the trick :)

Cheers,

Andrew

[jcbollinger]

On Apr 5, 6:11 am, Christophe L <cl.subscript...@gmail.com> wrote:
> Hello,
>

> Thanks for your answer.
>
> I understand the restriction about resource name uniqueness, but I
> don't understand the practical usage of it on the package resources.
>
> Let's say there is two modules totally different, written by two
> different developpers, but both depending on the same debian
> package :
> the two developper wills use the simplified syntax of the package
> resource such package { "MODULE_NAME" : ensure => installed }, like we
> can see it on usual puppet sample on internet.
>
> Then, when an administrator will include the two modules, it will fail
> because the same package is defined twice, even if the description of
> the resource is exactly the same.
>
> It doesn't sound as a normal behavior to me, unless if a best practice
> is to never use the simplified syntax for package resource ?
>
> Or is there an other best practice for avoiding this situation ?

Expanding somewhat on Andrew's response:

From a conceptual perspective, if two separate and independent modules
both require the same package, then clearly it is incorrect to model
that package as belonging to or being the responsibility of either
module. It is a more fundamental resource than either module
represents, else you would not have such a conflict in the first
place.

The correct solution, both conceptually and practically, is to manage
the package via a separate module that both of the others rely upon.
In your particular case, it looks like it would be very natural to
create and use a "php" or "php5" module to manage all the PHP
packages. Your "apache" and "cms" modules would then both depend on
the new "php" module to ensure that the needed packages were
installed.

At the manifest level, suppose you put all the PHP package
declarations in class "php::packages". Then both "apache::install"
and "cms::installpackage" would put this line at the beginning of
their bodies:

include "php::packages"

IMPORTANT: Puppet's 'include' function is *not* analogous to the C
preprocessor's #include directive: it does not interpolate anything.
It's much closer to Python's 'import' statement, which makes Python
modules visible to other Python modules.

If you don't necessarily need all of the PHP packages for every PHP
user, as it appears may be the case, then you can additionally declare
the PHP packages virtually in "php::packages". Then, after
'include'ing php::packages, your other modules would also 'realize'
the packages they need.


John

[Bruno Harbulot]

Hello,

On Thursday, April 5, 2012 1:51:45 PM UTC+1, jcbollinger wrote:

Expanding somewhat on Andrew's response:

From a conceptual perspective, if two separate and independent modules
both require the same package, then clearly it is incorrect to model
that package as belonging to or being the responsibility of either
module.  It is a more fundamental resource than either module
represents, else you would not have such a conflict in the first
place.

The correct solution, both conceptually and practically, is to manage
the package via a separate module that both of the others rely upon.
In your particular case, it looks like it would be very natural to
create and use a "php" or "php5" module to manage all the PHP
packages.  Your "apache" and "cms" modules would then both depend on
the new "php" module to ensure that the needed packages were
installed.

(I'm trying to understand how Puppet works and facing a similar problem.)

I'm wondering what this implies in terms of module re-usability. I'd like to be able to provide relatively generic modules for certain applications (including my own, as a developer).

Let's say I write a PHP5 application (for example) and distribute it via a Puppet module. Since it depends on PHP, it seems to make sense to "require => Package["php5"]" at some point in my module. The problem is that it only seems to work if the module (or the set of modules available to the particular Puppet environment) declares "package { "php5": ensure => installed }" somewhere.
As the author of that individual module, which may or may not be used alongside other modules over which I don't have much control, I'm not sure where this package will be declared (whereas I would assume that the "require" directive would get the right one using apt-get or yum if available).

If another developer has written another module, completely independently, that also depends on PHP and that is also installed in this Puppet setup, how can it prevent conflicts or declaration?

Maybe I'm not thinking about it the right way, but say:
  - module app_a relies on package xyz;
  - module app_b relies on package xyz;
  - both modules are completely independently written, but installed in the modulepath;
  - node n1 installs app_a;
  - node n2 installs app_a and app_b;
  - node n3 installs app_b;
  - another different Puppet user wants to use app_a and other independent modules.

Is there any way for the authors of app_a and app_b to declare in their respective modules that they'd want package xyz to be deployed (in the apt-get/yum sense at least) without assuming that there is a separate module upon which both app_a and app_b should rely (since the full context of usage for those modules may not be known: another Puppet user could use it completely differently)?

More generally, as a software developer and potential puppet module publisher, what the most generic way of making sure you're declaring the dependencies this piece of software will need, without creating potential conflicts with other modules that your users may have obtained?

Best wishes,

Bruno.

[Vaidas Jablonskis]

What I normally do is I create a virtual @package resource which installs php5 for example within say "apache" class and then I realize the virtual resource within the same "apache" class.

Please, if you intend to write generic and reusable modules so other people can use, STOP putting hardcoded dependencies in your modules.

Either write in documentation that it depends on an abstract resource, for instance a mysql database and make it configurable so you can pass db_name, db_host, etc to your "apache" class. Or create virtual resources and realize them within the same class - works great for packages and maybe other resource types.

[jcbollinger]

On Tuesday, November 6, 2012 12:31:16 PM UTC-6, Vaidas Jablonskis wrote:

What I normally do is I create a virtual @package resource which installs php5 for example within say "apache" class and then I realize the virtual resource within the same "apache" class.

Please, if you intend to write generic and reusable modules so other people can use, STOP putting hardcoded dependencies in your modules.

Either write in documentation that it depends on an abstract resource, for instance a mysql database and make it configurable so you can pass db_name, db_host, etc to your "apache" class. Or create virtual resources and realize them within the same class - works great for packages and maybe other resource types.

If only it were that easy.  Cross-module dependencies are a particularly tricky issue for all past and present Puppet versions.  We have had some interesting discussions on that topic here, with some possibly-useful ideas arising from them, but none of that has made it into the codebase yet.

If you write all your own modules then you can probably keep the problem under control, but if you rely on others' modules then you simply have to be prepared to make local modifications to address compatibility problems.  Good module documentation is helpful for that, but there is no documentation or coding practice that can solve the problem under the current DSL.


John