package conflict resolution method:

[Tom Noonan]

Hello, list:
I have two puppet modules that are unrelated to each other, but
both have (unrelated) Python scripts that parse YAML. As such, both
have a block like the following in their manifests for the PyYAML script
dependency:

package { 'PyYAML':
ensure => installed,
}

If I try and include both modules on the same server this
causes an obvious conflict as the PyYAML package is now defined in two
different package{} blocks.
Can the list please advise on what best practice is in this
case? I'd prefer not to create a whole other module just to do a class
dependency for PyYAML, but if that is best practice so be it. Please
let me know if I'm overlooking any other solutions. Thanks!

--Tom N.

[Ryan Coleman]

On Fri, Nov 1, 2013 at 2:03 PM, Tom Noonan <t...@tjnii.com> wrote:

        Can the list please advise on what best practice is in this
case?

Hi Tom, 

The puppetlabs/stdlib module includes the ensure_packages() and ensure_resource() functions. Both are intended for expressing common resources which will only be included in your catalog if they're not already in it. You can read more about it here: http://forge.puppetlabs.com/puppetlabs/stdlib

--

Ryan Coleman | Modules & Forge | ryanycoleman on twitter & #puppet IRC

 

[Tom Noonan]

That looks like a good solution, thanks!

[Jason Antman]

Tom,

I've actually been working with similar issues lately (and am in the
process of working on a virtualenv module).

I have a python module that includes classes for the common dependencies
(i.e. "require python::pyyaml") and have been pretty happy with that
pattern so far, but if you want, I believe the stdlib ensure_packages()
can also do this.

-jantman

[jcbollinger]

On Friday, November 1, 2013 4:16:29 PM UTC-5, Tom Noonan wrote:

That looks like a good solution, thanks!

If by "good" you mean "quick and dirty" then you are right.  Relying on ensure_packages() and especially ensure_resource() may get you in trouble, however, and at minimum doing so carries an ongoing maintenance burden that better alternatives avoid.  They are fundamentally the wrong solution, because the simple fact that two independent, nonexclusive classes or modules both rely on the same resource demonstrates that the resource doesn't properly belong to either one of them.  The correct solution, therefore, is to factor out such resources into separate classes / modules.

The ensure_*() functions are encumbered by at least these issues:

• they only solve the problem if they are applied to every declaration of the resources they guard
• they destabilize the state of guarded resources(*) unless you are careful to keep all the guarded resource declarations identical
• resource and class parameter overrides may fail to have the expected effect on resources that are guarded by ensure_*() functions
  

(*) The target resource state may change when you modify your manifest set, but otherwise will be stable from run to run.


John

[jcbollinger]

On Monday, November 4, 2013 9:14:02 AM UTC-6, jcbollinger wrote:

resource and class parameter overrides may fail to have the expected effect on resources that are guarded by ensure_*() functions

Sorry, I was a little too brief there.  Class parameter overrides don't work anyway, but what I meant on that side was that class parameters that are meant to influence the parameters of guarded resources may fail to have their intended effect.


John

[Matt Simmons]

Hi John, 

I'm new around here, but I'm also in the same situation as Tom, who started this thread. 

I was wondering if you could expound a little bit on the better solution that you mention. I write what I could refer to as "third grade puppet", but I'd like to get better. 

When you suggest factoring out these resources into separate classes and modules, do you mean that, in essence, all possibly-shared resources should be in discrete modules or classes? How does that jive with modules included from Puppet Forge? Doesn't that mean refactoring everything you include there, as well? 

Thanks, and sorry for what I'm sure is an overly-basic question,  

--Matt Simmons 

[Jason Antman]

Matt,

I can't speak for exactly what John was implying, but I've been working on something similar recently, and facing more or less the same problems that you're implying below, so I'll give you my take on it.

1) Yes, it means factoring "all" possibly-shared resources out into discrete classes. The definition of "all" is flexible depending on how fine-grained your package management needs to be, and how willing you are to possibly have some "extra" packages floating around.

2) Including modules from the Forge is another beast. At this point, at $work, we're moving in the direction of *only* writing our own modules if there isn't a suitable Forge module. So, we've ended up with a few cases where we have two conflicting modules that can't be used on the same node (usually a Forge module that is intended to replace an older in-house module as it's phased out).

The "proper" way to manage this is when you add a Forge module that replaces existing functionality, you should remove any existing declarations of those resources and replace them with a require, or something like that.

A specific example... we recently started using the puppetlabs-postgresql Forge module (https://github.com/puppetlabs/puppetlabs-postgresql/). It has a postgresql::lib::python class that installs the python-psycopg2 package (the python postgres driver), and a postgresql::lib::devel class that installs postgresql-devel. So, when we decided we were going to use the new Forge module, we grepped through all of our existing modules, and replaced any Package resources for these packages with "require postgresql::lib::python" and "require postgresql::lib::devel", respectively.

I assume I speak for many in saying that I hope that with the ever-increasing usefulness of the Forge, the community begins to converge on a single "canonical" module for a given use case or package.

-jason antman

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7f66b5b7-6e20-4835-a057-b62c1add08e7%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[jcbollinger]

On Tuesday, November 19, 2013 1:48:26 PM UTC-6, Matt Simmons wrote:

Hi John, 

I'm new around here, but I'm also in the same situation as Tom, who started this thread. 

I was wondering if you could expound a little bit on the better solution that you mention. I write what I could refer to as "third grade puppet", but I'd like to get better. 

When you suggest factoring out these resources into separate classes and modules, do you mean that, in essence, all possibly-shared resources should be in discrete modules or classes? How does that jive with modules included from Puppet Forge? Doesn't that mean refactoring everything you include there, as well? 

I do not mean that possibly-shared resources must necessarily go into their own tiny classes, though that would be one way to go about it.  I mean that any given resource must belong to exactly one module, in such a way that using the module via its public-facing classes and types cannot by itself cause multiple declaration.

The module owning any particular resource might reasonably be one obtained from the Forge, or it might be a home-grown one.  That multiple modules might attempt to own the same resource is a thorny and longstanding module compatibility issue -- two modules that both attempt to own the same resource are not compatible.  Such a pair can be rendered compatible either by making one module depend on the other for the resource, or by making both depend on some third module for it.  Needless to say, that involves modifying one or both of the modules.


John

[Thomas Noonan II]

I personally went the tiny class route, creating a module with a bunch
of little classes for various packages. I opted for this route as it
let me deal with naming differences between apt and yum, and let me do
repo dependencies for stuff in Epel.

Here is an example:
https://github.com/TJNII/puppet-commonpackages/blob/master/manifests/python/argparse.pp

I'm currently not making use of the forge, at this time I'm still
rolling my own modules. So that's a bridge I'll likely have to cross
later.

--