Class declaration ordering causes duplicate resource error

[Jon McKenzie]

Has anyone run into this issue before? http://projects.puppetlabs.com/issues/5046#note-17

Is there something obvious that I'm missing?

[jcbollinger]

On Friday, November 15, 2013 10:08:54 AM UTC-6, Jon McKenzie wrote:

Has anyone run into this issue before? http://projects.puppetlabs.com/issues/5046#note-17

Is there something obvious that I'm missing?

Yes.  The 'require' function is a form of class declaration, functionally equivalent to 'include' + an automatic relationship.  You therefore do have multiple declarations of class foo::baz: one in foo::bar and one in foo::bam.  As long as the parameterized-style declaration is evaluated first, all is well, but otherwise you will get a duplicate declaration error.

This general problem is one of my principal reasons for advising folks to not use parameterized-style class declarations, as I have done since parameterized classes were introduced.  In Puppet 3 you can usefully create and use parameterized classes without using parameterized-style declarations by relying on automatic parameter binding through Hiera.


John

[Jon McKenzie]

Thanks John,

Unfortunately, we're using Foreman in our shop for the ENC, so using Hiera is currently not possible (AFAIK)

For some reason, I thought the "require" statement was analogous to the "require" metaparameter. But looking at the documentation, clearly that's not the case. Still, this seems like a bug to me. If this is a duplicate declaration, shouldn't it error regardless of the ordering within the manifest?

Anyways, would writing something like this work?

class { "foo::bar":
    Class["foo:baz"] -> Class["foo::bar"]
}

[Felix Frank]

Hi,

no this cannot work, you're mixing two syntaxes.

In theory, this may work:

class { "foo::bar": }
Class["foo:baz"] -> Class["foo::bar"]

But I'm with John: class { "name": } declarations should only be used if
absolutely necessary, which it's not! Do this:

include foo::bar

Class["foo:baz"] -> Class["foo::bar"]

I think what you had in mind was the following. It might work in theory,
but the above version is preferable.

class { "foo::bar": require => Class["foo::baz"] }

Again, don't do this.

HTH,
Felix

[jcbollinger]

On Friday, November 15, 2013 5:50:16 PM UTC-6, Jon McKenzie wrote:

Thanks John,

Unfortunately, we're using Foreman in our shop for the ENC, so using Hiera is currently not possible (AFAIK)

What does one have to do with the other?

 

For some reason, I thought the "require" statement was analogous to the "require" metaparameter. But looking at the documentation, clearly that's not the case. Still, this seems like a bug to me. If this is a duplicate declaration, shouldn't it error regardless of the ordering within the manifest?

For classes, it is not duplicate declaration generally that is the problem.  It is a fundamental design feature of Puppet that the same class may be 'include'd (or 'require'd, or, now, 'contain'ed) multiple times, and more generally that it is OK to include (etc.) a class that has already been declared.

Parameterized-style declarations are a bit of a fly in the ointment, however, because they do two distinct things that are only loosely related: (1) declare a class, and (2) bind values to the class's parameters.  It is the second function that causes trouble.

If Puppet could parse the whole manifest set to determine what classes were declared first, then go back and set class parameters, then the constraint you ran into could be solved.  Unfortunately, that is not possible because class parameter values can influence which classes are declared.  There is no reliable, general way to defer class parameter assignments, so parameters are bound to a class by the first declaration evaluated, and subsequent parameter list declarations (even empty ones) cannot be honored.

tl;dr: it's not a bug, it's a design shortcoming.

 

Anyways, would writing something like this work?

class { "foo::bar":
    Class["foo:baz"] -> Class["foo::bar"]
}

Puppet should not object to that, provided that Class['foo::baz'] is declared somewhere else in the manifest set.  It should not be sensitive to evaluation order.  With that said, the constraint that foo::baz must be declared, but foo::bar does not cause it to be, constitutes a weakness that may bite you later.  I would suggest omitting the ordering relationship from foo::bar, and instead declaring the relationship in some place where you can know that both classes have been declared.  For example,

class foo::bam {
  include 'foo::bar'
  class {'foo::baz': }
  Class['foo::baz'] -> Class['foo::bar']
}

If indeed the relationship declaration is safe in foo::bar in the first place, then there must be one or more such higher-level places that would be suitable for declaring the relationship.


John

[Jon McKenzie]

Thanks for the replies and being patient with me!

Maybe I'm thinking about this incorrectly, but it seems to me that announcing a dependency ("I need x defined somewhere in order to work properly") shouldn't require a class to declare the dependency as well. It seems to me that the dependent class should not have to know anything about how a particular dependency is defined, just that it is defined.

As an analogy to RPM package dependencies, if I have a package called Django that requires a package which provides the "python" capability, the Django package shouldn't need to include it's own version of Python. It should be able to re-use any package which has the capability.

What does one have to do with the other?

You were suggesting using auto-lookups via Hiera to populate class parameters, but we're using Foreman to populate those parameters. AFAIK, there's no interoperability between Foreman's ENC and Hiera (without writing my own).

Jon

[Felix Frank]

Hi,

that's basically correct, but I'd like to ask you to get more specific
than that.

Both can be desirable:

1. Require a whole class: I don't care which resource makes sure my
apache is installed - I require the whole class to be successfully
evaluated before my dependent resource is applied (or not, in the
presence of failures within the apache class).

2. Require a certain resource: You don't care which module and/or class
declares the resource, as long as it's there.

Personally, I find (2) rather distasteful, since if used consequently,
it will leave you in a jumble of cross-module dependencies at some
point, and if any of those break, you will have one hell of a time
determining the reason.

Cheers,
Felix

[jcbollinger]

On Tuesday, November 19, 2013 9:23:21 AM UTC-6, Jon McKenzie wrote:

Thanks for the replies and being patient with me!

Maybe I'm thinking about this incorrectly, but it seems to me that announcing a dependency ("I need x defined somewhere in order to work properly") shouldn't require a class to declare the dependency as well. It seems to me that the dependent class should not have to know anything about how a particular dependency is defined, just that it is defined.

The 'include' function, and its cousins 'require' and 'contain', serve the purpose of requiring that a certain class be declared without needing to know the details of its declaration.  However, they are bona fide declarations, not mere assertions, so if the named class(es) has not yet otherwise been declared then they serve also to declare it with default parameters.

An other way to look at it is that the 'include' statement should not be read as applying to the class or definition in which it appears, but rather to the target node's catalog.  The statement "include 'foo'" means "ensure that class foo is in the target node's catalog".  The 'require' and 'contain' functions should be understood by analogy with 'include', though they add ordering requirements that do relate to the class in which they appear.

This is in part historical.  Prior to Puppet 2.6, 'include' and 'require' were the only means available to declare classes.  The parameterized-style declarations added in v2.6 were not a good fit, and they still aren't today.  Today, however, you can use parameterized classes without the parameterized-style declaration syntax.

 

As an analogy to RPM package dependencies, if I have a package called Django that requires a package which provides the "python" capability, the Django package shouldn't need to include it's own version of Python. It should be able to re-use any package which has the capability.

The 'include' function is more analogous to an RPM requirement than not.  In particular, note that
1) 'include' is NOT analogous to the C preprocessor's #include directive -- it does not perform interpolation, and
2) Puppet classes are idempotent.  Declaring the same class (successfully) multiple times has no different effect than declaring it exactly once.

 

What does one have to do with the other?

You were suggesting using auto-lookups via Hiera to populate class parameters, but we're using Foreman to populate those parameters. AFAIK, there's no interoperability between Foreman's ENC and Hiera (without writing my own).

Foreman will not populate an Hiera data file, as far as I am aware, but that does not mean you cannot use both.  Puppet performs an hiera lookup for every parameter that is not explicitly assigned a value, ENC or no ENC.  Surely Foreman does not force you to assign a value for every parameter of every class?

In any case, an ENC is not relevant to the sample code about which you inquired.  That code issues explicit declarations of the class in question.  If instead you are having difficulties related to classes declared via an ENC colliding with 'include'd or 'require'd classes then let's direct the discussion that way, but I am not certain whether such an issue occurs.

John