Catalog compilation improper ordering and anchoring

[Jesi Major]

We're having an issue where the catalog doesn't appear to be compiling in the proper order, causing a resource to be defined twice (since the if ! defined block runs first) and the catalog compilation to fail:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: File[/data] is already declared in file /etc/puppet/environments/jesim/modules/cassandra/manifests/config.pp:58; cannot redeclare at /etc/puppet/environments/jesim/modules/lvm/manifests/mount.pp:60

We are aware of the behavior of resources creeping out of included classes, so have anchored all classes involved in an attempt to resolve this issue. Although doing this initially seemed to be ordering these resources properly, another commit seems to have changed the order of catalog compilation, causing the above error.

This is the code snippet from the cassandra::config class, which is included and anchored properly in the cassandra class:

if ! defined( File['/data'] ) {
file { '/data':
ensure => 'directory',
before => File['cassandra-data-dir2'],
}
}

And from lvm::mount defined type, which is defined in base::storage, which is included and anchored in base, which is required by cassandra. $mount_point is being passed as '/data'

file { $mount_point:
ensure => 'directory',
}

As far as I can tell, lvm should be running first, then cassandra, so the if ! defined should work and it shouldn't be getting declared twice. And up until the last commit I did, this is exactly how it was working. (We were getting another error, as it appears the defined function doesn't look at variables properly, so was still declaring it twice, but that's another issue, and at least it was compiling in the appropriate order.)

We were under the impression that proper anchoring would make the catalog compile in the appropriate order, but I do not believe it is working. Is this a known issue? Is there any way to work around this? Should this definitely work and really our anchors might be fubared? I've looked through them pretty closely, but there are defined types at play within base::storage, and I'm not sure if those need anchoring as well.

[Chris Spence]

That's parse order dependent during compile (i.e. the order stuff gets
included, parsed and magicked during compile etc) - anchors aren't
really relevant here. So your commit has changed the include order I
would expect.

Best either make them virtual and realize them, or wrap them both in
if ! defined

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-dev+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/a4017005-a002-4235-8a04-50b88eac21e3%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[John Bollinger]

On Friday, November 1, 2013 11:52:36 AM UTC-5, Jesi Major wrote:

We're having an issue where the catalog doesn't appear to be compiling in the proper order

It took me a minute to parse Chris's response, so in case it's taking you longer, it boils down to this: there is no such thing as "proper" compilation order.  Puppet makes no particular guarantees about the order in which manifest files will be processed during catalog compilation, other than that the site manifest is processed first.

 

, causing a resource to be defined twice (since the if ! defined block runs first) and the catalog compilation to fail:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: File[/data] is already declared in file /etc/puppet/environments/jesim/modules/cassandra/manifests/config.pp:58; cannot redeclare at /etc/puppet/environments/jesim/modules/lvm/manifests/mount.pp:60

We are aware of the behavior of resources creeping out of included classes,

Resources do not "creep out" of included classes, but included classes themselves are not inherently constrained relative to the class declaring them.  In any case, that is strictly an order of application issue; it has nothing to do with compilation order.

> We were under the impression that proper anchoring would make the catalog compile in the appropriate order, but I do not believe it is working. Is this a known issue?

It is known behavior, but it is not an "issue".  Like everything else involving resource relationships, anchors constrain order of resource application, not order of catalog compilation.

This topic is not well suited to puppet-dev, which is about development of the Puppet program.  Issues related to writing manifests for Puppet would be on-topic next door in puppet-users.  If you ask there then I'm sure you will get some help.  Probably even from me.


John

[Jesi Major]

Thanks very much for the explanation. This is what I was starting to suspect, but was hoping someone would tell me I was wrong.

Could you possibly elaborate on what you mean by "make them virtual and realize them"? I considered wrapping them both in if ! defined, but the /data directory really has to be created by the lvm class (i.e. before the Cassandra class) or it won't be mounted properly.

[Jesi Major]

John, apologies for placing this in the wrong group. Would you like me to repost there, or is it okay if Chris answers my question here?

You explained more eloquently what I was trying to; wasn't quite sure how to explain the functionality with classes not inherently constrained relative to the declaring class.

[John Bollinger]

On Friday, November 1, 2013 1:58:59 PM UTC-5, Jesi Major wrote:

John, apologies for placing this in the wrong group. Would you like me to repost there, or is it okay if Chris answers my question here?

I don't mean to try to police this group.  I'm just saying that it may be to your benefit to ask in puppet-users.  Personally, I do not intend to engage in further discussion of this topic here in puppet-dev.

John

[Jesi Major]

No problem! And sorry again, think I have answered my own question to Chris anyway :) http://docs.puppetlabs.com/guides/virtual_resources.html