containment questions

[Peter Bauer]

hi,

i was recently bitten by ordering issues caused by missing contain/anchor declarations in a PostgreSQL module which should have finished its job before configuring and starting the PuppetDB which is done by another module.
So here are my questions:
- is it possible to use the contain function also for defined resource types/defines? I ended up using anchors since i could not find out how to use contain here.
- am i the only one thinking that it is too error-prone and cumbersome to add a contain everytime a sub-class is instantiated in a module? Since it is not transitive it has to be added on every abstraction layer of a module, from the init.pp down to all sub-classes until there are just classes left declaring resources directly. And in case one contain is missing, that code may be executed in the wrong time.

thx,
Peter

[Jeremiah Powell]

You can also use stages to define strong barriers to order large parts of a task.  The canonical example is configuring a database before the application that needs it.

Using contain on a defined type is not a syntax error.  But you won't provide any of the parameters on the defined type when you use contain.  This includes parameter like a unique name.  So the semantic meaning is rather curious. It adds a tag to the define type with the name of the class then adds a dependency between instances of the defined type and the class.  All of the defined type instances. That won't help you, though, for two reasons.  First, depending on the version of Puppet the resources in the defined type won't get the relationship.

Look at the graph for your node.

You should see the resources in your sub-classes 'floating' off the graph.  They should have no relationship to anything else.  

Secondly, this will still have the same sub-class issue. You can see this here:

https://ask.puppet.com/question/18868/containment-with-role-and-profile-pattern/

Sub-sub-classes in Puppet are not tagged with their ancestor (parent and parent's parent, etc) tags. 

Setting containment manually adds all the tags for the current class explicitly.  It also adds the relationship in the catalog between those tags.

In the link above: role class -> profile class -> utility class -> resource.  The resource is tagged with the utility class's name but not the profile or role name.  So you could not create relationships at the meta-level of roles or profiles.

Requires is another way to do this.

https://puppet.com/docs/puppet/5.4/lang_classes.html#using-include

The function create_resources() was a great way to trigger this behavior for a long time. Resources generated by this function did not have tags from the class in which create_resources was run.  Using create_resources to get  'fake looping' means explicit resource dependencies were required. 

This is one reason to replace create_resources with using the hash (* =>) attribute and lambdas.

https://puppet.com/docs/puppet/4.9/lang_resources_advanced.html#implementing-the-createresources-function

I still order things explicitly at the resource level.   Abstractions like classes and defines are nice programmer candy.  The Puppet agent only cares about the resources in the catalog. If  I'm having to add contains everywhere that tells me my fancy class structure is not adding value.  It is just adding more work for me.

In the end it is about making a server do something correct.  Do I want a service to start after another service? I include the dependency between the service resources directly.  The result is that I have much smaller modules with obvious behavior.

[Peter Bauer]

Am Freitag, 16. Februar 2018 04:09:34 UTC+1 schrieb Jeremiah Powell:

You can also use stages to define strong barriers to order large parts of a task.  The canonical example is configuring a database before the application that needs it.

But stages are not transitive either so the stage has to be given for every subclass like with the contain function.

 

Using contain on a defined type is not a syntax error.  But you won't provide any of the parameters on the defined type when you use contain.  This includes parameter like a unique name.  So the semantic meaning is rather curious. It adds a tag to the define type with the name of the class then adds a dependency between instances of the defined type and the class.  All of the defined type instances. That won't help you, though, for two reasons.  First, depending on the version of Puppet the resources in the defined type won't get the relationship.

So the bottom-line is that for defined types the new contain function is useless and we need to stick to the anchor pattern. Sounds like an incomplete feature to me.

 

Look at the graph for your node.

You should see the resources in your sub-classes 'floating' off the graph.  They should have no relationship to anything else.  

Secondly, this will still have the same sub-class issue. You can see this here:

https://ask.puppet.com/question/18868/containment-with-role-and-profile-pattern/

thx for the link, good read. The light-weight version of the anchor pattern shown by Piennar should cover most cases and looks better than the anchors everywhere.

 

Sub-sub-classes in Puppet are not tagged with their ancestor (parent and parent's parent, etc) tags. 

Though i guess it would be possible to implement a transitive version of the contain function. This would make it easier for module writers and also for the users since it is clearly visible in the main class that everything happens in there and no resources are floating around.

[jcbollinger]

On Thursday, February 15, 2018 at 9:09:34 PM UTC-6, Jeremiah Powell wrote:

You can also use stages to define strong barriers to order large parts of a task.  The canonical example is configuring a database before the application that needs it.

Stages are the wrong tool for almost every task, and especially for containment problems.  Stages best serve cases where you can formulate universal, generic rules, such as "Package repositories must be installed and configured before anything (else) that requires managing packages," but which you cannot easily implement those in terms of specific relationships or collector-based chaining rules.

 

Using contain on a defined type is not a syntax error.

Do you mean "using contain in a defined type is not a syntax error"?  Maybe it's not, but the semantics of such usage will be troublesome at best.  But no, contain is not meant for use declaring resource instances, whether of defined or plugin types.  It is not useful for that purpose, because resources are automatically contained by whatever class or resource declares them.  The containment question arises only for classes, which, by design, are not automatically contained.

 

  But you won't provide any of the parameters on the defined type when you use contain.  This includes parameter like a unique name.

 

If (pointlessly) declaring a resource via the contain function in fact works at all, then that resource instance must get some default name, such as an empty one.  That would mean that such a contain could only be evaluated once, lest an attempt to declare a duplicate resource occur.

  So the semantic meaning is rather curious. It adds a tag to the define type with the name of the class then adds a dependency between instances of the defined type and the class.  All of the defined type instances.

I don't know what you're talking about.  Puppet does perform automatic tagging of classes and resources, including tagging each resource with the name of the class that declared it, and tags are passed along via containment, but this has nothing to do with the contain function in particular.

 

That won't help you, though, for two reasons.  First, depending on the version of Puppet the resources in the defined type won't get the relationship.

Look at the graph for your node.

You should see the resources in your sub-classes 'floating' off the graph.  They should have no relationship to anything else.

I think you're confused.  Classes may float off the containment graph, but resources can do so only if they are declared at top scope in the first place, because they are automatically contained by their declaring object -- node, class, or other resource -- as already discussed.

And at this point I'm stopping.

John

[jcbollinger]

On Wednesday, February 14, 2018 at 12:02:41 PM UTC-6, Peter Bauer wrote:

hi,

i was recently bitten by ordering issues caused by missing contain/anchor declarations in a PostgreSQL module which should have finished its job before configuring and starting the PuppetDB which is done by another module.
So here are my questions:
- is it possible to use the contain function also for defined resource types/defines? I ended up using anchors since i could not find out how to use contain here.

Resources, whether of defined or plugin type, are automatically contained by the object that declares them.  You don't need to use contain, anchors, or any other special measure to make that happen.  Containment needs to be explicitly managed only for classes.

On the other hand, it generally does not make sense for a resource instance to contain a class, because classes are singletons.  If a resource (of a defined type) contained a class, then all resources of the same type would need to have overlapping containment, which often is not possible (and whose feasibility in practice is typically unpredictable).  Therefore, although it sometimes makes sense for a defined type to include or even require a class, if you think containment is warranted then convert the defined type into a class itself.

 

- am i the only one thinking that it is too error-prone and cumbersome to add a contain everytime a sub-class is instantiated in a module? Since it is not transitive it has to be added on every abstraction layer of a module, from the init.pp down to all sub-classes until there are just classes left declaring resources directly. And in case one contain is missing, that code may be executed in the wrong time.

If you really mean "sub-class" in the class inheritance sense, then either your Puppet usage is highly suspect or your terminology is.  I suspect you just mean "class", or maybe "module class" if you want to emphasize that the class in question belongs to the same module as the one declaring it.  Do not mistake Puppet DSL for an object-oriented language along C++ lines.  The DSL is object oriented in its own fashion, but Puppet's "classes" are not very analogous to C++'s.

Anyway, it is an intentional design feature that classes are not automatically contained by the object that declares them.  This follows necessarily from the fact that classes can be declared multiple times in different scopes.  If containment were automatic then exercising that multiple-declaration freedom would rapidly create unresolvable dependency cycles.

Inasmuch as you complain about multiple layers, I'm inclined to suspect that your problems are partly attributable to your approach to module design.  A module with deeply nested containment has bad code smell to me, so consider flattening.

John