Wrapper classes, ordering & anchors

[Mohit Chawla]

Hello,

I have a class like:

class wrapper {
include foo
include bar
include baz
}

And a node like:

node x {
include someclass
include wrapper
Class["someclass"]->Class["wrapper"]
}

The class chaining in node x doesn't get respected. In irc I was
suggested there being a possibility of this being related to #8040.
Can anyone suggest if that indeed might be the case ? Is there a clear
process to tell if certain chaining of classes or resources would
mandate using anchors or not ? Just to be clear, there is no order
required in the classes inside the wrapper class. But just that to
ensure before any of these, the class "someclass" gets applied. Any
ideas, and possible approaches would be nice.

[llowder]

From what I can tell, this looks like the main use case for the "anchor pattern" in stdlib.

https://github.com/puppetlabs/puppetlabs-stdlib
 

[Luke Bigum]

As I found out recently (Dan Bode clarified this on Oct 4th) anchors should only be needed for nested classes, unless I've misinterpreted that. The problem above is that you are defining a relationship on your 'wrapper' class, but your wrapper class has no relationship with the classes it itself declares (feature of the 'include' syntax). I think this should work:

class wrapper {
  include foo
  include bar
  include baz

  Class[foo] -> Class[wrapper]
  Class[bar] -> Class[wrapper]
  Class[baz] -> Class[wrapper]

}

node x {
  include someclass
  include wrapper
  Class["someclass"]->Class["wrapper"]
}

Or in your wrapper class you could change all the include statements to require statements, but then you are mixing two types of creating dependencies and personally I think it's "purer" to stay with the relationship chaining operators.

[Jeff McCune]

Yes, this is a perfect example of when to employ the anchor pattern. It's also a perfect example of the bug we need to fix in puppet.

Class foo, bar, an baz will "float off" in the relationship graph because class wrapper contains no other resources to "anchor" them down.

If you add a begin an end anchor resources to class wrapper and establish relationships between them and the three contained classes, it should work as you expect

I will give a concrete example, if you wish, once I get to my desk.

-Jeff

https://github.com/puppetlabs/puppetlabs-stdlib
 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/vd9gYqJEADgJ.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Mohit Chawla]

Hello,

On Thu, Oct 11, 2012 at 8:39 PM, Jeff McCune <je...@puppetlabs.com> wrote:
> Yes, this is a perfect example of when to employ the anchor pattern. It's
> also a perfect example of the bug we need to fix in puppet.
>
> Class foo, bar, an baz will "float off" in the relationship graph because
> class wrapper contains no other resources to "anchor" them down.
>
> If you add a begin an end anchor resources to class wrapper and establish
> relationships between them and the three contained classes, it should work
> as you expect
>
> I will give a concrete example, if you wish, once I get to my desk.
>

Thank you for the response, an example for the wrapper class would
indeed be nice.

And although this might be a difficult question to give a generalized
(or rather, a slightly less technical) answer, but this behaviour of
floating off of the graph, is it easy to attribute this to particular
scenarios ? For example, as you noticed in the previous replies - Luke
suggested this for nested classes, while llowder thought this was a
perfect example, as so you ! :)

Also, @Luke, your suggestion to be explicit about the relationships like so:

class wrapper {
include foo
include bar

class["foo"]->Class["wrapper"]
class["bar"]->Class["wrapper"]

}

node x {
include someclass
include wrapper

class["someclass"]->Class["wrapper"]
}

... didn't work. "someclass" was still applied after the wrapper
class's classes.

[Jeff McCune]

Here's how I'd do it:

class wrapper {

  anchor { 'wrapper::begin': }

  anchor { 'wrapper::end':

    require => Anchor['wrapper::begin'],

  }

  # Note, foo, bar, baz still do not have relationships between one another.

  class { foo:

    require => Anchor['wrapper::begin'],

    before => Anchor['wrapper::end'],

  }

  class { bar:

    require => Anchor['wrapper::begin'],

    before => Anchor['wrapper::end'],

  }

  class { baz:

    require => Anchor['wrapper::begin'],

    before => Anchor['wrapper::end'],

  }

}

And a node like:

node x {
  include someclass
  include wrapper

[Mohit Chawla]

Thank you, that worked nicely !

[jcbollinger]

On Thursday, October 11, 2012 10:32:29 AM UTC-5, alcy wrote:

And although this might be a difficult question to give a generalized
(or rather, a slightly less technical) answer, but this behaviour of
floating off of the graph, is it easy to attribute this to particular
scenarios ?

Yes.  When one class declares another, whether via the 'include' or 'require' function or via a parametrized-style declaration, that (intentionally) does not establish any ordering relationship between the declaring and declared classes.  Without something else, such as anchors, to establish an order between them, the two classes are disconnected in the relationship graph; that has come to be described as the declared class "floating off", especially when the declaring class is connected to multiple others in the relationship graph.

Such disconnectedness is not necessarily a problem, because it may indeed be that no relative ordering of the classes involved is necessary or expected.  If, however, the purpose of the one class declaring the other is to aggregate the latter into a larger unit, then it is a indeed an issue that needs to be addressed.  That's where you need to use the anchor pattern.


John

[Mohit Chawla]

Hello John,

On Thu, Oct 11, 2012 at 11:53 PM, jcbollinger <John.Bo...@stjude.org> wrote:
> Yes. When one class declares another, whether via the 'include' or
> 'require' function or via a parametrized-style declaration, that
> (intentionally) does not establish any ordering relationship between the
> declaring and declared classes. Without something else, such as anchors, to
> establish an order between them, the two classes are disconnected in the
> relationship graph; that has come to be described as the declared class
> "floating off", especially when the declaring class is connected to multiple
> others in the relationship graph.
>
> Such disconnectedness is not necessarily a problem, because it may indeed be
> that no relative ordering of the classes involved is necessary or expected.
> If, however, the purpose of the one class declaring the other is to
> aggregate the latter into a larger unit, then it is a indeed an issue that
> needs to be addressed. That's where you need to use the anchor pattern.
>

Thanks for the explanation, much appreciated. I suppose containing
resources is a fundamentally solved problem by virtue of the design
and goal, but containing classes seems to be a little different.
Nevertheless, I always do try to not having to enforce any order, but
at times, I suppose it becomes necessary. Thanks again for taking the
time to explain the behaviour.

[R.I.Pienaar]

i find the anchor pattern both a royal pain and too strict, i dont generally
care to bind classes between two resources but instead just before some
resource or class

class wrapper {
include one, two, three

Class["one"] -> Class["two"] -> Class["three"] -> Class["wrapper"]
}

at this point 'include wrapper' will have those 3 classes and their
resources completed before anything that requires the wrapper class

this is often sufficient enough and satisfies 'just enough ordering hints'
to me.

[Mohit Chawla]

Hello,

On Fri, Oct 12, 2012 at 2:44 AM, R.I.Pienaar <r...@devco.net> wrote:
>
>
> i find the anchor pattern both a royal pain and too strict, i dont generally
> care to bind classes between two resources but instead just before some
> resource or class
>
> class wrapper {
> include one, two, three
>
> Class["one"] -> Class["two"] -> Class["three"] -> Class["wrapper"]
> }
>
> at this point 'include wrapper' will have those 3 classes and their
> resources completed before anything that requires the wrapper class
>
> this is often sufficient enough and satisfies 'just enough ordering hints'
> to me.

I suppose this is similar to what Luke suggested, that is:

class wrapper {
include foo
include bar

Class["foo"]->Class["wrapper"]
Class["bar"]->Class["wrapper"]
}

But that didn't work for me if I tried to make sure some class is
executed before the wrapper class. node x { class xyz;
class["xyz"]->Class["wrapper"] }

Except that I did not try Class["foo"]->Class["bar"]->Class["wrapper"]
in the wrapper class ( because there isn't any relationship between
foo & bar ). So, maybe I am missing something in your approach ?

[R.I.Pienaar]

----- Original Message -----
> From: "Mohit Chawla" <mohit.cha...@gmail.com>
> To: puppet...@googlegroups.com
> Sent: Thursday, October 11, 2012 10:31:28 PM
> Subject: Re: [Puppet Users] Re: Wrapper classes, ordering & anchors
>

his example should work fine

here's a more detailed example with some notifies, i can freely shuffle
my classes around in the chaining and it seems to do the right thing

http://p.devco.net/213/

[Mohit Chawla]

Weird, the only difference is I didn't define any relationship between
one, two, three, but made them all depend on wrapper individually, and
then in the node def, made the wrapper depend on some other class -
and it didn't work. Probably a bug (2.7.19 client and 2.7.11 server) ?

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.

[R.I.Pienaar]

----- Original Message -----
> From: "Mohit Chawla" <mohit.cha...@gmail.com>
> To: puppet...@googlegroups.com
> Sent: Thursday, October 11, 2012 10:49:50 PM
> Subject: Re: [Puppet Users] Re: Wrapper classes, ordering & anchors
>

> Weird, the only difference is I didn't define any relationship
> between one, two, three, but made them all depend on wrapper individually,
> and then in the node def, made the wrapper depend on some other class -
> and it didn't work. Probably a bug (2.7.19 client and 2.7.11 server)
> ?

if i change mine to one->wrapper, two->wrapper, three->wrapper it works
fine.

puppet 2.7.17

[Mohit Chawla]

Hmm, thanks, I'll try upgrading the master to the same minor version
and see if it works differently, would perhaps be surprising if it
does.

[R.I.Pienaar]

----- Original Message -----
> From: "Mohit Chawla" <mohit.cha...@gmail.com>
> To: puppet...@googlegroups.com
> Sent: Thursday, October 11, 2012 10:59:33 PM
> Subject: Re: [Puppet Users] Re: Wrapper classes, ordering & anchors
>

> Hmm, thanks, I'll try upgrading the master to the same minor version
> and see if it works differently, would perhaps be surprising if it
> does.

try my code with puppet apply like i showed? does it work?

[Mohit Chawla]

Hi, it works with code like in your paste. But check this out :
http://pastie.org/5037832, the original situation I found myself in,
and you can see the "floating off" behaviour again.

[R.I.Pienaar]

----- Original Message -----
> From: "Mohit Chawla" <mohit.cha...@gmail.com>
> To: puppet...@googlegroups.com
> Sent: Thursday, October 11, 2012 11:19:48 PM
> Subject: Re: [Puppet Users] Re: Wrapper classes, ordering & anchors
>

> Hi, it works with code like in your paste. But check this out :
> http://pastie.org/5037832, the original situation I found myself in,
> and you can see the "floating off" behaviour again.
>

yes, our method does not constrain the left of the relationship
only the right - ie. classes before the wrapper class but at any
time before the wrapper class

if you then insert another class before the wrapper you dont have
a guarantee where in the graph it gets inserted.

In this specific case you could just swap the chain around putting
the classes after the wrapper

Class["wrapper"] -> Class["three"] -> Class["two"] -> Class["one"]

or whatever.

The difference between the two is:

for the anchor pattern as per Jeff:

<before anchor> classes <after anchor>

the simpler one:

classes <wrapper as anchor>

or

<wrapper as anchor> classes

its less strict and I prefer this because if you're overdoing
the require hints you're just really turning puppet into a bash
script

[Mohit Chawla]

Thanks for illustrating the difference in the approaches which govern
the placement of a class in the catalog graph. And indeed, it was only
my unassuming intuition about chaining (and unawareness of #8040) that
led me to think of simplifying some of the the code to use class
chaining. Nevertheless will keep all the mentioned points & opinions
in mind, they've been helpful to understand this behaviour better.

[Stefan Schulte]

On Fri, Oct 12, 2012 at 03:49:48AM +0530, Mohit Chawla wrote:
> Hi, it works with code like in your paste. But check this out :
> http://pastie.org/5037832, the original situation I found myself in,
> and you can see the "floating off" behaviour again.
>

Because you are saying that Class[abc] should be done *before*
Class[wrapper] and Class[three,two,one] should also be done *before*
Class[wrapper]. That does not imply any relationship between Class[abc]
and Class[three,two,one].

The example that was mentioned earlier does only work when you specify
that Class[abc] depends on Class[wrapper] because then you define that
Class[one,two,three] should run *before* Class[wrapper] and Class[abc]
should run *after* Class[wrapper]. That does indeed imply a relationship
between Class[one,two,three] and Class[abc].

So to let your example work

class 'wrapper' {
include one,two,three

Class['wrapper']->Class['one']
Class['wrapper']->Class['two']
Class['wrapper']->Class['three']
}

class 'abc' {
}

class 'xyz' {
include abc
include wrapper
Class['abc']->Class['wrapper']
}

-Stefan

[Mohit Chawla]

Ah ! Having come across this before in my resource dependencies with
normal types (just correlating with some of the modules), this makes
it even clearer, thanks.

[Jeff McCune]

Yeah, but sometimes the need to manage both sides of the wrapper class is un-avoidable.  I often find this to be the case when I'm dealing with interpreter packages like java, ruby, perl, python, etc...  The classes that setup the packaging repositories need to happen before the class managing the ruby / java / python / go / whatever interpreter... and the classes that actually manage your application need to come after the class that manages the interpreter.

So the interpreters are what is modeled in the "wrapper" class in this discussion.

This pattern is necessary in other situations too; web servers that host applications for example.  Basically, I think to myself, "I may need to anchor down this class..." if the thing the class is modeling sits between the infrastructure layer and the application layer.

So, not quite as common as a bash script, but not so uncommon that you never need to use it.

Again, the fact that it's necessary at all is definitely a bug in Puppet.  We just haven't fixed it yet, but not for lack of people trying.  =(

-Jeff

[Nick Fagerlund]

Also, for sake of concision, I should point out that chaining arrows can accept both resource declarations and multi-resource references (https://docs.puppetlabs.com/puppet/3/reference/lang_relationships.html#operands), which gives you the option of a one-liner workaround:

class wrapper { 
  include foo 
  include bar 

  include baz 

  anchor {'wrapper_start':} -> Class['foo', 'bar', 'baz'] -> anchor {'wrapper_end':}
} 

[jcbollinger]

On Thursday, October 11, 2012 6:27:07 PM UTC-5, Jeff McCune wrote:

Yeah, but sometimes the need to manage both sides of the wrapper class is un-avoidable.  I often find this to be the case when I'm dealing with interpreter packages like java, ruby, perl, python, etc...  The classes that setup the packaging repositories need to happen before the class managing the ruby / java / python / go / whatever interpreter... and the classes that actually manage your application need to come after the class that manages the interpreter.

So the interpreters are what is modeled in the "wrapper" class in this discussion.

This pattern is necessary in other situations too; web servers that host applications for example.  Basically, I think to myself, "I may need to anchor down this class..." if the thing the class is modeling sits between the infrastructure layer and the application layer.

So, not quite as common as a bash script, but not so uncommon that you never need to use it.

Again, the fact that it's necessary at all is definitely a bug in Puppet.  We just haven't fixed it yet, but not for lack of people trying.  =(

It struck me just now that the conversation never touched on the 'include' function's neglected little brother, 'require'.  The 'require' function does everything 'include' does, plus sets up a relationship that the required class be applied before the requiring one.  That's half of the effect of the anchor pattern right there.

One could imagine a complement to the 'require' function that was the same except for setting up an oppositely-directed relationship.  For continued parallelism with resource metaparameters, one might name such a function 'before'.  I'm not convinced that such a function would have even as much use as little-used 'require', but at least there would be symmetry.

The same theme does not work for full containment of one class by another, because classes appear as single nodes in the relationship graph.  You would need separate nodes for the beginning of a class and its end to model containment directly in the graph.  I have heard talk before about implementing a true container formalism for Puppet, but inasmuch as such a thing has not yet appeared, perhaps an alternative would be a better target.

Consider, then, another potential new function, "contain".  Like 'require', it would have the all the effects of include, but it would have the additional effect of copying all relationships to or from the containing class to corresponding relationships with the contained class -- much the same way that relationships affect classes' resources.

That would not require any new DSL syntax or object type, nor any change to the graph analysis code.  I think it would support multiple containment for no additional (development) cost.  It would allow users easily to control whether classes are contained or merely associated, and I think it could be applied even when classes are declared via (evil) parametrized-class syntax.  That is, inasmuch as this works:

class { 'foo': }
include 'foo'

there's no reason why this would not work too:

class { 'foo': }
contain 'foo'

The only down side I see right now is an increase in the number of relationships in the graph, relative to many (but not all) uses of the anchor pattern.  I am inclined to think that's not too bad a problem, however, because containment is only needed in relatively few places, and because I anticipate that most containing classes don't need to contain very many other classes.  The increase would be less than two relationships per contained class in the worst case, and zero in the best case.

Any takers?


John