Reconciling resource defaults, dynamic scope, and the singleton nature of classes

[Jeff McCune]

Hello,

I've been thinking about how best to handle the situation where a
single class is included multiple times from multiple classes who
themselves set resource defaults. This situation poses an interesting
problem in Puppet because of dynamic scoping rather than lexical
scoping and appears to come up frequently as I travel and work with
the puppet community. It's also come up quite a bit in my own work
and development of modules for the forge.

I'm looking for feedback from the community as to how best to handle
the unique situation in a maintainable and elegant manner.

This is pretty long, but it's a rather complex problem and fairly
common in my experience, so thanks in advance for taking the time to
read through if you do.

I believe an use case will illustrate the point.

Consider a module, perhaps publicly available in the Forge, which is
extremely common and many other modules require. A good example is
the configuration of the packaging system for the node to use a local
repository.

# /etc/puppet/modules/localpackagerepository/manifests/init.pp
class localpackagerepository {
$module = "localpackagerepository"
file { "/etc/yum/yum.repos.d/local.repo": source =>
"puppet:///modules/${module}/local.repo"; }
}

Other classes will likely establish a relationship in the graph to
this localpackagerepository class to ensure custom packages are
available before trying to manage their state. For example:

# /etc/puppet/modules/apache/manifests/init.pp
class apache {
$module = "apache"
include localpackagerepository
File {
owner => "apache",
group => "apache",
require => Package["apache"],
notify => Service["apache"],
}
###
package { "apache":
ensure => installed,
require => Class["localpackagerepository"];
}
service { "apache": ensure => running }
file { "/etc/httpd/httpd.conf": source =>
"puppet:///modules/${module}/apache.conf"; }
}

# /etc/puppet/modules/postfix/manifests/init.pp
class postfix {
$module = "postfix"
include localpackagerepository
File {
owner => "mail",
group => "mail",
require => Package["postfix"],
notify => Service["postfix"],
}
package { "postfix":
ensure => installed,
require => Class["localpackagerepository"];
}
service { "postfix": ensure => running; }
file { "/etc/postfix/main.cf": source =>
"puppet:///modules/${module}/main.cf"; }
}

Now, with these three modules we may include them in the node
classification like so:
include apache
include postfix

However, this will behave very differently than the classes included
in a different order:
include postfix
include apache

In the first ordering the localpackagerepository class will take on
the resource defaults of the apache class, while in the second
ordering the localpackagerepository class will take on the resource
defaults of the postfix class.

In both cases there will be a circular dependency (local.repo will
require a package, and the package will require local.repo), which is
clearly an issue. In either case, local.repo will take on defaults
for owner and group which are invalid.

How are you handling this situation? How do you think this situation
*should* be handled by puppet?

I can think of a number of work arounds, but they're not ideal and
they don't feel to me like they'll play well with public modules
posted to the forge.

I make the recommendation that classes should include the classes they
depend on, but this appears to be problematic if the class doing the
include sets resource defaults, particularity when setting a
relationship in a default like require, before, subscribe and notify.
This opens up the door for resource cycles.

Perhaps it might be useful to set resource defaults only for the local
scope, and not for any classes which get included into this scope.
How do you feel about this change to the language?

Is there some other organization of the class structure I'm overlooking?

Thanks for your feedback,
--
Jeff McCune
http://www.puppetlabs.com/

[Nigel Kersten]

On Thu, Jul 29, 2010 at 10:48 AM, Jeff McCune <je...@puppetlabs.com> wrote:

> Perhaps it might be useful to set resource defaults only for the local
> scope, and not for any classes which get included into this scope.
> How do you feel about this change to the language?

So this would mean that you'd only be able to set truly global
resource defaults with an import in site.pp? and no longer inside a
base class that includes all your other classes?

We make use of the current behavior of defaults being applied to
included classes quite a lot, and I find it useful just to provide a
counterpoint to RIP :)

>
> Is there some other organization of the class structure I'm overlooking?
>
> Thanks for your feedback,
> --
> Jeff McCune
> http://www.puppetlabs.com/
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>
>

--
nigel

[Jeff McCune]

On Thu, Jul 29, 2010 at 12:37 PM, Nigel Kersten <nig...@google.com> wrote:
> On Thu, Jul 29, 2010 at 10:48 AM, Jeff McCune <je...@puppetlabs.com> wrote:
>
>> Perhaps it might be useful to set resource defaults only for the local
>> scope, and not for any classes which get included into this scope.
>> How do you feel about this change to the language?
>
> So this would mean that you'd only be able to set truly global
> resource defaults with an import in site.pp? and no longer inside a
> base class that includes all your other classes?
>
> We make use of the current behavior of defaults being applied to
> included classes quite a lot, and I find it useful just to provide a
> counterpoint to RIP :)

How do you "know" what the scope is of those included classes will end
up as? Are you not including the "edge" classes multiple times?

[Nigel Kersten]

Nope. That would be insane due to what sparked this thread off :)

--
nigel

[Jeff McCune]

On Thu, Jul 29, 2010 at 12:58 PM, Nigel Kersten <nig...@google.com> wrote:

> How do you "know" what the scope is of those included classes will end
> up as?  Are you not including the "edge" classes multiple times?

Nope. That would be insane due to what sparked this thread off :)

I think this is something that is really becoming an issue with the increasing number of modules published to the Forge.  Modules need some way to satisfy their dependencies on other modules, and the current behavior of include just doesn't seem to cut it.

What about include ::localyumrepository ?

It's valid syntax in 0.24.9, 0.25.5, and 2.6.0

And...  Believe it or not, it actually already does almost exactly what I want it to do in 2.6, except I think I found a bug in 2.6.

Here's my test manifest:

#

class leaf { notify { "leaf": } }

class baseone {

  Notify { message => "baseone" }

  notify { "baseone": }

  include ::leaf

}

class basetwo {

  Notify { message => "basetwo" }

  notify { "basetwo": }

  include ::leaf

}

include baseone, basetwo

# include basetwo, baseone

Here's the result in 0.25.5 and 0.24.9:

notice: baseone

notice: //baseone/Notify[baseone]/message: defined 'message' as 'baseone'

notice: baseone

notice: //baseone/leaf/Notify[leaf]/message: defined 'message' as 'baseone'

notice: basetwo

notice: //basetwo/Notify[basetwo]/message: defined 'message' as 'basetwo'

Notice how Notify[leaf]'s scope is within baseone, which was the first class to include it.

Now, for the pleasant surprise: (in 2.6.0)

notice: basetwo

notice: /Stage[main]/Basetwo/Notify[basetwo]/message: defined 'message' as 'basetwo'

notice: baseone

notice: /Stage[main]/Leaf/Notify[leaf]/message: defined 'message' as 'baseone'

notice: baseone

notice: /Stage[main]/Baseone/Notify[baseone]/message: defined 'message' as 'baseone'

It *appears* Notify[leaf] is not within the scope of Baseone or Basetwo, however it's still getting the default value of the message parameter from the baseone class...  Flipping in the order around to include basetwo, baseone we get:

notice: /Stage[main]/Leaf/Notify[leaf]/message: defined 'message' as 'basetwo'

Again, it looks like the class is included into top scope, but the behavior is that we're still getting resource defaults from basetwo's scope.

Thoughts?  Am I insane for wanting include ::leaf to place Leaf at top scope?

[R.I.Pienaar]

I think so :) ::topclass isnt about scope its about resolution? I never considered it would have scope implications.

Only time you tend to see it is in:

class monitor { }
class foo::monitor { }
class foo::bar { include ::monitor }

to include class monitor and not foo::monitor, you wouldnt want that to do weird scope things

[Jeff McCune]

On Thu, Jul 29, 2010 at 2:00 PM, Jeff McCune <je...@puppetlabs.com> wrote:

And...  Believe it or not, it actually already does almost exactly what I want it to do in 2.6, except I think I found a bug in 2.6.

Hrm, it appears to be just a tweak to the display...

include leaf and include ::leaf both result in the same display of:

notice: /Stage[main]/Leaf/Notify[leaf]/message: defined 'message' as 'baseone'

So it's still nesting the scope of leaf, however it's just no longer clearly displaying it.

Rats.

[Jeff McCune]

On Thu, Jul 29, 2010 at 2:04 PM, R.I.Pienaar <r...@devco.net> wrote:
>
> I think so :) ::topclass isnt about scope its about resolution? I never considered it would have scope implications.

Right, forgot about that case.

So what about another keyword in addition to require and include
(feels like we're running out...) that adds a class to the catalog at
top scope? Something like "addclass" perhaps.

Modules would then avoid using "include mydependency" and prefer
"addclass mydependency"