Duplicate definition + parameterized classes + class scope

[chris_...@sra.com]

I apologize if this horse has already been beaten to death, but I'm
new here and very, very confused. I'm just starting to work with
Puppet and I can not make heads or tails of the language: specifically
how to use parameterized classes. I've spent a week reading the docs
and testing manifests and I can't make any progress. I have a feeling
that my confusion comes from the fact I have a programming background
and that my understanding of certain terms (i.e. 'class' and 'scope')
don't mean the same thing for Puppet as they do everywhere else.
(And I thought I understood the concept of 'declarative language', but
maybe not.)

Here's an example of what I feel should work:

class bar ($x='default') {
notify { "x=${x}": }
}

class foo {
notify { 'Inside class foo': }
class { 'bar' : x => 'inside foo', }
}

class baz {
notify { 'Inside class baz': }
class { 'bar' : x => 'inside baz', }
}

class { 'foo' : }
class { 'baz' : }

However, when I run this I get the following error:

Duplicate definition: Class[Bar] is already defined in file
test5.pp at line 10; cannot redefine at test5.pp:15

As I understand it, each class definition has it's own scope. So why
can't I declare the same parameterized class from two different
classes, especially when the parameters are different? If you can't
do this then what's the point of having them?

My understanding of the docs and how the scoping rules are moving
towards 2.8, seems to imply that 'include' is bad and 'parameterized
classes' are good. I'm cool with that, in fact I prefer that - it
matches more of style of coding for other languages.

Can somebody please explain what is going on?

thx
Chris.

[Denmat]

Hi, 

Here's what the docs say:

"Okay, we can pass parameters into classes now and change their behavior. Great! But classes are still always singletons; you can’t declare more than one copy and get two different sets of behavior simultaneously. And you’ll eventually want to do that! What if you had a collection of resources that created a vhost definition for a web server, or cloned a Git repository, or managed a user account complete with group, SSH key, home directory contents, sudoers entry, and .bashrc/.vimrc/etc. files? What if you wanted more than one Git repo, user account, or vhost on a single machine?

Well, you’d whip up a defined resource type."

So have a look at changing the bar class to a define instead.

Cheers,

Den

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[chris_...@sra.com]

I don't understand Puppet Language.  How can you take object-oriented constructs such as 'class' and 'inheritance' and then not allow things like multiple instances of a class, albeit with differing parameters.  Defined resource types don't help me as they don't have inheritance (which is something I very much want).

[Pablo Fernandez]

It must be some kind of "sales department" decision, you name things to be attractive, not because they represent reality.

> > type<http://docs.puppetlabs.com/learning/definedtypes.html> ."

[Bruce Richardson]

On Tue, Mar 06, 2012 at 05:53:57AM -0800, chris_...@sra.com wrote:
> I don't understand Puppet Language. How can you take object-oriented
> constructs such as 'class' and 'inheritance' and then not allow things like
> multiple instances of a class, albeit with differing parameters. Defined
> resource types don't help me as they don't have inheritance (which is
> something I very much want).

It would probably have been less troublesome, in the long run, if
Puppet classes had been called something else. Too late now.

One thing you need to get used to is that Puppet's DSL is a declarative
language, not an imperative one. Some of the things you want, there,
aren't actually appropriate.

The Puppet DSL works well enough when you use it to create the simplest
complete description of what you have. Try and use it like Java or Ruby
and it'll fight you all the way.

--
Bruce

The ice-caps are melting, tra-la-la-la. All the world is drowning,
tra-la-la-la-la. -- Tiny Tim.

[chris_...@sra.com]

Crap.  I'm trying to dump Bcfg2 and move to something reasonable.  But so far, all my initial assumptions and patterns for Puppet fail.  I think in terms of heirarchy and inheritence for my systems (all nodes install a core set of packages, some have exceptions for those core set of packages, etc) and as best as I can understand it Puppet's DSL really wants me to create a set of flat, non-hierarchial, non-inheritable set of nodes/classes. And for me that's completely un-managable. 

I'm reviewing the Puppet-user archives now and I'm seeing a lot of people with similar problems but no good patterns for solutions.

I want to be able do something like this (hierarchial, inheritance with overloading):

class base {
   package { 'sshd' : ensure => present }
   package {'ntp:      ensure => present }
}

node a,b,c {
   class { 'base' : }
}

node d {
   class { 'base' : }
   Package{'sshd': ensure => false }  
}

What I'm afraid I have to do is this (flat, redefine lots of nodes and duplicate data):

class base
   package {'ntp:      ensure => present }
   # More common packages defined
}

node a,b,c {
   class { 'base' : }
   package { 'sshd' : ensure => present }
}

node d {
   class { 'base' : }
   package { 'sshd' : ensure => false}
}

or worse (sometype of parameter passing in the worst, un-managable way):

class base ( # list ever possible ensure parameter, etc ) {
   package { 'sshd' : ensure => $ssh_ensure }
   package {'ntp:      ensure => $ntp_ensure }
   # More common packages defined
}

node a,b,c {
   class { 'base' : }}
}

node d {
   class { 'base' : ssh_ensure => false}
}


I'm  open to any and all suggestions.

thx
Chris.

[Craig White]

----
I use theforeman which has an ENC that allows nested classes (called hostgroups in Foreman) so I have a 'base' class and many 'groups' which are pre-defined collections of what you call the flat modules. Nesting is definitely permitted and useful in Foreman.

Craig

[jcbollinger]

On Mar 5, 2:23 pm, "chris_sny...@sra.com" <chris_sny...@sra.com>
wrote:

> I apologize if this horse has already been beaten to death, but I'm
> new here and very, very confused. I'm just starting to work with
> Puppet and I can not make heads or tails of the language: specifically
> how to use parameterized classes.

My usual advice is to avoid using parameterized classes. There is a
host of problems associated with their current design. My
understanding is that there will be substantial improvements on that
front in 'Telly', the next major release, but for now, just don't do
it.

Instead of using class parameters, either make your classes smarter,
or use an exteral data store via hiera or extlookup.

> I've spent a week reading the docs
> and testing manifests and I can't make any progress.  I have a feeling
> that my confusion comes from the fact I have a programming background
> and that my understanding of certain terms (i.e. 'class' and 'scope')
> don't mean the same thing for Puppet as they do everywhere else.

You are probably right.

The usage of the term 'class' in Puppet is not derived from its usage
in many (but not all) object-oriented languages. Instead, both are
independently derived from from the more general-purpose meaning of
the word (a synonym of "kind"). I suspect Puppet's choice was
intended to evoke the concept of classification (of nodes).

Other documentation and language-design choices make it worse by
muddying the waters. In particular, I think it was a poor idea to
name Puppet's facility for declaring variant classifications
"inheritance". Puppet class inheritance is quite different from OO
class inheritance, and you just have to get over the hurdle that the
word means something different here. Fortunately, class inheritance
is rarely needed or appropriate in Puppet, so you can probably just
pretend it doesn't exist. In fact, I recommend that you do.

The word "scope" is probably not the same kind of problem for you, but
you may be running into one of these scope-related gotchas:

1) Most blocks in Puppet DSL do not establish their own scopes. If
you're used to curly-brace programming languages then this takes some
getting used to, but it is useful and appropriate for Puppet. Only (I
think) these Puppet DSL constructs establish their own scopes: node
declaration bodies, class bodies, and definition bodies.

2) Through version 2.7.x, Puppet uses dynamic scoping. This is not a
Puppet innovation, but it *has* proved to be a sore thumb. You will
make your life better if you write DSL code as if there were only two
scopes: local scope and global scope. Use a fully-qualified name to
refer to any variable not belonging to the local lexical scope.
Starting in Telly, that will be required.

3) That brings us to to the point that all declared classes,
resources, and variables have global visibility. Thus, Puppet scoping
establishes name spaces and affects name resolution, but does not
limit variable accessibility.

That last point might seem problematic until you grasp that Puppet
classes cannot be multiply declared. The Puppet docs continue their
flirtation with OO terminology by describing this as all classes being
Singletons. There is therefore never any issue of choosing the right
"instance" of a class from which to read variable values. Instances
of defined types could present a problem in this regard, except that
there is no syntax in the DSL for addressing their variables.

> (And I thought I understood the concept of 'declarative language', but
> maybe not.)

There are two main points there:

1) Once anything is defined, it cannot be redefined (even to the same
value) in the same manifest set.

2) Puppet DSL is not executable. This is a rather fine distinction,
because the DSL interacts with the execution of the Puppet manifest
compiler, and the compiled catalogs strongly influence the execution
of the Puppet agent. Nevertheless, to become a Puppet master you must
jettison the idea of classes and resources "running".

> Here's an example of what I feel should work:
>
> class bar ($x='default') {
>     notify { "x=${x}": }
>
> }
>
> class foo {
>     notify { 'Inside class foo': }
>     class { 'bar' : x => 'inside foo', }
>
> }
>
> class baz {
>     notify { 'Inside class baz': }
>     class { 'bar' : x => 'inside baz', }
>
> }
>
> class { 'foo' : }
> class { 'baz' : }
>
> However, when I run this I get the following error:
>
>    Duplicate definition: Class[Bar] is already defined in file
> test5.pp at line 10; cannot redefine at test5.pp:15
>
> As I understand it, each class definition has it's own scope.  So why
> can't I declare the same parameterized class from two different
> classes, especially when the parameters are different?

Because all declared classes are global. Declaring a class in Puppet
is not analogous to instantiating one in, say, C++. Instead,
declaring a class simply says "the node (is a member of / has) class
<foo>". (Contrast this with declaring an instance of a defined type.)

It is perfectly fine and consistent for different classes A and B to
both declare that the node is a member of a third, non-parameterized
class (i.e. Puppet allows it). On the other hand, it is wholly
inconsistent for A to say "the node is a member of class C, and class
C's parameter is 'foo'" while B says "the node is a member of class C,
and class C's parameter is 'bar'". If ever both A and B are declared
for some node, then they contradict each other about what class C's
parameter is.

A better question would be why classes A and B cannot both declare
class C with the *same* parameters. That would at least be
consistent, but it is a shortcoming of Puppet's parameterized class
implementation that you cannot do it.

> If you can't
> do this then what's the point of having them?

Class parameters were implemented to solve some specific problems
related to class data, and especially to work around problems
associated with relying on dynamically-scoped variables to povide data
to classes.

For example, each node has at most one set of DNS servers, but
different nodes supported by the same Puppet manifests may need to
have different sets. You could write all the data and selection
criteria into the class managing /etc/resolv.conf, but your class is
more easily maintained and reused if instead it pulls the data from
some source outside itself. Class parameters are one of the places
from which classes can pull such data. There are others -- most
notably, external data stores such as you can access via Hiera.

> My understanding of the docs and how the scoping rules are moving
> towards 2.8, seems to imply that 'include' is bad and 'parameterized
> classes' are good.  I'm cool with that, in fact I prefer that - it
> matches more of style of coding for other languages.

'Include' vs. parameterized classes has nothing to do with scoping. I
agree that the current docs seem to push parameterized classes over
'include', but I have it on good authority that that is not PuppetLabs
policy or intention, nor even consistent PuppetLabs internal
practice. As I understand it, a lot of improvements to parameterized
classes are planned for Telly, among them some that should greatly
reduce the impedance mismatch between 'include' and parameterized
classes.

'Include' is definitely not "bad". Quite the opposite: at least until
Telly, I strongly recommend that you use only 'include' or its sibling
'require' to assign classes to nodes. This implies that you should
avoid using parameterized classes, which I also consider excellent
advice, at least for now.

As far as matching coding styles with which you are familiar, I
suggest that you may be better off disciplining yourself to a style
that is less familiar. You said yourself that your experience with OO
programming languages sometimes inclines you in the wrong direction
with Puppet, so a style that reinforces Puppet's differentness to you
may help you learn faster and more effectively.

> Can somebody please explain what is going on?

Was that sufficient?


John

[jcbollinger]

On Mar 6, 7:53 am, "chris_sny...@sra.com" <chris_sny...@sra.com>
wrote:

> I don't understand Puppet Language.  How can you take object-oriented
> constructs such as 'class' and 'inheritance' and then not allow things like
> multiple instances of a class, albeit with differing parameters.

That would be a travesty, but it's not what Puppet has done. Instead,
Puppet has taken entirely different constructs and named them in a
manner that confuses you. You know, just to keep you on your toes.

> Defined
> resource types don't help me as they don't have inheritance (which is
> something I very much want).

No, you don't. Or at least, the object-oriented kind of inheritance
that you think you want is not available from Puppet, so the fact that
neither defined types nor classes offer it does not distinguish
between the two.

Puppet's class inheritance is designed expressly for allowing the
properties of resources declared in a superclass to be overridden by a
subclass. Even that probably excites the OO programmer in you more
than it should. In practice, overriding resource properties is never
more than a convenience. Puppet offers several alternatives, some of
them cleaner.

Supposing that you're a good OO programmer, you will be aware of the
classic question of inheritance vs. composition. In my experience,
the better choice in OO programming is more often "composition" than
many practitioners recognize, but in Puppet DSL, "composition" is
*almost always* the better choice.


John

[jcbollinger]

On Mar 6, 8:51 am, "chris_sny...@sra.com" <chris_sny...@sra.com>
wrote:

Surprisingly, you have named one of the few types of problems for
which Puppet's class inheritance offers a reasonable solution:

class ssh {
package { 'sshd': ensure => 'present' }
}

class ssh::absent inherits ssh {
Package[ 'sshd' ] { ensure => 'absent' }
}

node default {
include 'sshd'
}

node d inherits default {
include 'ssh::absent'
# no problem that class ssh is also declared
}


All the same, a more forward-looking, probably better solution would
be to rely on hierarchical data instead of hierarchical nodes and
classes. Using the 'hiera' module recently adopted into Puppet, you
could achieve the same effect via just

class ssh {
package { 'sshd': ensure => hiera('sshd_ensure') }
}

node default {
include 'sshd'
}

CAVEAT: hiera configuration and data not shown. It isn't hard to set
up and use, but it isn't automagical, either.


John

[chris_...@sra.com]

I would like to thank everyone for their feedback, especially JCBollinger.  Things are definitely different than I thought they were. I need to step back and re-evaluate how I'm approaching Puppet and figure out my next steps.

I have to say I'm very disapointed right now with the state of Puppet. It seems that the official documentation is pushing parameterized classes but at the same time there are very serious drawbacks to their usage.  Additionally, they are trying to sunset the use of dynamically scoped variables which appear (to me anyway) to be the preferred method of the community at large (based upon my research on the web and this mailing list) to completing tasks. (I've lost track of the number of references I've seen that basically say, 'ignore them and go on'.) 

Regardless, it appears to me that no matter what I do, I will probably find my self having to refactor large portions of my code when 2.8 is released; either I'll be removing lots of 'includes' or changing lots of class defs and updating usage.  This does not instill confidence in me, nor does it inspire me to dive right in.  I really feel I've approached Puppet at a very unstable point in it's existence where there is no 'right answer' and the 'recommended patterns' will probably radically change in a year or so and at this time I'm not sure it's worth the effort.

I hate to say it, but I think I need to go investigate a few other tools (a la Chef, CFEngine, etc.) before I commit any more time to Puppet. 

However, you guys get serious kuddos for being immediately available, helpful, polite and incredible knowledgble about the tool. This does go a long way in determining the best tool for the job. (It beats the hell out of Bcfg2 where it was only an IRC channel and only one or two really smart guys.)

thx
Chris.

[Justin Lloyd]

John,

I'm running into some snags of my own and your explanations have been helpful. However, I'd like to ask if you can comment a bit more on the emphasis Puppet Labs has on parameterized classes versus include. For one, I'm thinking of modules available via github. Take the puppetlabs/mcollective module, for example. It's highly parameterized and I have some "wrapper classes" like this:

    class puppet_base ( $puppet_master = false, $mcollective_client = false ) {
       
        class { 'system_base': } # basics needed by all systems

        class { 'puppet':
            master  => $puppet_master,
            require => Class['system_base'],
        }

        class { 'mcollective_server':
            mcollective_client => $mcollective_client,
        }
    }

    class mcollective_server ( $mcollective_client => false ) {
        package { 'stomp':
            ensure   => present,
            provider => 'gem',
        }

        if $mcollective_client {
            class { 'activemq':
                require => Package['stomp'],
                before  => Class['mcollective'],
            }
        }

        class { 'mcollective': # puppetlabs/mcollective module
            client => $mcollective_client,
        }
    }

This seemed to me like an appropriate use of parameterized classes and a way of keeping node definitions simple and readable, like so:

    node 'regular-host' {
        class { 'puppet_base':
            stage => 'first'
        }
    }

    node 'mco-admin-host': {
        class { 'puppet_base':
            stage => 'first',
            mcollective_client => true,
        }
    }

    node 'puppet-master' {
        class { 'puppet_base':
            stage => 'first',
            puppet_master => true,
        }
    }

I am running into some snags with ensuring curl and rubygems packages are installed prior to the puppet_base being evaluated (that's a long story), but that may be more of an issue with better use of stages.

Can you comment on the above and how it is impacted by your explanations of includes (and smarter classes) vs. parameterization?

Thanks,
Justin

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--
“We don’t need to increase our goods nearly as much as we need to scale down our wants. Not wanting something is as good as possessing it.” -- Donald Horban

[Gary Larizza]

Hey there,

You're correct that we're in a sort of 'in-between' state, but that's because we're trying to optimize the way that Puppet handles data.  Check out this blog post --> http://puppetlabs.com/blog/the-problem-with-separating-data-from-puppet-code/  where we outline the various ways that you can separate your configuration data from your Puppet code.

I agree that there's a better way, and abstracting this data-lookup AWAY from your Puppet code will do a great deal to help you out.  Hiera is the method we're advocating now ( http://puppetlabs.com/blog/first-look-installing-and-using-hiera/ ) and it will be built-in to the next release of Puppet (targeted for the String).  In this way, you can expose parameters to your modules, but use Hiera as a data lookup mechanism within the Module itself.  By defaulting all your parameters to Hiera lookups, you can then just use the include() function and let Hiera do the legwork of returning configuration data.  It gives you the benefits of exposing parameters to your modules without the downfalls that can come of parameterized classes.

Dynamic Scoping is being deprecated because it can cause issues with the way you handle data within Puppet manifests, and Hiera is an excellent way to replace this functionality in a better-structured manner.  Hiera works great in Puppet 2.6 or 2.7, and you don't need to wait until version 2.8 to try it out.

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/zuwRgX20J5EJ.

To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--

Gary Larizza

Professional Services Engineer

Puppet Labs

[Gary Larizza]

*Targeted for the SPRING, not String. Thank you Lion autocorrect...

[jcbollinger]

On Mar 6, 12:40 pm, "chris_sny...@sra.com" <chris_sny...@sra.com>
wrote:

> I have to say I'm very disapointed right now with the state of Puppet. It
> seems that the official documentation is pushing parameterized classes but
> at the same time there are very serious drawbacks to their usage.

I agree that that is a sore spot, and I can assure you that the
PuppetLabs staff is aware of it. As with any small company, however,
there is more work to be done than hands to do it, even if you factor
in community contributions. The staff is very responsive to community
comment, so perhaps you can see your way clear to cutting PuppetLabs
some slack.

> Additionally, they are trying to sunset the use of dynamically scoped
> variables which appear (to me anyway) to be the preferred method of the
> community at large (based upon my research on the web and this mailing
> list) to completing tasks. (I've lost track of the number of references
> I've seen that basically say, 'ignore them and go on'.)

Dynamic scoping is a preferred method because at one time it was the
*only* method. Even then it posed unavoidable consistency problems,
however, and these sometimes bit real users.

Many uses of dynamically scoped variables can trivially be converted
to fully-qualified variables. You should not be writing new code that
relies on dynamic scoping, and you should be wary of third-party code
that relies on it. Getting rid of dynamic scoping is a good thing.

> Regardless, it appears to me that no matter what I do, I will probably find
> my self having to refactor large portions of my code when 2.8 is released;
> either I'll be removing lots of 'includes' or changing lots of class defs
> and updating usage.  This does not instill confidence in me, nor does it
> inspire me to dive right in.  I really feel I've approached Puppet at a
> very unstable point in it's existence where there is no 'right answer' and
> the 'recommended patterns' will probably radically change in a year or so
> and at this time I'm not sure it's worth the effort.

I do not work for PL, but I am confident in assuring you that you can
write modules that will work fine now and for the foreseeable future.
I think it's also safe to assume that most third-party modules that
are worth using will be updated for Telly soon after it is released
(though I confess that my criteria for "worth using" include being
supported by the author).

Puppet remains an actively developed project. I think that's a good
thing, but it has meant approimately one new major version every year
since I have used the software. Backwards compatibility has been
pretty good, with the upcoming removal of dynamic scoping being the
only absolutely breaking change I can think of over the last few
years. Nevertheless, Puppet development is fast-moving.

> I hate to say it, but I think I need to go investigate a few other tools (a
> la Chef, CFEngine, etc.) before I commit any more time to Puppet.

By all means do consider your alternatives. I think Puppet is a great
tool, but no one tool is best for everyone.


John

[jcbollinger]

First, I think run stages are a better tool in principle than they are
in practice. There is nothing you can do with stages that you cannot
also do with ordinary resource / class relationships, and people
sometimes get into trouble with run stages because they end up with
more implied relationships than they needed or wanted. If people have
run stages working well for them then great, but I recommend
approaching them with caution. If ordinary relationships can do the
job without too much trouble, then I think that's a better bet. Above
all, do not use run stages without specific need.

Second, here are some of my design principles for classes:

1) 'include' classes wherever a class or definition has either a
logical or a functional dependency. That is,
1a) If one class (or definition) refers to variables or resources
declared in another, then the former should directly or indirectly
'include' the latter
1b) If the configuration described by one class (or definition)
depends for client-side correctness or proper function on separate
configuration described by a different class, then the former should
directly or indirectly 'include' the latter (even if (1a) does not
apply)

2) Declare all needed resource relationships, as specifically as
possible. In particular, prefer resource / resource relationships to
resource / class and class / class relationships.

3) Define and document modules' and classes' scope and external
interfaces.

4) Avoid use of dynamic scoping.


Regarding (1): as of 2.7.x, this principle is not usable with
parameterized classes. That is my own biggest reason for avoiding
them. Also, principle (1) implies that some classes will be
'include'd more than once, which is just fine. The overall result of
applying this principle is that you can declare (via 'include' or
'require') any class in any place in your manifest without worrying
about parse-order issues or broken configurations (for lack of needed
other classes). Not being able to rely on that result is a
significant weakness of any class that does not apply principle (1),
among them all classes that declare parameterized classes.

Regarding (2): this one wil be controversial, at least with regard to
cross-module relationships. Looking ahead to (3), however, I consider
those resources available for cross-module relationships to be part of
modules' external interfaces, to be referenced only if documented.
Declaring relationships as precisely as possible gives Puppet the
greatest freedom to find an acceptable order of resource application.
Ideally, one module would not have to worry about the details of
application order within a different module, but it is very hard to
keep modules so self-contained without allowing them to balloon to
cover huge swaths of configuration space.


As for the specifics of your code:

A) Classes that are certain to not be subject to my principle (1) are
the least problematic when parameterized, though that does not mean
that they should not themselves apply principle (1) in their bodies.
Perhaps your wrapper classes are in that category.

B) I urge you to consider feeding data to your classes via Hiera
instead of via class parameters. Becoming familiar with Hiera and
using it effectively will pay off. Among other things, Hiera
integration into Telly will contribute to enabling you to apply
principle (1) to parameterized classes (or so I understand).

C) Other than generally objecting to parameterized classes, I don't
see anything awful about what you presented.


John