Re: [Puppet Users] problem with class include order

[Luke Bigum]

Hi Nikita,

On 25/09/12 10:05, Nikita Burtsev wrote:
> Hello,
>
> We have a weird problem with includes:
> err: Could not retrieve catalog from remote server: Error 400 on
> SERVER: Could not find resource 'Class[Common_software]' for
> relationship from 'Class[Default_repositories]'
>
> My wild guess would be that "common_software" gets in before
> "default_repositories" and thus the error message. If i run agent
> again it sometimes goes away and configuration gets applied, sometimes
> it does not. I even tried using stages, did not help.
>

No, this is not an ordering problem, because ordering (generally)
doesn't come into it when talking classes and relationships. It doesn't
matter in which order your classes are declared. It might be more
helpful if I reword the error to:

"The class Default_repositories has a dependency on the class
Common_software, but I can't find the class Common_software declared
anywhere"

You say below you use inheritance. If your site classes inherit from
common_software, then common_software is implicitly declared in your
manifests, so you shouldn't see this problem. Are you *sure* all your
child classes inherit properly?

When you say "it sometimes gets applied", is this for the same node or
different nodes? I would expect this problem to always be there, not be
intermittent. If it's intermittent, do you have other conditionals
around the inclusion of your site software classes?

> nodes.pp looks like this;
>
> stage { pre: before => Stage[main] }
>
> node 'basenode' {
>
> class { 'default_repositories' : stage => pre }
> <some stuff here>
> include common_software-site
> <some more stuff here>
> }
>
> We have multiple (read: many) sites with configuration which varies
> here and there. To make things a bit more sane and to reduce
> duplication we decided to have common code base we ship to each site
> which then changes from site to site using inheritance mechanism, so,
> for example, there is class called "common_software" which defines
> some resources and then there is "common_software-site" which inherits
> base class and adds some functionality.
>
> If I include child class in nodes.pp problem is there, but including
> parent class fixes the problem.
>
> Any thoughts on the matter?
>
> BR,
> Nikita
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/X2-ki-AVTZ0J.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.


--
Luke Bigum
Senior Systems Engineer

Information Systems
Ph: +44 (0) 20 3192 2520
luke....@lmax.com | http://www.lmax.com
LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


FX and CFDs are leveraged products that can result in losses exceeding
your deposit. They are not suitable for everyone so please ensure you
fully understand the risks involved. The information in this email is not
directed at residents of the United States of America or any other
jurisdiction where trading in CFDs and/or FX is restricted or prohibited
by local laws or regulations.

The information in this email and any attachment is confidential and is
intended only for the named recipient(s). The email may not be disclosed
or used by any person other than the addressee, nor may it be copied in
any way. If you are not the intended recipient please notify the sender
immediately and delete any copies of this message. Any unauthorised
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden.

LMAX operates a multilateral trading facility. Authorised and regulated
by the Financial Services Authority (firm registration number 509778) and
is registered in England and Wales (number 06505809).
Our registered address is Yellow Building, 1A Nicholas Road, London, W11
4AN.

[Nikita Burtsev]

Hi Luke, 

"sometimes" means this: 

[root@test-hpc puppet]# puppetd -vt --noop

info: Retrieving plugin

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find resource 'Class[Common_software]' for relationship from 'Class[Default_repositories]' on node hpc.tornado

warning: Not using cache on failed catalog

err: Could not retrieve catalog; skipping run

[root@test-hpc puppet]# vi manifests/nodes.pp 

[root@test-hpc puppet]# puppetd -vt --noop

info: Retrieving plugin

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find resource 'Class[Common_software]' for relationship from 'Class[Default_repositories]' on node hpc.tornado

warning: Not using cache on failed catalog

err: Could not retrieve catalog; skipping run

[root@test-hpc puppet]# puppetd -vt --noop

info: Retrieving plugin

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find resource 'Class[Common_software]' for relationship from 'Class[Default_repositories]' on node hpc.tornado

warning: Not using cache on failed catalog

err: Could not retrieve catalog; skipping run

[root@test-hpc puppet]# puppetd -vt --noop

info: Retrieving plugin

info: Caching catalog for hpc.tornado

info: Applying configuration version '1348562251'

info: create new repo ofed in file /etc/yum.repos.d/ofed.repo

notice: /Stage[pre]/Default_repositories::Configure/Yumrepo[ofed]/des 

<cut>

This is in the test lab, in this particular case agent and server are the same host. 

Every parent class has one child '-site' class, each defined in its own module directory, so for this example there is modules/common_software/manifests/init.pp and modules/common_software-site/manifests/init.pp

What do you mean by "inherit properly"? 

BR, 

Nikita 

To unsubscribe from this group, send email to puppet-users+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[jcbollinger]

On Tuesday, September 25, 2012 4:20:26 AM UTC-5, Luke Bigum wrote:

Hi Nikita,

On 25/09/12 10:05, Nikita Burtsev wrote:
> Hello,
>
> We have a weird problem with includes:
> err: Could not retrieve catalog from remote server: Error 400 on
> SERVER: Could not find resource 'Class[Common_software]' for
> relationship from 'Class[Default_repositories]'
>
> My wild guess would be that "common_software" gets in before
> "default_repositories" and thus the error message. If i run agent
> again it sometimes goes away and configuration gets applied, sometimes
> it does not. I even tried using stages, did not help.
>

No, this is not an ordering problem, because ordering (generally)
doesn't come into it when talking classes and relationships. It doesn't
matter in which order your classes are declared.

No, that's incorrect.  Resource references must not be parsed before the declarations of the underlying resources, else you get an error along the lines of the one the OP reported.

That's rather easy to address for dependencies on non-parametrized classes: because multiple declaration of such classes is safe, you can always just 'include' the needed class earlier in the same scope.  That it is not safe to do so with parametrized classes is one of my biggest complaints about them.  (Puppet 3 will make this situation somewhat better, but does not resolve it completely.)

 

It might be more
helpful if I reword the error to:

"The class Default_repositories has a dependency on the class
Common_software, but I can't find the class Common_software declared
anywhere"

Naturally, if the referenced resource is not declared anywhere then its declaration cannot be parsed before the reference, but that's a special case.

 

You say below you use inheritance. If your site classes inherit from
common_software, then common_software is implicitly declared in your
manifests, so you shouldn't see this problem.

That's correct: a subclass can always assume that its superclass will be parsed first.  If the superclass has not already been declared and cannot be autoloaded then the resulting error should identify the inheritance declaration, not some reference to the parent class in the subclass's body.

John

[jcbollinger]

On Tuesday, September 25, 2012 4:05:26 AM UTC-5, Nikita Burtsev wrote:

Hello, 

We have a weird problem with includes: 

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find resource 'Class[Common_software]' for relationship from 'Class[Default_repositories]'

My wild guess would be that "common_software" gets in before "default_repositories" and thus the error message. If i run agent again it sometimes goes away and configuration gets applied, sometimes it does not. I even tried using stages, did not help.

nodes.pp looks like this; 

stage { pre: before => Stage[main] }

node 'basenode' {

        class { 'default_repositories' : stage => pre }

        <some stuff here>

        include common_software-site

       <some more stuff here> 

} 

We have multiple (read: many) sites with configuration which varies here and there. To make things a bit more sane and to reduce duplication we decided to have common code base we ship to each site which then changes from site to site using inheritance mechanism, so, for example, there is class called "common_software" which defines some resources and then there is "common_software-site" which inherits base class and adds some functionality. 

If I include child class in nodes.pp problem is there, but including parent class fixes the problem.   

Any thoughts on the matter? 

Yes:

1. You really should arrange your classes in modules.
   
2. Class names must not contain hyphens (-).  They work accidentally in some versions of Puppet, but they do not work in others.  I doubt that's the source of your present problem, but it will likely cause you trouble in the future.
   
3. Although it has its use cases, class inheritance is usually not the best choice in Puppet.  If you're not using it to override parameters declared in a superclass, or to force parsing of the superclass to make it safe for a parametrized class to use the superclass's variables as parameter defaults, then you are misusing it.  You almost always want composition instead of inheritance (i.e. "include foo" in the body instead of "inherits foo" in the header).
   
4. It is safe to explicitly include a superclass and its subclass on the same node, in any order.
5. It is safe to 'include' a non-parametrized class any number of times.
   
6. Puppet parses any given manifest from top to bottom.  In particular, given your node declaration, it will parse the declaration of class 'default_repositories' before anything else in the node declaration.  In particular, it will parse that declaration before it parses "include common_software-site"
7. Puppet parses a class's definition immediately the first time it encounters a declaration of that class.  In your case, that means it will always start parsing the definition of class 'default_repositories' before the definition of class 'common_software-site'.  If it has no other reason to parse class 'common_software' than class 'common_software-site' inheriting from it, then it will (attempt to) parse all of the definition of 'default_repositories' before it parses either of the others.
8. The appearance of a reference to a class is not a reason to parse that class.
9. Therefore, if class 'default_repositories' contains a reference to Class['common_software'] but does nothing to ensure that it has been parsed (for example, "include 'common_software'") then I would expect exactly the error you describe.

John

[windowsrefund]

Luke,

You should really refrain from adding that bloated signature when communicating with mailing lists. It's basically a small DoS :)