Class Execution order

[Douglas Garstang]

Trying to force puppet class execution order with:

Class['lvm'] -> Class['network'] -> Class['apt'] -> Class <| |>

This is giving me:

err: Could not retrieve catalog from remote server: Error 400 on
SERVER: Resource type class doesn't exist at
/truth/sauce/env/prod/modules/role/manifests/common.pp:37 on node
testweb07.us1.xxx.com
warning: Not using cache on failed catalog

I did have it working with run stages, but when trying to also use the
relationship syntax with the following in site.pp

Apt::Source <| |> -> Exec["apt-update"]
Exec['apt-update'] -> Package <| |>

got the dreaded cyclic dependency errors... :(

Doug.

[jcbollinger]

Yes, both run stages and chaining from/to collections are prone to causing cycles.  They have their uses, but you do need to think very carefully about how you use them.  Moreover, the more you use them, the more likely you are to have trouble.  It is often quite hard to make correct blanket statements about resource relationships, more so the more complex your manifests become.

It looks and sounds as if you are trying to re-express your stages-based manifests using resource chaining.  If that's the case then you are on a wild goose chase.  Declaring the same relationships via a different syntax must necessarily produce the same cycle(s).  To fix the problem you need to remove (the right) unneeded relationships.


John

[earthgecko]

Dreaded.  I have found that using the .dot files that puppet --graph option produces sometimes helps see those dreaded dependency cycle issues (and some times it does not).  I recommend graphviz for anyone on linux for generating png files for the .dot files (not sure for Windows).

On my setup I have:

 # If graph is passed the graphviz .dot files are written to
 # /var/lib/puppet/state/graphs/ and you generate a .png image file
 # on your machine with graphviz e.g.:
 # dot -Tpng expanded_relationships.dot -o expanded_relationships.png

And that can help.  We feel for you, good luck, hopefully you have figured it by now.

[Douglas Garstang]

Not really. I have three run stages that have been working fine. It
was when I tried to add

Apt::Source <| |> -> Exec["apt-update"]
Exec['apt-update'] -> Package <| |>

to site.pp to globally enforce apt source installs, an apt-get update
and then installation of packages, in that order, that the dependency
cycles started. I thought maybe mixing the two was bad, which was when
I tried replacing the run stages with Class->, which doesn't seem to
be allowed syntactically btw. Looks like a bug?

I could put the installation of apt sources in their own run stage
that runs first. However, different classes of servers need different
apt sources, and therefore I need the ability to be able to add apt
sources at arbitrary points after the initial run stage, yet still
ensuring that an apt-get update happens only once after all the apt
sources have been installed, but before any packages are installed.

This must be a general problem. Wonder how people have solved it...?

Doug.

[jcbollinger]

On Tuesday, August 14, 2012 11:37:26 AM UTC-5, Douglas wrote:

Not really. I have three run stages that have been working fine. It
was when I tried to add

Apt::Source <| |> -> Exec["apt-update"]
Exec['apt-update'] -> Package <| |>

to site.pp to globally enforce apt source installs, an apt-get update
and then installation of packages, in that order, that the dependency
cycles started. I thought maybe mixing the two was bad, which was when
I tried replacing the run stages with Class->, which doesn't seem to
be allowed syntactically btw. Looks like a bug?

There is no inherent problem with mixing stages and chaining, as far as I know.  Both are simply facades on top of Puppet's underlying relationship machinery, so fundamentally they are parts of the same thing.

I am pretty confident that the problem is with Class<| |>.  There are at least two factors in play there:

1. Classes are not resources.  PL has made a concerted effort since the introduction of v. 2.6 to blur, downplay, and obscure the distinction, but they have not removed it.
2. Resource collections are conflated with realization of virtual resources (or collection of exported resources, where there isn't even a distinct name for the concept).  There is a longstanding ticket on this issue.

Since classes are not resources, but collection syntax specifies, in part, realization of virtual resources of the specified type, I do not find it surprising that Puppet rejects Class<| |>.  It looks like it should work (because PL has done a good job of making classes appear to be resources), but I'm not surprised that it doesn't.  I would not consider that a bug per se, but that it should work as you expected seems a reasonable feature request.

I could put the installation of apt sources in their own run stage
that runs first. However, different classes of servers need different
apt sources, and therefore I need the ability to be able to add apt
sources at arbitrary points after the initial run stage, yet still
ensuring that an apt-get update happens only once after all the apt
sources have been installed, but before any packages are installed.

This must be a general problem. Wonder how people have solved it...?

I agree that the problem seems general.  Does it not work to put your Apt::Source resources into their own classes, and assign those classes to your initial stage?  That seems the natural solution.


John

[Douglas Garstang]

Not really. The setup of base apt sources is handled during one of the
initial run stages. However, additional repo's are added later as the
function of the server is further refined.

Doug.

[jcbollinger]

I seem to be missing something here.  Perhaps there is a subtlety that I do not appreciate, on account of my intentional avoidance of run stages in my own manifests.  Based on the nature of the relationships you are trying to set up (with all the apt sources applied before any packages) I infer that "later" refers to parse order, but what does that have to do with which stages your classes are assigned to?

Here is an example of what I would expect to work, where "work" in this case means that on node 'myserver', class 'role42' is assigned to stage 'main', whereas class 'role42::apt' that it declares is assigned to stage 'apt-config':


stage { 'apt-config': before => Stage['main'] }

node myserver {
  class { 'role42': stage => 'main' }
}

class role42 {
  class { 'role42::apt'
    stage => 'apt-config'
  }
}

class role42::apt {
  apt::source { 'role42-source':
    # ...
  }
}


In other words, you should at any point in your manifest, from a class assigned to any stage, be able to declare a class into whatever stage you want (provided it has not already been declared elsewhere, of course).  Or that's my expectation, anyway.  Does it not actually work?


John