Puppet & class dependency

[Jean Baptiste Favre]

Hello,
There's something I don't understand with classes dependency.
I saw many thread about this subject, but still can not figure how it
works exactly.

I'm trying to setup a debian package repository. For that, I need:
- a vhost, served by nginx
- repo managment tools as well as GPG key copied into directory tree

So, that makes 2 classes:
include nginx
include pkgrepo

"nginx" class install nginx package and provide a define to deal with
vhost definition:
nginx::vhost { 'website.domain.tld':
port => 80,
docroot => '/var/www',
priority => 00,
}
This define will create directory /var/www/website.domain.tld as well as
deploying vhost file in /etc/nginx/sites-enabled/00-website.domain.tld

"pkgrepo" class will install repo managment tools and deploy repository
GPG key in /var/www/website.domain.tld/key.gpg

Therefore, it will require vhost to be created before being able to copy
GPG key.

Here's my setup:
node nodetest {
include nginx
nginx::vhost { 'website.domain.tld':
port => 80,
docroot => '/var/www',
priority => 00,
}
include pkgrepo
pkgrepo::repository { "test":
docroot => "/var/www/website.domain.tld",
gpgkeyid => "C78033BD",
}
}
And for classes definition, I got:
class pkgrepo {
Nginx::Vhost["packages.iscoolapp.com"] -> Class['Pkgrepo']
include pkgrepo::install,pkgrepo::config
}

Problem is, when I try to executed it twice, I don't alway have
dependency respected. For sure, I'm doing something wrong, but can not
figure what. Bellow are the execution output.

Best regards,
Jean Baptiste Favre

Wrong behaviour:
# /usr/sbin/puppetd --no-daemonize --logdest console --onetime --verbose
--preferred_serialization_format marshal
notice: Ignoring --listen on onetime run
info: Caching catalog for nodetest
info: Applying configuration version '1319805719'
notice: /Stage[main]/Pkgrepo::Install/Package[reprepro]/ensure: ensure
changed 'purged' to 'present'
notice:
/Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/nginx/sites-enabled/00-website.domain.tld]/ensure:
defined content as '{md5}bad3fd5741e335516e2e51aaefe2963c'
info:
/Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/nginx/sites-enabled/00-website.domain.tld]:
Scheduling refresh of Service[nginx]
notice: /Stage[main]/Nginx::Service/Service[nginx]: Triggered 'refresh'
from 1 events
notice: Finished catalog run in 6.25 seconds

Good behaviour:
# /usr/sbin/puppetd --no-daemonize --logdest console --onetime --verbose
--preferred_serialization_format marshal
notice: Ignoring --listen on onetime run
info: Caching catalog for nodetest
info: Applying configuration version '1319805719'
notice:
/Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/nginx/sites-enabled/00-website.domain.tld]/ensure:
defined content as '{md5}bad3fd5741e335516e2e51aaefe2963c'
info:
/Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/nginx/sites-enabled/00-website.domain.tld]:
Scheduling refresh of Service[nginx]
notice: /Stage[main]/Pkgrepo::Install/Package[reprepro]/ensure: ensure
changed 'purged' to 'present'
notice: /Stage[main]/Nginx::Service/Service[nginx]: Triggered 'refresh'
from 1 events
notice: Finished catalog run in 6.23 seconds

[Andrew Hendry]

Hi Jean,

Is Nginx::Vhost["packages.iscoolapp.com"] getting set up somewhere
else in your code?

If not should your pkgrepo be like this?
class pkgrepo {
Nginx::Vhost["website.domain.tld"] -> Class['Pkgrepo']
include pkgrepo::install,pkgrepo::config
}

Or

pkgrepo::repository { "test":
docroot => "/var/www/website.domain.tld",
gpgkeyid => "C78033BD",

require => Nginx::Vhost["website.domain.tld"],
}

Good luck.

On Oct 29, 12:41 am, Jean Baptiste Favre

> /Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/ngi nx/sites-enabled/00-website.domain.tld]/ensure:

> defined content as '{md5}bad3fd5741e335516e2e51aaefe2963c'
> info:
> /Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/ngi nx/sites-enabled/00-website.domain.tld]:
> Scheduling refresh of Service[nginx]
> notice: /Stage[main]/Nginx::Service/Service[nginx]: Triggered 'refresh'
> from 1 events
> notice: Finished catalog run in 6.25 seconds
>
> Good behaviour:
> # /usr/sbin/puppetd --no-daemonize --logdest console --onetime --verbose
> --preferred_serialization_format marshal
> notice: Ignoring --listen on onetime run
> info: Caching catalog for nodetest
> info: Applying configuration version '1319805719'
> notice:

> /Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/ngi nx/sites-enabled/00-website.domain.tld]/ensure:

> defined content as '{md5}bad3fd5741e335516e2e51aaefe2963c'
> info:
> /Stage[main]//Node[nodetest]/Nginx::Vhost[website.domain.tld]/File[/etc/ngi nx/sites-enabled/00-website.domain.tld]:

[Jean Baptiste FAVRE]

Hello Andrew,
Thanks for your answer.

Infact, pkgrepo *is* already like you suggest:

class pkgrepo {
Nginx::Vhost["website.domain.tld"] -> Class['Pkgrepo']
include pkgrepo::install,pkgrepo::config
}

Just forget about Nginx::Vhost["packages.iscoolapp.com"] and replace it
wherever you see it with Nginx::Vhost["website.domain.tld"] (my bad for
the typo).

And, that's why I don't understand because
"Nginx::Vhost['website.domain.tld'] -> Class['Pkgrepo']" is supposed to
make vhost applied before pkgrepo. But in my output examples, package
reprepro definied in pkgrepo::install got installed before vhosts is set up.

About setting require for pkgrepo::repository, I will try asap.
My first thought was to specify dependencies between classes in "root"
class (ie, in pkgrepo and not in pkgrepo::install) for readability purpose.
But, it that case, I could specify it pkgrepo::repository since it
defined in node definition. Just wonder why I did not tought about it
before :-/

Will let you updated right after my tests.

Regards,
JB

[jcbollinger]

On Oct 29, 2:50 am, Jean Baptiste FAVRE

> >> notice: Finished catalog run in 6.23 seconds- Hide quoted text -


Your setup is a bit strange in using arrow syntax within a class to
declare a dependency of that class itself, though I'm not sure that
shouldn't work. Instead, the key problem may be capitalization: to
the best of my knowledge, when you reference a resource you must
capitalize the resource *type*, but not the resource name, thus I
think you should use

Nginx::Vhost["website.domain.tld"] -> Class['pkgrepo']

Nevertheless, it shouldn't just fail silently. If you see nothing
relevant in either the master or the client's log about the
relationship not being honored, the resource not being found, or a
dependency cycle being detected, then you should file a ticket about
that.

Meanwhile, there are a couple of alternative, more conventional ways
to define your relationship. For example, you might find this to work
more reliably:

node nodetest {
include nginx
# NOTE relocation:
include pkgrepo

nginx::vhost { 'website.domain.tld':
port => 80,
docroot => '/var/www',
priority => 00,

# NOTE:
require => Class['pkgrepo']

}
pkgrepo::repository { "test":
docroot => "/var/www/website.domain.tld",
gpgkeyid => "C78033BD",
}
}

John

[devon]

> And for classes definition, I got:
> class pkgrepo {
>          Nginx::Vhost["packages.iscoolapp.com"] -> Class['Pkgrepo']
>          include pkgrepo::install,pkgrepo::config
>
> }

The way I understand it, all "included" classes are basically included
at the same time before catalog compilation. And then the resource
dependencies don't take effect until the catalog is being applied to
the node...or in short - "includes" aren't treated like resources.

You basically end up with a node that includes pkgrepo,
pkgrepo::install, and pkgrepo::config classes, then puppet ensures
that Nginx::Vhost["packages.iscoolapp.com"] occurs before
Class["pkgrepo"] - ignoring any sort of ordering for classes that
pkgrepo included.

To ensure the strict ordering when classes include other classes, you
need to specify the requirements for all the classes - something like
this should work:

class pkgrepo {
       Nginx::Vhost["packages.iscoolapp.com"] ->

Class['pkgrepo::install'] -> Class['pkgrepo::config'] ->
Class['pkgrepo']
       include pkgrepo::install,pkgrepo::config
}

[Jean Baptiste FAVRE]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Devon,
You're tight and that's the point I was looking for.
I discovered that point when doing some tests after Andrew's contribution.
So, now it works and I'll be able to deal with class ordering.

Thanks for your answer,
JB

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6u6IMACgkQM2eZoKJfKd3SwwCgi5nukmVgCL8FtbF6AVq5btbO
pRYAoITEGc4mzjAAMtYtxtE3hKKKAnjt
=1dAq
-----END PGP SIGNATURE-----

[Jean Baptiste FAVRE]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello John,
Answers inline

On 31/10/2011 15:40, jcbollinger wrote:
>
> Your setup is a bit strange in using arrow syntax within a class to
> declare a dependency of that class itself, though I'm not sure that
> shouldn't work. Instead, the key problem may be capitalization: to
> the best of my knowledge, when you reference a resource you must
> capitalize the resource *type*, but not the resource name, thus I
> think you should use
>
> Nginx::Vhost["website.domain.tld"] -> Class['pkgrepo']
>

About dependency of a class declared in the class itself, it bascially a
matter of modules/class organization:
- - Each module should be as much independent as possible. If they need
some other modules/classes, all requirement should be setted up in
"root" class.
For example, it's a simple module but still relevant: pkgrepo
Its goal is to set up a debian package repository. For that, it'll
install all relevant package and configure everything it can.
But, it need nginx::vhost to set up directory tree so that it'll be able
to deploy GPG key.
To make code maintenance easier, I've choosen to put all dependancies in
"root" class (in my example pkgrepo and not pkgrepo::install or
pkgrepo::configure)

The example here is quite simple, but I'm working on other modules like
zabbix, which needs specific MySQL database/user.

About capitalization, it's basically an error from me :-/

> Nevertheless, it shouldn't just fail silently. If you see nothing
> relevant in either the master or the client's log about the
> relationship not being honored, the resource not being found, or a
> dependency cycle being detected, then you should file a ticket about
> that.
>
> Meanwhile, there are a couple of alternative, more conventional ways
> to define your relationship. For example, you might find this to
> work more reliably:
>
> node nodetest { include nginx # NOTE relocation: include pkgrepo
>
> nginx::vhost { 'website.domain.tld': port => 80, docroot =>
> '/var/www', priority => 00, # NOTE: require => Class['pkgrepo'] }
> pkgrepo::repository { "test": docroot =>
> "/var/www/website.domain.tld", gpgkeyid => "C78033BD", } }

This example should work, but won't for more complicated modules
(example above with zabbix module I'm working on)

Regards,
JB

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6u654ACgkQM2eZoKJfKd22yQCdFFNg2uqgBohKOOCFZmrfSuum
vdwAnAxETA97w/g7A3cZjJMpaeKcAEFR
=DI+f
-----END PGP SIGNATURE-----