puppet agent --tag (and hiera) during Scientific Linux installation

[Arnau]

Hello all,

I'm running puppet 3.7.5 + hiera 1.3.4 and facter 2.3.0 .

My site.pp:

$ cat manifests/site.pp
node default {
  hiera_include('classes')
}

in hiera, I've defined the default classes to be included:

$ cat hieradb/defaults.yaml
---
#Default classes
classes:
 - profiles::base

and the base proflie includes few classes like:

class profiles::base {

  contain ntp
  contain resolv
  contain smtp
  contain ssh
  contain common
  contain puppet::agent
  contain repos
  contain nagios::client

...
}

in hiera defaults.yaml I've also defined a couple of nagios::client[1] variables (for not including selinux):

$cat hieradb/defaults.yaml
[...]
nagios::client::selinux: 'false'
nagios::client::selinux_enforced: 'false'
[...]

*Default for selinux is true.

the relevant code from nagios[1] module:

# nagios/manifests/client.pp 

[...]
  # With selinux, some nrpe plugins require additional rules to work
  if $selinux and $::selinux_enforced {
    selinux::audit2allow { 'nrpe':
      source => "puppet:///modules/${module_name}/messages.nrpe",
    }
  }
[...]

and ONLY NTP class has the tag kickstart::bootstrap defined.

* Hiera works and returns the expected values:

#  hiera -c /etc/puppet/hiera.yaml -d  classes environment=basic_conf clientcert=XX
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Hiera YAML backend starting
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking up classes in YAML backend
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/global
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/cert/XX
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/env/basic_conf
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Cannot find datafile /var/lib/puppet-deploy/.../basic_conf.yaml, skipping
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/defaults
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Found classes in basic_conf/hieradb/defaults
["profiles::cb::base"]

#  hiera -c /etc/puppet/hiera.yaml -d  nagios::client::selinux environment=basic_cb_conf clientcert=inhas01883.eu.boehringer.com

[...]

false

My kickstart posinstall section runs puppet like:

puppet agent --test --tags=kickstart::bootstrap --report --pluginsync --no-noop 

I expect puppet to run, not include selinux and ONLY configure ntp

But it gives me an error (failed catalog) because selinux::audit2allow is an invlaid resource type. (which means that the nagios variables are not picked up from hiera (false)  so selinux is included)

If I reboot the node, and run the same exact puppet agent line, then puppet runs (no selinux complain) and only NTP class is configured:

#  /usr/bin/puppet agent --test --environment=basic_cb_conf --tags=kickstart::bootstrap
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for XXX
Info: Applying configuration version '....'
Notice: /Stage[main]/Ntp/File[/etc/ntp.conf]/content:
--- /etc/ntp.conf       2015-11-10 12:23:14.946909373 +0000
+++ /tmp/puppet-file20151110-5619-gw8wio-0      2015-11-10 12:24:05.208909327 +0000
@@ -1,54 +1,10 @@
-# For more information about this file, see the man pages
-# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).
[...]

the expected behaivour.

So, what are (or could be) the differences between puppet runs inside the kickstart postinstall process and puppet runs outside it?

Why is hiera ignored?

[1] (https://forge.puppetlabs.com/thias/nagios)

TIA,

[Martin Alfke]

Hi Arnau

On 10 Nov 2015, at 13:19, Arnau <lists...@gmail.com> wrote:

> …

> in hiera defaults.yaml I've also defined a couple of nagios::client[1] variables (for not including selinux):
>
> $cat hieradb/defaults.yaml
> [...]
>
> nagios
> ::client::selinux: 'false'
>
> nagios
> ::client::selinux_enforced: 'false'
> [...]
> *Default for selinux is true.

Are your hiera data generic or specific to environments?

> …

> * Hiera works and returns the expected values:
> # hiera -c /etc/puppet/hiera.yaml -d classes environment=basic_conf clientcert=XX

Here you make use of the ::environment.

> …

> My kickstart posinstall section runs puppet like:
> puppet agent --test --tags=kickstart::bootstrap --report --pluginsync --no-noop

Here you do not specify the environment.

>
> I expect puppet to run, not include selinux and ONLY configure ntp
> But it gives me an error (failed catalog) because selinux::audit2allow is an invlaid resource type. (which means that the nagios variables are not picked up from hiera (false) so selinux is included)
> If I reboot the node, and run the same exact puppet agent line, then puppet runs (no selinux complain) and only NTP class is configured:
> # /usr/bin/puppet agent --test --environment=basic_cb_conf --tags=kickstart::bootstrap

Here you again use the environment.

Can you provide the content and location of your hiera.yaml file?


Best,
Martin

> …

> So, what are (or could be) the differences between puppet runs inside the kickstart postinstall process and puppet runs outside it?
> Why is hiera ignored?
>
> [1] (https://forge.puppetlabs.com/thias/nagios)
>
> TIA,
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAM69jx_uAbfBc1aU0hYUv%3DGfkGu1oTCx%3D1%3DkgS1JUE1ifsxMxQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Arnau]

2015-11-10 14:39 GMT+01:00 Martin Alfke <tux...@gmail.com>:

Hi Arnau

Hi Martin,

 

Are your hiera data generic or specific to environments?

specific to hiera.

 

> …

> * Hiera works and returns the expected values:
> #  hiera -c /etc/puppet/hiera.yaml -d  classes environment=basic_conf clientcert=XX

Here you make use of the ::environment.

Yes, because the default environment is not "basic_conf" .

 

> My kickstart posinstall section runs puppet like:
> puppet agent --test --tags=kickstart::bootstrap --report --pluginsync --no-noop

Here you do not specify the environment.

I've tried to run the puppet command + --environment=basic_conf inside the kickstart process and it still gives me the error. I should have pasted the complete command and not this one. Sorry.

 

The reason why I specify the environment is becasue the default env in the puppet master is not basic_conf. The node env  (AND ONLY NODE ENV) is defined in foreman. (yes, maybe I should have talked about foreman before, but, as the same command with/without environment fails, I did not consider that foreman was interefering with my current issue).

 

Can you provide the content and location of your hiera.yaml file?

 # cat /etc/puppet/hiera.yaml

---

:backend:

  - yaml

:hierarchy:

  - "%{environment}/hieradb/global"

  - "%{environment}/hieradb/cert/%{clientcert}"

  - "%{environment}/hieradb/net/%{netlocation}"

  - "%{environment}/hieradb/location/%{location}"

  - "%{environment}/hieradb/env/%{environment}"

  - "%{environment}/hieradb/dist/%{operatingsystem}/%{operatingsystemrelease}"

  - "%{environment}/hieradb/dist/%{operatingsystem}/%{operatingsystemmajrelease}"

  - "%{environment}/hieradb/dist/%{operatingsystem}"

  - "%{environment}/hieradb/dist/%{lsbdistid}/%{lsbdistrelease}"

  - "%{environment}/hieradb/dist/%{lsbdistid}/%{lsbmajdistrelease}"

  - "%{environment}/hieradb/dist/%{lsbdistid}"

  - "%{environment}/hieradb/defaults"

:yaml:

  :datadir: "/var/lib/puppet-deploy/XX-environments"

Best,
Martin

 

Cheers,

Arnau

[Martin Alfke]

Hi Arnau,

On 10 Nov 2015, at 14:23, Arnau <lists...@gmail.com> wrote:

>
>
> 2015-11-10 14:39 GMT+01:00 Martin Alfke <tux...@gmail.com>:
> Hi Arnau
> Hi Martin,
>
> Are your hiera data generic or specific to environments?
>
> specific to hiera.
>
> > …
>
> > * Hiera works and returns the expected values:
> > # hiera -c /etc/puppet/hiera.yaml -d classes environment=basic_conf clientcert=XX
>
> Here you make use of the ::environment.
>
> Yes, because the default environment is not "basic_conf" .
>
>
> > My kickstart posinstall section runs puppet like:
> > puppet agent --test --tags=kickstart::bootstrap --report --pluginsync --no-noop
>
> Here you do not specify the environment.
>
> I've tried to run the puppet command + --environment=basic_conf inside the kickstart process and it still gives me the error. I should have pasted the complete command and not this one. Sorry.
>
>
> The reason why I specify the environment is becasue the default env in the puppet master is not basic_conf. The node env (AND ONLY NODE ENV) is defined in foreman. (yes, maybe I should have talked about foreman before, but, as the same command with/without environment fails, I did not consider that foreman was interefering with my current issue).

I see two possible entry points for your problem:

1. environment specific hiera data
according to your hiera.yaml file you have hieradata per environment.
can you please cross check that the nagios::client::selinux key is set to false in all environment default.yaml files.

2. foreman providing data to modules.
check whether you set the selinux parameter for the nagios::client class in foreman based upon environments.

>
>
> Can you provide the content and location of your hiera.yaml file?
>
> # cat /etc/puppet/hiera.yaml
> ---
> :backend:
> - yaml
> :hierarchy:
> - "%{environment}/hieradb/global"
> - "%{environment}/hieradb/cert/%{clientcert}"
> - "%{environment}/hieradb/net/%{netlocation}"
> - "%{environment}/hieradb/location/%{location}"
> - "%{environment}/hieradb/env/%{environment}"
> - "%{environment}/hieradb/dist/%{operatingsystem}/%{operatingsystemrelease}"
> - "%{environment}/hieradb/dist/%{operatingsystem}/%{operatingsystemmajrelease}"
> - "%{environment}/hieradb/dist/%{operatingsystem}"
> - "%{environment}/hieradb/dist/%{lsbdistid}/%{lsbdistrelease}"
> - "%{environment}/hieradb/dist/%{lsbdistid}/%{lsbmajdistrelease}"
> - "%{environment}/hieradb/dist/%{lsbdistid}"
> - "%{environment}/hieradb/defaults"
>
> :yaml:
> :datadir: "/var/lib/puppet-deploy/XX-environments"
>
>
>
>
> Best,
> Martin
>
> Cheers,
> Arnau
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAM69jx-JRhRtHhrSo92nW%3DgkQE4Sw7G1RVhnvh-JOjx8%2BX6axA%40mail.gmail.com.

[Arnau]

Hi,

>
> I see two possible entry points for your problem:
>
> 1. environment specific hiera data
> according to your hiera.yaml file you have hieradata per environment.
> can you please cross check that the nagios::client::selinux key is set to false in all environment default.yaml files.

The only env where it's defined is the one I'm using.
But, in any case,if I specify the env in command line, puppet should not mix data from other envs, am I wrong?

> 2. foreman providing data to modules.
> check whether you set the selinux parameter for the nagios::client class in foreman based upon environments.

Foreman has no parameter defined. Only the env.

What really confuses me is the fact that everything wlrks as expected once the node has been rebooted. i must compare facter outputs, and anything else?

Also, i'll try to remlve all hierarchy entries except defaults.yaml, just to be sure that that's the only file checked.

I've been trying to download the catalog, with no luck. If the compilation fails, is there any way for downloading it anyway?

Thanks,
Arnau

[Martin Alfke]

On 10 Nov 2015, at 20:33, Arnau <lists...@gmail.com> wrote:

> Hi,
>
> >
> > I see two possible entry points for your problem:
> >
> > 1. environment specific hiera data
> > according to your hiera.yaml file you have hieradata per environment.
> > can you please cross check that the nagios::client::selinux key is set to false in all environment default.yaml files.
>
> The only env where it's defined is the one I'm using.

Maybe that is the reason.
Can you please have a look at the nagios::client code and check whether there is a default set for $selinux?

> But, in any case,if I specify the env in command line, puppet should not mix data from other envs, am I wrong?

Data is hiera. Env is puppet code.
When using env in hiera, then data should not get mixed among several environments.

>
> > 2. foreman providing data to modules.
> > check whether you set the selinux parameter for the nagios::client class in foreman based upon environments.
> Foreman has no parameter defined. Only the env.

OK.

>
> What really confuses me is the fact that everything wlrks as expected once the node has been rebooted. i must compare facter outputs, and anything else?

The only information the node sends to the master is: facts.
Maybe you can verify whether facts change after the reboot.

>
> Also, i'll try to remlve all hierarchy entries except defaults.yaml, just to be sure that that's the only file checked.
>
> I've been trying to download the catalog, with no luck. If the compilation fails, is there any way for downloading it anyway?
>
> Thanks,
> Arnau
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAM69jx9VhnXR6S2J8uoosWy8bVDBGE27%2BeYSguBaa4z6BSfOiw%40mail.gmail.com.

[Arnau]

Hi Martin,

> The only env where it's defined is the one I'm using.

Maybe that is the reason.

Sorry, but I don't see why this could be the reason. Could you please explain it to me?

 

Can you please have a look at the nagios::client code and check whether there is a default set for $selinux?

the default is "true".

 

I've compared facts during installation and after reboot. there's a difference in selinux value

selinux = true during installation

selinux = false after installation

but this is ::selinux and not ::nagios::client::selinux

I've tried to set nagios::client::selinux = false  in the puppet master/environment/class. the source file that triggers the error. And the error is still there. The only thing that really fix teh problem is commenting out the if statement in the nagios client code:

  ## With selinux, some nrpe plugins require additional rules to work

  #if $selinux and $::selinux_enforced {

   # selinux::audit2allow { 'nrpe':

  #    source => "puppet:///modules/${module_name}/messages.nrpe",

 #   }

#  }

Even if the value is true/false the nagios client code refers to a type defined in a different module (selinux):

  # With selinux, some nrpe plugins require additional rules to work

  if $selinux and $::selinux_enforced {

    selinux::audit2allow { 'nrpe':

      source => "puppet:///modules/${module_name}/messages.nrpe",

    }

  }

and the parsers gives the error because the resource type is unknown.

I think this explains what's going on. Does it make sense?

but I'm still wondering why the parser does not complain after the node reboot. any answer to this?

Thanks,

Arnau

[Arnau]

Change ::selinux to false (append selinux=0 in the kernel line) and everything works as expected.

still (very) confused....

Arnau

[jcbollinger]

On Tuesday, November 10, 2015 at 7:19:49 AM UTC-6, Arnau wrote:

 

* Hiera works and returns the expected values:

#  hiera -c /etc/puppet/hiera.yaml -d  classes environment=basic_conf clientcert=XX
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Hiera YAML backend starting
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking up classes in YAML backend
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/global
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/cert/XX
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/env/basic_conf
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Cannot find datafile /var/lib/puppet-deploy/.../basic_conf.yaml, skipping
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Looking for data source basic_conf/hieradb/defaults
DEBUG: Tue Nov 10 14:14:39 +0100 2015: Found classes in basic_conf/hieradb/defaults
["profiles::cb::base"]

#  hiera -c /etc/puppet/hiera.yaml -d  nagios::client::selinux environment=basic_cb_conf clientcert=inhas01883.eu.boehringer.com

[...]

false

My kickstart posinstall section runs puppet like:

puppet agent --test --tags=kickstart::bootstrap --report --pluginsync --no-noop 

I expect puppet to run, not include selinux and ONLY configure ntp

You haven't given us enough information to judge whether that's a reasonable expectation.

First off, you talk about how you are using Hiera to set variables of class ::nagios::client, but that can be true only inasmuch as class parameters are a special kind of variable, and only if the particular variables you are trying to set are indeed class parameters.  Is that the case?  My apologies if I sound pedantic, but it is essential that we share a common understanding of the situation.

Secondly, all classes declared directly or indirectly by those tagged kickstart::bootstrap will also bear that tag, and if you have a class by that name or any with kickstart::bootstrap:: as a name prefix then those will bear that tag, too.  Are you certain there are no such classes that could explain the observed behavior?

 

But it gives me an error (failed catalog) because selinux::audit2allow is an invlaid resource type. (which means that the nagios variables are not picked up from hiera (false)  so selinux is included)

Well, maybe it does.  The actual error message emitted by the catalog builder might be helpful in tying the error to a specific line in a specific manifest.  Without such a clue, it is conceivable that the unwanted declaration is not the one you're looking at.  It is a catalog build failure you're talking about, right?  Not a catalog application failure?

 

If I reboot the node, and run the same exact puppet agent line, then puppet runs (no selinux complain) and only NTP class is configured:

#  /usr/bin/puppet agent --test --environment=basic_cb_conf --tags=kickstart::bootstrap
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for XXX
Info: Applying configuration version '....'
Notice: /Stage[main]/Ntp/File[/etc/ntp.conf]/content:
--- /etc/ntp.conf       2015-11-10 12:23:14.946909373 +0000
+++ /tmp/puppet-file20151110-5619-gw8wio-0      2015-11-10 12:24:05.208909327 +0000
@@ -1,54 +1,10 @@
-# For more information about this file, see the man pages
-# ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5).
[...]

the expected behaivour.

So, what are (or could be) the differences between puppet runs inside the kickstart postinstall process and puppet runs outside it?

Why is hiera ignored?

I think it's safe to say that Hiera is not ignored; the unexpected behavior surely arises for a different reason.

As to differences, the local machine's state is likely different after the initial reboot than it was in kickstart.  This is especially likely if your kickstart configures the agent to run automatically as a daemon, for in that case a catalog run will have been launched as soon as the machine restarted, so that by the time you manually perform your run, Puppet will have already applied the node's full configuration.  Even if no other Puppet run happens between the two you are comparing, the agent might well report different fact values in the two cases, and there could be other differences related to when the agent run occurs relative to other actions that take place during kickstart.

Also, as others have observed already, your two Puppet commands are not the same.  Specifying an environment in one case and not doing so in the other is a big distinction if your master is configured to accept the agent's assertion of its environment, and especially when your Hiera hierarchy is environment-specific, as yours is.  If neither agent nor ENC explicitly specifies an environment, then the agent is assigned to environment "production".  If that's what is being used during kickstart then the master will get different data for the node in the two cases (given the Hiera configuration you provided in another post).


John

[jcbollinger]

On Wednesday, November 11, 2015 at 7:21:44 AM UTC-6, Arnau wrote:

Change ::selinux to false (append selinux=0 in the kernel line) and everything works as expected.

still (very) confused....

It is plausible that disabling selinux on the machine would result in selinux-related classes declaring no resources, or otherwise behaving differently then when selinux is enabled.  That would be a function of the classes involved, but the enablement status would be visible to them via a node fact.  I'm inclined to think that the behavior change you describe arises from such a provision, but I cannot be sure without reviewing the classes involved.

More generally, it would be helpful both to you and to us if you could narrow down the failure case as much as possible, and present a complete, minimal example with which we could reproduce the problem.  There's far too much that we don't know about your manifest set and data; without (much) more, it's difficult for us to offer more than guesswork.


John

[Arnau]

Hi John,

2015-11-11 15:43 GMT+01:00 jcbollinger <John.Bo...@stjude.org>:

first of all, let me thanks for your answer,

[...]

First off, you talk about how you are using Hiera to set variables of class ::nagios::client, but that can be true only inasmuch as class parameters are a special kind of variable, and only if the particular variables you are trying to set are indeed class parameters.  Is that the case?  My apologies if I sound pedantic, but it is essential that we share a common understanding of the situation.

Sure, no problem at all. In the orginal question I refered to the module I was using (https://forge.puppetlabs.com/thias/nagios/dependencies). 

Let me try to give you all the information:

The relevant part of code :

The nagios class:

"nagios/manifests/client.pp"

class nagios::client (

[...tons of parmaeters....]

  $selinux                     = true

) inherits ::nagios::params {

[...]

my base profile:

class profiles::base {

  contain ntp

  contain resolv

  contain smtp

  contain ssh

  contain common

  contain puppet::agent

  contain repos

  contain nagios::client

[...Order...]

}

My hiera conf:

$ grep selinux hieradb/defaults.yaml

nagios::client::selinux: 'false'

nagios::client::selinux_enforced: 'false'

and the error (Parser error- > undefined type) refer to line 160 of client.pp:

  # With selinux, some nrpe plugins require additional rules to work

  if $selinux and $::selinux_enforced {

    selinux::audit2allow { 'nrpe':

      source => "puppet:///modules/${module_name}/messages.nrpe",

    }

  }

* I don't have the selinux module installed as I don't want to use it. The trivial solution would be installing it.

 

Secondly, all classes declared directly or indirectly by those tagged kickstart::bootstrap will also bear that tag, and if you have a class by that name or any with kickstart::bootstrap:: as a name prefix then those will bear that tag, too.  Are you certain there are no such classes that could explain the observed behavior?

I've created this tag and only ntp class includes it.

the ntp class is a in-house developed class that does not include any other one : package -> file ->  service .

 

But it gives me an error (failed catalog) because selinux::audit2allow is an invlaid resource type. (which means that the nagios variables are not picked up from hiera (false)  so selinux is included)

Well, maybe it does.  The actual error message emitted by the catalog builder might be helpful in tying the error to a specific line in a specific manifest.  Without such a clue, it is conceivable that the unwanted declaration is not the one you're looking at.  It is a catalog build failure you're talking about, right?  Not a catalog application failure?

I'll try to add some screenshot (sorry, VMware web client... can't copy and paste). 

But it's a parser error in line 160 of client.pp because it does not know the type. (Sorry, I should be more precise in this key point)

 

 

I think it's safe to say that Hiera is not ignored; the unexpected behavior surely arises for a different reason.

As to differences, the local machine's state is likely different after the initial reboot than it was in kickstart.  This is especially likely if your kickstart configures the agent to run automatically as a daemon, for in that case a catalog run will have been launched as soon as the machine restarted, so that by the time you manually perform your run, Puppet will have already applied the node's full configuration.  Even if no other Puppet run happens between the two you are comparing, the agent might well report different fact values in the two cases, and there could be other differences related to when the agent run occurs relative to other actions that take place during kickstart.

yes, both runs report  differnt facts. As you already know, selinux changes its value.

puppet is run in cron mode, so there are no puppet in between.

Also, as others have observed already, your two Puppet commands are not the same.  Specifying an environment in one case and not doing so in the other is a big distinction if your master is configured to accept the agent's assertion of its environment, and especially when your Hiera hierarchy is environment-specific, as yours is.  If neither agent nor ENC explicitly specifies an environment, then the agent is assigned to environment "production".  If that's what is being used during kickstart then the master will get different data for the node in the two cases (given the Hiera configuration you provided in another post).

No, I'm sure about the env it runs during the installation and after it. only my environment has the nagios class and only my uses hiera.

I should not pasted different commands becasue they are the same. I just copied & pasted from another terminal.

[... at this point I join your answer from the other e-mail....]

Change ::selinux to false (append selinux=0 in the kernel line) and everything works as expected.

still (very) confused....

It is plausible that disabling selinux on the machine would result in selinux-related classes declaring no resources, or otherwise behaving differently then when selinux is enabled.  That would be a function of the classes involved, but the enablement status would be visible to them via a node fact.  I'm inclined to think that the behavior change you describe arises from such a provision, but I cannot be sure without reviewing the classes involved.

yes, it's plausible, but looking into nagios code again, it refers to :

if $selinux and $::selinux_enforced {

::selinux_eforced = true because selinux is enabled during installation.

but I though that $selinux value inside nagios::client class  _should_ come from hiera as it's nagios::client::hiera.  this is my mistake, am I wrong?

and after the reboot puppet runs because all selinux variables (facts) equal to false as selinux has been disabled after reboot. I'm not setting the variable properly in hiera.

More generally, it would be helpful both to you and to us if you could narrow down the failure case as much as possible, and present a complete, minimal example with which we could reproduce the problem.  There's far too much that we don't know about your manifest set and data; without (much) more, it's difficult for us to offer more than guesswork.

I think I've asked the proper question now.... did I?

Again, thanks to all of you for your answers,

Best,

Arnau

[jcbollinger]

On Wednesday, November 11, 2015 at 11:58:19 AM UTC-6, Arnau wrote:

Hi John,

2015-11-11 15:43 GMT+01:00 jcbollinger <John.Bo...@stjude.org>:

first of all, let me thanks for your answer,

[...]

First off, you talk about how you are using Hiera to set variables of class ::nagios::client, but that can be true only inasmuch as class parameters are a special kind of variable, and only if the particular variables you are trying to set are indeed class parameters.  Is that the case?  My apologies if I sound pedantic, but it is essential that we share a common understanding of the situation.

Sure, no problem at all. In the orginal question I refered to the module I was using (https://forge.puppetlabs.com/thias/nagios/dependencies). 

Let me try to give you all the information:

The relevant part of code :

The nagios class:

"nagios/manifests/client.pp"

class nagios::client (

[...tons of parmaeters....]

  $selinux                     = true

) inherits ::nagios::params {

[...]

So, importantly, class nagios::client does have a class parameter $::nagios::client::selinux.  Its value defaults to the boolean value true if it is not otherwise specified.

On the other hand, checking the full source shows that there is no class parameter $::nagios::client::selinux_enforced.

 

my base profile:

class profiles::base {

  contain ntp

  contain resolv

  contain smtp

  contain ssh

  contain common

  contain puppet::agent

  contain repos

  contain nagios::client

[...Order...]

}

My hiera conf:

$ grep selinux hieradb/defaults.yaml

nagios::client::selinux: 'false'

nagios::client::selinux_enforced: 'false'

Because there is no class parameter $::nagios::client::selinux_enforced, providing data for key "nagios::client::selinux_enforced" probably has no observable effect on catalog building.  It's unlikely to be directly harmful to provide that data, but doing so could be confusing.

 

and the error (Parser error- > undefined type) refer to line 160 of client.pp:

  # With selinux, some nrpe plugins require additional rules to work

  if $selinux and $::selinux_enforced {

    selinux::audit2allow { 'nrpe':

      source => "puppet:///modules/${module_name}/messages.nrpe",

    }

  }

Note that the condition is based on global variable $::selinux_enforced, which takes its value from a node fact.  Setting a value in Hiera for key "nagios::client::selinux_enforced" has no effect on the value of that variable.  This also explains why disabling selinux on the client alters Puppet's observed behavior.

 

* I don't have the selinux module installed as I don't want to use it. The trivial solution would be installing it.

I submit that it is a poor choice to install a puppet module but to intentionally avoid installing all of that module's dependencies.  However, although installing the selinux module ought to resolve the problem manifestation, the real problem is that during kickstart, the master is not using the correct data for the node.  That should be resolved, as it might cause other, less visible, problems, either now or in the future.

 

 

Secondly, all classes declared directly or indirectly by those tagged kickstart::bootstrap will also bear that tag, and if you have a class by that name or any with kickstart::bootstrap:: as a name prefix then those will bear that tag, too.  Are you certain there are no such classes that could explain the observed behavior?

I've created this tag and only ntp class includes it.

the ntp class is a in-house developed class that does not include any other one : package -> file ->  service .

Ok.

 

 

But it gives me an error (failed catalog) because selinux::audit2allow is an invlaid resource type. (which means that the nagios variables are not picked up from hiera (false)  so selinux is included)

Well, maybe it does.  The actual error message emitted by the catalog builder might be helpful in tying the error to a specific line in a specific manifest.  Without such a clue, it is conceivable that the unwanted declaration is not the one you're looking at.  It is a catalog build failure you're talking about, right?  Not a catalog application failure?

I'll try to add some screenshot (sorry, VMware web client... can't copy and paste). 

But it's a parser error in line 160 of client.pp because it does not know the type. (Sorry, I should be more precise in this key point)

Ok, good enough for me.

 

 

 

I think it's safe to say that Hiera is not ignored; the unexpected behavior surely arises for a different reason.

As to differences, the local machine's state is likely different after the initial reboot than it was in kickstart.  This is especially likely if your kickstart configures the agent to run automatically as a daemon, for in that case a catalog run will have been launched as soon as the machine restarted, so that by the time you manually perform your run, Puppet will have already applied the node's full configuration.  Even if no other Puppet run happens between the two you are comparing, the agent might well report different fact values in the two cases, and there could be other differences related to when the agent run occurs relative to other actions that take place during kickstart.

yes, both runs report  differnt facts. As you already know, selinux changes its value.

puppet is run in cron mode, so there are no puppet in between.

Also, as others have observed already, your two Puppet commands are not the same.  Specifying an environment in one case and not doing so in the other is a big distinction if your master is configured to accept the agent's assertion of its environment, and especially when your Hiera hierarchy is environment-specific, as yours is.  If neither agent nor ENC explicitly specifies an environment, then the agent is assigned to environment "production".  If that's what is being used during kickstart then the master will get different data for the node in the two cases (given the Hiera configuration you provided in another post).

No, I'm sure about the env it runs during the installation and after it. only my environment has the nagios class and only my uses hiera.

I should not pasted different commands becasue they are the same. I just copied & pasted from another terminal.

The fact remains that by far the most plausible explanations for the observed behavior difference revolve around the variable interpolations on which your Hiera hierarchy relies.  I remain suspicious of the environment chosen, as you have data only for a non-default environment, and Hiera not providing any data would certainly produce the effect you observe.

I suggest switching your focus from the client to the master for a bit: kickstart a node, and look in the master's logs / puppetdb / node fact cache to see which environment the node was actually assigned to, and what facts it reported.  Running the master with debug logging enabled might help here.  Look for any Hiera data file that might override the value of the "nagios::client::selinux" key to true.  Don't overlook any files named ".yaml" anywhere in the Hiera data tree; if there are any, then by default they will not be presented in directory listings, but it's conceivable that Hiera would read and use such files under certain circumstances.

 

More generally, it would be helpful both to you and to us if you could narrow down the failure case as much as possible, and present a complete, minimal example with which we could reproduce the problem.  There's far too much that we don't know about your manifest set and data; without (much) more, it's difficult for us to offer more than guesswork.

I think I've asked the proper question now.... did I?

You have not provided all that I asked: a complete, minimal example with which we could reproduce the problem.  Nevertheless, what you did provide answers several questions, enough so to move forward.


John