Speaking of Nagios...

[Sean Alderman]

Al,
  I didn't want to hijack the other thread, but do I follow correctly that using your puppet modules the Nagios server will have it the hosts it monitors automatically updated as the nodes with the NRPE module deployed and your monitoring abstract configured for nagios?  I could globally set this up in Foreman with Parameters from HostGroups/Environments/Subnets/Hosts etc.?  If so this is a fascinating collision of tasks all made simple.

Thinking this is quite awesome, I just spun up a nagios server in my Foreman lab using the E42 nagios module with the node.  The puppet run fails to start nagios because it doesn't like something in the hostgroups/all.cfg ...

Error: Could not expand members specified in hostgroup (config file '/etc/nagios/auto.d/hostgroups/all.cfg', starting on line 3)

That file is extremely trivial, I can't seem to identify what could be wrong.  Have you run into this?  I admit this is my first stab with Nagios, but that file's got like 5 lines and they all look good as far as I can tell with the docs.

Thanks.

[Alessandro Franceschi]

Il giorno martedì 2 luglio 2013 22:38:39 UTC+2, Sean Alderman ha scritto:

Al,
  I didn't want to hijack the other thread, but do I follow correctly that using your puppet modules the Nagios server will have it the hosts it monitors automatically updated as the nodes with the NRPE module deployed and your monitoring abstract configured for nagios?  I could globally set this up in Foreman with Parameters from HostGroups/Environments/Subnets/Hosts etc.?  If so this is a fascinating collision of tasks all made simple.

Absolutely yes.
To enable automatic monitoring set these variables on The Foreman:

$monitor = true

$monitor_tool = [ 'nagios' , 'puppi' ]

(You can just set nagios as monitor tool, but since you'll have already the nagios Plugins and nrpe installed on all the nodes, you can have puppi checks for free (that is, if you enable puppi as monitor tool, you will be able to see the status of a node executing, locally, 'puppi check' ).

Note also that for Nagios Monitoring you need stored configs enabled on your PuppetMaster.

One quite powerful, IMHO, option of the Nagios module is the possibility to automatically split Nagios Servers according to custom variables.

That is useful when you've a lot of nodes to manage and a single Nagios server could be overloaded (expect long Puppet run times on it).

If you set the variable $nagios_grouplogic to 'env', for example

you'll have a Nagios server that monitors all the nodes who have the same value for the variable 'env' (so, for example if the $env of your nagios is 'prod' , it will automatically monitor all and only the nodes who have $env=prod).

There are some other options in the module , more or less, documented.

 

Thinking this is quite awesome, I just spun up a nagios server in my Foreman lab using the E42 nagios module with the node.  The puppet run fails to start nagios because it doesn't like something in the hostgroups/all.cfg ...

Error: Could not expand members specified in hostgroup (config file '/etc/nagios/auto.d/hostgroups/all.cfg', starting on line 3)

This will be solved as soon as new nodes are added.

 

That file is extremely trivial, I can't seem to identify what could be wrong.  Have you run into this?  I admit this is my first stab with Nagios, but that file's got like 5 lines and they all look good as far as I can tell with the docs.

Yes, I know the problem and I probably should manage it better. It should solve anyway when you have some nodes added to monitoring.

[Sean Alderman]

Al,
  Sorry, I had moved away from this in my test lab due to lack of memory to sustain another full-time VM running on my laptop.  I'm coming back to it now because I've got real data center resources for running Puppet and Foreman that I've spent the last few days building up.  I'm now running latest versions of both Puppet and Foreman, and while I'm still struggling a bit to get everything glued together with the small differences in versions between my lab and the new stuff, but anyway...

  The bits you mention about about the group logic and such, is this fully functional?  Are we talking about having a unique Nagios Server that monitors each Foreman Host Group?  If so, how do I create the logic that connects what Nagios server serves what Foreman Host Group?  What other options are the for the group logic?

Thanks!

[Alessandro Franceschi]

NAgios and Foreman hostgroups are 2 different things and generally they have now correlation.

BUT, you can somehow manage to make them map and have on your Nagios server your nodes grouped according to Foreman hostgroups.

Basically you have to:

- Set for each hostgroup in Foreman a variable tha identified it (ex: hostgroup = 'xxx' )

- Set the top scope variable $icinga_hostgrouplogic = 'hostgroup' - Hope 

I've just committed a small patch right on this thing, so take the latest release to test.

IT should work, but I've tested this module with this configuration only in one place.

[Sean Alderman]

Intersting...  Ok, I'm out of my depth, I've not looked into Icinga and before diving into another tool I want to get to where I can build a host via Foreman and see it monitored in Nagios later.  So if I could step back to the simpler concept with just the basic nagios/nrpe config.

For the servers to be monitored - including the nagios server - Do I need to set the nrpe_allowed_hosts to include the Nagios server, correct?  My simple experiment says yes, but throwing Foreman in the middle makes it more difficult.  Should I hardcode the Nagios Server's IP?  I still struggle with how parameters in your modules work through Foreman.  I have worked through some of those issues, but the latest has to do with the nrpe_allowed_hosts parameter. 

Your example shows something like -

nrpe_allowed_hosts => [ '127.0.0.1', $::ipaddress, '<nagios server ip>' ]

I'm guessing, since I don't know NRPE and Nagios well enough, that we can't drop the $::ipaddress, and the nagios IP has to be an IP, not a name.  I've tried setting a global parameter in Foreman called nagserver and setting it to the nagios server IP, but when the puppet agents run they get 400 Errors.  I'm not sure if this is over the nagserver parameter or $::ipaddress.  Both appear to show up in the host yaml as literal strings, not substituted with their values.

You wouldn't happen to have a suggestion?

P.S. Al, sorry if you got this off-list, the group is not enabled for posting via email and this always gets me as I tend to reply from my inbox.

[Alessandro Franceschi]

On Tuesday, July 16, 2013 10:52:19 PM UTC+2, Sean Alderman wrote:

Intersting...  Ok, I'm out of my depth, I've not looked into Icinga and before diving into another tool I want to get to where I can build a host via Foreman and see it monitored in Nagios later.  So if I could step back to the simpler concept with just the basic nagios/nrpe config.

For the servers to be monitored - including the nagios server - Do I need to set the nrpe_allowed_hosts to include the Nagios server, correct?

Yes, nrpe_allowed_hosts is a parameter you have to set (on all the monitored hosts) with the addess(es) of the Nagios server (and possibily add localhost and $ipaddress)

 

My simple experiment says yes, but throwing Foreman in the middle makes it more difficult.  Should I hardcode the Nagios Server's IP?  I still struggle with how parameters in your modules work through Foreman.  I have worked through some of those issues, but the latest has to do with the nrpe_allowed_hosts parameter. 

There are different approaches you can follow to integrate Example42 modules with Foreman (or generally with any other ENC) and they are related to how Example42 modules accept parameters:
- Via direct class declaration, with all the needed parameters explicitely set

- Via Hiera using key names like $modulename_$parameter (example: openssh_template) or (default behaviour on Puppet3) openssh::template

- Via Top Scope variables (as the ones that are set in Foreman), like $::openssh_template

On Foreman therefore you have to:
1- Clone Example42 modules in your PuppetMaster's modulepath

2- Have a Foreman-Proxy running on the PuppetMaster
3- Import all the classes in Foreman (Menu More/ Configuration / Puppet Classes / Import from <puppetmaster ). This may take a while.

At this point you have different alternatives on where to place your data:

4a- Once all the classes are imported Foreman can manage directly their parameters and you can use Smart Variables to set them. There parameters are used in parametrized classes declarations (via the ENC).

4b- You can use Foreman to Include classes and set "normal" variables (as the ones you can set per HostGroups/Node/OperatingSystem... in the Parameters tab). These parameters appear as top scope variables.

4c- You can use Foreman to include classes and place on Hiera all your parameters

4d- You can use Foreman to include classes and place your variables in Foreman and/or in your custom site module (where you already have to place your local files and templates).

There are some variations on these patterns, which basically relate to how much information you want to place in Foreman GUI and how much in external files (either hiera or custom modules), but they somehow depend on personal needs and choices.

 

Your example shows something like -

nrpe_allowed_hosts => [ '127.0.0.1', $::ipaddress, '<nagios server ip>' ]

I'm guessing, since I don't know NRPE and Nagios well enough, that we can't drop the $::ipaddress, and the nagios IP has to be an IP, not a name.  I've tried setting a global parameter in Foreman called nagserver and setting it to the nagios server IP, but when the puppet agents run they get 400 Errors.  I'm not sure if this is over the nagserver parameter or $::ipaddress.  Both appear to show up in the host yaml as literal strings, not substituted with their values.

You wouldn't happen to have a suggestion?

A possible approach can be:
- [Optional] Set on Foreman something like $nagiosserver=<ip of your server> 
- Include in Foreman a custom class (like 'site')

- In site/manifests/init.pp, where you have the site class, used by all your modules, you you place something like:
class { 'nrpe':
  nrpe_allowed_hosts => [ '127.0.0.1', $::ipaddress, '<nagios server ip>' ],
}
where <nagios server ip> is either $::nagiosserver (the top scope var that you set in Foreman) or directly the ip of your Nagios server.

[Sean Alderman]

A possible approach can be:
- [Optional] Set on Foreman something like $nagiosserver=<ip of your server> 
- Include in Foreman a custom class (like 'site')

- In site/manifests/init.pp, where you have the site class, used by all your modules, you you place something like:
class { 'nrpe':
  nrpe_allowed_hosts => [ '127.0.0.1', $::ipaddress, '<nagios server ip>' ],
}
where <nagios server ip> is either $::nagiosserver (the top scope var that you set in Foreman) or directly the ip of your Nagios server.

This seems to work well enough.  The caveat would be the NRPE class can not be included from Foreman.  I think when I had tried to insert this into the site class - well in my case it's a hostgroup class - Foreman would not import the class because of some strange syntax issue...which I resolved by removing the code from the module and going back to attempting to do inside of Foreman.  With your note, I've created a simple monitoring wrapper class that I can include with the host-group, then set the $::nagiosserver parameter by host group.  Seems to be working so far!

Thank you for the guidance!...and your kind patience!

[Alessandro Franceschi]

Do you have this "strange syntax issue" when you import the class into foreman the first time or when you try to add it to an host or hostgroup.

Right yesterday I did a general import of the modules on a Foreman installation without problems  so I wonder where's exactly might be the issue.

al

[Sean Alderman]

Within Foreman under Puppet Classes, when I click Import from <puppetmaster> I get these messages in /var/log/foreman-proxy/access.log:

Error while parsing /etc/puppet/environments/ops/modules/udmonitor/manifests/init.pp: Syntax error at '='; expected '}' at line 3
Error while parsing /etc/puppet/environments/common/profile/manifests/params.pp: Syntax error at ','; expected '}' at line 19
Error while parsing /etc/puppet/environments/common/syslog-ng/manifests/init.pp: Syntax error at '-'; expected '}' at line 261

Then Foreman came back with no changes detected.  So for whatever reason the profile and syslog-ng modules were already imported, but still had syntax errors, but my udmonitor module did not get imported.  I had several iterations of the init.pp the syntax error was always there, but with various at '<something>'; expected messages.

At the time I was also trying to set the monitor/monitor_tool globals within that module so that I didn't have to set them within Foreman.  I am no longer trying to do that, so it could be the issue as well.

[Fran Rodríguez]

Hi there,

I got a misunderstanding with this, im trying to setup the $nagios_grouplogic for getting a nagios sever for each environment. I define on global variables from foreman the nagios_grouplogic = environment but i dont get to work it. This doesnt work because foreman doesnt resolve it: 

classes:

  site:

    munin_cidr_allow:

    - 127.0.0.1

    - mun1.xxx.xxx.es

    munin_server_local: false

    nrpe_absent: false

  services::monitoring:

    nagios_disable: false

parameters:

  puppetmaster: foreman.xxx.xxx.es

  domainname: ''

  hostgroup: Default testing VMs

  root_pw: $1$vfQRzPw5$Uf2.AieBmI6PQj3ZUVDpQ0

  puppet_ca: foreman.xxx.xxx.es

  foreman_env: testing

  owner_name: Admin User

  owner_email: s...@xxx.es

  monitor: 'true'

  monitor_tool: ! '["puppi", "nagios", "munin"]'

  nagios_customconfigdir: /etc/nagios3/auto.d

  nagios_grouplogic: environment

  puppi: 'true'

  enable-puppetlabs-repo: 'true'

  static: 'true'

Im reading your article about how to use example42 modules for monitoring, and you said u can use a variable called env or dc on your top scope but then u have to setup two variables on you top scope, env and nagios_grouplogic. This is getting me lost... $environment variables is already on the top scope, i thought that if i assign $nagios_grouplogic = "$environment", this will be worked but it doesnt. So maybe i missing something or probably i dont understand how it works.

Please somebody could explain me how could do this what im trying.

Thanks,

[Fran Rodríguez]

Hi again,

I read how it works get_magicvar, and now i understand how we pass the variable... :), nagios_grouplogic = environment in the global parameters from foreman. Now the magic_tag has the environment value, but i dont get any effect on nagios, im still getting one grouplogic, nothing is changed. Any ideas what is happening?¿?¿

Cheers

[Alessandro Franceschi]

Probably the documentation is not clear on this.
The grouplogic stuff is used to automatically segment nodes to be monitored by a nagios server according to a custom parameter (typically their operational environment, but could be something different, such as data center, zone or whatever).

This is NOT useful to create automatically nagios hostgroups inside the same Nagios server according to the variable used for grouplogic.

So a nagios server using the production environment automatically monitors all the nodes of the production environment (if $grouplogic=production) and a nagios server using a test environment would collect and monitor all (and only) the servers in the test environment.

This is probably not what you are looking for (I suppose you want to  automatically create Nagios hostgroups according to the grouplogic variable).

I tried in the past to automate the management of hostgroups but it's a real PITA, as Nagios configuration syntax is very picky, as we all know, and it was difficult to manage in a consistent way the correct host and hostgroups with exported resources.
Some better integration on this is done on the icinga module: https://github.com/example42/puppet-icinga where you find another top scope variable $::hostgrouplogic which tries to automatically create hostgroups based on the variable defined there (with the same get_magicvar function).
Can't guarantee that it works well (at the end, in a place where I used this module I managed hostgroups via a custom hash on Hiera, using the variable icinga::hostgroups_hash).

Icinga and Nagios module have started from the same code base then diverged so some features are available only in one of them, even if they can be ported to the other module.

Both of them lack decent documentation :-I

Hope it helps

Al

--
You received this message because you are subscribed to the Google Groups "Example42 Puppet Modules" group.
To unsubscribe from this group and stop receiving emails from it, send an email to example42-puppet-m...@googlegroups.com.
To post to this group, send email to example42-pu...@googlegroups.com.
Visit this group at http://groups.google.com/group/example42-puppet-modules.
For more options, visit https://groups.google.com/d/optout.

[Fran Rodríguez]

Hi Al,

Dont worry about documentation, what i want to do its split Nagios Server for monitor different environments. I started with two Nagios server, one for production and other for testing, but im not getting to work right. I have all nodes on both Nagios Servers, and what i want its that you said, one Nagios server for one environment. I could be greate that you said about hostgroup but i realized that i couldnt be used it.

What i do its setting up nrpe server from the different environment allows to a Nagios server of that environment connects, now the grouplogic variable has the correct environment but im still watching all nodes on both Nagios servers, and other delete nodes hasnt disappeared. So i dont know how follow, i dont know if i miss something or what. Please could you point to me in correct way?¿

Cheers

To unsubscribe from this group and stop receiving emails from it, send an email to example42-puppet-modules+unsub...@googlegroups.com.

[Alessandro Franceschi]

It might be an issue with stale exported resources. Try to remote a node from PuppetDB (I suppose you are using it as backend) with:

puppet node deactivate

and see if it reappears on both the Nagios servers when Puppet is run (having the grouplogic variable correctly set as you need).

For the automatic removal of removed hosts try to set source_dir_purge => true
where you use the nagios class.

[Fran Rodríguez]

Hi Al,

It seems working right now. 

Thank you very much for the help, and congratulations for the fantastic modules.