Using hiera to configure the jgazeley/ossec module

[Todd Courtnage]

I'm in the process of refactoring our puppet to make use of r10k, hiera and roles/profiles, as seems to be the suggested methodology these days.

I've successfully got the ericsson/motd and puppetlabs/apt modules up and running and configured with appropriate with hiera, doing what I want in various environments.

I'm trying (so far unsuccessfully) to use hiera to configure this module (https://forge.puppetlabs.com/jgazeley/ossec). r10k is configured to pull in the ossec module and dependencies (which it has).

I'm simply attempting to set the ossec_server_ip parameter in the ossec::client class, but all I ever get is a "Must pass ossec_server_ip to Class[Ossec::Client]..." error. I get this from running a "puppet agent --test--noop" on an agent or running a puppet apply directly on the module. I feel like this should be incredibly simple and that I'm just missing something completely obvious.

This is with the open-source puppetserver 4.2.1 on Ubuntu 14.04, with an agent running puppet 4.2.1 as well.

/etc/puppetlabs/code/hiera.yaml (unchanged from default)

---
:backends:
  - yaml
:hierarchy:
  - "nodes/%{::trusted.certname}"
  - common

:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /etc/puppetlabs/code/environments/%{environment}/hieradata on *nix
# - %CommonAppData%\PuppetLabs\code\environments\%{environment}\hieradata on Windows
# When specifying a datadir, make sure the directory exists.
  :datadir:

/etc/puppetlabs/code/environments/development/hieradata/common.yaml:

---
classes:
  - 'profile::base'

motd::motd_content:
  - 'This is a development environment. Booya!'

apt::purge:
  sources.list.d: true
apt::update:
  frequency: daily

ossec::client::ossec_server_ip: <ip_address>

/etc/puppetlabs/code/environments/development/site/profile/manifests/base.pp

class profile::base {
  class { '::motd': }
  class { '::apt': }
  class { '::ossec::client': }
}

Trying to "make it go":

root@vm:/etc/puppetlabs/code/environments# puppet agent --test --noop
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Must pass ossec_server_ip to Class[Ossec::Client] at /etc/puppetlabs/code/environments/development/site/profile/manifests/base.pp:4 on node puppet.development.vm
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Any help/suggestions/pointers greatly appreciated.

[Luke Bigum]

I would isolate the Hiera lookup first, run it by hand on your Puppet Master:

hiera --debug -c /etc/puppetlabs/code/hiera.yaml -y $(puppet config print yamldir)/facts/vm.yaml ossec::client::ossec_server_ip ::environment=development

The above assumes the certname of your node is actually "vm", it's probably not, so change the path to your node's YAML Facts cache.

--
Luke Bigum

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5a2a8004-ae32-4af1-88e1-3fe1da352167%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
---

LMAX Exchange, Yellow Building, 1A Nicholas Road, London W11 4AN
http://www.LMAX.com/

#1 Fastest Growing Tech Company in the UK - Sunday Times Tech Track 100 (2014)

2015 Best Margin Sector Platform - Profit & Loss Readers' Choice Awards
2015 Best FX Trading Venue - ECN/MTF - WSL Institutional Trading Awards
2014 Best Margin Sector Platform - Profit & Loss Readers' Choice Awards
2014 Best FX Trading Venue - ECN/MTF - WSL Institutional Trading Awards
2014 Best Infrastructure/Technology Initiative - WSL Institutional Trading Awards
2013 #15 Fastest Growing Tech Company in the UK - Sunday Times Tech Track 100
2013 Best Overall Testing Project - The European Software Testing Awards
2013 Best Margin Sector Platform - Profit & Loss Readers' Choice Awards
2013 Best FX Trading Platform - ECN/MTF - WSL Institutional Trading Awards
2013 Best Executing Venue - Forex Magnates Awards

---

FX and CFDs are leveraged products that can result in losses exceeding your deposit. They are not suitable for everyone so please ensure you fully understand the risks involved.

This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). This message is not intended for any recipient(s) who based on their nationality, place of business, domicile or for any other reason, is/are subject to local laws or regulations which prohibit the provision of such products and services. This message is subject to the following terms (http://lmax.com/pdf/general-disclaimers.pdf), if you cannot access these, please notify us by replying to this email and we will send you the terms. If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.

LMAX Exchange is the trading name of LMAX Limited. LMAX Limited operates a multilateral trading facility. LMAX Limited is authorised and regulated by the Financial Conduct Authority (firm registration number 509778) and is a company registered in England and Wales (number 6505809).

LMAX Hong Kong Limited is a wholly-owned subsidiary of LMAX Limited. LMAX Hong Kong is licensed by the Securities and Futures Commission in Hong Kong to conduct Type 3 (leveraged foreign exchange trading) regulated activity with CE Number BDV088.

[jcbollinger]

On Tuesday, September 8, 2015 at 4:49:22 PM UTC-5, Todd C wrote:

I'm in the process of refactoring our puppet to make use of r10k, hiera and roles/profiles, as seems to be the suggested methodology these days.

I've successfully got the ericsson/motd and puppetlabs/apt modules up and running and configured with appropriate with hiera, doing what I want in various environments.

I'm trying (so far unsuccessfully) to use hiera to configure this module (https://forge.puppetlabs.com/jgazeley/ossec). r10k is configured to pull in the ossec module and dependencies (which it has).

I'm simply attempting to set the ossec_server_ip parameter in the ossec::client class, but all I ever get is a "Must pass ossec_server_ip to Class[Ossec::Client]..." error. I get this from running a "puppet agent --test--noop" on an agent or running a puppet apply directly on the module. I feel like this should be incredibly simple and that I'm just missing something completely obvious.

Are you certain that the apt and motd classes are correctly receiving their parameters from Hiera?  If their definitions happen to provide default values for all parameters, then a failure to look up parameter values in hiera would not cause catalog building to fail for them.  It could be that Hiera is misconfigured, so that it's not providing any data.

Have you restarted the master since last updating the data?  I'm never sure any more what exactly Puppet caches, or when, but you can ensure that its caches are flushed by restarting the master (the service, not necessarily the whole machine).

Have you looked in the master's logs for diagnostics that might be relevant?


John

[Todd C]

On Wednesday, September 9, 2015 at 1:34:32 AM UTC-6, Luke Bigum wrote:

I would isolate the Hiera lookup first, run it by hand on your Puppet Master:

  hiera --debug -c /etc/puppetlabs/code/hiera.yaml -y $(puppet config print yamldir)/facts/vm.yaml ossec::client::ossec_server_ip ::environment=development

The above assumes the certname of your node is actually "vm", it's probably not, so change the path to your node's YAML Facts cache.

Thanks Luke, while I did not mention it in my original post, I had indeed confirmed that hiera was getting the configuration value ok:

hiera --debug -c /etc/puppetlabs/code/hiera.yaml ossec::client::ossec_server_ip environment=development
DEBUG: 2015-09-09 15:10:15 +0000: Hiera YAML backend starting
DEBUG: 2015-09-09 15:10:15 +0000: Looking up ossec::client::ossec_server_ip in YAML backend
DEBUG: 2015-09-09 15:10:15 +0000: Looking for data source common
DEBUG: 2015-09-09 15:10:15 +0000: Found ossec::client::ossec_server_ip in common
<ip_address>

(I get the same proper result using the -y option in the hiera command line as well.)

So, hiera is getting the proper result.

[Todd C]

On Wednesday, September 9, 2015 at 7:28:40 AM UTC-6, jcbollinger wrote:

Are you certain that the apt and motd classes are correctly receiving their parameters from Hiera?  If their definitions happen to provide default values for all parameters, then a failure to look up parameter values in hiera would not cause catalog building to fail for them.  It could be that Hiera is misconfigured, so that it's not providing any data.

I am certain that the apt and motd classes are correctly receiving the parameters from hiera. I confirmed it numerous times by updating the hiera config with different values, doing the puppet agent run and confirm the changes made in hiera came through to the agent. I needed to ensure I wasn't going insane. :-)

Have you restarted the master since last updating the data?  I'm never sure any more what exactly Puppet caches, or when, but you can ensure that its caches are flushed by restarting the master (the service, not necessarily the whole machine).

I have restarted the master, yes, a number of times. I'm in the same boat, not quite sure when the master needs to be restarted, so I've been restarting it to make sure to cover all my bases.
 

Have you looked in the master's logs for diagnostics that might be relevant?

This I did not do. And I just did, and sure enough, there are some other (possibly relevant) errors showing up in the puppetserver.log.

==> puppetserver.log <==
2015-09-09 15:11:52,333 INFO  [puppet-server] Puppet Caching node for puppet.development.vm
2015-09-09 15:11:53,286 WARN  [puppet-server] Puppet You cannot collect exported resources without storeconfigs being set; the export is ignored
2015-09-09 15:11:53,448 ERROR [puppet-server] Puppet Evaluation Error: Error while evaluating a Function Call, ossec::agentkey:  is missing
2015-09-09 15:11:53,448 ERROR [puppet-server] Puppet Evaluation Error: Error while evaluating a Function Call, ossec::agentkey:  is missing at /etc/puppetlabs/code/environments/development/modules/ossec/manifests/agentkey.pp:8:20 on node puppet.development.vm
2015-09-09 15:11:53,449 ERROR [puppet-server] Puppet Evaluation Error: Error while evaluating a Function Call, ossec::agentkey:  is missing

Thanks for the pointer, I'll investigate this further! Hopefully this is the failure causing the problem. I might get more help from the author on this one too, then.

John

[Todd C]

 

Have you looked in the master's logs for diagnostics that might be relevant?

This I did not do. And I just did, and sure enough, there are some other (possibly relevant) errors showing up in the puppetserver.log.

==> puppetserver.log <==
2015-09-09 15:11:52,333 INFO  [puppet-server] Puppet Caching node for puppet.development.vm
2015-09-09 15:11:53,286 WARN  [puppet-server] Puppet You cannot collect exported resources without storeconfigs being set; the export is ignored
2015-09-09 15:11:53,448 ERROR [puppet-server] Puppet Evaluation Error: Error while evaluating a Function Call, ossec::agentkey:  is missing
2015-09-09 15:11:53,448 ERROR [puppet-server] Puppet Evaluation Error: Error while evaluating a Function Call, ossec::agentkey:  is missing at /etc/puppetlabs/code/environments/development/modules/ossec/manifests/agentkey.pp:8:20 on node puppet.development.vm
2015-09-09 15:11:53,449 ERROR [puppet-server] Puppet Evaluation Error: Error while evaluating a Function Call, ossec::agentkey:  is missing

Thanks for the pointer, I'll investigate this further! Hopefully this is the failure causing the problem. I might get more help from the author on this one too, then.

Hmmm, the error in the log is now the error I'm getting on my agent when doing the "puppet agent --test --noop" (it's no longer complaining about the missing ossec_server_ip value). I've obviously changed something, now I wonder if I'm further along or further behind. :-)

Anyway, I think my next step will be to post a ticket for the author of the module on github for this new error.

[Todd C]

Just to close the loop on this (for anyone interested), originally I was failing with my hiera-fu. Once I got the error about missing the ossec_server_ip value sorted, the second problem (Error while evaluating a Function Call, ossec::agentkey: is missing) seemed to be related to the way the fail() function works. I rewrote that bit and submitted a pull request to the author (along with other enhancements to the module).