Puppet3 | hiera | role -> Problem role/mysql.yaml not used -> example from rnelson0.com with mysql

[Denny Fuchs]

hi,

I try to understand the example from https://rnelson0.com/2015/02/02/deploying-mysql-with-puppet-without-disabling-selinux/comment-page-1/ with MySQL.

At this moment, most parts are working, except role/mysql.yaml . It seems, that this file isn't noticed by hiera/puppet:

:yaml:
  :datadir: "/etc/puppet/environments/%{environment}/hieradata"

:hierarchy:
  - "node/%{::fqdn}"
  - "role/%{::role}"
  - "profile/%{::profile}"
  - "operatingsystem/%{::operatingsystem}/%{::operatingsystemmajrelease}"
  - "operatingsystem/%{::operatingsystem}"
  - "osfamily/%{::osfamily}"
  - "datacenter/%{::datacenter}"
  - "virtual/%{::is_virtual}"
  - common

All other files are copied from the above example, except: hiera/puppet_role/ -> hieradata/role/

The problem is (puppet agent -t on the example host):

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Must pass backupuser to Class[Mysql::Server::Backup] at /opt/puppet/environments/test/modules/profile/manifests/mysql/server.pp:12 on node mon.example.com
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

What is wrong, so that /opt/puppet/environments/test/hieradata/role/mysql.yaml isn't working on the Puppet master.

• ./manifests/site.pp

hiera_include('classes', '')
node default {
  $role = hiera('role')
  include $role
}

• manifests/mon.example.com.pp

node 'mon.example.com' {
  include  role::mysql_server
}

• ...test/hieradata/node/mon.example.com.yaml

# Mon
---
classes: apt
apt::mirror: http://debianmirror.example.bla/pub/linux/debian/debian/
apt::source: true

---
mysql::server::override_options :
  'client':
[...]

Puppet Version is 3.8.7-1puppetlabs1 on Debian 7.11

Any suggestions?

cu denny

[Rob Nelson]

Denny,

Have you tried using the hiera command line utility to see what it returns for that key? `hiera mysql::server::backup::backupuser ::environment=production ::role=mysql --trace` should provide some clues as to how hiera is behaving and if it can find the right files. If that doesn't find anything, try `environment` and/or `role` (without the ::) as hiera does not normalize that, and sometimes it doesn't behave the way you would expect. You can also try looking up a key that is present in `common.yaml`, just to make sure you have the syntax right.

[Denny Fuchs]

hi Rob,

I had to use a slightly different   command:

hiera -c /etc/puppet/hiera.yaml mysql::server::backup::backupuser environment=test ::role=mysql_server --debug

[...]
DEBUG: 2016-12-21 19:14:39 +0100: Hiera JSON backend starting
DEBUG: 2016-12-21 19:14:39 +0100: Looking up mysql::server::backup::backupuser in JSON backend
DEBUG: 2016-12-21 19:14:39 +0100: Looking for data source role/mysql_server
DEBUG: 2016-12-21 19:14:39 +0100: Cannot find datafile /etc/puppet/hiera/role/mysql_server.json, skipping
DEBUG: 2016-12-21 19:14:39 +0100: Looking for data source common
DEBUG: 2016-12-21 19:14:39 +0100: Cannot find datafile /etc/puppet/hiera/common.json, skipping
DEBUG: 2016-12-21 19:14:39 +0100: Hiera YAML backend starting
DEBUG: 2016-12-21 19:14:39 +0100: Looking up mysql::server::backup::backupuser in YAML backend
DEBUG: 2016-12-21 19:14:39 +0100: Looking for data source role/mysql_server
DEBUG: 2016-12-21 19:14:39 +0100: Found mysql::server::backup::backupuser in role/mysql_server
dbbackup

So, that seems to be working, so something else must be wrong in my config files.

cu denny

[Rob Nelson]

Starting with the simplest of solutions, have you restarted the master service after updating hiera.yaml? If not, it may be using an older in-memory version of hiera.yaml.

Rob Nelson
rnel...@gmail.com

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/68e1bc79-6709-47a3-a3e3-7a9cbd483a84%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Denny Fuchs]

hi,

yep, restarting after every change.

[Rob Nelson]

I guess the next step is to try and run puppet master service in debug mode. I haven't done that in forever and certainly not on v3. It looks like https://docs.puppet.com/puppet/latest/services_master_webrick.html#run-puppet-master-on-the-command-line may be helpful there. You can also check on the agent with `facter -p role` to make sure it's receiving the role correctly through your custom fact.

Rob Nelson
rnel...@gmail.com

On Wed, Dec 21, 2016 at 2:21 PM, Denny Fuchs <denny....@gmail.com> wrote:

hi,

yep, restarting after every change.

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c1bc2d00-8c3e-411f-b8be-cf1136ddc320%40googlegroups.com.

[Denny Fuchs]

hi Rob,

maybe it helps: I found something in the Puppet server logs, while calling the agent on the node:

[...]
2016-12-21 20:59:16,496 INFO  [puppet-server] Puppet Caching node for mon.example.com
2016-12-21 20:59:17,236 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::role';
2016-12-21 20:59:17,236 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::profile';
2016-12-21 20:59:17,238 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::datacenter';
2016-12-21 20:59:17,239 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::role';
2016-12-21 20:59:17,240 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::profile';
2016-12-21 20:59:17,241 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::datacenter';
2016-12-21 20:59:17,243 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::role';
2016-12-21 20:59:17,243 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::profile';
2016-12-21 20:59:17,245 WARN  [puppet-server] Scope(Class[main]) Could not look up qualified variable '::datacenter';
2016-12-21 20:59:18,406 ERROR [puppet-server] Puppet Must pass backupuser to Class[Mysql::Server::Backup] at /opt/puppet/environments/test/modules/profile/manifests/mysql/server.pp:12 on node mon.example.com
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in `fail'
/usr/lib/ruby/vendor_ruby/puppet/resource.rb:482:in `validate_complete'
org/jruby/RubyHash.java:1342:in `each'
[...]

cu denny

[John Gelnaw]

On Wednesday, December 21, 2016 at 11:37:38 AM UTC-5, Denny Fuchs wrote:

hi,

I try to understand the example from https://rnelson0.com/2015/02/02/deploying-mysql-with-puppet-without-disabling-selinux/comment-page-1/ with MySQL.

At this moment, most parts are working, except role/mysql.yaml . It seems, that this file isn't noticed by hiera/puppet:

:yaml:
  :datadir: "/etc/puppet/environments/%{environment}/hieradata"

:hierarchy:
  - "node/%{::fqdn}"
  - "role/%{::role}"
  - "profile/%{::profile}"
  - "operatingsystem/%{::operatingsystem}/%{::operatingsystemmajrelease}"
  - "operatingsystem/%{::operatingsystem}"
  - "osfamily/%{::osfamily}"
  - "datacenter/%{::datacenter}"
  - "virtual/%{::is_virtual}"
  - common

Everything you're using in your hiera.yaml is OK except the role line-- "role" is not a fact (unless you have a custom fact defining it), or a top-level variable, so you'll need to define it before any hiera calls that might use it.

  ./manifests/site.pp

hiera_include('classes', '')
node default {
  $role = hiera('role')
  include $role
}




Here, you define $role, but you're pulling it from hiera as well-- where is this variable defined?




I suspect that if you add:

$role = 'mysql'

to your role/manifests/mysql_server.pp file, that things may improve-- but you're also declaring it in your default node definition (or trying to), and one thing puppet doesn't tolerate well is defining the same variable twice.

[John Gelnaw]

Hate replying to myself.  As you noticed, "profile" and "datacenter" are also not defined.

[vigi...@yahoo.com]

--------------------------------------------
On Wed, 12/21/16, Denny Fuchs <denny....@gmail.com> wrote:

Subject: Re: [Puppet Users] Re: Puppet3 | hiera | role -> Problem role/mysql.yaml not used -> example from rnelson0.com with mysql
To: "Puppet Users" <puppet...@googlegroups.com>
Date: Wednesday, December 21, 2016, 10:35 PM

--

You received this message because you are subscribed to the
Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails

from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1140e0f8-ee1a-4fe8-b3b8-3b33dbe1f954%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

ntinuare si el pe pozitii de dreapta. in cadrul L.A.N.C. s-a conturat o grupare

[Denny Fuchs]

hi,

thanks for the reply :-)

I added a custom fact via modules/role/lib/facter/role.rb

Facter.add(:role) do
   setcode do
     host = Facter.value(:fqdn)
     declared_role = Facter.value(host)
     declared_role.nil? ? 'base' : declared_role
   end
end

I copied that piece of code and it works, if I change "base" to "mysql_server". In the end, I can execute on the mon.example.com "facter -p role" and I get "mysql_server" back. So far so good. I saw many examples how to set the fact, but mostly I dislike them (e.g. create file /etc/system_role ... hardcoded in the Facter.add .... ). What I miss: how can I set the fact (role in my case) via hiera, or via node manifest ?


cu denny

[Rob Nelson]

Adding the fact to a module is a recommended method, which is what you are doing. That file will get synced to a client first, then facts are collected, then the catalog is generated using those facts, so this ensures the correct order of operations.

The only thing you may want to tweak is what module the fact is in. I put most of mine in my profile module since that's part of my controlrepo, but you could make a personal_facts module if you wanted to consolidate multiple custom facts in one place. I wouldn't prematurely optimize this, it's easy to change later.

--


You received this message because you are subscribed to the Google Groups "Puppet Users" group.


To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/2840db66-1c91-43c9-8e96-47f52b38f7e6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson

[armstron...@yahoo.com]

--------------------------------------------

On Tue, 12/27/16, Rob Nelson <rnel...@gmail.com> wrote:

Subject: Re: [Puppet Users] Re: Puppet3 | hiera | role -> Problem role/mysql.yaml not used -> example from rnelson0.com with mysql

To: puppet...@googlegroups.com
Date: Tuesday, December 27, 2016, 7:55 PM

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAC76iT_XWnDqK4Hzx-F49tcjYMpjvcfGRwo%2BVKhsLM3f1xiF7w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

n a doua jumatate a anului 1838 Alexandru Ghica a plecat din tara tentia declarata de a-si ingriji sanatatea in strainatate. in absenta sa din se anunta la 20 ianuarie 1839 ca o parte dintre boieri sunt hotarati sa-l ja domn pe Ion Campineanu candidatura insa repede abandonata in lipsa imitatii.

[armstron...@yahoo.com]

--------------------------------------------

On Wed, 12/28/16, armstrongjerry893 via Puppet Users <puppet...@googlegroups.com> wrote:

Subject: Re: [Puppet Users] Re: Puppet3 | hiera | role -> Problem role/mysql.yaml not used -> example from rnelson0.com with mysql
To: puppet...@googlegroups.com

Date: Wednesday, December 28, 2016, 5:03 AM

https://groups.google.com/d/msgid/puppet-users/287363314.1790332.1482894190612%40mail.yahoo.com.
For more options, visit
https://groups.google.com/d/optout.bia dupa 1867 hcrurile s-au schimbat radical ambele categorii de >prietan - satenii pe de o parte proprietarii de mosii pe de alta - acum aptati noii situatii au valorificat mai bine pamantul. Din sporul de productie -e a determinat si importante cresteri ale exportului s-au putut face acumulari nnificative - p=irte intrebuintate' pentru achizitia de masini agricole parte ntru cresterea investitiilor interne. Drumul spre societatea moderna deschis :a la inceputul secolului a fost astfel mult inlesnit.

[vigi...@yahoo.com]

https://groups.google.com/d/optout.atita sa progreseze era permanent impiedicata de institutii si o clasa politica

[Denny Fuchs]

Hi Rob,

I got it working :-) I don't know at this time, if the layout is quite OK, but for the moment ....

hieradata/node/mon.example.com.yaml
hieradata/role/mon.yaml
hieradata/global.yaml

thanks a lot for the great help :-)

I used also the role/profile howto from you :-)

cu denn

[Rob Nelson]

It looks like a great start to me. Glad the howto helped!

--


You received this message because you are subscribed to the Google Groups "Puppet Users" group.


To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/06466f47-ef76-40d9-b61c-a99e3ac01f3e%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Nelson