Stupid hiera.yaml wildcard question.

[Robert Davidson]

Note: We're still running puppet 3.6.2 and hiera 1.3.

What I would like to do is to have hiera load all yaml files inside a directory - ie, have all the profile specific data split up into independent files sitting inside a 'profile' directory, without having to tell hiera to load them individually. This would purely be for ease of working with data for each profile module, rather than having all data changes committed against a single profile.yaml file.

Is that a thing that can be done? I don't see a way to do it, but it seems so staggeringly obvious a thing to want to do that I'm sure I'm missing something.

--
Robert Davidson

[Christopher Wood]

On Mon, Jun 01, 2015 at 08:12:33PM +0000, Robert Davidson wrote:
> Note: We're still running puppet 3.6.2 and hiera 1.3.
>
> What I would like to do is to have hiera load all yaml files inside a directory - ie, have all the profile specific data split up into independent files sitting inside a 'profile' directory, without having to tell hiera to load them individually. This would purely be for ease of working with data for each profile module, rather than having all data changes committed against a single profile.yaml file.

I have never heard of such a thing, but you could probably scribble together your own hiera backend to do this. Or copy something from the net:

https://ask.puppetlabs.com/question/1149/using-wildcards-in-hierayaml/
https://ask.puppetlabs.com/question/11151/merging-multiple-files-with-hiera/

(I searched for (no quotes) "hiera multiple files". If you read those links there are always caveats.)

> Is that a thing that can be done? I don't see a way to do it, but it seems so staggeringly obvious a thing to want to do that I'm sure I'm missing something.

It sounds like more pain than it's worth, figuring out which hiera key to use in two dimensions of lookup instead of one.

>
> --
> Robert Davidson
>
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/1EE73329D6577F44A3C2FB0F7D4ACAE98CEE3058%40MBX-02.WIN.NOMINUM.COM.
> For more options, visit https://groups.google.com/d/optout.

[Philip Wigg]

I think what you're suggesting might have been rejected in the past
(i.e. a * wildcard in hiera.yaml):-

https://tickets.puppetlabs.com/browse/HI-120

Are you using a custom fact for your roles? Maybe something like the
hiera.yaml example is a better solution anyway?

http://rnelson0.com/2014/07/14/intro-to-roles-and-profiles-with-puppet-and-hiera/

Regards,
Phil.

[Robert Davidson]

On Monday, June 01, 2015 at 1:48 PM, Christopher Wood <christop...@pobox.com> said:

*snip*

> It sounds like more pain than it's worth, figuring out which hiera key to use
> in two dimensions of lookup instead of one.

It wouldn't be two dimensions, though. As it currently stands, my profile.yaml file contains keys that look like so:

# Logrotate Profile
profile::logrotate::package::present:
'logrotate':
'ensure': 'present'
profile::logrotate::serviceconf: "/etc/logrotate.conf"
profile::logrotate::servicefile: "logrotate.conf"
profile::logrotate::cronloc: "/etc/cron.daily/logrotate"
profile::logrotate::cronfile: "logrotate.cron"

# NTP Profile
profile::ntp::servicename: "ntpd"
profile::ntp::serviceconf: "/etc/ntp.conf"
profile::ntp::servicefile: "ntp.conf"
profile::ntp::package::present:
'ntp':
'ensure': 'present'
'ntpdate':
'ensure': 'present'

. . . with a whole bunch more profile::$MODULE::$KEY entries for other profile modules following that. (We're making an extremely concerted effort to minimize the amount of data built into our modules, due to past experience with how much of a pain in the ass that becomes.)

What I would rather have is a profile directory in my hiera.yaml, where all the files there are loaded automatically by hiera, so I can have profile/logrotate.yaml with the logrotate keys, profile/ntp.yaml with ntp related keys, and so on down the line. This would make it simpler to keep things clear between different modules when setting the assorted defaults.

Unless there's a dead simple way to set a fact that will load a a yaml associated with each assigned class, which would make everything so much easier. . .


--
Robert Davidson

[jamese]

If you use calling_class as part of the hierarchy in your hiera.yaml file, you can have files named after the calling class.

https://docs.puppetlabs.com/hiera/1/puppet.html#special-pseudo-variables

This would allow you to create a yaml file named after each class in question.

You can also use calling_module, but that probably wouldn't give the granularity that you want.

[jcbollinger]

Yes, one of these is the standard way to approach such problems, provided that you're not looking to provide for cross-module data directly via Hiera.  You use the calling_class or calling_module variable to define one or more levels of your hierarchy (i.e. in hiera.yaml), so that lookups choose the correct data file for their context.  Note that this is actually more efficient than what you asked: instead of loading all the data files, it will load exactly the one it needs.


John

[Craig Dunn]

+1 to calling_module - I nearly always use this approach when using
the YAML back end for Hiera. You are right, it comes with the
limitation that I cant look up tango::bar from class foo without some
very nasty hackery, but this isn't such a bad thing. If you are
relying on this kind of cross-module data sharing then you probably
have a larger design issue, modules that require data from other
modules should get them by including the class and referencing them in
Puppet ($::tango::bar), not circumventing the module and going
straight to Hiera, thus keeping any logic (such as params.pp defaults)
that the module provides.

In a small number of cases I've required a variable from a class that
is not included, and therefore this pattern failed - but in all of
those cases a small re-think revealed a few design flaws and things
were re-factored.

Craig

--
Enviatics | Automation and configuration management
http://www.enviatics.com | @Enviatics
Puppet Training http://www.enviatics.com/training/

[Robert Davidson]

On Tuesday, June 02, 2015 at 12:55 AM, jamese <james.e...@gmail.com> sent:

Hmm. That had been something that I looked at, but this bit in the documentation:
"Note that these variables are effectively local scope, as they are pseudo-variables that only exist within the context of a specific class, and only inside of hiera. Therefore, they must be called as %{variable_name} and never %{::variable_name}. They are not top-scope."

Followed by:
"Do not use local Puppet variables in Hiera’s hierarchy or data sources. Only use facts and ENC-set top-scope variables."

Led me to discard them as useful in constructing a hiera.yaml hierarchy. From your response (and the replies from Craig and John) it appears that I was incorrect to do so. Thanks!


--
Robert Davidson