Hiera 4 problems

[Jonathan Gazeley]

I'm still setting up my new PE 2016.4 installation and I'm running into
a problem with Hiera data per environment. I followed these steps:

https://docs.puppet.com/puppet/4.7/reference/lookup_quick.html#if-you-already-use-hiera-in-environments

Every Puppet run errors with:

Error: Could not retrieve catalog from remote server: Error 500 on
SERVER: Server Error: Evaluation Error: Error while evaluating a
Function Call, Lookup of key 'classes' failed: Cannot load backend
module_data: no such file to load -- hiera/backend/module_data_backend
in Cannot load backend module_data: no such file to load --
hiera/backend/module_data_backend at
/etc/puppetlabs/code/environments/pe/manifests/site.pp:32:1 on node
puppet4-prod.resnet.bris.ac.uk

site.pp:32 contains:

lookup('classes', Array[String], 'unique').include

My per-environment hiera.yaml contains:

---
version: 4
datadir: data

hierarchy:
- name: "Nodes"
backend: yaml
path: "node/%{trusted.certname}"

- name: "Operating System"
backend: yaml
path: "operatingsystem/%{operatingsystem}"

- name: "Operaring System Release"
backend: yaml
path: "operatingsystem/%{operatingsystem}/%{operatingsystemmajrelease}"

- name: "Virtual"
backend: yaml
path: "virtual/%{virtual}"

- name: "Manufacturer"
backend: yaml
path: "manufacturer/%{manufacturer}"

- name: "Module"
backend: yaml
path: "module/%{calling_module}"

- name: "Common"
backend: yaml
path: "common"


The file data/node/puppet4-prod.resnet.bris.ac.uk.yaml contains:

---
classes:
- 'uob_pe'


So there should definitely be a value for "classes". I tried a command
to give some debug output:

[root@puppet4-prod ~]# puppet lookup classes --node
puppet4-prod.resnet.bris.ac.uk --explain
Error: Could not run: Evaluation Error: Error while evaluating a
Resource Statement, Evaluation Error: Error while evaluating a Method
call, 'dig' parameter 'data' expects a Collection value, got String at
/opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42

That manifest ships with PE and I haven't messed with it. Anyone know
what's going on?

Thanks!
Jonathan

--
Jonathan Gazeley
Senior Systems Administrator
IT Services
University of Bristol

[R.I.Pienaar]

----- Original Message -----
> From: "Jonathan Gazeley" <Jonathan...@bristol.ac.uk>
> To: "puppet-users" <puppet...@googlegroups.com>
> Sent: Wednesday, 30 November, 2016 16:59:16
> Subject: [Puppet Users] Hiera 4 problems

> I'm still setting up my new PE 2016.4 installation and I'm running into
> a problem with Hiera data per environment. I followed these steps:
>
> https://docs.puppet.com/puppet/4.7/reference/lookup_quick.html#if-you-already-use-hiera-in-environments
>
> Every Puppet run errors with:
>
> Error: Could not retrieve catalog from remote server: Error 500 on
> SERVER: Server Error: Evaluation Error: Error while evaluating a
> Function Call, Lookup of key 'classes' failed: Cannot load backend
> module_data: no such file to load -- hiera/backend/module_data_backend
> in Cannot load backend module_data: no such file to load --
> hiera/backend/module_data_backend at
> /etc/puppetlabs/code/environments/pe/manifests/site.pp:32:1 on node
> puppet4-prod.resnet.bris.ac.uk
>

you need to completely remove the old ripienaar-module_data backend as
its not 4 compatible - looks like this is what is happening here

if you do use it, you can adjust your modules before hand to be puppet 4
compatible by adding a puppet 4 hiera.yaml for its own module data feature

[Henrik Lindberg]

On 30/11/16 16:59, Jonathan Gazeley wrote:
> I'm still setting up my new PE 2016.4 installation and I'm running into
> a problem with Hiera data per environment. I followed these steps:
>
> https://docs.puppet.com/puppet/4.7/reference/lookup_quick.html#if-you-already-use-hiera-in-environments
>
>
> Every Puppet run errors with:
>
> Error: Could not retrieve catalog from remote server: Error 500 on
> SERVER: Server Error: Evaluation Error: Error while evaluating a
> Function Call, Lookup of key 'classes' failed: Cannot load backend
> module_data: no such file to load -- hiera/backend/module_data_backend
> in Cannot load backend module_data: no such file to load --
> hiera/backend/module_data_backend at
> /etc/puppetlabs/code/environments/pe/manifests/site.pp:32:1 on node
> puppet4-prod.resnet.bris.ac.uk
>

You are using the "hack" that R.I wrote that has been superseded by the
"data in modules" provided by puppet.

That is : this thing
https://github.com/ripienaar/puppet-module-data/blob/master/lib/hiera/backend/module_data_backend.rb

That should not be used at all.

The above does not work. It is not how it should be done.
You should place the data in the module instead.
You cannot vary what is bound to a key based on the caller.
(super bad idea that works in hiera 3).

> - name: "Common"
> backend: yaml
> path: "common"
>
>
> The file data/node/puppet4-prod.resnet.bris.ac.uk.yaml contains:
>
> ---
> classes:
> - 'uob_pe'
>
>
> So there should definitely be a value for "classes". I tried a command
> to give some debug output:
>
> [root@puppet4-prod ~]# puppet lookup classes --node
> puppet4-prod.resnet.bris.ac.uk --explain
> Error: Could not run: Evaluation Error: Error while evaluating a
> Resource Statement, Evaluation Error: Error while evaluating a Method
> call, 'dig' parameter 'data' expects a Collection value, got String at
> /opt/puppetlabs/puppet/modules/puppet_enterprise/manifests/master/puppetserver.pp:673:42
>
>
> That manifest ships with PE and I haven't messed with it. Anyone know
> what's going on?
>

You have old stuff that needs to be removed, and the hierarchy you have
is designed for the old hiera (the module part). Fix those things and it
should work better.

- henrik

> Thanks!
> Jonathan
>


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Jonathan Gazeley]

On 30/11/16 16:03, R.I.Pienaar wrote:
> you need to completely remove the old ripienaar-module_data backend as
> its not 4 compatible - looks like this is what is happening here
>
> if you do use it, you can adjust your modules before hand to be puppet 4
> compatible by adding a puppet 4 hiera.yaml for its own module data feature

Yes, I removed the ripienaar-module_data module from all branches - we
weren't using it anyway. I checked that it is not being referenced in
any of my manifests. I can't find any reference to module_data_backend
with a recursive grep in /etc/puppetlabs.

Any more suggestions?

Many thanks,

[R.I.Pienaar]

----- Original Message -----
> From: "Jonathan Gazeley" <Jonathan...@bristol.ac.uk>
> To: "puppet-users" <puppet...@googlegroups.com>

> Sent: Wednesday, 30 November, 2016 17:16:50
> Subject: Re: [Puppet Users] Hiera 4 problems

> On 30/11/16 16:03, R.I.Pienaar wrote:
>> you need to completely remove the old ripienaar-module_data backend as
>> its not 4 compatible - looks like this is what is happening here
>>
>> if you do use it, you can adjust your modules before hand to be puppet 4
>> compatible by adding a puppet 4 hiera.yaml for its own module data feature
>
> Yes, I removed the ripienaar-module_data module from all branches - we
> weren't using it anyway. I checked that it is not being referenced in
> any of my manifests. I can't find any reference to module_data_backend
> with a recursive grep in /etc/puppetlabs.
>
> Any more suggestions?
>

whats in your classic hiera.yaml?

[Jonathan Gazeley]

On 30/11/16 16:17, Henrik Lindberg wrote:
>>
>> - name: "Module"
>> backend: yaml
>> path: "module/%{calling_module}"
>>
>
> The above does not work. It is not how it should be done.
> You should place the data in the module instead.
> You cannot vary what is bound to a key based on the caller.
> (super bad idea that works in hiera 3).

Thanks - I removed this block and it now works! Onto the next bug :)

Cheers,