Puppet 3.8 to 4.2 migration issue...

[Jakov Sosic]

Hi guys,

I'm having problems migrating from 3.8 to 4.2...


# puppet agent -tv --tags snmp --noop
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: Evaluation Error: Error while evaluating a Function Call, Error
from DataBinding 'hiera' while looking up 'yum::conf::stage':
(<unknown>): sequence entries are not allowed here at line 13 column 25
at
/etc/puppetlabs/code/environments/production/modules/yum/manifests/init.pp:11:3
on node kickstart.neogov.net
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run


# cat /etc/puppetlabs/code/hiera.yaml
---
:backends:
- yaml
:yaml:
:datadir: "/etc/puppetlabs/code/environments/%{environment}/hieradata"
:hierarchy:
- "host/%{::fqdn}"
- "domain/%{::domain}"
- useraccounts
- "osfamily/%{::family}"
- common



Now, if I state a wrong hiera directory, for example hiera_data, my
manifests start working again, but without Hiera, agent wants to make
changes to defaults...

So, my question is - any ideas why do I get this error?

[Jakov Sosic]

Also if I disable the class that's mentioned in the error, next class in
the manifest bombs out:

Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: Evaluation Error: Error while evaluating a Function Call, Error

from DataBinding 'hiera' while looking up 'centos::iptables::source': ...



This is centos::iptables::source:

class centos::iptables (
$source = 'puppet:///modules/centos/ipv4_iptables',
){
...
}


Any ideas?

[R.I.Pienaar]

----- Original Message -----
> From: "Jakov Sosic" <jso...@gmail.com>
> To: "puppet-users" <puppet...@googlegroups.com>
> Sent: Tuesday, October 13, 2015 3:48:24 PM
> Subject: [Puppet Users] Re: Puppet 3.8 to 4.2 migration issue...

> On 10/13/2015 04:45 PM, Jakov Sosic wrote:
>> Hi guys,
>>
>> I'm having problems migrating from 3.8 to 4.2...
>>
>>
>> # puppet agent -tv --tags snmp --noop
>> Info: Retrieving pluginfacts
>> Info: Retrieving plugin
>> Info: Loading facts
>> Error: Could not retrieve catalog from remote server: Error 400 on
>> SERVER: Evaluation Error: Error while evaluating a Function Call, Error
>> from DataBinding 'hiera' while looking up 'yum::conf::stage':
>> (<unknown>): sequence entries are not allowed here at line 13 column 25
>> at
>> /etc/puppetlabs/code/environments/production/modules/yum/manifests/init.pp:11:3
>> on node kickstart.neogov.net
>> Warning: Not using cache on failed catalog
>> Error: Could not retrieve catalog; skipping run

The YAML parser in the newer ruby has changed and implements the spec a bit differently
so this error is about malformed YAML - which could indeed have worked before.

You can test all your yamls like:

for i in $(find /etc/puppetlabs/code/hieradb -name \*.yaml)
do
/opt/puppetlabs/puppet/bin/ruby -r yaml -e "YAML.load(File.read('$i')) && puts('$i: OK') rescue puts('$i: NOT OK')"
done

change /etc/puppetlabs/code/hieradb to your hiera dir

[Jakov Sosic]

On 10/13/2015 05:07 PM, R.I.Pienaar wrote:

> You can test all your yamls like:
>
> for i in $(find /etc/puppetlabs/code/hieradb -name \*.yaml)
> do
> /opt/puppetlabs/puppet/bin/ruby -r yaml -e "YAML.load(File.read('$i')) && puts('$i: OK') rescue puts('$i: NOT OK')"
> done
>
> change /etc/puppetlabs/code/hieradb to your hiera dir

Yeah that's it!
I've fixed my yamls and everything is fine on that ground now.


But, I have new issues:

Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: Evaluation Error: Error while evaluating a Function Call,

undefined method `sub' for
Logstash::Configfile[]{:name=>""}:Puppet::Resource at
/etc/puppetlabs/code/environments/production/modules/helpers/manifests/logstash.pp:19:28
on node xxx


This is the snippet:

include ::logstash

# load configs from hiera
$logstash_configs = hiera_hash('logstash::configfiles', undef)
if ($logstash_configs) { create_resources(::Logstash::Configfile,
$logstash_configs) }
$logstash_patterns = hiera_hash('logstash::patternfiles', undef)
if ($logstash_patterns) { create_resources(::Logstash::Patternfile,
$logstash_patterns) }


And this is from hiera:

logstash::configfiles:
'input_header':
content: "input {\n"
order: 11
'input_postfix':
template: 'helpers/logstash/inputs/mongodb.erb'
order: 12
...


I have general problem with importing hashes from yaml in manifests...
It's bombing out on other stuff too...

[Jakov Sosic]

On 10/14/2015 02:06 PM, Jakov Sosic wrote:

> # load configs from hiera
> $logstash_configs = hiera_hash('logstash::configfiles', undef)
> if ($logstash_configs) { create_resources(::Logstash::Configfile,
> $logstash_configs) }
> $logstash_patterns = hiera_hash('logstash::patternfiles', undef)
> if ($logstash_patterns) { create_resources(::Logstash::Patternfile,
> $logstash_patterns) }
>
>
> And this is from hiera:
>
> logstash::configfiles:
> 'input_header':
> content: "input {\n"
> order: 11
> 'input_postfix':
> template: 'helpers/logstash/inputs/mongodb.erb'
> order: 12

Ah, I only lowered case and it seems it works:


if ($logstash_configs) { create_resources(::logstash::configfile,
$logstash_configs) }

[Thomas Müller]

I would recommend you to run puppet-lint against all your manifests. It will detect other possible problems.

- Thomas