Jira (PUP-9965) can't use Block Style Indicator in nodes description

[Lukasz (JIRA)]

Lukasz created an issue

Puppet / PUP-9965 can't use Block Style Indicator in nodes description Issue Type: Bug Assignee: Unassigned Created: 2019/08/13 2:56 PM Priority: Normal Reporter: Lukasz Puppet Version: 5.5.16 Puppet Server Version: 5.5.16 OS Name/Version: ubuntu xenial I tried to use hiera variable which is in format: --- site::roles::puppet::rsync_ssh_key: " -----BEGIN OPENSSH PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx -----END OPENSSH PRIVATE KEY----- " site::roles::puppet::gitlab_ssh_key: " -----BEGIN RSA PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxx -----END RSA PRIVATE KEY----- this format works in 4.10. Desired Behavior: I want to work this format in new puppet Actual Behavior: # puppet agent -t Info: Using configured environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Retrieving locales Info: Loading facts Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Lookup of key 'lookup_options' failed: Unable to parse (/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml): could not find expected ':' while scanning a simple key at line 13 column 1 on node puppet-01.example.com Warning: Not using cache on failed catalog Error: Could not retrieve catalog; skipping run when I run puppet lookup I get following error: puppet lookup --explain site::roles::puppet::rsync_ssh_key Error: Could not run: Lookup of key 'lookup_options' failed: Unable to parse (/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml): could not find expected ':' while scanning a simple key at line 3 column 1 but if I change format to and variable looks like this: site::roles::puppet::rsync_ssh_key: " -----BEGIN OPENSSH PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx -----END OPENSSH PRIVATE KEY----- " puppet lookup (and puppet agent -t) works: puppet lookup --explain site::roles::puppet::rsync_ssh_key Searching for "lookup_options" Global Data Provider (hiera configuration version 5) Using configuration "/etc/puppetlabs/puppet/hiera.yaml" Merge strategy hash Hierarchy entry "Per-node data (yaml version)" Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml" Original path: "nodes/%{trusted.certname}.yaml" No such key: "lookup_options" Hierarchy entry "Other YAML hierarchy levels" Merge strategy hash Path "/etc/puppetlabs/code/environments/production/hieradata/roles/KVM.yaml" Original path: "roles/%{facts.role}.yaml" Path not found Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml" Original path: "common.yaml" No such key: "lookup_options" Module "site" not found Searching for "site::roles::puppet::rsync_ssh_key" Global Data Provider (hiera configuration version 5) Using configuration "/etc/puppetlabs/puppet/hiera.yaml" Hierarchy entry "Per-node data (yaml version)" Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml" Original path: "nodes/%{trusted.certname}.yaml" Found key: "site::roles::puppet::rsync_ssh_key" value: " -----BEGIN OPENSSH PRIVATE KEY----- xxxx deleted content xxxxxxxxxxx -----END OPENSSH PRIVATE KEY----- "   what is proper syntax for such variables in hiera ? Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Lukasz (JIRA)]

Lukasz updated an issue

Puppet / PUP-9965 can't use Block Style Indicator in nodes description

Change By: Lukasz *Puppet Version: 5.5.16* *Puppet Server Version: 5.5.16* *OS Name/Version: ubuntu xenial*

I tried to use hiera variable which is in format:

{code:java}

--- site::roles::puppet::rsync_ssh_key: " | -----BEGIN OPENSSH PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx -----END OPENSSH PRIVATE KEY----- " site::roles::puppet::gitlab_ssh_key: " |

-----BEGIN RSA PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxx

-----END RSA PRIVATE KEY-----{code}

this format works in 4.10.

*Desired Behavior:*

I want to work this format in new puppet

*Actual Behavior:* {code:java}

# puppet agent -t Info: Using configured environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Retrieving locales Info: Loading facts Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Lookup of key 'lookup_options' failed: Unable to parse (/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml): could not find expected ':' while scanning a simple key at line 13 column 1 on node puppet-01.example.com Warning: Not using cache on failed catalog Error: Could not retrieve catalog; skipping run

{code}

when I run puppet lookup I get following error:

{code:java}

puppet lookup --explain site::roles::puppet::rsync_ssh_key

Error: Could not run: Lookup of key 'lookup_options' failed: Unable to parse (/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml): could not find expected ':' while scanning a simple key at line 3 column 1{code}

but if I change format to and variable looks like this:

{code:java}

site::roles::puppet::rsync_ssh_key: " -----BEGIN OPENSSH PRIVATE KEY----- xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx -----END OPENSSH PRIVATE KEY-----

"{code}

puppet lookup (and puppet agent -t) works:

{code:java}

puppet lookup --explain site::roles::puppet::rsync_ssh_key Searching for "lookup_options" Global Data Provider (hiera configuration version 5) Using configuration "/etc/puppetlabs/puppet/hiera.yaml" Merge strategy hash Hierarchy entry "Per-node data (yaml version)" Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml" Original path: "nodes/%{trusted.certname}.yaml" No such key: "lookup_options" Hierarchy entry "Other YAML hierarchy levels" Merge strategy hash Path "/etc/puppetlabs/code/environments/production/hieradata/roles/KVM.yaml" Original path: "roles/%{facts.role}.yaml" Path not found Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml" Original path: "common.yaml" No such key: "lookup_options" Module "site" not found Searching for "site::roles::puppet::rsync_ssh_key" Global Data Provider (hiera configuration version 5) Using configuration "/etc/puppetlabs/puppet/hiera.yaml" Hierarchy entry "Per-node data (yaml version)" Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet-01.example.com.yaml" Original path: "nodes/%{trusted.certname}.yaml"

Found key: "site::roles::puppet::rsync_ssh_key" value: " -----BEGIN OPENSSH PRIVATE KEY----- xxxx deleted content xxxxxxxxxxx -----END OPENSSH PRIVATE KEY----- "{code}

what is proper syntax for such variables in hiera ?

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-9965

Re: can't use Block Style Indicator in nodes description Thoughts Henrik Lindberg?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9965

Re: can't use Block Style Indicator in nodes description

The question is really "What is proper syntax for such values in YAML" - has nothing to do with hiera. The error comes from the YAML parser. Either use YAML double quotes or YAML blocks - see YAML documentation: https://yaml.org/YAML_for_ruby.html#blocks It is recommended to run a YAML linter on yaml files to ensure they have the correct syntax.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-9965

Re: can't use Block Style Indicator in nodes description

While yaml text can be broken up on multiple lines when indented it is not allowed to contain yaml tokens - in this case it contained hyphens. It must therefore be written as a quoted string or a block.

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-9965

Re: can't use Block Style Indicator in nodes description

Ah that makes sense. Here's the different psych and libyaml versions: [root@d9rsmnam0lbw8pv bin]# /opt/puppetlabs/puppet/bin/puppet --version 4.10.12 [root@d9rsmnam0lbw8pv bin]# /opt/puppetlabs/puppet/bin/ruby -ryaml -e "puts YAML::VERSION" 2.0.5 [root@d9rsmnam0lbw8pv bin]# /opt/puppetlabs/puppet/bin/ruby -ryaml -e "puts YAML::LIBYAML_VERSION" 0.1.6 Versus: [root@nhoggf2wjohsey8 ~]# /opt/puppetlabs/puppet/bin/puppet --version 5.5.2 [root@nhoggf2wjohsey8 ~]# /opt/puppetlabs/puppet/bin/ruby -ryaml -e "puts YAML::VERSION" 2.2.2 [root@nhoggf2wjohsey8 ~]# /opt/puppetlabs/puppet/bin/ruby -ryaml -e "puts YAML::LIBYAML_VERSION" 0.1.7

Add Comment

[Lukasz (JIRA)]

Lukasz commented on PUP-9965

Re: can't use Block Style Indicator in nodes description

Thank You for answers - Yours thoughts and hints was helpful, after checking by yamllint the files with these variables it showed the same error as puppet lookup - there were wrong indentations (some spurious white spaces) and that make fool on yaml interpter. And as Henrik pointed out - it wasn't related to hiera.

Add Comment