Jira (PUP-11194) Global variable hostgroup passed from foreman is not resolved in hiera.yaml

24 views
Skip to first unread message

Tejas Rao (Jira)

unread,
Jul 30, 2021, 9:24:02 PM7/30/21
to puppe...@googlegroups.com
Tejas Rao created an issue
 
Puppet / Bug PUP-11194
Global variable hostgroup passed from foreman is not resolved in hiera.yaml
Issue Type: Bug Bug
Affects Versions: PUP 7.8.0
Assignee: Unassigned
Components: Hiera & Lookup
Created: 2021/07/30 6:23 PM
Priority: Normal Normal
Reporter: Tejas Rao

Puppet Version:7.9.0-1
Puppet Server Version:7.2.1-1
OS Name/Version:RHEL7.9

hostgroup variable passed from foreman is resolved correctly in classes using $::hostgroup but not resolving in hiera.yaml (%{::hostgroup})

 

Puppet lookup complains with the below error - 

Warning: Undefined variable '::hostgroup';
(file & line not available)

With --explain , you can see that variable is not resolving. 

Hierarchy entry "Host Group Data"
Path "/etc/puppetlabs/code/environments/production/hieradata/.yaml"
Original path: "%{::hostgroup}.yaml"
Path not found

Desired Behavior: Variable should resolve correctly. 

Actual Behavior:

[root@master06 bin]# puppet lookup puppet_agent::package_version --node node22 --environment raot
Warning: Undefined variable '::hostgroup';
(file & line not available)
— 6.24.0-1.el8
[root@master06 bin]#
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Ciprian Badescu (Jira)

unread,
Aug 2, 2021, 10:13:02 AM8/2/21
to puppe...@googlegroups.com
Ciprian Badescu commented on Bug PUP-11194
 
Re: Global variable hostgroup passed from foreman is not resolved in hiera.yaml

Tejas Rao, Can you provide us a simple was to reproduce this error? Did it worked on earlier puppet versions?

Tejas Rao (Jira)

unread,
Aug 2, 2021, 10:30:11 AM8/2/21
to puppe...@googlegroups.com
Tejas Rao commented on Bug PUP-11194

It does work on our old puppet server running a very old version (puppet 3).

The puppet server was installed using the foreman installer, nothing special was done and it is a rather standard setup. 

Using the below versions - 

foreman-2.3.5-1.el7.noarch

puppet-agent-6.24.0-1.el7.x86_64
puppetserver-6.16.1-1.el7.noarch

[root@master06 ~]# hiera -V
3.7.0
[root@master06 ~]# 

 

[root@master06 ~]# cat /etc/puppetlabs/code/environments/production/hiera.yaml
version: 5
defaults: # Used for any hierarchy level that omits these keys.
datadir: hieradata # This path is relative to hiera.yaml's directory.
data_hash: yaml_data # Use the built-in YAML backend.

hierarchy:

  • name: "Per-node data" # Human-readable name.
    path: "%{trusted.certname}.yaml" 
  • name: "Host Group Data"
    path: "%{::hostgroup}.yaml"

 

Josh Cooper (Jira)

unread,
Aug 2, 2021, 5:24:03 PM8/2/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11194

Tejas Rao I suspect the issue is related to how exactly the hostgroup variable is defined. See the caution in https://puppet.com/docs/puppet/6/hiera_merging.html#interpolate_puppet_variable about "pseudo-variables"

Also I'd suggest adding the --explain option to `puppet lookup`

Tejas Rao (Jira)

unread,
Aug 3, 2021, 1:12:03 PM8/3/21
to puppe...@googlegroups.com
Tejas Rao commented on Bug PUP-11194

[root@master06 ~]# puppet lookup puppet_agent::package_version --node node1.example.com --environment production --strict_variables --trace --explain


Warning: Undefined variable '::hostgroup';
(file & line not available)

Searching for "lookup_options"
Global Data Provider (hiera configuration version 5)
No such key: "lookup_options"
Environment Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"
Merge strategy hash
Hierarchy entry "Per-node data"
Path "/etc/puppetlabs/code/environments/production/hieradata/node1.example.com.yaml"
Original path: "%{trusted.certname}.yaml"
Path not found


Hierarchy entry "Host Group Data"
Path "/etc/puppetlabs/code/environments/production/hieradata/.yaml"
Original path: "%{::hostgroup}.yaml"
Path not found

Hierarchy entry "linuxfarm Per-node data"
Path "/etc/puppetlabs/code/environments/production/linuxfarm/hieradata/node1.example.com.yaml"
Original path: "%{trusted.certname}.yaml"
Path not found
Hierarchy entry "linuxfarm Host Group Data"
Path "/etc/puppetlabs/code/environments/production/linuxfarm/hieradata/RedHat.yaml"
Original path: "%{::osfamily}.yaml"
Path not found
Hierarchy entry "Secret data (encrypted)"
Path "/etc/puppetlabs/code/environments/production/hieradata/secure.yaml"
Original path: "secure.yaml"
Path not found
Hierarchy entry "linuxfarm Common data"
Path "/etc/puppetlabs/code/environments/production/linuxfarm/hieradata/common.yaml"
Original path: "common.yaml"
Path not found
Hierarchy entry "Common Data"
Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"
Original path: "common.yaml"
No such key: "lookup_options"
Module data provider for module "puppet_agent" not found
Searching for "puppet_agent::package_version"
Global Data Provider (hiera configuration version 5)
No such key: "puppet_agent::package_version"
Environment Data Provider (hiera configuration version 5)
Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"
Hierarchy entry "Per-node data"
Path "/etc/puppetlabs/code/environments/production/hieradata/node1.example.com.yaml"
Original path: "%{trusted.certname}.yaml"
Path not found


Hierarchy entry "Host Group Data"
Path "/etc/puppetlabs/code/environments/production/hieradata/.yaml"
Original path: "%{::hostgroup}.yaml"
Path not found

Hierarchy entry "linuxfarm Per-node data"
Path "/etc/puppetlabs/code/environments/production/linuxfarm/hieradata/node1.example.com.yaml"
Original path: "%{trusted.certname}.yaml"
Path not found
Hierarchy entry "linuxfarm Host Group Data"
Path "/etc/puppetlabs/code/environments/production/linuxfarm/hieradata/RedHat.yaml"
Original path: "%{::osfamily}.yaml"
Path not found
Hierarchy entry "Secret data (encrypted)"
Path "/etc/puppetlabs/code/environments/production/hieradata/secure.yaml"
Original path: "secure.yaml"
Path not found
Hierarchy entry "linuxfarm Common data"
Path "/etc/puppetlabs/code/environments/production/linuxfarm/hieradata/common.yaml"
Original path: "common.yaml"
Path not found
Hierarchy entry "Common Data"
Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"
Original path: "common.yaml"
Found key: "puppet_agent::package_version" value: "6.24.0-1.el8"
[root@master06 ~]#

 

You can see that other variables such as ::osfamily and ::environment resolve correctly using hiera 5

Josh Cooper (Jira)

unread,
Aug 3, 2021, 1:18:03 PM8/3/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11194

osfamily and environment are builtin facts, but hostgroup is not. Where is that variable defined?

Tejas Rao (Jira)

unread,
Aug 5, 2021, 9:27:02 AM8/5/21
to puppe...@googlegroups.com
Tejas Rao commented on Bug PUP-11194

hostgroup is a top level variable passed by external ENC (foreman). hostgroup variable passed from foreman is resolved correctly in classes using $::hostgroup but not resolving in hiera.yaml (%{::hostgroup})

 

[root@master06 ~]# sudo -u puppet /etc/puppetlabs/puppet/node.rb node1.example.com | grep hostgroup
hostgroup: bnlboxgpfs
[root@master06 ~]#

Josh Cooper (Jira)

unread,
Aug 17, 2021, 7:42:02 PM8/17/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11194

I can't reproduce. I have a simple ENC script that always prints:

# /etc/puppetlabs/puppet/node.rb node1.example.com
 
---
 
classes:
 
  group:
 
parameters:
 
  hostgroup: group1
 
environment: env11194

Hiera is configured to resolve the hierarchy using the hostgroup top-scope variable returned by the ENC:

# cat /etc/puppetlabs/code/environments/env11194/hiera.yaml
 
---
 
version: 5
 
hierarchy:
 
  - name: "Host Group Data"
 
    paths:
 
      - "%{::hostgroup}.yaml"
 
  - name: "Common"
 
    paths:
 
      - "common.yaml"

The hiera data contains a key-value:

# cat /etc/puppetlabs/code/environments/env11194/data/group1.yaml
 
---
 
puppet_agent::package_version: 6.24.0-1.el8

And the class group contains:

# cat /etc/puppetlabs/code/environments/env11194/modules/group/manifests/init.pp
 
class group(
 
) {
 
  $var = lookup("puppet_agent::package_version")
 
  notify { "lookup variable is $var": }
 
}

The server queries the ENC and uses the hostgroup top-scope variable (defined to be group1) and uses that to lookup the hiera data for the node:

# puppet agent -t 
Notice: Local environment: 'production' doesn't match server specified node environment 'env11194', switching agent to 'env11194'.
 
Info: Retrieving pluginfacts
 
Info: Retrieving plugin
 
Info: Caching catalog for shabby-catchup.xxx
 
Info: Applying configuration version '1629243605'
 
Notice: lookup variable is 6.24.0-1.el8
 
Notice: /Stage[main]/Group/Notify[lookup variable is 6.24.0-1.el8]/message: defined 'message' as 'lookup variable is 6.24.0-1.el8'
 
Notice: Applied catalog in 0.01 seconds

Tejas Rao can you provide steps to reproduce?

Josh Cooper (Jira)

unread,
Sep 13, 2021, 8:05:02 PM9/13/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11194

Ah I was able to reproduce. I think the key is that you must use the compile option since the hierarchy using facts. See the CLI help:

* --compile
 
  Perform a full catalog compilation prior to the lookup. If your hierarchy and
 
  data only use the $facts, $trusted, and $server_facts variables, you don't
 
  need this option; however, if your Hiera configuration uses arbitrary
 
  variables set by a Puppet manifest, you might need this option to get accurate
 
  data. No catalog compilation takes place unless this flag is given.

Please give that a try and close this out if it's resolved.

Josh Cooper (Jira)

unread,
Nov 10, 2021, 11:09:02 PM11/10/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11194

I'm going to close this out based on my last comment. Please reopen if this is still an issue when using the --compile option.
Reply all
Reply to author
Forward
0 new messages