Jira (PUP-11207) lookup() does not seem to work as expected

[remco wouts (Jira)]

remco wouts created an issue

Puppet / PUP-11207 lookup() does not seem to work as expected Issue Type: Improvement Affects Versions: PUP 7.8.0, PUP 6.9.0 Assignee: Unassigned Components: Hiera & Lookup Created: 2021/08/17 2:29 AM Labels: hiera lookup Priority: Normal Reporter: remco wouts Puppet Version: 7.X Puppet Server Version: 7.X OS Name/Version: Debian 10,Ubuntu 20.04 With an environment hiera.yaml file that contains a hierarchy like: hierarchy: -name: per module config   path: %{role}/%{module_name}.yaml Automatic class parameter lookup works as expected (ie. it is possible to define per module data in hiera). However the lookup() function when called with  parameters <MODULE NAME>::<SOMETHING> from any other module sets %{module_name} to the current module and not to <MODULE NAME>. The result is that lookup() does not find the parameter if the only place it is defined in is data/%{role}/%{module_name}.yaml Desired Behavior: lookup(<MODULE NAME>::<SOMETHING>) should always set %{module_name} to <MODULE NAME>.     Proposed fix: lib/puppet/pops/lookup/invocation.rb | 2 +- modified lib/puppet/pops/lookup/invocation.rb @@ -72,7 +72,7 @@ class Invocation def lookup(key, module_name = nil) key = LookupKey.new(key) unless key.is_a?(LookupKey) @top_key = key @module_name = module_name.nil? ? key.module_name : module_name + @module_name = key.module_name.nil? ? module_name : key.module_name save_current = self.class.current if save_current.equal?(self) yield Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-11207 lookup() does not seem to work as expected

Change By: Josh Cooper *Puppet Version: 7.X* *Puppet Server Version: 7.X* *OS Name/Version: Debian 10,Ubuntu 20.04*

With an environment {{hiera.yaml}} file that contains a hierarchy like:

{ quote code:yaml }hierarchy: -name: per module config

path: %\{role}/%\{module_name}.yaml

{ quote code }

Automatic class parameter lookup works as expected (ie. it is possible to define per module data in hiera). However the {{lookup()}} function when called with  parameters {{<MODULE NAME>::<SOMETHING>}} from any other module sets {{%\{module_name}}} to the current module and not to {{<MODULE NAME>}}. The result is that {{lookup()}} does not find the parameter if the only place it is defined in is {{data/%\{role}/%\{module_name}.yaml}}

*Desired Behavior:*

lookup({{<MODULE NAME>::<SOMETHING>) should always set %\{module_name} to <MODULE NAME>.}}

h3. Proposed fix: { quote code:diff } lib/puppet/pops/lookup/invocation.rb | 2 +-

modified lib/puppet/pops/lookup/invocation.rb @@ -72,7 +72,7 @@ class Invocation def lookup(key, module_name = nil) key = LookupKey.new(key) unless key.is_a?(LookupKey) @top_key = key

- @module_name = module_name.nil? ? key.module_name : module_name

+ @module_name = key.module_name.nil? ? module_name : key.module_name save_current = self.class.current if save_current.equal?(self) yield

{ quote code }

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-11207 lookup() does not seem to work as expected Change By: Josh Cooper

*Puppet Version: 7.X* *Puppet Server Version: 7.X* *OS Name/Version: Debian 10,Ubuntu 20.04* With an environment {{hiera.yaml}} file that contains a hierarchy like:

{code:yaml}hierarchy:

-name: per module config   path: %

\ {role}/% \ {module_name}.yaml {code}

Automatic class parameter lookup works as expected (ie. it is possible to define per module data in hiera). However the {{lookup()}} function when called with  parameters {{<MODULE NAME>::<SOMETHING>}} from any other module sets {{%

\ {module_name}}} to the current module and not to {{<MODULE NAME>}}. The result is that {{lookup()}} does not find the parameter if the only place it is defined in is {{data/%\{role}/%\{module_name}.yaml}}

*Desired Behavior:* lookup({{<MODULE NAME>::<SOMETHING>) should always set %\{module_name} to <MODULE NAME>.}}   h3. Proposed fix:

{code:diff}

lib/puppet/pops/lookup/invocation.rb | 2 +- modified lib/puppet/pops/lookup/invocation.rb @@ -72,7 +72,7 @@ class Invocation def lookup(key, module_name = nil) key = LookupKey.new(key) unless key.is_a?(LookupKey) @top_key = key - @module_name = module_name.nil? ? key.module_name : module_name + @module_name = key.module_name.nil? ? module_name : key.module_name save_current = self.class.current if save_current.equal?(self) yield

{code}

Add Comment

[Josh Cooper (Jira)]

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-11207

Re: lookup() does not seem to work as expected I can't reproduce the issue. I have a module a that does a lookup of module b which has per-module hiera data, and it looks up the key as expected: # tree /etc/puppetlabs/code/environments/production/ /etc/puppetlabs/code/environments/production/ ├── data ├── environment.conf ├── hiera.yaml ├── manifests └── modules ├── a │   └── manifests │   └── init.pp └── b ├── data │   └── common.yaml └── hiera.yaml   7 directories, 5 files # cat /etc/puppetlabs/code/environments/production/hiera.yaml --- version: 5 hierarchy: - name: "Common" paths: - "common.yaml" # cat /etc/puppetlabs/code/environments/production/modules/a/manifests/init.pp class a { notice(lookup("b::key")) } # cat /etc/puppetlabs/code/environments/production/modules/b/hiera.yaml --- version: 5 hierarchy: - name: "Common values" path: "common.yaml" # cat /etc/puppetlabs/code/environments/production/modules/b/data/common.yaml --- b::key: from module b # puppet lookup b::key --- from module b # puppet apply -e "include a" [Notice: Scope(Class[A]): from module b

Add Comment

[remco wouts (Jira)]

remco wouts commented on PUP-11207

Re: lookup() does not seem to work as expected

The issue is not about per-module data. That works fine. The issue is about environment data specifically when one uses %{module_name} in hiera.yaml. Here is a simple example: vagrant@ubuntu-focal:/vagrant/environment/testing$ tree . . ├── data │   ├── a.yaml │   └── b.yaml

├── hiera.yaml ├── manifests └── modules ├── a │   └── manifests │   └── init.pp └── b

└── manifests └── init.pp7 directories, 5 files vagrant@ubuntu-focal:/vagrant/environment/testing$ cat hiera.yaml version: 5 hierarchy: - name: "Configure each module seperately" path: "%{module_name}.yaml" vagrant@ubuntu-focal:/vagrant/environment/testing$ cat modules/{a,b}/manifests/init.pp class a ($key) { notice($key) notice(lookup("b::key",undef,undef,"lookup for b::key failed")) } class b ($key) { notice($key) notice(lookup('b::key',undef,undef,"lookup for b::key failed")) } This shows the issue, automatic parameter lookup works fine, lookup of data for another module does not. vagrant@ubuntu-focal:/vagrant/environment/testing$ puppet apply --codedir=. -e 'include a' Notice: Scope(Class[A]): keyA Notice: Scope(Class[A]): lookup for b::key failed Notice: Compiled catalog for ubuntu-focal.workspace.rug.nl in environment production in 0.05 seconds Notice: Applied catalog in 0.01 seconds this shows that module b works correctly: vagrant@ubuntu-focal:/vagrant/environment/testing$ puppet apply --codedir=. -e 'include b' Notice: Scope(Class[B]): keyB Notice: Scope(Class[B]): keyB Notice: Compiled catalog for ubuntu-focal.workspace.rug.nl in environment production in 0.04 seconds Notice: Applied catalog in 0.01 seconds I added a default value for{{ lookup()}} to prevent the commands from failing. On close examination of the documentation it seems that the lookup() function works as described, that is not the issue. It is more that this usage should also work. Currently I can work around the problem by adding a per-module hiera.yaml file with a datadir pointing back to the environment datadir. But that seems abit hackish.

Add Comment