Hiera fails to interpolate data path for a specific node

[Jianwen Wei]

Hi,

After upgrading openproductionource Puppet to version 4.9, I followed the Hiera guide https://docs.puppet.com/puppet/4.9/hiera_hierarchy.html#most-hierarchies-interpolate-variables to feed some data to it. I found that for some node, 'puppet lookup' failed to interpolate variables in yaml setting, which lead to incorrect lookup results.

Any suggestion is welcome.

Best,

Jianwen

Steps to reproduce this issue

1. Install Puppet server on puppet.pi.sjtu.edu.cn .

2. Install Puppet agents on puppet.pi.sjtu.edu.cn, slurm1.pi.sjtu.edu.cn and mu05.pi.sjtu.edu.cn. Sign certificates for them.

3. Delete global hiera setting `/etc/puppetlabs/code/hiera.yaml`.

# rm -f /etc/puppetlabs/code/hiera.yaml

4. Create an environment named `production`. 

# mkdir -p /etc/puppetlabs/code/environments/production/{hieradata,maifests,modules}

5. Create the Hiera conf file for the production environment in `/etc/puppetlabs/code/environments/production/hiera.yaml`

---

version: 5

defaults:

  datadir: hieradata

  data_hash: yaml_data

hierarchy:

  - name: "Per-node data"

    path: "nodes/%{trusted.certname}.yaml"

  - name: "common"

    path: "common.yaml"

6. Add common data in `/etc/puppetlabs/code/environments/production/hieradata/common.yaml`:

---

group: Compute Nodes

7a. Add per-node hiera data for slurm1.pi.sjtu.edu.cn in `/etc/puppetlabs/code/environments/production/hieradata/nodes/slurm1.pi.sjtu.edu.cn.yaml`:

---

group: SLURM

7b. Add per-node hiera data for puppet.pi.sjtu.edu.cn in `/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet.pi.sjtu.edu.cn.yaml`:

---

group: Puppet Server

8a. Lookup group for puppet.pi.sjtu.edu.cn and successfully get "Puppet Server" stored in `puppet.pi.sjtu.edu.cn.yaml`.

# puppet lookup group --node puppet.pi.sjtu.edu.cn --explain

Searching for "lookup_options"

  Global Data Provider (hiera configuration version 5)

    No such key: "lookup_options"

  Environment Data Provider (hiera configuration version 5)

    Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"

    Merge strategy hash

      Hierarchy entry "Per-node data"

        Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet.pi.sjtu.edu.cn.yaml"

          Original path: "nodes/%{trusted.certname}.yaml"

          No such key: "lookup_options"

      Hierarchy entry "common"

        Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"

          Original path: "common.yaml"

          No such key: "lookup_options"

Searching for "group"

  Global Data Provider (hiera configuration version 5)

    No such key: "group"

  Environment Data Provider (hiera configuration version 5)

    Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"

    Hierarchy entry "Per-node data"

      Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet.pi.sjtu.edu.cn.yaml"

        Original path: "nodes/%{trusted.certname}.yaml"

        Found key: "group" value: "Puppet Server"

8b. Lookup group for puppet.pi.sjtu.edu.cn and successfully get "Compute Nodes" stored in `common.yaml`.

# puppet lookup group --node mu05.pi.sjtu.edu.cn --explain

Searching for "lookup_options"

  Global Data Provider (hiera configuration version 5)

    No such key: "lookup_options"

  Environment Data Provider (hiera configuration version 5)

    Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"

    Merge strategy hash

      Hierarchy entry "Per-node data"

        Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/mu05.pi.sjtu.edu.cn.yaml"

          Original path: "nodes/%{trusted.certname}.yaml"

          Path not found

      Hierarchy entry "common"

        Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"

          Original path: "common.yaml"

          No such key: "lookup_options"

Searching for "group"

  Global Data Provider (hiera configuration version 5)

    No such key: "group"

  Environment Data Provider (hiera configuration version 5)

    Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"

    Hierarchy entry "Per-node data"

      Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/mu05.pi.sjtu.edu.cn.yaml"

        Original path: "nodes/%{trusted.certname}.yaml"

        Path not found

    Hierarchy entry "common"

      Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"

        Original path: "common.yaml"

        Found key: "group" value: "Compute Nodes"

8c. Lookup group for puppet.pi.sjtu.edu.cn and fail to interpolate %{trusted.certname} which results in incorrect lookup fallback to `common.yaml`. It is expected to use `slurm1.pi.sjtu.edu.cn.yaml` instead thus the correct result should be "SLURM", not "Compute Nodes".

# puppet lookup group --node slurm1.pi.sjtu.edu.cn --explain

Searching for "lookup_options"

  Global Data Provider (hiera configuration version 5)

    No such key: "lookup_options"

  Environment Data Provider (hiera configuration version 5)

    Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"

    Merge strategy hash

      Hierarchy entry "Per-node data"

        Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/.yaml"

          Original path: "nodes/%{trusted.certname}.yaml"

          Path not found

      Hierarchy entry "common"

        Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"

          Original path: "common.yaml"

          No such key: "lookup_options"

Searching for "group"

  Global Data Provider (hiera configuration version 5)

    No such key: "group"

  Environment Data Provider (hiera configuration version 5)

    Using configuration "/etc/puppetlabs/code/environments/production/hiera.yaml"

    Hierarchy entry "Per-node data"

      Path "/etc/puppetlabs/code/environments/production/hieradata/nodes/.yaml"

        Original path: "nodes/%{trusted.certname}.yaml"

        Path not found

    Hierarchy entry "common"

      Path "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"

        Original path: "common.yaml"

        Found key: "group" value: "Compute Nodes"

[Eric Thompson]

On Monday, March 13, 2017 at 11:26:41 AM UTC-7, Wei Jianwen wrote:

Hi,

After upgrading openproductionource Puppet to version 4.9, I followed the Hiera guide https://docs.puppet.com/puppet/4.9/hiera_hierarchy.html#most-hierarchies-interpolate-variables to feed some data to it. I found that for some node, 'puppet lookup' failed to interpolate variables in yaml setting, which lead to incorrect lookup results.

Hi Jianwen, 

what version of puppet are you using?  This is a known issue in puppet 4.9.0.  have you tried upgrading to puppet 4.9.3?

[Wei Jianwen]

    Thank you, Eric. I am using Puppet version 4.9.4 on all nodes and this issue persists.

Best,

Jianwen

在 2017年3月15日星期三 UTC+8上午12:33:09，Eric Thompson写道：

[Henrik Lindberg]

On 16/03/17 05:35, Wei Jianwen wrote:
>
> Thank you, Eric. I am using Puppet version 4.9.4 on all nodes and
> this issue persists.
>
> Best,
>
> Jianwen

Jianwen, please file a ticket in Puppet's Jira for the PUP project and
include the information you posted here.

Best,
- henrik

>
> 在 2017年3月15日星期三 UTC+8上午12:33:09，Eric Thompson写道：
>
>
>
> On Monday, March 13, 2017 at 11:26:41 AM UTC-7, Wei Jianwen wrote:
>
> Hi,
>
> After upgrading openproductionource Puppet to version 4.9, I
> followed the Hiera
> guide https://docs.puppet.com/puppet/4.9/hiera_hierarchy.html#most-hierarchies-interpolate-variables
> <https://docs.puppet.com/puppet/4.9/hiera_hierarchy.html#most-hierarchies-interpolate-variables> to
> feed some data to it. I found that for some node, 'puppet
> lookup' failed to interpolate variables in yaml setting, which
> lead to incorrect lookup results.
>
>
> Hi Jianwen,
>
> what version of puppet are you using? This is a known issue in
> puppet 4.9.0. have you tried upgrading to puppet 4.9.3?
>
>
> Any suggestion is welcome.
>
>
> Best,
>
> Jianwen
>
>
> Steps to reproduce this issue
>
> 1. Install Puppet server on puppet.

> <http://puppet.sjtu.edu.cn>pi.sjtu.edu.cn <http://pi.sjtu.edu.cn> .

> 2. Install Puppet agents on puppet.pi.sjtu.edu.cn

> <http://puppet.pi.sjtu.edu.cn>, slurm1.pi.sjtu.edu.cn
> <http://slurm1.pi.sjtu.edu.cn> and mu05.pi.sjtu.edu
> <http://mu05.pi.sjtu.edu>.cn. Sign certificates for them.

> 3. Delete global hiera setting `/etc/puppetlabs/code/hiera.yaml`.
>
> # rm -f /etc/puppetlabs/code/hiera.yaml
> 4. Create an environment named `production`.
>
> # mkdir
> -p /etc/puppetlabs/code/environments/production/{hieradata,maifests,modules}
>
> 5. Create the Hiera conf file for the production environment in
> `/etc/puppetlabs/code/environments/production/hiera.yaml`
>

> *---*
> *version: 5*
> *defaults:*
> * datadir: hieradata*
> * data_hash: yaml_data*
> *
> *
> *hierarchy:*
> * - name: "Per-node data"*
> * path: "nodes/%{trusted.certname}.yaml"*
> *
> *
> * - name: "common"*
> * path: "common.yaml"*

>
> 6. Add common data in
> `/etc/puppetlabs/code/environments/production/hieradata/common.yaml`:
>
> ---
> group: Compute Nodes
>
>
> 7a. Add per-node hiera data for slurm1.pi.sjtu.edu.cn

> <http://slurm1.pi.sjtu.edu.cn> in

> `/etc/puppetlabs/code/environments/production/hieradata/nodes/slurm1.pi.sjtu.edu.cn.yaml`:
>
> ---
>
> group: SLURM
>
>
> 7b. Add per-node hiera data for puppet.pi.sjtu.edu.cn

> <http://puppet.pi.sjtu.edu.cn> in

> `/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet.pi.sjtu.edu.cn.yaml`:
>
>
> ---
>
> group: Puppet Server
>
>
> 8a. Lookup group for puppet.pi.sjtu.edu.cn

> <http://puppet.pi.sjtu.edu.cn> and successfully get "Puppet
> Server" stored in `puppet.pi.sjtu.edu.cn
> <http://puppet.pi.sjtu.edu.cn>.yaml`.
>
> *# puppet lookup group --node puppet.pi.sjtu.edu.cn
> <http://puppet.pi.sjtu.edu.cn> --explain*
>
> *Searching for "lookup_options"*
>
> * Global Data Provider (hiera configuration version 5)*
>
> * No such key: "lookup_options"*
>
> * Environment Data Provider (hiera configuration
> version 5)*
>
> * Using configuration
> "/etc/puppetlabs/code/environments/production/hiera.yaml"*
>
> * Merge strategy hash*
>
> * Hierarchy entry "Per-node data"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet.pi.sjtu.edu.cn.yaml"*
>
> * Original path: "nodes/%{trusted.certname}.yaml"*
>
> * No such key: "lookup_options"*
>
> * Hierarchy entry "common"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"*
>
> * Original path: "common.yaml"*
>
> * No such key: "lookup_options"*
>
> *Searching for "group"*
>
> * Global Data Provider (hiera configuration version 5)*
>
> * No such key: "group"*
>
> * Environment Data Provider (hiera configuration
> version 5)*
>
> * Using configuration
> "/etc/puppetlabs/code/environments/production/hiera.yaml"*
>
> * Hierarchy entry "Per-node data"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/nodes/puppet.pi.sjtu.edu.cn.yaml"*
>
> * Original path: "nodes/%{trusted.certname}.yaml"*
>
> * Found key: "group" value: "Puppet Server"*

>
> 8b. Lookup group for puppet.pi.sjtu.edu.cn

> <http://puppet.pi.sjtu.edu.cn> and successfully get "Compute

> Nodes" stored in `common.yaml`.
>

> *# puppet lookup group --node mu05.pi.sjtu.edu.cn
> <http://mu05.pi.sjtu.edu.cn> --explain*
>
> *Searching for "lookup_options"*
>
> * Global Data Provider (hiera configuration version 5)*
>
> * No such key: "lookup_options"*
>
> * Environment Data Provider (hiera configuration
> version 5)*
>
> * Using configuration
> "/etc/puppetlabs/code/environments/production/hiera.yaml"*
>
> * Merge strategy hash*
>
> * Hierarchy entry "Per-node data"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/nodes/mu05.pi.sjtu.edu.cn.yaml"*
>
> * Original path: "nodes/%{trusted.certname}.yaml"*
>
> * Path not found*
>
> * Hierarchy entry "common"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"*
>
> * Original path: "common.yaml"*
>
> * No such key: "lookup_options"*
>
> *Searching for "group"*
>
> * Global Data Provider (hiera configuration version 5)*
>
> * No such key: "group"*
>
> * Environment Data Provider (hiera configuration
> version 5)*
>
> * Using configuration
> "/etc/puppetlabs/code/environments/production/hiera.yaml"*
>
> * Hierarchy entry "Per-node data"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/nodes/mu05.pi.sjtu.edu.cn.yaml"*
>
> * Original path: "nodes/%{trusted.certname}.yaml"*
>
> * Path not found*
>
> * Hierarchy entry "common"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"*
>
> * Original path: "common.yaml"*
>
> * Found key: "group" value: "Compute Nodes"*

>
>
> 8c. Lookup group for puppet.pi.sjtu.edu.cn

> <http://puppet.pi.sjtu.edu.cn> and fail to interpolate

> %{trusted.certname} which results in incorrect lookup fallback
> to `common.yaml`. It is expected to use `slurm1.pi.sjtu.edu.cn

> <http://slurm1.pi.sjtu.edu.cn>.yaml` instead thus the correct

> result should be "SLURM", not "Compute Nodes".
>

> *# puppet lookup group --node slurm1.pi.sjtu.edu.cn
> <http://slurm1.pi.sjtu.edu.cn> --explain*
>
> *Searching for "lookup_options"*
>
> * Global Data Provider (hiera configuration version 5)*
>
> * No such key: "lookup_options"*
>
> * Environment Data Provider (hiera configuration
> version 5)*
>
> * Using configuration
> "/etc/puppetlabs/code/environments/production/hiera.yaml"*
>
> * Merge strategy hash*
>
> * Hierarchy entry "Per-node data"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/nodes/.yaml"*
>
> * Original path: "nodes/%{trusted.certname}.yaml"*
>
> * Path not found*
>
> * Hierarchy entry "common"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"*
>
> * Original path: "common.yaml"*
>
> * No such key: "lookup_options"*
>
> *Searching for "group"*
>
> * Global Data Provider (hiera configuration version 5)*
>
> * No such key: "group"*
>
> * Environment Data Provider (hiera configuration
> version 5)*
>
> * Using configuration
> "/etc/puppetlabs/code/environments/production/hiera.yaml"*
>
> * Hierarchy entry "Per-node data"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/nodes/.yaml"*
>
> * Original path: "nodes/%{trusted.certname}.yaml"*
>
> * Path not found*
>
> * Hierarchy entry "common"*
>
> * Path
> "/etc/puppetlabs/code/environments/production/hieradata/common.yaml"*
>
> * Original path: "common.yaml"*
>
> * Found key: "group" value: "Compute Nodes"*
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/53687fb0-7bb3-4eac-be99-84b4b5357c1c%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/53687fb0-7bb3-4eac-be99-84b4b5357c1c%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/