Jira (PUP-11527) puppet lookup -E doesn't execute the ENC

50 views
Skip to first unread message

Remko Catersels (Jira)

unread,
May 2, 2022, 11:57:02 AM5/2/22
to puppe...@googlegroups.com
Remko Catersels created an issue
 
Puppet / Bug PUP-11527
puppet lookup -E doesn't execute the ENC
Issue Type: Bug Bug
Affects Versions: PUP 6.26.0, PUP 6.27.0
Assignee: Unassigned
Components: Hiera & Lookup
Created: 2022/05/02 8:56 AM
Labels: puppet-lookup
Priority: Normal Normal
Reporter: Remko Catersels

Puppet Version:
Puppet Server Version:
OS Name/Version:

 puppet lookup --compile --node `hostname -f` -E production somevalue

Doesn't execute the ENC at all. So all variables that are defined via the ENC's parameters are not available. Oddly enough '-E production' fails too. Not providing '-E production' (so it defaults to the production environment) does execute the ENC.

 

Desired Behavior:

 puppet lookup -E <environment> --compile

 ** should execute the ENC.

Actual Behavior:

ENC is never executed (and its values aren't passed). Even with '-E production'. It does work without -E. But this makes it impossible to check hiera in a dev environment.

 
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)
Atlassian logo

Remko Catersels (Jira)

unread,
May 2, 2022, 11:58:01 AM5/2/22
to puppe...@googlegroups.com
Remko Catersels updated an issue
Change By: Remko Catersels
*Puppet Version:*
*Puppet Server Version:*
*OS Name/Version:*
{code:java}
puppet lookup --compile --node `hostname -f` -E production somevalue{code}

Doesn't execute the ENC at all. So all variables that are defined via the ENC's parameters are not available. Oddly enough '-E production' fails too. Not providing '-E production' (so it defaults to the production environment) does execute the ENC.

 

*Desired Behavior:*
{code:java}
puppet lookup -E <environment> --compile{code}

 ** should execute the ENC.

*Actual Behavior:*

ENC is never executed (and its values aren't passed). Even with '-E production'. It does work without -E. But this makes it impossible to check hiera in a dev environment. Some of our modules are depending on variables that are set via the ENC. Code just fails to compile because all of the ENC's information is missing.


 

Remko Catersels (Jira)

unread,
May 2, 2022, 12:00:04 PM5/2/22
to puppe...@googlegroups.com
Remko Catersels updated an issue
*Puppet Version: 4.27.0 (should be 4.27.1 but puppet -V shows an 'unreleased' version 4.27.0) *
*Puppet Server Version:* 6.19.0
*OS Name/Version: RHEL7 *

{code:java} puppet lookup --compile --node `hostname -f` -E production somevalue{code}
Doesn't execute the ENC at all. So all variables that are defined via the ENC's parameters are not available. Oddly enough '-E production' fails too. Not providing '-E production' (so it defaults to the production environment) does execute the ENC.

 

*Desired Behavior:*
{code:java} puppet lookup -E <environment> --compile{code}
 
**  should execute the ENC.


*Actual Behavior:*

ENC is never executed (and its values aren't passed). Even with '-E production'. It does work without -E. But this makes it impossible to check hiera in a dev environment. Some of our modules are depending on variables that are set via the ENC. Code just fails to compile because all of the ENC's information is missing.

Josh Cooper (Jira)

unread,
May 2, 2022, 1:07:01 PM5/2/22
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11527
 
Re: puppet lookup -E doesn't execute the ENC

This is due to the changes in PUP-7479. Passing an explicit environment now means "lookup data in this environment" always and ignore what the ENC says. However that means node parameters that would normally be set by the ENC will be absent. I'm not sure how we can satisfy both this request and PUP-7479 at the same time.

Remko Catersels (Jira)

unread,
May 3, 2022, 5:45:02 AM5/3/22
to puppe...@googlegroups.com

Not pass the environment from the ENC but do allow the classes and parameters to pass?

Nirupama Mantha (Jira)

unread,
May 3, 2022, 4:17:02 PM5/3/22
to puppe...@googlegroups.com
Nirupama Mantha updated an issue
 
Change By: Nirupama Mantha
*Puppet Version: 4 6 .27.0 (should be 4 6 .27.1 but puppet -V shows an 'unreleased' version 4 6 .27.0)*

*Puppet Server Version:* 6.19.0
*OS Name/Version: RHEL7*
{code:java} puppet lookup --compile --node `hostname -f` -E production somevalue{code}
Doesn't execute the ENC at all. So all variables that are defined via the ENC's parameters are not available. Oddly enough '-E production' fails too. Not providing '-E production' (so it defaults to the production environment) does execute the ENC.

 

*Desired Behavior:*
{code:java} puppet lookup -E <environment> --compile{code}
 should execute the ENC.

*Actual Behavior:*

ENC is never executed (and its values aren't passed). Even with '-E production'. It does work without -E. But this makes it impossible to check hiera in a dev environment. Some of our modules are depending on variables that are set via the ENC. Code just fails to compile because all of the ENC's information is missing.

 

Remko Catersels (Jira)

unread,
May 6, 2022, 5:00:01 AM5/6/22
to puppe...@googlegroups.com
 
Re: puppet lookup -E doesn't execute the ENC

I've been trying to come up with a solution as enabling the ENC again will probably cause way too much other issues.

How about adding an option to provide the parameters on the command line? Something like ```puppet lookup --environment <dev env> --parameters parameters.yaml ...```

It would fairly easy to prepare such a file, it would also allow one to test various responses that would otherwise be difficult to do if the parameters only came from the ENC.

Michael Hashizume (Jira)

unread,
May 10, 2022, 4:42:01 PM5/10/22
to puppe...@googlegroups.com
Michael Hashizume updated an issue
 
Change By: Michael Hashizume
Story Points: 5

Michael Hashizume (Jira)

unread,
May 10, 2022, 4:42:02 PM5/10/22
to puppe...@googlegroups.com
Michael Hashizume updated an issue
Change By: Michael Hashizume
Team: Phoenix

Nirupama Mantha (Jira)

unread,
May 17, 2022, 4:39:02 PM5/17/22
to puppe...@googlegroups.com
Nirupama Mantha commented on Bug PUP-11527
 
Re: puppet lookup -E doesn't execute the ENC

SirDice Would you like to submit a PR for this?

Josh Cooper (Jira)

unread,
May 26, 2022, 12:00:03 PM5/26/22
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11527

SirDice I think this request "the ability to set node parameters" is duplicated by PUP-6400, so I'm inclined to close this ticket. Can you confirm?

Josh Cooper (Jira)

unread,
Sep 2, 2022, 11:45:02 AM9/2/22
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11527

Sorry I'm conflating "node parameters returned by the ENC" vs "setting arbitrary node parameters from the CLI". This ticket is the former, while PUP-6400 is the latter.
This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)
Atlassian logo

April Murphy (Jira)

unread,
Sep 2, 2022, 12:07:02 PM9/2/22
to puppe...@googlegroups.com
April Murphy updated an issue
 
Change By: April Murphy
Release Notes: Known Issue
Release Notes Summary: The issue is if you specify puppet lookup with an explicit environment ( '--environment web' ) then lookup will not call to the classifier, causing any node parameters set in the classifier to be omitted. The reason for the behavior is because if we call the classifier, then the classifier will by default assign a different environment to the node, returning a lookup result for a different environment than was requested. See PUP-7479

No work around - Unfortunately it doesn't work when specifying an explicit environment ( 'puppet lookup --environment' ), a node parameter is defined in the classifier and your hiera.conf interpolates the node parameter, e.g. 'data/%{::tier}.yaml'

It also affects open source (replace the word 'classifier' with 'ENC')

Josh Cooper (Jira)

unread,
Sep 2, 2022, 12:19:02 PM9/2/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Affects Version/s: PUP 7.16.0
Affects Version/s: PUP 7.17.0
Affects Version/s: PUP 7.18.0

David Piekny (Jira)

unread,
Oct 19, 2022, 11:51:01 AM10/19/22
to puppe...@googlegroups.com
David Piekny updated an issue
Change By: David Piekny
Priority: Normal Major

Josh Cooper (Jira)

unread,
Oct 19, 2022, 12:23:03 PM10/19/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Epic Link: PUP-11619

Josh Cooper (Jira)

unread,
Oct 19, 2022, 2:52:02 PM10/19/22
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11527
 
Re: puppet lookup -E doesn't execute the ENC

In order to satisfy this regression and to satisfy the requirements in PUP-7479 ("puppet lookup needs to work when the server specifies the environment"), we will need to send additional request options when making the node request to differentiate between "trusted puppet lookup command" vs "compiling a catalog on behalf of an agent". And the PE classifier terminus will need to check/verify.

April Murphy (Jira)

unread,
Oct 19, 2022, 3:04:03 PM10/19/22
to puppe...@googlegroups.com
April Murphy updated an issue
 
Change By: April Murphy
Labels: known-issue-added puppet-lookup

April Murphy (Jira)

unread,
Oct 19, 2022, 3:05:01 PM10/19/22
to puppe...@googlegroups.com
April Murphy updated an issue
Change By: April Murphy
Labels: known-issue-added known_issue_added puppet-lookup

David Piekny (Jira)

unread,
Oct 20, 2022, 1:22:03 PM10/20/22
to puppe...@googlegroups.com
David Piekny commented on Bug PUP-11527
 
Re: puppet lookup -E doesn't execute the ENC

Micah Wilson we'll need to create a complimentary PE ticket for this escalation. There will be some amount of Dumpling work we'll need to coordinate, FYI.

Gareth McGrillan (Jira)

unread,
Oct 24, 2022, 8:16:02 AM10/24/22
to puppe...@googlegroups.com

Folks - can this be qualified as a regression or a bug?

Josh Cooper (Jira)

unread,
Oct 25, 2022, 3:41:01 PM10/25/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
Sprint: Phoenix 2022-11-09

Josh Cooper (Jira)

unread,
Oct 25, 2022, 3:41:02 PM10/25/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Story Points: 5 3

Josh Cooper (Jira)

unread,
Oct 25, 2022, 3:41:03 PM10/25/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Fix Version/s: PUP 7.21.0

Aria Li (Jira)

unread,
Oct 26, 2022, 6:33:03 PM10/26/22
to puppe...@googlegroups.com
Aria Li assigned an issue to Aria Li
Change By: Aria Li
Assignee: Aria Li

Aria Li (Jira)

unread,
Oct 27, 2022, 2:11:02 PM10/27/22
to puppe...@googlegroups.com
Aria Li assigned an issue to Unassigned

Aria Li (Jira)

unread,
Oct 27, 2022, 2:21:02 PM10/27/22
to puppe...@googlegroups.com
Aria Li assigned an issue to Aria Li

Josh Cooper (Jira)

unread,
Oct 27, 2022, 2:30:02 PM10/27/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Acceptance Criteria: It should be possible to set a node parameter in an external node classifier, reference that node parameter in hiera.yaml and ensure the value is resolved correctly when running "puppet lookup -E <env>" for the current node, for a different node and with/without the --compile option.  If "-E" is omitted, then it should default to the environment specified in puppet.conf or default to "production"

By default, it should be possible to interpolate the "environment" parameter in hiera. However, if an explicit environment is specified (via -E) then it should take precedence over what the ENC returned.

Aria Li (Jira)

unread,
Oct 27, 2022, 4:06:03 PM10/27/22
to puppe...@googlegroups.com
Aria Li assigned an issue to Unassigned
Change By: Aria Li
Assignee: Aria Li

Aria Li (Jira)

unread,
Nov 1, 2022, 7:35:02 PM11/1/22
to puppe...@googlegroups.com
Aria Li assigned an issue to Aria Li

Morgan Rhodes (Jira)

unread,
Nov 9, 2022, 1:06:02 PM11/9/22
to puppe...@googlegroups.com
Morgan Rhodes updated an issue
Change By: Morgan Rhodes
Sprint: Phoenix 2022-11-09 , Phoenix 2022-11-23

April Murphy (Jira)

unread,
Nov 28, 2022, 10:56:01 AM11/28/22
to puppe...@googlegroups.com
April Murphy updated an issue
Change By: April Murphy
Labels: docs-reviewed known_issue_added puppet-lookup

April Murphy (Jira)

unread,
Nov 28, 2022, 10:56:03 AM11/28/22
to puppe...@googlegroups.com

Parker Leach (Jira)

unread,
Dec 7, 2022, 4:24:03 PM12/7/22
to puppe...@googlegroups.com
Parker Leach updated an issue
Change By: Parker Leach
Labels: docs_reviewed known_issue_added puppet-lookup
Reply all
Reply to author
Forward
0 new messages