Jira (PUP-8094) Lookup ignores environment from the classifier when using rule with trusted facts

[Thomas Hallgren (JIRA)]

Thomas Hallgren moved an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts Change By: Thomas Hallgren Key: HI PUP - 586 8094 Project: Hiera Puppet Add Comment

This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)

[Laura Macchi (JIRA)]

Laura Macchi commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts Puppet apply doesn't have an option --node: [root@10-32-175-155 ~]# puppet apply --node 10-32-174-86.rfc1918.puppetlabs.net -e 'notice(lookup("message")' Error: Could not parse application options: ambiguous option: --node Anything else I can try?

Add Comment

[Thomas Hallgren (JIRA)]

Thomas Hallgren commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

Sorry, i meant --certname <the node>.

Add Comment

[Laura Macchi (JIRA)]

Laura Macchi commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

[root@10-32-175-155 ~]# puppet query 'inventory ['environment'] { certname = "10-32-174-86.rfc1918.puppetlabs.net" }' [ { "environment": "acceptance" } ]   [root@10-32-175-155 ~]# puppet apply --certname 10-32-174-86.rfc1918.puppetlabs.net -e 'notice(lookup("message"))' Notice: Scope(Class[main]): This node is using common data Notice: Compiled catalog for 10-32-174-86.rfc1918.puppetlabs.net in environment production in 0.13 seconds Notice: Applied catalog in 0.40 seconds

Add Comment

[Thomas Hallgren (JIRA)]

Thomas Hallgren commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

OK, so from the looks of it, puppet apply doesn't honor the environment set in the classifier either then.

Add Comment

[Laura Macchi (JIRA)]

Laura Macchi commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

Puppet apply does not talk to the classifier at all tho, all the catalog compilation happens locally at the agent level.

Add Comment

[Thomas Hallgren (JIRA)]

Thomas Hallgren commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

Right, and lookup does a half-hearted attempt to do so but then fails to switch environment based on the outcome. The root cause is probably that although it does talk to the classifier, it never involves the Puppet::Configurer. A puppet agent does, and as a result, the configurer performs the actual environment switch after the catalog has been compiled. The environment is changed to the environment appointed by the catalog before processing continues (which I guess produces a new catalog from the correct environment). This means that in order to fix this we must transform lookup --compile to a full agent -T run before the actual lookup takes place. Perhaps we should instead add an option to puppet agent so that it can perform a lookup? In essence, that would be a puppet agent -T with a catalog amended by the desired lookup.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

While it is desirable to have a puppet agent --lookup to lookup from the agent side that does not help when you want to run this on the master. I think the lookup CLI needs to honour an environment change performed by the classifier. If not done the result can be seen as a lie and user has no way of knowing this. It is however not a lie if lookup is seen as answering which values were used when last compiling with a set of facts - but then we have an odd mix of some historic data and fresh data from classifier. When considering how to solve this it is worth also considering that a real future request from an agent to get a catalog will be based on the facts at that point in time. A simulation with lookup using last known facts + current classifier is only a simulation - to get the right answer more things need to be given on the command line (simulation of the future fact values, and specifically giving the resulting environment). A possible thing to do could be to issue a warning when the environment is changed - "The answer is for the node's current environment 'foo'. To get the answer for the environment as dictated by the classifier please use --environment <what was returned from classifier>". (And we do not warn when --environment is given on the command line).

Add Comment

[Thomas Hallgren (JIRA)]

Thomas Hallgren commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

I'm just concerned that no matter how much we simulate in lookup, there will always be differences between the real outcome and that simulation. It's also not 100% clear to me how such a simulation would be made in this case. How do we get lookup to negotiate the environment with the master the same way an agent does?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

We could call the ENC and it returns a value for environment. Not sure what we should do by default: use the env stored in the node data we got for the node, or the one ENC stipulates? (A given environment to the CLI should naturally win). I am not so concerned with 100% accuracy as I am with the user understanding under which conditions the value was produced - clearly if all inputs are known it will be an accurate answer, but when you don't know what inputs have been used...

Add Comment

[Laura Macchi (JIRA)]

Laura Macchi commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

From the customer point of view, we need consistency. If lookup uses the classifier to retrieve the environment for pinned nodes, then it should also look at the classifier for nodes matched via a rule, regardless of the rule. As I see it, the options would be: 1. Always use the classifier to determine environment. 2. Always use production. I think option 1 allows the user to have a tool to troubleshoot hiera if things go wrong and allows lookup to behave as close as the puppet run as possible. Now we're stuck in a state that is in between those 2 options: sometimes we get real environment, sometimes we get production. That is what this ticket is trying to address. Last but not least, users come from using hiera, and hiera could give a reasonably accurate answer only from the master, so I'm not opposed to the idea of the master being able to retrieve the real environment but the agents using production if the classifier cannot be reached. That being said, I'd also like to see lookup accept a --environment flag to override the default, since right now that flag only works if the node is in the "agent-specified" node group, but I believe that's being covered in another ticket.

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

Laura Macchi I think the choice is if by default the lookup CLI should show values using the node's current state (what lookup does now) or its future state (ENC gives it a different env than what is stored). I also suggested a route where user would be told that the ENC says it should have a different env, and that in order to see those values, the user will need to override the env on the command line. (Naturally, the opposite is also possible if we want the "future env" to be the default). What to do here is a UX/Product decision - technically it can be made to work either way. Ping Eric Sorenson.

Add Comment

This message was sent by Atlassian JIRA (v7.0.2#70111-sha1:88534db)

[Thomas Hallgren (JIRA)]

Thomas Hallgren assigned an issue to Eric Sorenson

Puppet / PUP-8094

Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Thomas Hallgren Assignee: Thomas Hallgren Eric Sorenson

Add Comment

[Jarret Lavallee (JIRA)]

Jarret Lavallee updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Jarret Lavallee CS Priority: Needs Priority

Add Comment

[Owen Rodabaugh (JIRA)]

Owen Rodabaugh updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Owen Rodabaugh CS Priority: Needs Priority Major CS Impact: Similar to PUP-7479 this issue makes it difficult to troubleshoot hiera issues using `puppet lookup`. CS Severity: 3 - Serious CS Business Value: 5 - $$$$$$ CS Frequency: 3 - 25-50% of Customers

Add Comment

[Owen Rodabaugh (JIRA)]

Owen Rodabaugh updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Owen Rodabaugh CS Impact: Similar to PUP-7479 this issue makes it difficult to troubleshoot hiera issues using `puppet lookup`.  Users expect that with the correct set of switches they can get lookup to perfectly emulate what the agent will do at runtime.

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts Laura Macchi Henrik Lindberg Sorry, I missed the earlier question and just came across this during the CS Top 10 triage. To answer Henrik's question above, I strongly agree with Laura that lookup should as closely as possible mimic what the agent would receive, that is, it should treat the ENC as authoritative.

Add Comment

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)

[Owen Rodabaugh (JIRA)]

Owen Rodabaugh updated an issue

Puppet / PUP-8094

Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Owen Rodabaugh Team: Platform Core

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts Moving back to Ready for Eng, unless there are further design questions from Thomas Hallgren or Henrik Lindberg - is there a clear path to implementation now?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

Eric Sorenson See my comment above There is a choice between using a) "current/given env" (what it does now) or b) "the env it would switch to" (what was asked in this ticket), or c) or some other combination because just switching to b) isn't quite right either since it depends on why you are looking up; is it for debugging "the values you got", or "the values you will get". At the same time, it is not right to just get values from ENC but ignore what it says about env, since the other things ENC gives may then be completely wrong. So, something needs to be done about this - but what? So - no clear path yet.

Add Comment

[Eric Sorenson (JIRA)]

Eric Sorenson commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

The user need for lookup is to simulate, as closely as possible, the values that a node will get when a catalog is compiled for it. Therefore, lookup should behave as similar to the catalog compilation data flow as possible – including the fact that the ENC would be authoritative for the environment. Am I missing something?

Add Comment

[Henrik Lindberg (JIRA)]

Henrik Lindberg commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

That would be option b) "the env it will get". Thus if your current catalog contains a value and you want to know how it was looked up, you would need to give the now-previous env manually to override the ENC. I am fine with that.

Add Comment

[Adam Bottchen (JIRA)]

Adam Bottchen updated an issue

Puppet / PUP-8094

Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Adam Bottchen Labels: cstop10

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Josh Cooper Sprint: Coremunity Grooming

Add Comment

[Josh Cooper (JIRA)]

Josh Cooper commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts That makes sense to me too. It's not hard to have lookup perform a node request and either use the returned node's environment or continue along using its configured environment.

Add Comment

[Rob Braden (JIRA)]

Rob Braden assigned an issue to Unassigned

Puppet / PUP-8094

Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Rob Braden Assignee: Eric Sorenson

Add Comment

[Jarret Lavallee (JIRA)]

Jarret Lavallee updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Jarret Lavallee CS Rank: 20000

Add Comment

[Yasmin Rajabi (JIRA)]

Yasmin Rajabi commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts Josh Cooper could we take the minimal approach here to address this?

Add Comment

[Nick Walker (Jira)]

Nick Walker commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

Josh Cooper do you think we can proceed forward with what you proposed?

Add Comment

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-8094

Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Josh Cooper Sprint: Coremunity Grooming Hopper

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts So this ticket seems to cover two things. First, puppet lookup completely ignores the ENC environment when --compile is omitted. Second, rules with trusted facts don't seem to work. It's not clear to me yet if the second issue is a consequence of the first, or a separate issue. The first problem is caused by PUP-8502, see https://github.com/puppetlabs/puppet/commit/efb3ce8391c76e4f3403e66294626be686e4738f#diff-80625c6f3829dfdf6a8561bb4bf8f761R327-R329. If --compile is omitted, then lookup ignores the current node terminus. The reason for that change was to prevent classes from being included when looking up a value for a node. I think the fix was incorrect. I think we should always ask the node terminus for the node object (which will fix this issue), and when --compile is omitted, set the node's classes to an empty array (so we don't regress on PUP-8502). If we do this, then the ENC assigned environment will be authoritative, which matches the agent's behavior. If the node is in an agent-specified environment, then running puppet lookup --environment <env> should take precedence. I'll see if that resolves the second issue too.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper commented on PUP-8094

Re: Lookup ignores environment from the classifier when using rule with trusted facts

The second issue is because the lookup application does not specify trusted node information when "finding" the node. As a result the classifier's node terminus creates an empty trusted information object. So any classification based on trusted facts will be inaccurate. The lookup application should load facts via Puppet::Node::Facts.indirection.find for the node whose values are being looked up (we're in a delegation context). The lookup application should create a trusted information object using the trusted facts stored in puppetdb. So long as puppetdb's facts are relatively current, then lookup should be able to classify a node in the same way that the agent running on that node would.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-8094

Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Josh Cooper Acceptance Criteria: Lookup will honor the environment set in the classifier when assigning nodes to an environment group using trusted facts. Lookup uses facts from the fact terminus (when looking up for the local host this uses facter, when looking up another node this typically uses puppetdb). If a fact file is specified, then facts are merged/override. If there are no cached facts and no fact file is specified then error. Lookup retrieves the node from the node terminus (typically classifier) passing in all of the facts including trusted information. When the --compile option is omitted, remove classes from the node so that we don't regress on PUP-8502.

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Josh Cooper Epic Link: PUP-6870

Add Comment

[Jarret Lavallee (Jira)]

Jarret Lavallee updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Jarret Lavallee CS Priority: Major Needs Priority

Add Comment

[Jarret Lavallee (Jira)]

Jarret Lavallee updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Jarret Lavallee CS Rank: 20000

Add Comment

[Jarret Lavallee (Jira)]

Jarret Lavallee updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Jarret Lavallee

CS Business Value: 5 - $$$$$$ CS Frequency: 3 - 25-50% of Customers

CS Impact: Similar to PUP-7479 this issue makes it difficult to troubleshoot hiera issues using `puppet lookup`. Users expect that with the correct set of switches they can get lookup to perfectly emulate what the agent will do at runtime.

CS Priority: Needs Priority Reviewed

CS Severity: 3 - Serious

Add Comment

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Josh Cooper Epic Link: PUP- 6870 11052

Add Comment

This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)

[Josh Cooper (Jira)]

Josh Cooper updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Josh Cooper Sprint: Coremunity Hopper , Community PRs 2

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Ciprian Badescu Story Points: 5

Add Comment

[Victor Bobosila (Jira)]

Victor Bobosila assigned an issue to Victor Bobosila

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Victor Bobosila Assignee: Victor Bobosila

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Ciprian Badescu Sprint: NW - 2021-10-20 , NW - 2021-11-03

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Ciprian Badescu Sprint: NW - 2021-10-20, NW - 2021-11-03 , NW - 2021-11-17

Add Comment

[Ciprian Badescu (Jira)]

Ciprian Badescu updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Ciprian Badescu Sprint: NW - 2021-10-20, NW - 2021-11-03, NW - 2021-11-17 , NW - 2021-12-06

Add Comment

[Victor Bobosila (Jira)]

Victor Bobosila updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Victor Bobosila Release Notes: Bug Fix

Add Comment

[Victor Bobosila (Jira)]

Victor Bobosila updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Victor Bobosila Release Notes Summary: Fix an issue where trusted facts couldn't be used as rules for classification. This was fixed by gathering the trusted facts from the puppetdb query result, and overriding the trusted facts context.

Add Comment

[Michael Hashizume (Jira)]

Michael Hashizume updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Michael Hashizume Fix Version/s: PUP 6.26.0 Fix Version/s: PUP 7.13.0

Add Comment

[Parker Leach (Jira)]

Parker Leach updated an issue

Puppet / PUP-8094 Lookup ignores environment from the classifier when using rule with trusted facts

Change By: Parker Leach Labels: docs_reviewed

Add Comment

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)