Jira (PUP-8589) Puppet apply or agent should list hiera keys, values and their paths

0 views
Skip to first unread message

Naeem (JIRA)

unread,
Mar 22, 2018, 5:10:03 PM3/22/18
to puppe...@googlegroups.com
Naeem created an issue
 
Puppet / New Feature PUP-8589
Puppet apply or agent should list hiera keys, values and their paths
Issue Type: New Feature New Feature
Assignee: Unassigned
Created: 2018/03/22 2:09 PM
Priority: Normal Normal
Reporter: Naeem

When running puppet apply or puppet agent in noop mode, the output should list all the hiera keys listed in the module along with their values and also the hiera data path(s) they were found in.

In our scenario, we have a central hiera repository and the component modules do not have all the hiera data inside the module.

We can run puppet apply/agent with --debug flag and it shows the hiera keys/values in the output alongwith the paths but the output is too verbose and not easily readable.

The ask here is to have another flag with puppet apply/agent (let's call it --demgstify) to only lookup hiera keys required to run the module and list them along with their values and paths.

This will greatly help us troubleshooting puppet runs.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Henrik Lindberg (JIRA)

unread,
Mar 23, 2018, 10:11:02 AM3/23/18
to puppe...@googlegroups.com
Henrik Lindberg commented on New Feature PUP-8589
 
Re: Puppet apply or agent should list hiera keys, values and their paths

I think we have another ticket where it is asked that an agent can ask for --explain without having to turn on debugging on the server side. You would then end up with the explain output in the log. Would that be sufficient?

I doubt that a much simpler format for the explain output is really doable (without loss of fidelity) as the backends can be quite complex along with all the processing that hiera can do (merges, defaults, aliases etc.)

You can also use the lookup CLI with --explain and use --compile to compile a catalog - maybe we should augment that with something like an --explain-all. You could then get all of it as JSON for post processing with some json queries.

Naeem (JIRA)

unread,
Mar 23, 2018, 3:17:02 PM3/23/18
to puppe...@googlegroups.com
Naeem commented on New Feature PUP-8589

Thanks for the quick response. I understand that --explain does the job but it also gives you the output of all the hiera data paths it is traversing through to lookup the hiera keys. What we are thinking that if we could get the simplified output like below:

  • Key name
  • Key value
  • Path the key and value was found

If this is not easily doable then your suggestion about --explain-all might work as well with some post processing on our side (which we are trying to avoid). How much time do you anticipate it might take to get this feature in.

Henrik Lindberg (JIRA)

unread,
Mar 23, 2018, 6:30:03 PM3/23/18
to puppe...@googlegroups.com

Naeem Output like you suggest is too simplistic - many lookup results are composed out of multiple paths from multiple locations (a deep merge for example resulting in a hash, a merge with knockout values that removes information, a value that is composed out of interpolation of other lookups, etc. etc. A backend has access to the explain feedback and can produce its own set of information - it may for instance operate on a complex key, it may have its own kind of hierarchy and set of paths. It is simply not doable to reduce it to the simple information you want as it will be wrong in a very large percentage of all lookups. So - I don't see that we can provide that in puppet.

You will naturally face the same challenge if you want to attempt to simplify the result of explanations down to that. It may work in your particular case if you are not using deep merges, aliases, or any fancy backends.

I am not sure how much work it would be to support a combination of --explain-all and --compile, (not sure if it perhaps already works by just using --explain and --compile. By also specifying a log destination that ends with .json you could perhaps get all of the explain output logged to that file. (I have not tried, in theory it could work). If that does not work, we may need to tweak the implementation we have. Naturally, we need to prioritize such a feature against all other features being requested. If we do prioritize this it will be to "better support understanding and debugging hiera lookup used when compiling" and I think we need to really understand what that means - given that it is not doable to reduce it to the simple case you wish is it really meaningful to dump all of the information - did that just lead to exchanging one problem for another? What is a good user interface in this case? etc.

I added this ticket to our Epic for things to consider for Puppet in the 6.y series. However, I don't think it is clear yet how a better debugging support would actually work (given all the things I already commented on).

Craig Gomes (JIRA)

unread,
Apr 2, 2018, 5:38:04 PM4/2/18
to puppe...@googlegroups.com

Craig Gomes (JIRA)

unread,
Apr 2, 2018, 5:41:11 PM4/2/18
to puppe...@googlegroups.com

Maggie Dreyer (Jira)

unread,
Mar 31, 2021, 1:28:02 PM3/31/21
to puppe...@googlegroups.com
Maggie Dreyer commented on New Feature PUP-8589
 
Re: Puppet apply or agent should list hiera keys, values and their paths

We're not going to get to this any time soon.

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages