Jira (BOLT-836) Support Hiera lookup() in plans outside of apply blocks

0 views
Skip to first unread message

Alex Dreyer (JIRA)

unread,
May 16, 2019, 11:58:03 AM5/16/19
to puppe...@googlegroups.com
Alex Dreyer updated an issue
 
Puppet Task Runner / New Feature BOLT-836
Support Hiera lookup() in plans outside of apply blocks
Change By: Alex Dreyer
Summary: Support Hiera lookup() in plans outside of apply blocks
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Alex Dreyer (JIRA)

unread,
May 29, 2019, 5:34:03 PM5/29/19
to puppe...@googlegroups.com
Alex Dreyer updated an issue
h2. Feature Request

It would be handy to support the hiera lookup() function within a plan.
h2. Open Question

What goes into the hierarchy for a plan? Without facter how are those values set? 
h2. Usecases
h4. Lookup different values to use the same plan in different environments or contexts

As a plan author i need to store data about how to execute the plan that may be environment or deployment specific. These settings would be looked up from the "perspective" of the bolt controller node. Example: may need different API keys, credentials, etc to use for API calls. Example 2: may be deploying in a different manner based on a setting in a config that is easy to switch.
h4. Use hiera to lookup task parameters for different targets

As a plan author i need to perform actions differently depending on certain facts and/or attributes of a node. These facts may be gather by the `facts` task. From these facts/attributes i then need to lookup() data to determine how the action should be run.  These facts would be looked up from the "perspective" of specific targets. Example: I'm deploying a software pacakge against a variety of hosts, some may be 32-bit some 64-bit. I need to deploy different versions of software to the 32-bit systems. I would run a 'facts' task to determine architecture, then choose which software package to copy and install. The data hierarchy would be used to uniquely specify which software package to utilize.

h2. Learning from spike
* currently just exposing hiera allows backends to work without interpolation as long as all interpolation is based on $facts and $trusted. The fact that those are set during plan execution feels wrong and may be a bug.
* We probably need the ability for users to specify a separate hiearchy for the plan level so we don't limit interpolation options.
* There is value in exposing backends so that hiera-eyaml can be used but this problem may be solved by supporting eyaml in inventory.


h1 Questions for commentors
* What levels of heirarchy would you use at the plan level?
* Would there be any fact like interpolations in the hierarchy? What values would be used.
* Do you expect values from plan level data to be automatically set anywhere with data bindings like classes in manifests.

Henrik Lindberg (JIRA)

unread,
May 30, 2019, 3:39:03 AM5/30/19
to puppe...@googlegroups.com
Henrik Lindberg commented on New Feature BOLT-836
 
Re: Support Hiera lookup() in plans outside of apply blocks

Some thoughts about implementation...

One thing to consider here is that switching "perpective" (bolt controller, each node) using a single hiera instance will lead to its cache being evicted on each switch. This would potentially be quite bad for performance if many such switches are required as a bolt run progresses. Imagine looping over a set of nodes and in each iteration some plan specific, and some node specific value needs to be obtained.

From a UX perspective everything could be hidden unless it should be possible to lookup node specific values in a plan. For that, it must be possible to use a different perspective. This could be another function for example (lookup_for_node()), or a method on a node/target.

When designing the "perpectives", a big question is if it should be possible to override a node perspective in the plan perspective. If not, it may be possible to have one hiera/lookup-context per node in memory if performance is bad using switching of the set of values used to determine the paths in the hierarchy. Maybe it is possible to make hiera switch between named (i.e. "perspective specific") caches in an effective way.

It seems like a design with one hierarchy for the plan perspective, and one for all node specific perspectives would be best, and if override of node perspective is needed in plan perspective, then that can be done manually in the plan's logic.

Yasmin Rajabi (JIRA)

unread,
Aug 15, 2019, 7:16:03 PM8/15/19
to puppe...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages