Jira (PUP-7428) Implement knockout prefix for unique lookups

22 views
Skip to first unread message

Henrik Lindberg (JIRA)

unread,
Apr 5, 2017, 11:30:02 AM4/5/17
to puppe...@googlegroups.com
Henrik Lindberg moved an issue
 
Puppet / New Feature PUP-7428
Implement knockout prefix for unique lookups
Change By: Henrik Lindberg
Issue Type: Bug New Feature
Key: HI PUP - 568 7428
Project: Hiera Puppet
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Henrik Lindberg (JIRA)

unread,
Apr 5, 2017, 11:31:02 AM4/5/17
to puppe...@googlegroups.com
Henrik Lindberg updated an issue
Change By: Henrik Lindberg
Team: Puppet Developer Experience
Sprint: PDE Triage

Henrik Lindberg (JIRA)

unread,
Apr 5, 2017, 11:32:02 AM4/5/17
to puppe...@googlegroups.com
Henrik Lindberg commented on New Feature PUP-7428
 
Re: Implement knockout prefix for unique lookups

Ping Thomas Hallgren - thoughts? Is this something that it would be better if the deep merge gem did?

Thomas Hallgren (JIRA)

unread,
Apr 5, 2017, 6:12:02 PM4/5/17
to puppe...@googlegroups.com

I think the concept of knock_out_prefix is quite horrible and should be replaced by filter functions like the ones we discuss in PUP-7332. So I would rather spend efforts making that work.

Matthias Baur (JIRA)

unread,
Apr 6, 2017, 3:09:02 AM4/6/17
to puppe...@googlegroups.com
Matthias Baur commented on New Feature PUP-7428

Fine with me, would be just nice to have a knock_out_prefix like behavior which works without a workaround

Eric Sorenson (JIRA)

unread,
Apr 20, 2017, 8:25:03 PM4/20/17
to puppe...@googlegroups.com
Eric Sorenson assigned an issue to Matthias Baur
 
Change By: Eric Sorenson
Assignee: Matthias Baur

Eric Sorenson (JIRA)

unread,
Apr 20, 2017, 8:26:01 PM4/20/17
to puppe...@googlegroups.com
Eric Sorenson commented on New Feature PUP-7428
 
Re: Implement knockout prefix for unique lookups

Matthias Baur can you please describe the use case or the problem you're running into? This ticket suggests a particular implementation but doesn't provide enough context.

Matthias Baur (JIRA)

unread,
Apr 21, 2017, 4:16:02 AM4/21/17
to puppe...@googlegroups.com
Matthias Baur commented on New Feature PUP-7428

Sure, happy to help. We use the lookup() function for getting all client classes. Sometimes the hierarchy makes it necessary to remove the class in a higher level.

Simple example:

hiera.yaml:

---
version: 5
defaults:
  data_hash: yaml_data
  datadir: data
hierarchy:
  - name: "Customer node"
    paths:
      - "%{::trusted.extensions.customer}/%{::trusted.extensions.project}/%{::trusted.extensions.customerenv}/nodes/%{::trusted.certname}.yaml" # Node
  - name: "Common"
    paths:
      - "common.yaml" # Global defaults

common.yaml:

---
classes:
  - base
  - ntp
  - firewall

node.yaml:

classes:
  - REMOVE_firewall
  - fancy_custom_firewall

What we like to have:

lookup( { 'name'  => 'classes',
               'merge' => {
                 'strategy'        => 'unique',
                 'knockout_prefix' => 'REMOVE_',
             }
           }
         ).include

What we currently have to to:

# Hiera 5 lookup. Used to be hiera_include('classes')
$classes_tmp = lookup('classes', { merge => 'unique' })
 
# This is an ugly hack implementing knockout prefix for arrays.
# See https://tickets.puppetlabs.com/browse/HI-223 and https://tickets.puppetlabs.com/browse/HI-568 for more informations
$remove_classes_tmp = $classes_tmp.filter |$c| { $c =~ /^REMOVE_/ }
$remove_classes = $remove_classes_tmp.map |$c| { regsubst($c, '^REMOVE_', '') }
$classes = (($classes_tmp - $remove_classes) - $remove_classes_tmp)
 
$classes.include

Hope that helps. If I can provide more information, just let me know.

Matthias Baur (JIRA)

unread,
Apr 21, 2017, 4:46:02 AM4/21/17
to puppe...@googlegroups.com
Matthias Baur assigned an issue to Eric Sorenson
 
Change By: Matthias Baur
Assignee: Matthias Baur Eric Sorenson

Ethan Brown (JIRA)

unread,
May 17, 2017, 2:11:03 PM5/17/17
to puppe...@googlegroups.com
Ethan Brown updated an issue
Change By: Ethan Brown
Team: Puppet Developer Experience Agent

Ethan Brown (JIRA)

unread,
May 17, 2017, 2:12:05 PM5/17/17
to puppe...@googlegroups.com

Ethan Brown (JIRA)

unread,
May 17, 2017, 2:12:12 PM5/17/17
to puppe...@googlegroups.com
Ethan Brown updated an issue
Change By: Ethan Brown
Sprint: PDE Agent  Triage

Henrik Lindberg (JIRA)

unread,
May 31, 2017, 9:10:06 AM5/31/17
to puppe...@googlegroups.com
Henrik Lindberg updated an issue
Change By: Henrik Lindberg
Sprint: Agent Language  Triage

Kenyon Ralph (JIRA)

unread,
Nov 27, 2018, 5:54:03 PM11/27/18
to puppe...@googlegroups.com
Kenyon Ralph commented on New Feature PUP-7428
 
Re: Implement knockout prefix for unique lookups

knockout_prefix should really be removed from the documentation so that people don't waste time rediscovering this bug like I just did.

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Sean Millichamp (Jira)

unread,
Apr 10, 2020, 7:42:03 AM4/10/20
to puppe...@googlegroups.com

I just encountered the brokenness with knockout_prefix in Hiera's deep merge. I understand that might be a tricky problem to fix, but adding knockout_prefix to Hiera's unique lookup would be relatively easy code.

Since Eric Sorenson asked for use-cases above, I was trying to implement a "feature_flag" Hiera setting, implemented as an array of strings. I wanted to be able to list the feature flags but also to negate them as an earlier (more specific) Hiera level, in this case to opt out a small set of customers (who are currently in an extended change freeze) from what would be an otherwise global change on a particular platform type. The deep merge knockout_prefix appeared to work in simple testing, but it is a good thing I did a final full-catalog compile test with octocatalog-diff and caught the problem or I would have ended up violating the customers' change freezes.

I ended up whipping up a quick Ruby function applied in the module after Hiera APL that still allowed me to use Hiera's unique merge and get the knockout feature flag behavior for the final set of enabled feature flags, but it would be much cleaner if it were just an option in Hiera's unique merge itself.

 

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Molly Waggett (Jira)

unread,
Mar 1, 2022, 1:35:02 PM3/1/22
to puppe...@googlegroups.com
Molly Waggett updated an issue
 
Change By: Molly Waggett
Team: Froyo Phoenix
This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)
Atlassian logo

Josh Cooper (Jira)

unread,
Mar 2, 2022, 12:04:02 PM3/2/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Language Triage

Josh Cooper (Jira)

unread,
Apr 27, 2022, 12:25:02 AM4/27/22
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Epic Link: PUP-6870

Alvin Rodis (Jira)

unread,
Dec 5, 2022, 12:18:02 PM12/5/22
to puppe...@googlegroups.com
Alvin Rodis updated an issue
Change By: Alvin Rodis
Labels: jira_escalated
This message was sent by Atlassian Jira (v8.20.11#820011-sha1:0629dd8)
Atlassian logo

Alvin Rodis (Jira)

unread,
Dec 5, 2022, 12:21:02 PM12/5/22
to puppe...@googlegroups.com
Alvin Rodis updated an issue
Change By: Alvin Rodis
Zendesk Ticket Count: 1
Zendesk Ticket IDs: 50335
Reply all
Reply to author
Forward
0 new messages