Jira (HI-599) Hiera cannot interpolate non-string values

4 views
Skip to first unread message

Carl Caum (JIRA)

unread,
Apr 5, 2018, 11:44:02 AM4/5/18
to puppe...@googlegroups.com
Carl Caum created an issue
 
Hiera / Bug HI-599
Hiera cannot interpolate non-string values
Issue Type: Bug Bug
Assignee: Unassigned
Created: 2018/04/05 8:43 AM
Priority: Normal Normal
Reporter: Carl Caum

It is not possible for Hiera to interpolate non-string values from Facter. For example, the `selinux.enabled` fact has a Boolean value. However, in Hiera you cannot do something like:

 docker::selinux_enabled: %{facts.selinux.enabled}

Instead, the user has to quote the facter fact code so it can be interpolated, like this:

 docker::selinux_enabled: "%{facts.selinux.enabled}"

which inadvertently casts any non-string values to a string.

Example of a user experiencing this problem: https://github.com/puppetlabs/puppetlabs-docker/issues/212

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Trevor Vaughan (JIRA)

unread,
Apr 5, 2018, 12:43:02 PM4/5/18
to puppe...@googlegroups.com
Trevor Vaughan commented on Bug HI-599
 
Re: Hiera cannot interpolate non-string values

It would be ideal if, instead of overriding particular lookups at the configuration layer, that this could be exposed at interpolation time so that it is less confusing for users.

For instance, something like the following would be nice:

"%{typed('facts.selinux.enabled', 'Boolean')}"

Craig Gomes (JIRA)

unread,
Apr 16, 2018, 5:21:02 PM4/16/18
to puppe...@googlegroups.com
Craig Gomes updated an issue
 
Change By: Craig Gomes
Team: Platform Core

Craig Gomes (JIRA)

unread,
Apr 16, 2018, 5:21:03 PM4/16/18
to puppe...@googlegroups.com
Craig Gomes updated an issue
Change By: Craig Gomes
Sub-team: Language

Henrik Lindberg (JIRA)

unread,
Apr 17, 2018, 2:03:03 AM4/17/18
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-599
 
Re: Hiera cannot interpolate non-string values

There is the alias/lookup hiera interpolation function which when used retains the data type of the aliased value - for that to work the string with the interpolation cannot contain anything besides that alias lookup.

Henrik Lindberg (JIRA)

unread,
Apr 17, 2018, 2:04:02 AM4/17/18
to puppe...@googlegroups.com
Henrik Lindberg assigned an issue to Carl Caum
 
Change By: Henrik Lindberg
Assignee: Carl Caum

Trevor Vaughan (JIRA)

unread,
Apr 17, 2018, 8:54:02 AM4/17/18
to puppe...@googlegroups.com
Trevor Vaughan commented on Bug HI-599
 
Re: Hiera cannot interpolate non-string values

Henrik Lindberg Yes, unfortunately, that doesn't work with facts. Since the use of typed facts is now the norm, Hiera should be able to have a way to ensure that the data type of facts is preserved.

Henrik Lindberg (JIRA)

unread,
Apr 17, 2018, 9:38:07 AM4/17/18
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-599

Right, alias does not actually work because that is a lookup in hiera, not a variable reference. The documentation shows that there also is a scope interpolation function (See https://puppet.com/docs/puppet/5.0/hiera_interpolation.html#using-interpolation-functions).

Have you tried:

docker::selinux_enabled: "%{scope(facts.selinux.enabled)}"

That seems to be what you are after? Not sure if it retains the data type of the variable though.

Henrik Lindberg (JIRA)

unread,
Apr 17, 2018, 9:41:09 AM4/17/18
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-599

In a custom backend you can do whatever you want. The lookup_key kind does not automatically perform hira interpolation of the returned values. For a data_hash you would return a hash where the variable reference is already resolved using for example the {[dig}} function to drill down into the value.

Trevor Vaughan (JIRA)

unread,
Apr 17, 2018, 10:01:10 AM4/17/18
to puppe...@googlegroups.com
Trevor Vaughan commented on Bug HI-599

Henrik Lindberg Unfortunately, using scope still results in a String instead of a Boolean value.

Henrik Lindberg (JIRA)

unread,
Apr 17, 2018, 11:43:03 AM4/17/18
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-599

I could argue that scope should work the same way as alias - it is not backwards compatible though to make such a change as some may rely on it interpolating the variable as a string.

I am very reluctant to introduce new interpolation functions (especially when they are not actual string interpolations) and I am also very reluctant to introduce any syntax into the data values used in hiera data.

What we could do is to solve this with lookup_options - we have a ticket for a real alias expressed in lookup_options. Would not be terribly hard to also support a variable/dig reference (mutually exclusive with use of alias).

Trevor Vaughan (JIRA)

unread,
Apr 17, 2018, 12:58:03 PM4/17/18
to puppe...@googlegroups.com
Trevor Vaughan commented on Bug HI-599

Henrik Lindberg I think that scope should work the same way as alias. It makes sense. I can see that some people might rely on that but that seems like a bug and should be treated as such. In theory, your Hiera data is getting passed into your classes and everything should be strongly typed at this point (hopefully).

Rune Juhl Jacobsen (JIRA)

unread,
Jun 11, 2018, 4:50:03 AM6/11/18
to puppe...@googlegroups.com

Since you're already using YAML, maybe it would make sense to make use of the existing syntax for YAML anchors? Instead of making the Hiera-specific syntax more convoluted by adding an optional argument for type coercion, as suggested in comment #1, this would keep the syntax (relatively) simple.

According to the YAML 1.2 spec there shouldn't be any problem with anchors containing any of the valid characters for Puppet identifiers, i.e. an anchor like *::some::weird_identifier::here is perfectly fine. With that it would be possible to use any types in facts without having to explicitly state the type.

I imagine it's possible to hook into the YAML deserialization to make this possible. Of course, this means that Puppet YAML breaks the YAML spec by abusing anchors, but I imagine that not many people use YAML anchors, and even less people use YAML anchors with :: in them, making this less of an issue.

Henrik Lindberg (JIRA)

unread,
Jun 11, 2018, 5:05:02 AM6/11/18
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-599

Rune Juhl Jacobsen Modifying YAML to be non standard is not a happy path. Going there, you would need to invent a new hiera format.
Note that if you write a lookup_key kind of hiera backend function you handle interpolations yourself, and you can either call the built in standard "hiera 3 like" interpolation or do it some other way. That means you can process results with your own interpolation mechanism.

Rune Juhl Jacobsen (JIRA)

unread,
Jun 11, 2018, 5:31:02 AM6/11/18
to puppe...@googlegroups.com

Henrik Lindberg Absolutely, I completely understand the reluctance to do something like this – I just thought I'd mention it, since it's what popped into my head as soon as I saw this issue
I'd love to (have the time to) try implementing it as a proof of concept new Hiera backend...

Henning Andersen (JIRA)

unread,
Jan 21, 2019, 11:35:03 AM1/21/19
to puppe...@googlegroups.com

Henrik Lindberg (JIRA)

unread,
Jan 21, 2019, 1:35:03 PM1/21/19
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug HI-599

The solution is to write a simple hiera backend function (in the puppet language) that returns the value. Alternatively if the String form can be parsed (for example for Integer, Boolean and Float values), the existing type casting feature in lookup_options works - i.e. transforming the string value back to the original data type.

This in wait of the possible alias feature mentioned above (possibly) being introduced.

Thomas Hallgren (JIRA)

unread,
Jan 28, 2020, 1:48:04 PM1/28/20
to puppe...@googlegroups.com
Thomas Hallgren commented on Bug HI-599

Perhaps we should have something identical to the strict_alias proposed in PUP-8593?. Given that an interpolation without any function is identical to scope, we could make strict imply a strict lookup in scope. E.g.

docker::selinux_enabled: "%{strict('facts.selinux.enabled')}"

would give the desired result.

Reply all
Reply to author
Forward
0 new messages