Puppet Feature Request (lookup with keys containing ".")

[John Baird]

Problem:

  So, currently as has been explained to me, doing a lookup with keys containing "." chops the key at first "." because of scoping and namespace.  This makes file-based lookups impossible (as far as I can tell).  I would like to be able to use HTTP or other backends to serve files, which by their very nature WILL contain file extensions.

lookup('/path/to/somefile.txt') will lookup "/path/to/somefile" 

Expectation:

  While I understand that changing the very function of lookup to accomodate this is out of scope, I would still think that providing a way to tell lookup to NOT chomp or namespace the lookup would be appropriate?  

Something like "lookup('/path/to/somefile.txt', scope=false)"... In this manner, the user and more specifically, each lookup call can determine whether or not to include the dotted key.

Is this something we could push forward?  How would I go about doing that ?

[jcbollinger]

On Wednesday, October 4, 2017 at 10:54:30 AM UTC-5, John Baird wrote:

Problem:

  So, currently as has been explained to me, doing a lookup with keys containing "." chops the key at first "." because of scoping and namespace.

Well, that seems a bit of an odd characterization.  I think a better explanation would be that lookup() chops keys at the period character because that character is not allowed in (simple) keys.  It can be regarded as an operator when it appears in the key expression.  That has nothing to do with namespace or scope in Puppet's sense of those terms.  I realize that this limitation on keys is not well documented, and I sympathize with you about the confusion that it may have caused.  FWIW, essentially the same issue was previously raised against Hiera as HI-485.

 

  This makes file-based lookups impossible (as far as I can tell).  I would like to be able to use HTTP or other backends to serve files, which by their very nature WILL contain file extensions.

Yes, it does make general filenames unsuitable for use as keys.  If you want to serve files as you describe then you'll need to perform some kind of translation between filenames and keys.  Do note, by the way, that in that case, the '.' character is not the only one you would need to be prepared to translate.

I don't quite follow why you want to do as you describe, but you should consider whether the best alternative would be to take an altogether different approach.  We could talk about how that might look if you fill us in on the bigger picture.

 

Expectation:

  While I understand that changing the very function of lookup to accomodate this is out of scope, I would still think that providing a way to tell lookup to NOT chomp or namespace the lookup would be appropriate?  

Something like "lookup('/path/to/somefile.txt', scope=false)"... In this manner, the user and more specifically, each lookup call can determine whether or not to include the dotted key.

Is this something we could push forward?  How would I go about doing that ?

The lookup() function already has a form that accepts an options hash.  In principle, that could be leveraged for what you want instead of adding a new argument.  If you want to pursue that then the Puppet Developers group would be a better place to discuss it, and Puppet's issue tracker would be the place to submit a formal feature request.  I'm inclined to doubt that such a request would be accepted, but I've been wrong before.

John

[Henrik Lindberg]

You get what you want if you quote the entire key.

lookup("'/path/to/somefile.txt'")

Note the quotes inside of the string. In lookup any quoted part is taken
verbatim and will not be subject to interpretation as a 'dig' into the
value being looked up.

- henrik

> Is this something we could push forward?  How would I go about doing that ?
>

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/c4e8c719-5367-4da4-a7d5-a95f0684e5ea%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/c4e8c719-5367-4da4-a7d5-a95f0684e5ea%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Craig Dunn]

On Thu, Oct 5, 2017 at 6:44 PM, Henrik Lindberg <henrik....@puppet.com> wrote:

On 04/10/17 08:54, John Baird wrote:

Something like "lookup('/path/to/somefile.txt', scope=false)"... In this manner, the user and more specifically, each lookup call can determine whether or not to include the dotted key.

You get what you want if you quote the entire key.

  lookup("'/path/to/somefile.txt'")

Note the quotes inside of the string. In lookup any quoted part is taken verbatim and will not be subject to interpretation as a 'dig' into the value being looked up.

- henrik

Hi Henrik.

This is not the behaviour that I see.   The fact that interpolated strings still dig is actually useful for dynamically digging into hashes, that doesn't help the OP though.....

[root@puppet hieradata]# cat common.yaml

---

foo:

  bar:

    tango: delta

[root@puppet hieradata]# puppet apply -e 'notify { lookup("foo.bar.tango"): }'

...

Notice: delta

Notice: /Stage[main]/Main/Notify[delta]/message: defined 'message' as 'delta'

Regards

Craig

[Craig Dunn]

Apologies, I didn't spot the extra quotes (or read properly what you said about them)....  I clearly need more coffee... Yes, that does work...

---

"foo.bar.tango": bar

$bar = lookup("'foo.bar.tango'")

notify { $bar: }

Notice: /Stage[main]/Main/Notify[bar]/message: defined 'message' as 'bar'

--

Enviatics     |      Automation and Configuration Management

Puppet Labs Service Delivery Partner & Certified Consultant

http://www.enviatics.com | @Enviatics | cr...@enviatics.com