Inserting Data back into hiera

[Spencer Krum]

Hello,

The other day I was on IRC and someone asked how to export strings in PuppetDB. As far as I am aware this isn't possible with PuppetDB. It is possible with certain hiera backends.

I've created a hiera_insert module at https://github.com/nibalizer/hiera-insert. This module only has one function right now, redis_insert(). When using the redis backend to hiera(and some assumptions I've hardcoded for now), its possible to push key:value pairs into hiera and pull them out.

On its own this isn't very interesting. But it enables multi node communication. And since you can configure the hiera lookups on 'subscriber' nodes to fail catalog compilation until the key they're looking up on is present, you have some guarantees of in order execution across your infrastructure.

I'm wondering if other people think this is a good idea? Are there other implementations I should be contributing to? Would people use it if it supported different hiera backends? Really just interested in feedback.

It's not on the forge right now because I still consider it experimental and proof of concept. But if there is interest I will be happy to publish it.

Thanks,
Spencer

[Xav Paice]

This is something we've been attempting to overcome and have been
thinking about for some time - so far about 99% of our use cases have
been taken care of by exported resources and PuppetDB - e.g. rather than
using a fact and creating the resource on box B that refers to something
in box A, we simply export the resource itself on box A and collect it
on box B, filtering if needs be with tags. I've not looked at the IRC
logs but I'm sure that came up.

The remaining examples have included things which we've, so far, put
statically into hiera (we use yaml).

Your hiera_insert function looks to be the sort of thing we'd be able to
make good use of, even if we do have to change backend.

Would love to see the ability to incorporate encryption as well - then
we can hiera_insert an encrypted password/key/etc with some greater
confidence.

Thanks
Xav

[Spencer Krum]

so far about 99% of our use cases have
been taken care of by exported resources and PuppetDB

Where controlling a system that is well modeled by puppet resources, host entries in /etc/hosts for example, exported resources work well. I find myself often wanting to just pick up strings and use them in templates. The only way I know how to do this in PuppetDB is to export a concat fragment resource.

Your hiera_insert function looks to be the sort of thing we'd be able to
make good use of, even if we do have to change backend.

Of the 'dynamic' backends, redis, postgres, etc which backend would you be most likely to use?

Would love to see the ability to incorporate encryption as well - then
we can hiera_insert an encrypted password/key/etc with some greater
confidence.

Can you expand a bit more on this? Do you mean a transparent encryption layer? Right now you could put an encrypted password into a redis key. We could also write a function to encrypt a plaintext string to the puppet master's public gpg key. That would enable hostA to have the plaintext and export only the encrypted version into redis, then hostB would receive the cleartext as well. I'm not sure how 'encrypted' hiera really works, since it always seems pointless to me.

Thanks for your feedback!

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/532916B9.8080307%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
Spencer Krum
(619)-980-7820

[Xav Paice]

On 19/03/14 17:11, Spencer Krum wrote:

so far about 99% of our use cases have
been taken care of by exported resources and PuppetDB

Where controlling a system that is well modeled by puppet resources, host entries in /etc/hosts for example, exported resources work well. I find myself often wanting to just pick up strings and use them in templates. The only way I know how to do this in PuppetDB is to export a concat fragment resource.

Templates - yeah that's where it gets really hard using exported resources.

Your hiera_insert function looks to be the sort of thing we'd be able to
make good use of, even if we do have to change backend.

Of the 'dynamic' backends, redis, postgres, etc which backend would you be most likely to use?

we love postgres here, so I guess that would be it (plus it's already on our puppetmaster for puppetdb).  Others are an option, just more effort.

Would love to see the ability to incorporate encryption as well - then
we can hiera_insert an encrypted password/key/etc with some greater
confidence.

Can you expand a bit more on this? Do you mean a transparent encryption layer? Right now you could put an encrypted password into a redis key. We could also write a function to encrypt a plaintext string to the puppet master's public gpg key. That would enable hostA to have the plaintext and export only the encrypted version into redis, then hostB would receive the cleartext as well. I'm not sure how 'encrypted' hiera really works, since it always seems pointless to me.

sorry - wasn't very clear.  We use eyaml for passwords and one or two ssl certs.  The private key to decrypt the content of that yaml is located on the puppetmaster (not on the client) and the client receives the cleartext as the puppetmaster does the decryption as part of the eyaml backend.  It's not ideal, but suits our needs and was really easy to set up.  I've not explored the code to figure out how it works (yet).

Encrypting a string using the puppetmaster's public gpg key sounds like a better way to go, I'm not sure how the backend might trigger the decryption before sending the cleartext to hostB - but I'm sure we can work that one out.  It's a function of the hiera backend, and wouldn't fit into hiera_insert anyway.

Thanks
Xav