hiera - multiple yaml backends

[Brett Swift]

I'm wondering if anyone has this unique use case. 

We're going to experiment by giving our ops team their own hieradata repository, and keep our internal repository separate. 

(If you're curious,  we'll be giving them control over the %{::hostname}  tier,  and we'll keep common  / roles / project  layers in the hierarchy tied at the 'dev' hieradata). 

The reason we're trying this is to experiment with different controls over different repositories.  Ops owns one, so they don't have to have the rigor of feature branches and pull requests and the rest of the SDLC that comes with our dynamic environment testing approach. 

Ultimately: I want this: 

:backends:
    - opsyaml
    - yaml
:yaml:
    :datadir: /etc/puppetlabs/puppet/hieradata/%{::environment}
:opsyaml:
    :datadir: /etc/puppetlabs/puppet/hostdata
    :extension: yaml

but am limited because of this code: https://github.com/puppetlabs/hiera/blob/60f9d2d4b8b36e6dd47c3713dd64dc793b9745c0/lib/hiera/config.rb#L74-L82

Possible solutions:

1) create our own simple backend

2) symlink hostyaml_backend.rb  to yaml_backend.rb

3) use eyaml, and yaml backends,  just because it's easy .. :)

Does anyone have the same requirements?   What worked for you?

[Ramin K]

In the past I've used yaml for Ops and json for Dev. That worked well
and it was mostly automated scripts that we dropping files into a
different path.

While it's much more work you might consider Redis as a Hiera backend
coupled with an http user interface and api. I did some work around
allowing Jenkins to post to an API to update new version for Puppet to
push. I stole the idea directly from this talk at the 2013 Puppet COnf.

Ramin

On 2/9/15 12:12 PM, Brett Swift wrote:
> I'm wondering if anyone has this unique use case.
>
> We're going to experiment by giving our ops team their own hieradata
> repository, and keep our internal repository separate.
>
> (If you're curious, we'll be giving them control over the %{::hostname}
> tier, and we'll keep common / roles / project layers in the
> hierarchy tied at the 'dev' hieradata).
>
> The reason we're trying this is to experiment with different controls
> over different repositories. Ops owns one, so they don't have to have
> the rigor of feature branches and pull requests and the rest of the SDLC
> that comes with our dynamic environment testing approach.
>
>

> *Ultimately: *I want this:
>
> |
> :backends:
> -opsyaml
> -yaml

> :yaml:
> :datadir:/etc/puppetlabs/puppet/hieradata/%{::environment}
> :opsyaml:
> :datadir:/etc/puppetlabs/puppet/hostdata
> :extension:yaml
> |
>
>
> but am limited because of this
> code: https://github.com/puppetlabs/hiera/blob/60f9d2d4b8b36e6dd47c3713dd64dc793b9745c0/lib/hiera/config.rb#L74-L82
>
>

> *Possible solutions:*
> *
> *

> 1) create our own simple backend
> 2) symlink hostyaml_backend.rb to yaml_backend.rb
> 3) use eyaml, and yaml backends, just because it's easy .. :)
>
> Does anyone have the same requirements? What worked for you?
>

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/73086f39-fbe9-4a51-bdde-e562de88efe8%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/73086f39-fbe9-4a51-bdde-e562de88efe8%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[Ramin K]

gah, link is https://www.youtube.com/watch?v=7NBJAC10ato

[Brett Swift]

Yea I've tested a redis backend as an ENC but we haven't enabled it yet as we'll likely ditch it for the console enc in 3.7. 

The problem with Redis is lack of version control, which our Ops guys do want.   ( unless you did an http backend that served up a file system with your yaml files..  errr.   )

Sounds like I'm not the only one with this use case.   I might put in a feature request and/or hiera code if I can find the time. 

>> <mailto:puppet-users+unsub...@googlegroups.com>.

[Matthew Burgess]

On 9 February 2015 at 20:12, Brett Swift <brett...@gmail.com> wrote:
> I'm wondering if anyone has this unique use case.
>
> We're going to experiment by giving our ops team their own hieradata
> repository, and keep our internal repository separate.

We have a similar requirement whereby our infrastructure team
(handling core FW, networking and OS-level resources) and project
teams (largely deploying application-layer RPMs, but perhaps also
opening up additional host-level firewall ports) need access to 2
different Git repositories so the baddies (app dev) can't stomp over
the goodies' (infrastructure) hard work :-)

We've ended up rolling our own synchronisation script; a common Git
user has read-only access to both repositories and clones/pulls them
to a staging area. It then composes them into a single YAML backend
hierarchy, unlike your example where you have a 1-1 mapping between
sources and backends.

Regards,

Matt.