hiera deep hash merges broken

[Daniel Urist]

To get deep hash merges in hiera, I have been using the following in my hiera.yaml:

 

---
:merge_behavior: deeper
:backends:
  - eyaml

And in one of my classes, a call to hiera_hash() like this:

$if_merged = hiera_hash('profiles::pserver::interfaces')

However, with  puppetserver version 2.7.2-1puppetlabs1, this is now broken and fails with this error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Internal Server Error: org.jruby.exceptions.RaiseException: (Exception) Hiera type mismatch: expected Hash and got Array

I've tried commenting out ":merge_behavior: deeper" in hiera.yaml and using a lookup function instead:

$if_merged = lookup('profiles::pserver::interfaces', Hash, 'deep')

However, I'm not getting a merged hash when I do this.

How can I get a deep merge hash?

 

[Daniel Urist]

Just found this issue, which seems to describe what's going on: https://github.com/TomPoulton/hiera-eyaml/issues/222

So eyaml is broken wrt hash merges? Are there any workarounds available?

[Francois Lafont]

Hi,

On 02/14/2017 08:51 PM, Daniel Urist wrote:

> Just found this issue, which seems to describe what's going on:
> https://github.com/TomPoulton/hiera-eyaml/issues/222
> So eyaml is broken wrt hash merges? Are there any workarounds available?

I have exactly the same problem here, and no solution. And the problem is
that "eyaml" seems to be abandoned. So there is a little problem here.

I'm very interesting in any information. I think eyaml is widely used.

François Lafont

[Francois Lafont]

Oops, I have forgotten to say this:

On 02/14/2017 08:51 PM, Daniel Urist wrote:

> Just found this issue, which seems to describe what's going on:
> https://github.com/TomPoulton/hiera-eyaml/issues/222
> So eyaml is broken wrt hash merges? Are there any workarounds available?

Currently, the problem happens with puppet-agent version 1.9.x.
No problem with the version 1.8.3. So it could be a temporarily
workaround.

François Lafont

[Moses Mendoza]

Thanks all for the reports. A fix is in progress / en route, trackable via https://tickets.puppetlabs.com/browse/PUP-7215

François Lafont

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c473132b-b49f-2081-44dd-26fecc216b7f%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Moses Mendoza
Puppet

[Francois Lafont]

On 02/15/2017 05:44 PM, Moses Mendoza wrote:

> Thanks all for the reports. A fix is in progress / en route, trackable via
> https://tickets.puppetlabs.com/browse/PUP-7215

Ah ok, thanks Moses for the information.

I have tested in my testing VM and the commit of Thomas Hallgren seems
to work well. :)

François Lafont

[Eric Sorenson]

That's great to hear François, thank you for testing the patch! Our plan at this point is to accumulate a couple more fixes and ship a new build by Tuesday 21 Feb -

I don't want to generate a new build containing only the fix in PUP-7215 because 

(a) there is QA work underway on the current release that may turn up new things that need fixing by the end of the week

(b) there is a simple workaround in PUP-7216 which is that you can s/hiera_hash/lookup/ as the function that you call.  

--eric0

[Francois Lafont]

On 02/15/2017 09:24 PM, Eric Sorenson wrote:

> That's great to hear François, thank you for testing the patch!

You are welcome. :)

> Our plan at
> this point is to accumulate a couple more fixes and ship a new build by
> Tuesday 21 Feb -
>
> I don't want to generate a new build containing only the fix in PUP-7215
> because
> (a) there is QA work underway on the current release that may turn up new
> things that need fixing by the end of the week
> (b) there is a simple workaround in PUP-7216 which is that you can
> s/hiera_hash/lookup/ as the function that you call.

Ok, no problem, it's better to hold your workflow.

Bye.
François Lafont

[Daniel Urist]

It turns out my issue was caused by eyaml no longer supporting deep hash merges. I've worked around my issue by separating out plain yaml and eyaml. Fortunately none of my existing code does a deep hash merge on the eyaml keys. 

It would be great to get some reassurance that eyaml will continue to be properly supported going forward-- my impression is that it's pretty popular; certainly it's very useful.

François Lafont

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/442a6d8a-387b-1dbc-031b-4605842b706f%40gmail.com.

[Henrik Lindberg]

On 16/02/17 08:16, Daniel Urist wrote:
> It turns out my issue was caused by eyaml no longer supporting deep hash
> merges. I've worked around my issue by separating out plain yaml and
> eyaml. Fortunately none of my existing code does a deep hash merge on
> the eyaml keys.
>
> It would be great to get some reassurance that eyaml will continue to be
> properly supported going forward-- my impression is that it's pretty
> popular; certainly it's very useful.
>

We are adding support for eyaml in hiera 5 which means you will get
explain support and that you can use eyaml when also using the new
format. We now also test with eyaml.

You still need to install the eyaml gem separately.

Hope that helps.

- henrik

> On Wed, Feb 15, 2017 at 4:51 PM, Francois Lafont

> <francois.l...@gmail.com <mailto:francois.l...@gmail.com>>

> wrote:
>
> On 02/15/2017 09:24 PM, Eric Sorenson wrote:
>
> > That's great to hear François, thank you for testing the patch!
>
> You are welcome. :)
>
> > Our plan at
> > this point is to accumulate a couple more fixes and ship a new build by
> > Tuesday 21 Feb -
> >
> > I don't want to generate a new build containing only the fix in PUP-7215
> > because
> > (a) there is QA work underway on the current release that may turn up new
> > things that need fixing by the end of the week
> > (b) there is a simple workaround in PUP-7216 which is that you can
> > s/hiera_hash/lookup/ as the function that you call.
>
> Ok, no problem, it's better to hold your workflow.
>
> Bye.
> François Lafont
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it,

> send an email to puppet-users...@googlegroups.com
> <mailto:puppet-users%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/442a6d8a-387b-1dbc-031b-4605842b706f%40gmail.com

> <https://groups.google.com/d/msgid/puppet-users/442a6d8a-387b-1dbc-031b-4605842b706f%40gmail.com>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/CAEo6%3DKZ%3D4LPZQJBtD8UZoNxrVS8gu_E5uZD7am8RSJu4abZ%2BUg%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-users/CAEo6%3DKZ%3D4LPZQJBtD8UZoNxrVS8gu_E5uZD7am8RSJu4abZ%2BUg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Francois Lafont]

Hi,

On 02/16/2017 09:03 PM, Henrik Lindberg wrote:

> We are adding support for eyaml in hiera 5 which means you will get explain support and that you can use eyaml when also using the new format. We now also test with eyaml.

By the way, and just for information ;), here an issue I have currently with eyaml
and Puppet 4 (_but_ I don't use hiera 5 yet):

https://github.com/TomPoulton/hiera-eyaml/issues/186

But I absolutely don't know if this issue is a Puppet side problem or a eyaml
side problem. If it's a Puppet side problem, it would be great to fix it with
hiera 5 \o/. If it's a eyaml side problem, unfortunately eyaml no longer seems
to be really maintained.

François Lafont

[Henrik Lindberg]

On 16/02/17 15:37, Francois Lafont wrote:
> Hi,
>
> On 02/16/2017 09:03 PM, Henrik Lindberg wrote:
>
>> We are adding support for eyaml in hiera 5 which means you will get explain support and that you can use eyaml when also using the new format. We now also test with eyaml.
>
> By the way, and just for information ;), here an issue I have currently with eyaml
> and Puppet 4 (_but_ I don't use hiera 5 yet):
>
> https://github.com/TomPoulton/hiera-eyaml/issues/186
>
> But I absolutely don't know if this issue is a Puppet side problem or a eyaml
> side problem. If it's a Puppet side problem, it would be great to fix it with
> hiera 5 \o/.

As I understand it, hiera 5 implementation is a reimplementation of the
topmost layer in eyaml and it should be resposible for handling merges
etc. The new functionality is available from Puppet 4.9.3 where you will
also get explain support for eyaml.

> If it's a eyaml side problem, unfortunately eyaml no longer seems
> to be really maintained.
>

There are talks about Vox Pupuli taking over stewardship of eyaml.

Best,
- henrik

> François Lafont