hiera_hash lookup returns struct, how to pass to function written in puppet?

[pokemon.mas...@gmail.com]

I'm relatively new to puppet but I used hiera_hash to lookup what I would consider a "hash" in yaml.

The data I tested with is rather complex in nature but it basically boils down to a hash of hashes with sub hashes / arrays.

Ex:  (my hieradata yaml file, pardon the lack of creativity in the names)

stuff:
  FirstNamedStuff:
    foo: bar
    foo2:
      - bara
      - barb
    foo3:
       anotherthing: hi
       yetanotherthing: hi2

Puppet (v4.3.2) is returning it as a struct (ex:  Struct[{FirstNamedStuff => Struct[] }] ) when I was expecting it to be returned as a hash.  I'm using functions written in puppet (not be confused with puppet functions written in ruby which if you google is what you'll get mostly) with type information for my parameter.  I pass in this value gotten from hiera_hash("stuff") to my custom function but it fails because it expected a hash and instead received a struct.

Ex:

function module::get_foo(Hash[String, Struct[{
     foo => String,
     foo2 => Array[String]
     foo3 => Hash[String, String]
   }] $myvar)
{
}

I think what's happening is hiera is always being treated as a struct for any hash.  I assumed they were interchangeable but it doesn't look like they are.

There are some cases where I want to enforce a hash, there are other cases where I want to enforce a struct.  Ex:

  FirstNameStuff for instance might be user defined, it could be anything.  But foo, foo2, foo3 are all properties that I expect and so a struct makes sense.  foo3's properties might be userdefined so a hash makes sense.


So the question is, can I somehow get this functionality from hieradata and a custom function?  Maybe this is just a bug with 4.3.2?  Or is my only hope to strip off the type information?

[Henrik Lindberg]

On 10/08/16 18:28, pokemon.mas...@gmail.com wrote:
> I'm relatively new to puppet but I used hiera_hash to lookup what I
> would consider a "hash" in yaml.
>
> The data I tested with is rather complex in nature but it basically
> boils down to a hash of hashes with sub hashes / arrays.
>
> Ex: (my hieradata yaml file, pardon the lack of creativity in the names)
> |
> stuff:
> FirstNamedStuff:
> foo:bar
> foo2:
> -bara
> -barb
> foo3:
> anotherthing:hi
> yetanotherthing:hi2
> |
>
> Puppet (v4.3.2) is returning it as a struct (ex:
> Struct[{FirstNamedStuff => Struct[] }] ) when I was expecting it to be
> returned as a hash. I'm using functions written in puppet (not be
> confused with puppet functions written in ruby which if you google is
> what you'll get mostly) with type information for my parameter. I pass
> in this value gotten from hiera_hash("stuff") to my custom function but
> it fails because it expected a hash and instead received a struct.
>

A Struct data type is a Hash, only with more details about the keys and
the type of the value per key.

> Ex:
>
> |
> functionmodule::get_foo(Hash[String,Struct[{

> foo =>String,
> foo2 =>Array[String]
> foo3 =>Hash[String,String]

> }]$myvar)
> {
> }
> |
>
>
> I think what's happening is hiera is /always/ being treated as a struct

> for any hash. I assumed they were interchangeable but it doesn't look
> like they are.
>

They are.

> There are some cases where I want to enforce a hash, there are other
> cases where I want to enforce a struct. Ex:
>
> FirstNameStuff for instance might be user defined, it could be
> anything. But foo, foo2, foo3 are all properties that I expect and so a
> struct makes sense. foo3's properties might be userdefined so a hash
> makes sense.
>

You can use Hash or Struct - a Hash[String, Any] would allow any data
that has string keys.

The Hash type applies the same key and value data type rule for all
entries, the Struct allows you to define the value types specifically
per key, as well as the optionality of the keys (missing entries).

>
> So the question is, can I somehow get this functionality from hieradata
> and a custom function? Maybe this is just a bug with 4.3.2? Or is my
> only hope to strip off the type information?
>

There have been some bugs in the type system that have been fixed. It
may be that you are bitten by one of those. Some examples are needed of
actual input and error messages to be able to tell.

Hope that helps.
- henrik

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-users/21d19acd-b054-41eb-9c56-da1d6c6bfd44%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/21d19acd-b054-41eb-9c56-da1d6c6bfd44%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[pokemon.mas...@gmail.com]

Thanks Henrik,

I created a sample, here's what you need to reproduce the error (which is mentioned at the bottom)

function

function test::myfunc(Struct[{
  propertya => Hash[String, Struct[{propertya_1 => Array[Integer], propertya_2 => Array[String]}]]
}] $sample
{
  1
}

hieradata

classes:
  - test
myhash:
  something:
    propertya:
      propertya_1:
        -1
        -2
      propertya_2:
        - "foo"

manifest (for module test)

class test
{
  $empty_hash = {}
  $data = hiera_hash("myhash", $empty_hash)
  $output = $data.reduce(0) |$memo, $value|
  {
    test::myfunc($value[1])
  }
  notify{"Value is ${$output}":}
}

Error received:

"Evaluation Error: Error while evaluating a Function Call, 'test::myfunc' parameter 'sample' struct member propertya expects a Hash value, got Struct at init.pp:8:5"

[pokemon.mas...@gmail.com]

That 8:5 in the error message should be an 7:5, I had a commented out line in the test class to print the type of $data

[Henrik Lindberg]

On 11/08/16 06:13, pokemon.mas...@gmail.com wrote:
> Thanks Henrik,
>
> I created a sample, here's what you need to reproduce the error (which
> is mentioned at the bottom)
>
> function
> |
> functiontest::myfunc(Struct[{
> propertya =>Hash[String,Struct[{propertya_1

> =>Array[Integer],propertya_2 =>Array[String]}]]

> }]$sample
> {
> 1
> }
> |
>
> hieradata
> |
> classes:
> -test
> myhash:
> something:
> propertya:
> propertya_1:
> -1
> -2
> propertya_2:
> -"foo"
> |
>
> manifest (for module test)
> |
> classtest
> {
> $empty_hash ={}
> $data =hiera_hash("myhash",$empty_hash)
> $output =$data.reduce(0)|$memo,$value|
> {
> test::myfunc($value[1])
> }
> notify{"Value is ${$output}":}
> }
> |
>
>
> Error received:
>

> *"Evaluation Error: Error while evaluating a Function Call,

> 'test::myfunc' parameter 'sample' struct member propertya expects a Hash

> value, got Struct at init.pp:8:5"*
>

Thanks, as I mentioned there were some issues with Struct/Hash
type-checking that are now fixed. (Need a bit of detective work to
figure out which ticket is applicable to your case).

I tested the following on latest 4.x (to be 4.6.0).

test.pp
-------
class foo(Struct[{
x => Array[Integer],
y => Array[String]}] $arg
){
notice $arg
}

class { 'foo':
arg => {x => [1,2,3], y => ["foo"]}
}


If you try that sample (a puppet apply of that) it takes hiera out of
the equation (just to be sure there is no glitch/typo there) and it
ensures that only the code snippet in question is evaluated.


This is what I get on 4.x latest:

run
---
puppet apply test.pp
Notice: Scope(Class[Foo]): {x => [1, 2, 3], y => [foo]}
Notice: Compiled catalog for henrik2 in environment production in
0.08 seconds
Notice: Applied catalog in 0.01 seconds

Regards,

- henrik


>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/fb0d655b-d586-4e67-a204-407016dd7480%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/fb0d655b-d586-4e67-a204-407016dd7480%40googlegroups.com?utm_medium=email&utm_source=footer>.

[pokemon.mas...@gmail.com]

Hi Henrik, thanks again for your help.  I tried it and go the same result as you on puppet 4.3.2

[pokemon.mas...@gmail.com]

I ended up coming up with a way to reproduce it without hieradata:

init.pp

class foo(Hash[String, Struct[{

  x => Array[Integer],
  y => Array[String]}]] $arg
)
{

  notify{"$arg":}
}

class test
{
  class
  {
   'foo':
   arg => {propertya => {propertya_1 => [1, 2], propertya_2 => ["foo"]}}
  }
}

The error I get is:

"Evaluation Error: Error while evaluating a Resource Statement, Class[Foo] : parameter 'arg' expects a Hash value, got Struct"

Hopefully that helps!

[Henrik Lindberg]

On 11/08/16 17:07, pokemon.mas...@gmail.com wrote:
> Hi Henrik, thanks again for your help. I tried it and go the same
> result as you on puppet 4.3.2
>

Then it must be something wrong with the data from data binding since
clearly, if you give it valid data it works.

- henrik

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/d39213e6-7a2c-4b3c-9501-d02900e7902e%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/d39213e6-7a2c-4b3c-9501-d02900e7902e%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Henrik Lindberg]

On 11/08/16 17:36, pokemon.mas...@gmail.com wrote:
> I ended up coming up with a way to reproduce it without hieradata:
>
> init.pp
>
> |
> classfoo(Hash[String,Struct[{
> x =>Array[Integer],
> y =>Array[String]}]]$arg
> )
> {
> notify{"$arg":}
> }
>
> classtest
> {
> class
> {
> 'foo':
> arg =>{propertya =>{propertya_1 =>[1,2],propertya_2 =>["foo"]}}
> }
> }
> |
>

That example is flawed, you are passing propertya and propertya_1 etc.
when the Struct expects 'x' and 'y'.

The problem here is the bad error message as it is not telling you what
is missing. In the latest version of puppet you get this:

Error: Evaluation Error: Error while evaluating a Resource Statement,
Class[Foo]:
parameter 'arg' entry 'propertya' expects a value for key 'x'
parameter 'arg' entry 'propertya' expects a value for key 'y'
parameter 'arg' entry 'propertya' unrecognized key 'propertya_1'
parameter 'arg' entry 'propertya' unrecognized key 'propertya_2'

- henrik

>
> The error I get is:
>

> *"Evaluation Error: Error while evaluating a Resource Statement,

> Class[Foo] : parameter 'arg' expects a Hash value, got Struct"
>

> *Hopefully that helps!

>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com
> <mailto:puppet-users...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/acb9a227-2581-4b9d-8904-d65d882734ec%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/acb9a227-2581-4b9d-8904-d65d882734ec%40googlegroups.com?utm_medium=email&utm_source=footer>.

[pokemon.mas...@gmail.com]

Thanks Henrik.  That was a flawed example.  And as a matter of fact, since the error message outputted was so similar, I went back to my issue and noticed that it was indeed a typo!  I'm sorry for all the trouble caused but I just assumed there was a bigger issue at hand.  Any idea on when the error message changed?

Anyway, thank you again for all your help.  I really appreciate it!