Hiera Questions: Virtual User Resources and Hiera

[Dan White]

One way I have seen for setting up system users is to create them virtually and then realize them with the spaceship operator, say by ggroup -- like this:

User <| groups == 'wheel' |>

Reference: http://www.mail-archive.com/puppet...@googlegroups.com/msg29719.html

In the referenced posting, the OP was trying to create a bunch of virtual users with all the parameter data in hiera.
But this is usually done with create_resources and that don't do virtual :(

So, I am asking a couple of questions going in different directions:

Can anyone show me a way to create a virtual resource from hiera ?
Actually, that should be: Hot to create a bunch of virtual resources from a hiera-hash

Contrarywise, it was suggested in the referenced posting that one
"could find all users in all hierarchies with hiera_hash and then declare them at once".
The down side of that is that using the spaceship operator,
one can filter by class parameters and there is no problem if the same virtual resource is realized more than once.
I am not certain that the hash-merge would produce a list of unique resource parameters.
If the same user was in more than one hiera file, would it appear multiple times in the hash ?

[Gary Larizza]

That's exactly what I was suggesting.  If you declared the users as create_resources expected it, and included the same user in multiple levels of the hierarchy, the hash merging functionality would leave you with a single user declaration.  One caveat (as of right now): you need to declare ALL the parameters in ALL the levels of the hierarchy.

 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--

Gary Larizza

Professional Services Engineer

Puppet Labs

[Dan White]

Ah, Gary ! Just the brain I wish to pick ! :)

Thanks for the response. It makes sense.
However, if I perceive this properly, it would provide an All-Or-Nothing implementation of users.

I am looking for for a way to have a master list of users in hiera and then realize/instantiate a group-keyed-subset of the master list on each node.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

[Luke Bigum]

On 18/05/12 13:46, Dan White wrote:
> Ah, Gary ! Just the brain I wish to pick ! :)
>
> Thanks for the response. It makes sense.
> However, if I perceive this properly, it would provide an All-Or-Nothing implementation of users.
>
> I am looking for for a way to have a master list of users in hiera and then realize/instantiate a group-keyed-subset of the master list on each node.
>

> �Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.�
> Bill Waterson (Calvin& Hobbes)
>

Why not use a define to wrap the virtualised declaration of your users,
or tried and doesn't work? Theoretically like this:

--------------------
define virtualise_user ($name, $uid, ...) {
@user { $name:
uid => $uid,
}
}

$all_users = hiera_hash('users')

create_resources('virtualise_user', $all_users)
--------------------

-Luke

--
Luke Bigum

Information Systems
Ph: +44 (0) 20 3192 2520
luke....@lmax.com | http://www.lmax.com
LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN


FX and CFDs are leveraged products that can result in losses exceeding
your deposit. They are not suitable for everyone so please ensure you
fully understand the risks involved. The information in this email is not
directed at residents of the United States of America or any other
jurisdiction where trading in CFDs and/or FX is restricted or prohibited
by local laws or regulations.

The information in this email and any attachment is confidential and is
intended only for the named recipient(s). The email may not be disclosed
or used by any person other than the addressee, nor may it be copied in
any way. If you are not the intended recipient please notify the sender
immediately and delete any copies of this message. Any unauthorised
copying, disclosure or distribution of the material in this e-mail is
strictly forbidden.

LMAX operates a multilateral trading facility. Authorised and regulated
by the Financial Services Authority (firm registration number 509778) and
is registered in England and Wales (number 06505809).
Our registered address is Yellow Building, 1A Nicholas Road, London, W11
4AN.

[Dan White]

----- Luke Bigum <Luke....@lmax.com> wrote:
> On 18/05/12 13:46, Dan White wrote:
> > Ah, Gary ! Just the brain I wish to pick ! :)
> >
> > Thanks for the response. It makes sense.
> > However, if I perceive this properly, it would provide an All-Or-Nothing implementation of users.
> >
> > I am looking for for a way to have a master list of users in hiera and then realize/instantiate a group-keyed-subset of the master list on each node.
> >

> > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”

> > Bill Waterson (Calvin& Hobbes)
> >
>
> Why not use a define to wrap the virtualised declaration of your users,
> or tried and doesn't work? Theoretically like this:
>
> --------------------
> define virtualise_user ($name, $uid, ...) {
> @user { $name:
> uid => $uid,
> }
> }
>
> $all_users = hiera_hash('users')
>
> create_resources('virtualise_user', $all_users)
> --------------------

The original posting tried it like this:

class users {
$system_users = hiera('system_users')
$system_groups = hiera('system_groups')

create_resources("@users::mkuser",$system_users)
create_resources("@users::mkgroup",$system_groups)
} # class users

... and puppet screamed "could not create resource of unknown type @users::mkuser"

I can certainly try that and see what happens.

Thanks.

[jcbollinger]

On May 17, 11:16 pm, Gary Larizza <g...@puppetlabs.com> wrote:
> That's exactly what I was suggesting.  If you declared the users as
> create_resources expected it, and included the same user in multiple levels
> of the hierarchy, the hash merging functionality would leave you with a
> single user declaration.  One caveat (as of right now): you need to declare
> ALL the parameters in ALL the levels of the hierarchy.

That's because hiera hash merging is shallow, right? That is, every
value in the resulting hash is the exact one given for the associated
key at some level of the hierarchy? I'm not complaining; I just want
to confirm that I understand correctly.


John

[Gary Larizza]

Yep, Hiera doesn't currently support deep merges. 

 

John

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Dan White]

----- Luke Bigum <Luke....@lmax.com> wrote:

> On 18/05/12 13:46, Dan White wrote:
> > Ah, Gary ! Just the brain I wish to pick ! :)
> >
> > Thanks for the response. It makes sense.
> > However, if I perceive this properly, it would provide an All-Or-Nothing implementation of users.
> >
> > I am looking for for a way to have a master list of users in hiera and then realize/instantiate a group-keyed-subset of the master list on each node.
> >

> > “Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”

> > Bill Waterson (Calvin& Hobbes)
> >
>
> Why not use a define to wrap the virtualised declaration of your users,
> or tried and doesn't work? Theoretically like this:
>
> --------------------
> define virtualise_user ($name, $uid, ...) {
> @user { $name:
> uid => $uid,
> }
> }
>
> $all_users = hiera_hash('users')
>
> create_resources('virtualise_user', $all_users)
> --------------------
>

Closer, but no cigar yet !

user.pp:
-------------------------
define add_user ( $username, $uid, $ingroups, $info ) {
<clip>
}

define add_virtual_user ( $v_username, $v_uid, $v_ingroups, $v_info ) {
@add_user { "${name}":
username => "${v_username}",
uid => "${v_uid}",
ingroups => "${v_ingroups}",
info => "${v_info}",
}
}
-------------------------
/etc/puppet/hierdata/common.yaml:
-------------------------
users:
beast:
v_username : beast
v_uid : 6666
v_ingroups :
v_info : Let's see if this works
boo:
v_username : boo
v_uid : 6667
v_ingroups :
v_info : Let's see if this works also

-------------------------

And if I say:
hiera users -c /etc/puppet/hiera.yaml

I get:
{"beast"=>{"v_info"=>"Let's see if this works", "v_username"=>"beast", "v_ingroups"=>nil, "v_uid"=>6666}, "boo"=>{"v_info"=>"Let's see if this works also", "v_username"=>"boo", "v_ingroups"=>nil, "v_uid"=>6667}}

Which is out of order, but everything is there. That's how hashes are supposed to work !

But when I try to run the catalog with
$the_users = hiera('users') or $the_users = hiera_hash('users')
followed by :
create_resources ( 'add_virtual_user', $the_users )
I get:
Must pass a parameter or all necessary values at /etc/puppet/manifests/nodes/puppetmaster-node.pp:15 on node puppetmaster.foo.org

[Gary Larizza]

I wonder if the nil value is not being accepted as 'being passed' - can you populate the nil value and try again?

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Luke Bigum]

I agree with Gary, Dan, it's probably the lack of data in the 'v_ingroups' key in your YAML that create_resources() is complaining about. If it truly can't pass an empty key/val pair you could do something hacky like use the string "undef" then explicitly check for it in the define.

define add_virtual_user ( $v_username, $v_uid, $v_ingroups, $v_info ) {

�� if ($v_ingroups == "undef") {
������ $real_v_ingroups� = undef
�� } else {
������ $real_v_ingroups = $v_ingroups
�� }
� �@add_user { "${name}":
� � � �username => "${v_username}",
� � � �uid � � �=> "${v_uid}",
� � � �ingroups => "${real_v_ingroups}",
� � � �info � � => "${v_info}",
� �}

}

On 18/05/12 23:16, Gary Larizza wrote:

I wonder if the nil value is not being accepted as 'being passed' - can you populate the nil value and try again?

On Fri, May 18, 2012 at 2:41 PM, Dan White <yg...@comcast.net> wrote:

----- Luke Bigum <Luke....@lmax.com> wrote:

> On 18/05/12 13:46, Dan White wrote:

> > Ah, Gary ! �Just the brain I wish to pick ! �:)

> >
> > Thanks for the response. It makes sense.
> > However, if I perceive this properly, it would provide an All-Or-Nothing implementation of users.
> >
> > I am looking for for a way to have a master list of users in hiera and then realize/instantiate a group-keyed-subset of the master list on each node.
> >

> > �Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.�
> > Bill Waterson (Calvin& �Hobbes)

> >
>
> Why not use a define to wrap the virtualised declaration of your users,
> or tried and doesn't work? Theoretically like this:
>
> --------------------
> define virtualise_user ($name, $uid, ...) {

> � �@user { $name:
> � � �uid => $uid,
> � �}

> }
>
> $all_users = hiera_hash('users')
>
> create_resources('virtualise_user', $all_users)
> --------------------
>

Closer, but no cigar yet !

user.pp:
-------------------------
define add_user ( $username, $uid, $ingroups, $info ) {
<clip>
}

define add_virtual_user ( $v_username, $v_uid, $v_ingroups, $v_info ) {

� �@add_user { "${name}":
� � � �username => "${v_username}",
� � � �uid � � �=> "${v_uid}",
� � � �ingroups => "${v_ingroups}",
� � � �info � � => "${v_info}",
� �}
}
-------------------------
/etc/puppet/hierdata/common.yaml:
-------------------------
users:
� �beast:
� � � �v_username : beast
� � � �v_uid � � �: 6666
� � � �v_ingroups :
� � � �v_info � � : Let's see if this works
� �boo:
� � � �v_username : boo
� � � �v_uid � � �: 6667
� � � �v_ingroups :
� � � �v_info � � : Let's see if this works also

-------------------------

And if I say:
hiera users -c /etc/puppet/hiera.yaml

I get:
{"beast"=>{"v_info"=>"Let's see if this works", "v_username"=>"beast", "v_ingroups"=>nil, "v_uid"=>6666}, "boo"=>{"v_info"=>"Let's see if this works also", "v_username"=>"boo", "v_ingroups"=>nil, "v_uid"=>6667}}

Which is out of order, but everything is there. �That's how hashes are supposed to work !

But when I try to run the catalog with

�$the_users = hiera('users') �or $the_users = hiera_hash('users')
followed by :
�create_resources ( 'add_virtual_user', $the_users )
I get:
�Must pass a parameter or all necessary values at /etc/puppet/manifests/nodes/puppetmaster-node.pp:15 on node puppetmaster.foo.org

�Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.�

Bill Waterson (Calvin & Hobbes)

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

--

Gary Larizza

Professional Services Engineer

Puppet Labs

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

-- 
Luke Bigum

Information Systems
Ph: +44 (0) 20 3192 2520

luke....@lmax.com | http://www.lmax.com
LMAX, Yellow Building, 1A Nicholas Road, London W11 4AN

[Jeff McCune]

On Mon, May 21, 2012 at 1:24 AM, Luke Bigum <Luke....@lmax.com> wrote:

I agree with Gary, Dan, it's probably the lack of data in the 'v_ingroups' key in your YAML that create_resources() is complaining about. If it truly can't pass an empty key/val pair you could do something hacky like use the string "undef" then explicitly check for it in the define.

define add_virtual_user ( $v_username, $v_uid, $v_ingroups, $v_info ) {

   if ($v_ingroups == "undef") {

Do you really mean to be comparing to the string "undef" rather than the keyword undef (no quotes)?

There's a big difference...

If you want to test if a variable is undefined the best way is to do this:

if ($foo == undef) { notice "\$foo is undef" }

else { notice "\$foo is defined as ${foo}" }

-Jeff

[Luke Bigum]

On 22/05/12 00:22, Jeff McCune wrote:

On Mon, May 21, 2012 at 1:24 AM, Luke Bigum <Luke....@lmax.com> wrote:

I agree with Gary, Dan, it's probably the lack of data in the 'v_ingroups' key in your YAML that create_resources() is complaining about. If it truly can't pass an empty key/val pair you could do something hacky like use the string "undef" then explicitly check for it in the define.

define add_virtual_user ( $v_username, $v_uid, $v_ingroups, $v_info ) {

   if ($v_ingroups == "undef") {

Do you really mean to be comparing to the string "undef" rather than the keyword undef (no quotes)?

Yes, unfortunately I did. It's because when using Hiera 0.3 it's a bit difficult to figure out what a Ruby nil gets passed into Puppet as. Consider the following manifest using Dan's example YAML (v_ingroups is a nil value):

----------------
#---
#users:
#   beast:
#       v_username : beast
#       v_uid      : 6666
#       v_ingroups :
#       v_info     : Let's see if this works

define add_virtual_user ( $v_username, $v_uid, $v_ingroups, $v_info ) {

   notify { $name:
       message => "username = ${v_username}, uid = ${v_uid}, ingroups = ${v_ingroups}, info = ${v_info}",

   }
}

$the_users = hiera_hash('users')

notice($the_users["beast"])
notice("prints as ${the_users[beast][v_ingroups]}")

if ($the_users["beast"]["v_ingroups"] == undef) {
  notice("is == undef")
}

if (defined($the_users["beast"]["v_ingroups"])) {
  notice("is not defined")
}

if ($the_users["beast"]["v_ingroups"] == "") {
  notice("is empty string")
}

if (! $the_users["beast"]["v_ingroups"]) {
  notice("is false")
}

if ($the_users["beast"]["v_ingroups"]) {
  notice("is true")
}

if ($the_users["beast"]["v_ingroups"] == nil) {
  notice("is nil?")
}

create_resources('add_virtual_user', $the_users)
---------------

It's not an empty string, it's not undef (but when you print it it comes out as undef), it's not nil (which doesn't exist in Puppet), it's not false but it *is* true? I've came across this once before and can't remember what "nil" actually gets interpreted as.

So if you feed that Puppet hash directly into the create_resources() function, it complains about a missing parameter:

---------------------
biguml@biguml-laptop:~$ puppet apply test.pp
notice: Scope(Class[main]): v_usernamebeastv_uid6666v_ingroupsundefv_infoLet's see if this works
notice: Scope(Class[main]): undef
notice: Scope(Class[main]): is true
Must pass a parameter or all necessary values at /home/biguml/test.pp:40 on node biguml-laptop
---------------------

So my suggestion was to explicitly set "undef" as a string in the yaml, then match on that in the Puppet manifests. It's horrible but would work.

-Luke

There's a big difference...

If you want to test if a variable is undefined the best way is to do this:

if ($foo == undef) { notice "\$foo is undef" }

else { notice "\$foo is defined as ${foo}" }

-Jeff

[Dan White]

I found an answer to this particular issue. Thanks for the reminder so I can share the answer:

I found the hiera/yaml way to indicate an empty array !
So, to use my earlier example:

users:
beast:
username : beast
uid : 6666
ingroups :
- ''
info : Let's see if this works

Then, with a hiera call, I get :

{"beast"=>{"ingroups"=>[""], "uid"=>6666, "username"=>"beast", "info"=>"Let's see if this works"}

I was able to more forward past this problem after figuring that out.

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”

Bill Waterson (Calvin & Hobbes)

----- Luke Bigum <Luke....@lmax.com> wrote:

[Jeff McCune]

On Tuesday, May 22, 2012, Dan White wrote:

I found an answer to this particular issue.  Thanks for the reminder so I can share the answer:

I found the hiera/yaml way to indicate an empty array !
So, to use my earlier example:

users:
 beast:
     username : beast
     uid      : 6666
     ingroups :
         - ''
     info     : Let's see if this works

Then, with a hiera call, I get :

{"beast"=>{"ingroups"=>[""], "uid"=>6666, "username"=>"beast", "info"=>"Let's see if this works"}

This is actually a non-empty array hat had one element, the empt string.

This clearly seems like a bug in puppet and how it is handling Hash values. I'll take a look more as soon as I get into the office.

-Jeff

 

> > To post to this group, send email to puppet-users@

[Jeff McCune]

On Tue, May 22, 2012 at 8:50 AM, Jeff McCune <je...@puppetlabs.com> wrote:

On Tuesday, May 22, 2012, Dan White wrote:

I found an answer to this particular issue.  Thanks for the reminder so I can share the answer:

I found the hiera/yaml way to indicate an empty array !
So, to use my earlier example:

users:
 beast:
     username : beast
     uid      : 6666
     ingroups :
         - ''
     info     : Let's see if this works

Then, with a hiera call, I get :

{"beast"=>{"ingroups"=>[""], "uid"=>6666, "username"=>"beast", "info"=>"Let's see if this works"}

This is actually a non-empty array hat had one element, the empt string.

OK, I had a look today.  Much of the behavior of hashes and arrays whose elements are not defined has been resolved in Puppet 3.0.0rc.  If you could try that out it would help us make sure your problem has actually been solved in Puppet.

As to how to specify an empty array as the value of a hash key using Hiera and Puppet, this is the way:

--- 

  username: beast

  uid: 6666

  ingroups: []

  info: Let's see if this works

Notice it's just an empty set of square braces, no empty string.

 

This clearly seems like a bug in puppet and how it is handling Hash values. I'll take a look more as soon as I get into the office.

It is a bug, luckily we've fixed it in Puppet 3.0.x.  Please give the release candidates a try.

-Jeff 

[Abhay]

This issue is still unresolved as of 3.1.0

# hiera -h resources -c /etc/puppet/hiera.yaml

{"webapplication2"=>{"Consumer"=>"undef"}, "webapplication"=>{"Consumer"=>{"2a"=>nil, "1a"=>{"offset"=>"3"}, "3a"=>"undef"}, "indexer"=>nil}}

____hieranil.pp________

$fullhash = hiera_hash(resources)

create_resources(noundefparams,$fullhash)

define noundefparams($Consumer="abc", $indexer="def"){

if $Consumer {  notify {"$Consumer":}   }

}

_________________

]# puppet apply hieranil.pp

Error: Received incomplete information - no value provided for parameter indexer at /tmp/experiment/hieranil.pp:15 on node puppet-master

Wrapped exception:

Received incomplete information - no value provided for parameter indexer

Error: Received incomplete information - no value provided for parameter indexer at /tmp/experiment/hieranil.pp:15 on node puppet-master

[Dan White]

Great !
Now all I need is some deep hash merging...

“Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us.”
Bill Waterson (Calvin & Hobbes)

---

From: "Abhay" <chrungo...@gmail.com>
To: puppet...@googlegroups.com
Sent: Thursday, February 14, 2013 9:15:38 AM
Subject: Re: [Puppet Users] Hiera Questions: Virtual User Resources and Hiera

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.