chaining of create_resources

[kashif]

Hi
I am using create_resource to create a dir and then mount it. I am using two create_resources and want one to be completed before other.

$mount_point = hiera('test::mount_point', [])
$defaults = {
 'ensure' => 'directory',
}
$mountit = hiera('test::mountit')
create_resources (file, $mount_point, $defaults)
create_resources (mount, $mountit)

hiera_yaml is

test::mount_point:
  '/local_area':
        owner: root
        group: root
        mode: 0644
 
test::mountit:
   '/local_area':
         device  : "mymachine:/software"
         fstype  : "nfs"
         ensure  : "mounted"
         options : "nfsvers=3,tcp,noatime,hard,intr,exec,rw,bg,nosuid"
         atboot  : true
#         require : [ File['/local_area'] ],
  

It works if I comment out require line but fail with this error if I un-comment require line

Error: Failed to apply catalog: Could not find dependency File['/local_area'] for Mount[/local_area]

Is there any other way to chain two create_resources statements ?

Thanks
Kashif

[Adam Stacey]

I would be interested to know if you found a solution for this...? 

Adam 

This message may contain confidential material. If you are not the intended recipient, please notify the sender and destroy all copies.
We may monitor communications to and from our network.

[jcbollinger]

I didn't think it was possible to record resource references in an Hiera data file.  Or more precisely, I know YAML has no concept of resource references, and I don't think either Hiera itself or create_resources() has any special tooling to convert strings that have the form of resource references into bona fide resource references. 

 

Is there any other way to chain two create_resources statements ?

You cannot chain function calls because they happen during catalog compilation, not during catalog application.  You can declare resource relationships after the fact, however, with the chaining operator:

File['/local_area'] -> Mount['/local_area']

That you are using create_resources() (why?) suggests that you may not want to put explicit chains such as that into your manifests, but there are a number of ways to deal with that.  For example, if you can rely on all the mount titles to be the mount point names (i.e. you are not specifying any 'name' parameters for the mounts) then you can extract the mount point list via the keys() function provided by puppetlabs-stdlib.  You can then use a defined type to declare the needed relationships:

class site::mounts1 {
  # ...
  $mount_points = keys($mountit)
  site::mountpoint_relationship { $mount_points: }
}

define site::mountpoint_relationship() {
  File[$name] -> Mount[$name]
}

But you can do better than that.  Supposing that you want all your mount point directories to be configured the same (since they will be invisible anyway when the remote FS is mounted), why do you need to describe them separately in your hiera data?  Moreover, wouldn't it make sense to better connect declaration of the mount with declaration of the mount point?  To do that, just extend the scope of the definition to encompass full declaration of mount and mount point, based on the data for the mount.  Here's one slightly roundabout way to do that:

class site::mounts2 {
  # ...
  $mount_points = keys($mountit)
  site::mountpoint { $mount_points: mount_data => $mountit }
}

define site::mountpoint($mount_data) {
  # Declare one Mount
  $my_mount_data = { $name => $mount_data[$name] }
  create_resources('mount', $my_mount_data)

  # Declare its mount point
  file { $name:
    ensure => 'directory',
    owner => 'root',
    group => 'root',
    mode => '0755',
    # declare the relationship here
    before => Mount[$name]
  }
}

I don't particularly like using create_resources() there, but it helps highlight both the similarities with and the differences from your starting point, and it simplifies the code for dealing with the parameters.  If you don't like the details then I'm sure you can come up with several variations on the same theme.


John

[jcbollinger]

On Monday, September 2, 2013 5:57:44 AM UTC-5, kashif wrote:

Hi
I am using create_resource to create a dir and then mount it. I am using two create_resources and want one to be completed before other.

$mount_point = hiera('test::mount_point', [])
$defaults = {
 'ensure' => 'directory',
}
$mountit = hiera('test::mountit')
create_resources (file, $mount_point, $defaults)
create_resources (mount, $mountit)

Do be aware, however, that it is distinctly problematic to manage a mount point directory with Puppet, because the operating system actively obfuscates the distinction between the mount point itself and the root of the remote filesystem mounted on it (if any).  This is an intentional feature of UNIX and similar systems.  That will present a problem for you if you want to manage the mount point, but you don't want to (or can't) manage the mounted filesystem root, which is a rather common situation.

As it happens, another current thread suggests as good a solution as I've ever come across: use an Exec to create the mount point directory if it is absent.  That would look something like this:

exec { "Mount point $mountpoint":
  command => "/usr/bin/install -u root -g root -m 0755 -d $mountpoint",
  creates => $mountpoint,
  before => Mount[$mountpoint]
}


John

[Adam Stacey]

John - many thanks for the detailed answer. I wasn't aware of the 'keys' function of stdlib - this is very useful. For my particular problem, using an Exec (with the Before metaparameter) fits perfectly.

Regards

Adam

[José Luis Ledesma]

There are other options. As Jon suggest file(mountpoint) and mount are related, so perhaps it is easy to do something like:

hiera:

 
test::mountit:
   '/local_area':
         device  : "mymachine:/software"
         fstype  : "nfs"
         ensure  : "mounted"
         options : "nfsvers=3,tcp,noatime,hard,intr,exec,rw,bg,nosuid"
         atboot  : true

        owner: root
        group: root
        mode: 0644
#         require : [ File['/local_area'] ],

define megamount ( $mp=$title, $device, $fstype, $options, $atboot, $owner, $group, $mode)

 file { $mp:

    owner => $owner,

    group => $group,

    mode => ·mode.

}

mount { $mp:

   device => $device ....

   ... more parameters here...

 }

File[$mp] -> Mount[$mp]

}

then in the manifests

$mounts=hiera(test::mountit)

create_resources(megamount,$mounts)

}

and there is yet another option (although I'm not fan of it). You can set tags in the hiera and use a resource collector. This would be something like:

$mount_point = hiera('test::mount_point', []) 
$defaults = {
 'ensure' => 'directory',
}
$mountit = hiera('test::mountit')
create_resources (file, $mount_point, $defaults)
create_resources (mount, $mountit)

File<|tag==first|> -> Mount<|tag==second|>       #<<<<<<<<<<<<<<<<<<<<<<<<<<<<-------------

hiera_yaml is

test::mount_point:
  '/local_area':
        owner: root
        group: root
        mode: 0644

        tag:  "first"  #<<<<<<<<<<<<<<<<<<<<<<<<<<<<-------------

 
test::mountit:
   '/local_area':
         device  : "mymachine:/software"
         fstype  : "nfs"
         ensure  : "mounted"
         options : "nfsvers=3,tcp,noatime,hard,intr,exec,rw,bg,nosuid"
         atboot  : true
#         require : [ File['/local_area'] ],

         tag: "second"   #<<<<<<<<<<<<<<<<<<<<<<<<<<<<-------------
   

About using the "require" I think the problem is the parser doesn't found the File['/local_area'] beacuse it's created in the "create_resources", although IMHO it should work (although I don't know the complexity of make this possible)

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/a4755499-86d7-40ef-915a-0c094c74d3a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
José Luis Ledesma

[Baptiste]

Hi,

Le mercredi 26 mars 2014 16:03:05 UTC+1, Jose Luis Ledesma a écrit :

About using the "require" I think the problem is the parser doesn't found the File['/local_area'] beacuse it's created in the "create_resources", although IMHO it should work (although I don't know the complexity of make this possible)

It seems that it is just a matter of correctly writing the dependency. The following two forms should work equally:

require "File[/local_area]"

require: File[/local_area]

See https://ask.puppetlabs.com/question/3130/trying-to-set-dependencies-with-hiera-and-create_resources/?answer=4362#post-id-4362

[jcbollinger]

No, that's very mixed up.

There are three different, but related, things that are being commingled there: the 'require' statement/function of Puppet DSL, a hash key 'require' in the hiera data describing a resource, and, by context and implication, the 'require' metaparameter that all resource types have.

The first form given, "require <quoted string>", can only be a use of the 'require' function (http://docs.puppetlabs.com/references/3.stable/function.html#require).  That function is for declaring classes (and simultaneously declaring a relationship to declared classes), therefore its argument must be a class name or an array of them.  It does not accept resource references, so that form will not work.

In this context, the second form given, "require: <resource reference string>" can be only a key/value pair appearing in an Hiera data file.  It may be valid YAML, but to YAML the value is just a string.  Notwithstanding one claim in one (wrong) answer to that ask.puppetlabs.com question, all reports from the field -- including this very thread -- are consistent that that approach does not work.  Hiera does not automagically coerce the string value to a resource reference.

Not only will neither of those work, but it is misleading to compare them as if they were direct alternatives.  One is a function call that (if it worked) would need to appear in an appropriate place in a Puppet manifest file, whereas the other is a YAML fragment that could make sense only in an Hiera data file.  They are not different forms of the same thing.


John

[Baptiste]

Le mardi 8 avril 2014 15:27:13 UTC+2, jcbollinger a écrit :

No, that's very mixed up.

There are three different, but related, things that are being commingled there: the 'require' statement/function of Puppet DSL, a hash key 'require' in the hiera data describing a resource, and, by context and implication, the 'require' metaparameter that all resource types have.

The first form given, "require <quoted string>", can only be a use of the 'require' function (http://docs.puppetlabs.com/references/3.stable/function.html#require).  That function is for declaring classes (and simultaneously declaring a relationship to declared classes), therefore its argument must be a class name or an array of them.  It does not accept resource references, so that form will not work.

In this context, the second form given, "require: <resource reference string>" can be only a key/value pair appearing in an Hiera data file.  It may be valid YAML, but to YAML the value is just a string.  Notwithstanding one claim in one (wrong) answer to that ask.puppetlabs.com question, all reports from the field -- including this very thread -- are consistent that that approach does not work.  Hiera does not automagically coerce the string value to a resource reference.

Not only will neither of those work, but it is misleading to compare them as if they were direct alternatives.  One is a function call that (if it worked) would need to appear in an appropriate place in a Puppet manifest file, whereas the other is a YAML fragment that could make sense only in an Hiera data file.  They are not different forms of the same thing.

OK, so there is something that I should do wrong. I did some tests prior to post this message as it was causing me troubles while trying to do a nodeless setup using hiera, with classes, defines and resources assigned using hiera.

I have a vagrant-based public project of a "node-less" project you can check/test to see how it behaves, and from what I can see, in the graph generated using the agent, the dependency is present in the relationship.dot: https://github.com/gnubila-france/puppet-vagrant-playground . (before generating the graph I removed the classes declaration from the hieradata/common.yaml file to simplify the graph output)
 
Here is the png of the generated relationships.dot for the client VM: https://files.bapt.name/relationships.png . It contains the relationship between mysql service and package.

Am I mis-interpreting this code and graph?

John

Best,
Baptiste

[jcbollinger]

Other reports suggest so, but there are other possibilities, such as the graph not going with the version of the manifests and/or data presented.  I am not inclined to set up a test environment for your code, but all past reports -- of which there have been several -- are consistent that resource references cannot be expressed in Hiera data.  And that's perfectly sensible.


John

[Baptiste Grenier]

Le 09/04/14 à 15:15, jcbollinger téléscripta :

> Other reports suggest so, but there are other possibilities, such as the
> graph not going with the version of the manifests and/or data presented.

That's why I provided a complete vagrant project allowing to easily and
confidently test such things.

> I am not inclined to set up a test environment for your code, but all past
> reports -- of which there have been several -- are consistent that resource
> references cannot be expressed in Hiera data. And that's perfectly
> sensible.

OK.

As it is very easy with vagrant, I did destroy the VM and restart from
scratch (I am using a debian base box from puppetlabs, with puppetlabs
and plopopertaions puppet modules) and added two more resources
(postgresql package and service) and the dependency are present in the
graphs files:
https://github.com/gnubila-france/puppet-vagrant-playground/blob/master/graphs/relationships.png

In case anyone is interested in checking or testing this I added the graphs
files (.dot and .png) to the github repository
https://github.com/gnubila-france/puppet-vagrant-playground

The README should be sufficient, only vagrant and builder/ruby/rubygems
(or puppet and r10k) are needed, and most of the things are documented
and automatized, there is just a documented workaround for an annoying
bug with VirtualBox 4.3.10.

> John

Best,
Baptiste




--
\,,/_[-_-]_\,,/

There was a young lady named Riddle
Who had an untouchable middle.
She had many friends
Because of her ends,
Since it isn't the middle you diddle.