Resource ordering only using Hiera?

[Joseph Swick]

Hello,
I'm trying to come up with a generic wrapper class for use with hiera so
that I can have a service that is completely managed and configured via
hiera and its own puppet module to be notified by Puppet of changes to a
file or other resource managed by a different class/module. The example
I'm using here is the management of /etc/security/limits.conf for a
service account that only needs to be changed on certain systems.

[Yes, I could put the configuration of the limits.conf into the module
for this particular service, but I'm trying to keep the dependencies of
that module to a minimum so that it is portable.]

Picture the following yaml hiera data for a node (the basic::limits
class is a wrapper for the puppet-limits module):

---
classes:
- foo
- basic::limits
foo::option_1: true
basic::limits::instance:
foo_limits:
user: 'foobar'
limit_type: 'nofile'
hard: '10000'
soft: '10000'


If I create a separate class, I can do the resource ordering normally as
expected:

class tmp {
Limits::Limits['foo_limits'] ~> Service['foo']
}


This separate class ensures that changes to the limits configuration
properly notifies the service to restart. However, I don't want to
clutter up our puppet configuration with a lot of singular
modules/classes just for resource ordering and would like to be able to
define it just in hiera. Therefore, I attempted to create a generic
wrapper define for resource ordering (the following doesn't actually
work, will get to the error):

#Define for resource ordering or subscribing
define basic::order_resource::ordering (
$resource1_type = undef,
$resource1_name = undef,
$resource2_type = undef,
$resource2_name = undef,
$notify = false,
){
if $notify {
$resource1_type[$resource1_name] ~> $resource2_type[$resource2_name]
} else {
$resource1_type[$resource1_name] -> $resource2_type[$resource2_name]
}
}


And a wrapper class to use it via hiera:

#basic::order_resource
class basic::order_resource ($instance){
$real_instance = hiera_hash(basic::order_resource::instance)
create_resources(basic::order_resource::ordering, $real_instance)
}


Sample yaml in hieradata:

basic::order_resource::instance:
foo_limits_sub:
resource1_type: Limits::Limits
resource1_name: foo_limits
resource2_type: Service
resource2_name: foo
notify: true


However, as I already stated, my generic define doesn't actually work
and returns the following error because Puppet can't/doesn't evaluate
the variable as an actual resource type:

Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: resource1_type is not a hash or array when accessing it with
foo_limits at
/etc/puppet/modules/basic/manifests/order_resource/ordering.pp:10 on
node vagrant-centos-6


What am I missing to get Puppet to evaluate the $resourceX_type
variables as a resource type [e.g: File, Service, etc.] to get this to
work? I'd like to try do it this way so that I don't have to make a
very long, messy and hard to maintain class with a lot of nested case
statements to perform a similar function. That is, unless there's some
other method that I haven't considered/found/thought of.

sample nested case code (this will get ugly fast and isn't very flexible):
case $resource1_type {
'File': {
case $resource2_type {
'Service': {
if $notify {
File[$resource1_name] ~> Service[$resource2_name]
} else {
File[$resource1_name] -> Service[$resource2_name]
}
}
default: {fail("Unknown resource type '${resource2_type}' for
\$resource2_type")}
}
}
default: {fail("Unknown resource type '${resource1_type}' for
\$resource1_type")}
}


Thank you for taking the time to wade through my lengthy question.
--
Joseph Swick <joseph...@meltwater.com>
Operations Engineer
Meltwater Group

[jcbollinger]

On Friday, December 13, 2013 3:56:50 PM UTC-6, Joseph Swick wrote:

[...]

What am I missing to get Puppet to evaluate the $resourceX_type
variables as a resource type [e.g: File, Service, etc.] to get this to
work?

Puppet DSL does not provide such a feature.  It is conceivable -- likely, even -- that you could instead create a custom function to apply generic ordering constraints such as you want but you should beware that unlike relationships declared via the DSL, relationships applied via a function would probably be sensitive to manifest evaluation order.

 

 I'd like to try do it this way so that I don't have to make a
very long, messy and hard to maintain class with a lot of nested case
statements to perform a similar function.  That is, unless there's some
other method that I haven't considered/found/thought of.

sample nested case code (this will get ugly fast and isn't very flexible):
case $resource1_type {
  'File': {
    case $resource2_type {
      'Service': {
        if $notify {
          File[$resource1_name] ~> Service[$resource2_name]
        } else {
          File[$resource1_name] -> Service[$resource2_name]
        }
      }
      default: {fail("Unknown resource type '${resource2_type}' for
\$resource2_type")}
    }
  }
  default: {fail("Unknown resource type '${resource1_type}' for
\$resource1_type")}
}

I suspect that the use case for the functionality you describe is not nearly so general as you suppose.  Module portability is more typically approached in Puppet by using facts -- custom facts, if necessary -- to probe the relevant characteristics of the target machine, and by using a small number of higher-level alternatives to determine what to declare.  For example, it is common to use the $::osfamily fact to guide what declarations to make, coding appropriate specifics for each supported OS family.


John

[Joseph Swick]

On 12/16/2013 10:59 AM, jcbollinger wrote:
>
>
> On Friday, December 13, 2013 3:56:50 PM UTC-6, Joseph Swick wrote:
>
> [...]
>
> What am I missing to get Puppet to evaluate the $resourceX_type
>> variables as a resource type [e.g: File, Service, etc.] to get this to
>> work?
>
>
>
> Puppet DSL does not provide such a feature. It is conceivable -- likely,
> even -- that you could instead create a custom function to apply generic
> ordering constraints such as you want but you should beware that unlike
> relationships declared via the DSL, relationships applied via a function
> would probably be sensitive to manifest evaluation order.
>
>
>>>
>

> I suspect that the use case for the functionality you describe is not
> nearly so general as you suppose. Module portability is more typically
> approached in Puppet by using facts -- custom facts, if necessary -- to
> probe the relevant characteristics of the target machine, and by using a
> small number of higher-level alternatives to determine what to declare.
> For example, it is common to use the $::osfamily fact to guide what
> declarations to make, coding appropriate specifics for each supported OS
> family.
>
>
> John
>

Thank you for the reply. However, I think you may not be fully
understanding what I was trying to describe (or I didn't describe it
well enough). What I'm trying to do has nothing do do with resource
facts on the system within a module. We're already using various facts
to define which portions of the hieradata should be considered for a
particular node. I would like to ensure that the resources that get
defined as a result are created in the correct order when required.

Tl;dr;: I'm looking to see if I can achieve resource chaining wholly
within hiera definitions without having to create singular
manifests/classes for nodes to include via hiera, which the sole purpose
is to order resources that are being realized and configured via hiera.

It's quite likely what I'm trying to achieve isn't currently possible in
Puppet, but I was hoping that someone could point out something I
overlooked. If not, I'll have to find an appropriate place in our
puppet site to create the resource ordering classes for various nodes
where the order of resource creation is critical.

Thank you.

[jcbollinger]

On the contrary, you were clear about that the first time.  You are looking for a general-purpose way to encode generic resource relationships in your hiera data.  You cast the reason for wanting to do so in terms of module portability.

As I wrote the first time, Puppet DSL+hiera does not support what you hope to do, though likely you could achieve it with the help of a custom function.  The rest of what I wrote was basically a recommendation against doing it even with a custom function.  I don't think it will serve your portability objective very well.

John