class parameters that depend on other parameters

[Tim Mooney]

Hi All!

We've been long-time users of puppet (opensource). A lot of our
home-grown modules were written to use direct hiera() calls (and before
that extlookup()) for loading config. Because of prior limitations
with class parameters, we also mostly avoided parameterized classes and
class inheritance.

Since I converted my site from puppet 3.8.7 to puppetserver 5.x and
puppet-agent 5.3.x, and converted us from hiera 3.x to hiera 5.x, a lot
of the "old ways" we were doing things can and should be modernized. For
example, I'm embarking on a project to convert all deprecated direct
hiera() calls to use lookup() instead, but before I do that I can also
greatly reduce the number of direct lookup() calls by making better use
of automatic parameter loading for classes, where appropriate.

One area where I've never found a good solution is class parameters that
depend on the value of other class parameters, especially when we want to
provide reasonable defaults for both but we also want to allow overriding
one or both of the parameters.

Here's an example:

modules/sandbox/manifests/init.pp:
#
# This module exists only to serve as a sandbox where we can experiment with
# puppet code.
#
class sandbox(
Enum['typeA', 'typeB', 'combined'] $server_type = 'combined',
String $service_name = $::sandbox::params::service_name,
) inherits ::sandbox::params {

notice("sandbox initialized.\n")

notice("\$server_type = ${server_type}\n")
notice("\$service_name = ${service_name}\n")

}

modules/sandbox/manifests/params.pp:
#
# a 'sandbox' class for the params pattern.
#
class sandbox::params {

$_server_type_actual = $::sandbox::server_type

case $_server_type_actual {
'combined': {
$service_name = 'sandbox-typeA+typeB'
}
'typeA': {
$service_name = 'sandbox-typeA'
}
'typeB': {
$service_name = 'sandbox-typeB'
}
default: {
fail("\n\nsandbox::server_type must be one of: combined, typeA, typeB\n")
}
}

}



Hopefully the *intent* is relatively clear: provide an intelligent default
value for $service_name based on what the value is for $server_type, but
allow our "intelligent default" value to be overridden. If
'sandbox::server_type' is set to 'typeB' in our hiera hierarchy, I want
the *default* value for 'sandbox::service_name' to become 'sandbox-typeB'.
If the person configuring the machine needs to override that too, they
should be able to, but setting just the first setting should provide
suitable defaults for the others.

This doesn't work, because there's a chicken-or-the-egg problem here.
Class sandbox inherits from sandbox::params to follow the "params
pattern", so settings in the parent class end up depending upon on
parameters to the child class.

Assuming I don't have any need to support old versions of puppet (anything
before 5.x), what's the current best practice for doing this?

Thanks,

Tim
--
Tim Mooney Tim.M...@ndsu.edu
Enterprise Computing & Infrastructure 701-231-1076 (Voice)
Room 242-J6, Quentin Burdick Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164

[Henrik Lindberg]

For starters you do not really need a params.pp or inheritance. Simply
configure all parameters in hiera.

Then, you can produce the default value by calling a function. For
example like this:

function mymodule::default_from_a($x) {
if $x == 'type-A' {
'sandbox-typeA'
}
}
class example($a, $b = mymodule::default_from_a($a)) {
notice $a
notice $b
}
class {example: a => 'type-A' }


With that, $b will get the value from the function if a value was not
given and there was no binding in hiera for it.

Running the above produces:

Notice: Scope(Class[Example]): type-A
Notice: Scope(Class[Example]): sandbox-typeA


Hope that helps.

- henrik

> Thanks,
>
> Tim


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Tim Mooney]

In regard to: Re: [Puppet Users] class parameters that depend on other...:

> On 2018-06-12 00:55, Tim Mooney wrote:

[snip some of my original context]

[snip some of my original context]

> For starters you do not really need a params.pp or inheritance. Simply
> configure all parameters in hiera.
>
> Then, you can produce the default value by calling a function. For example
> like this:
>
> function mymodule::default_from_a($x) {
> if $x == 'type-A' {
> 'sandbox-typeA'
> }
> }
> class example($a, $b = mymodule::default_from_a($a)) {
> notice $a
> notice $b
> }
> class {example: a => 'type-A' }

Thanks for your response Henrik! The function use is an interesting
approach that I would not have considered. It works well in the simple
example I presented and accomplishes what I was trying to do.

I'm not sure how I would scale this to something that's "real world" in
size, though.

Let's say you had *many* parameters that you wanted to set defaults for
(but allow overrides on an individual basis) based on a single parameter.
In my environment, the best example where we've used this is to support
the alternate versions of particular packages that are available for
RHEL (and respins) using Software Collections Library (SCL).

For example, on RHEL 6, the standard packages provide php 5.3.3 (plus
some backports and vendor "special sauce").

However, there are alternate versions available from SCL:

php54-php

php55-php

rh-php56-php

rh-php70-php

So to support various web application requirements, we have modules
that do stuff like (sorry for the lengthy code):

if $facts['os']['family'] == 'RedHat' {
if $facts['os']['release']['major'] == '5' {
#
# There's no easy httpd 2.4 option for RHEL 5, so barf
#
fail("\n\nRHEL 5 is not supported.\n\n")
} elsif $facts['os']['release']['major'] == '6' {

# PHP-related settings.
$php_variant = hiera('scl::php', 'php54')

if $php_variant == 'php54' or $php_variant == 'php55' {
# don't need to include the fpm package, as the phpfpm class gets it
$php_extra_packages = hiera('scl::php::extra_packages',
[
"${php_variant}-runtime",
$php_variant,
"${php_variant}-php-cli",
"${php_variant}-php-ldap",
"${php_variant}-php-mbstring",
"${php_variant}-php-pdo",
])
$php_fpm_package_name = "${php_variant}-php-fpm"
$php_fpm_service_name = $php_fpm_package_name
$php_fpm_pool_dir = "/opt/rh/${php_variant}/root/etc/php-fpm.d"
$php_fpm_pid_file = "/opt/rh/${php_variant}/root/var/run/php-fpm/php-fpm.pid"
$php_config_dir = "/opt/rh/${php_variant}/root/etc"
} elsif $php_variant != 'UNDEF' {
#
# for version 5.6.x and later, the name may include rh- at the start
# and many of the paths have changed.
#
# don't need to include the fpm package, as the phpfpm class gets it
$php_extra_packages = hiera('scl::php::extra_packages',
[
"${php_variant}-runtime",
$php_variant,
"${php_variant}-php-cli",
"${php_variant}-php-ldap",
"${php_variant}-php-mbstring",
"${php_variant}-php-pdo",
])
$php_fpm_package_name = "${php_variant}-php-fpm"
$php_fpm_service_name = $php_fpm_package_name
$php_fpm_pool_dir = "/etc/opt/rh/${php_variant}/php-fpm.d"
$php_fpm_pid_file = "/var/opt/rh/${php_variant}/run/php-fpm/php-fpm.pid"
$php_config_dir = "/etc/opt/rh/${php_variant}"
}
} else {

# hack to work around no support for true undef in hiera
$php_variant = hiera('scl::php', 'UNDEF')
#
# verify that $php_variant is either UNDEF or matches one of:
#
# php55
# rh-php56
# rh-php70
#
validate_re($php_variant, [
'^UNDEF$',
'^php55$',
'^rh-php56$',
'^rh-php70$'
],
'scl::php must be one of: UNDEF, php55, rh-php56, rh-php70'
)

if $php_variant == 'UNDEF' {
#
# RHEL 7 or later, with default php from the base operating system
#

# don't need to include the fpm package, as the phpfpm class gets it
$php_extra_packages = hiera('php::extra_packages',
[
'php-cli',
'php-ldap',
'php-mbstring',
'php-pdo',
])
$php_fpm_package_name = 'php-fpm'
$php_fpm_service_name = $php_fpm_package_name
$php_fpm_pool_dir = '/etc/php-fpm.d'
$php_fpm_pid_file = '/var/run/php-fpm/php-fpm.pid'
$php_config_dir = '/etc'
} elsif $php_variant == 'php55' {
#
# RHEL 7, with SCL php55
#

# don't need to include the fpm package, as the phpfpm class gets it
$php_extra_packages = hiera('scl::php::extra_packages',
[
"${php_variant}-runtime",
$php_variant,
"${php_variant}-php-cli",
"${php_variant}-php-ldap",
"${php_variant}-php-mbstring",
"${php_variant}-php-pdo",
])
$php_fpm_package_name = "${php_variant}-php-fpm"
$php_fpm_service_name = $php_fpm_package_name
$php_fpm_pool_dir = "/opt/rh/${php_variant}/root/etc/php-fpm.d"
$php_fpm_pid_file = "/opt/rh/${php_variant}/root/var/run/php-fpm/php-fpm.pid"
$php_config_dir = "/opt/rh/${php_variant}/root/etc"
} else {
#
# RHEL 7 or later, php from SCL, using the newer naming and layout
# conventions.
#
# don't need to include the fpm package, as the phpfpm class gets it
$php_extra_packages = hiera('scl::php::extra_packages',
[
"${php_variant}-runtime",
$php_variant,
"${php_variant}-php-cli",
"${php_variant}-php-ldap",
"${php_variant}-php-mbstring",
"${php_variant}-php-pdo",
])
$php_fpm_package_name = "${php_variant}-php-fpm"
$php_fpm_service_name = $php_fpm_package_name
$php_fpm_pool_dir = "/etc/opt/rh/${php_variant}/php-fpm.d"
$php_fpm_pid_file = "/var/opt/rh/${php_variant}/run/php-fpm/php-fpm.pid"
$php_config_dir = "/etc/opt/rh/${php_variant}"
}
}
} else {
fail("Unsupported osfamily=${$facts['os']['family']}\n")
}



As you can see, there are potentially dozens of settings that depend on
the "big feature switch" of 'scl::php', and we need to be able to override
some or all of the defaults.

While it would be possible to write a function to allow for each and
every one of these to have a dynamic default based on the setting
of scl::php, it's going to be fairly confusing to some of my coworkers
that don't work with puppet as much as I do. Maybe that's just the
trade-off I have to make to support this kind of thing.

Ultimately, I'm trying to modernize some of the now outdated practices
we've adopted in our modules, with an eye toward current best practices.
I don't want to have to keep explaining to our casual puppet users why
our modules look so different from stuff they see when they look at the
docs or examine a module from the forge.

[Henrik Lindberg]

Yeah, you cannot extend my simple example with one function being called
for one parameter to the general case of many parameters and possibly
with cross dependencies between them. While simpler cases would work it
would not look nice with a function call per parameter.

An alternative that almost works is to write a hiera backend, but it
cannot take given values into account so would only be able to compute
a default from "master-values" that come from hiera (or already assigned
variables - as it has no access to the context in which given values are
being bound to parameters of a class).

You could have a wrapper class that is the API, inside of it you would
compute all of the defaults and delegate to a private class to do the
actual work. This is kind of what params.pp pattern does but via
inheritance.

You could change the API of the class such that it accepts a struct.
You also write one function that computes the defaults given such a
struct. Users use both.

type MyStruct = Struct[a => Integer, Optional[b] => String]

class example( MyStruct $params) {
notice $params
}

function with_mystruct_defaults(MyStruct $input) >> MyStruct {
# given content of $input, returns the computed resulting hash
# with default values filled in
$input + case [$input['b'], $input['a']] {
[undef, Integer[101]] : { {'b' => 'x-large'} }
[undef, Integer[11,100]] : { {'b' => 'large'} }
[undef, Integer] : { {'b' => 'medium'} }
default : { { } }
}
}

# Declare it
class { 'example':
params => with_mystruct_defaults('a' => 42)
}

When running that it produces this:

Notice: Scope(Class[Example]): {a => 42, b => large}

If you want to use that with a class that does not use the struct:

class example(Integer $a, String $b) { }
# Declare it
class { 'example':
* => with_mystruct_defaults('a' => 42)
}

As that expands the struct into individual parameter value settings.
the downside is that you have to maintain the set of parameter names in
both a struct and as individual parameters.

Hope one of those examples gives you some inspiration.

Best,

[jcbollinger]

On Tuesday, June 12, 2018 at 3:14:45 PM UTC-5, Tim.Mooney wrote:

Let's say you had *many* parameters that you wanted to set defaults for
(but allow overrides on an individual basis) based on a single parameter.

The simplest and most straightforward alternative I can think of is simply to reposition your classes.  What I mean by that is instead of using a params class to set Puppet-level default parameter values, make the target class a wrapper that handles parameter defaults and adjustments, and passes them off to a private, internal class.  Truly, the internal class part is optional, but it provides for a separation of concerns, and maps more cleanly onto your example.

For instance,

modules/sandbox/manifests/internal.pp: 

#
# This module exists only to serve as a sandbox where we can experiment with
# puppet code.
#

# @api private
#
class sandbox::internal(
   # No parameter defaults needed or useful at this level
   Enum['typeA', 'typeB', 'combined'] $server_type,
   String                             $service_name,
) {
   notice("sandbox initialized.\n")

   notice("\$server_type  = ${server_type}\n")
   notice("\$service_name = ${service_name}\n")
}

modules/sandbox/manifests/init.pp: 

#
# a 'sandbox' class for managing class parameters

#
class sandbox(
  Enum['typeA', 'typeB', 'combined'] $server_type = 'combined',

  Optional[String]                   $service_name = undef,
) {
  if $service_name =~ NotUndef {
    $_service_name_actual = $service_name
  } else {
    $_type_to_name = {
      'combined' => 'sandbox-typeA+typeB',
      'typeA'    => 'sandbox-typeA',
      'typeB'    => 'sandbox-typeB',
      # No other alternatives are possible
    }
    $_service_name_actual = $_type_to_name[$service_type]
  }

  # Resource-like class declarations can be ok for private,
  # internal classes:
  class { 'sandbox::internal':
    server_type  => $server_type,
    service_name => $_service_name_actual,
  }
}

That's actually a variation on a tried and true pattern from before even parameterized classes (it was applied in defined types), though the separation of the parameter handling from the meat of the class / type is less common.  Data types are really helpful here, though, because they afford a much clearer and more convenient way to check whether the user supplied a parameter value than is possible without.

John

[Tim Mooney]

In regard to: Re: [Puppet Users] class parameters that depend on other...:

> Hope one of those examples gives you some inspiration.

They do. Thank you very much for taking the time to read and consider
the wall of text and code I posted, and then come back with some
insightful suggestions. The options you considered and then rejected,
and the reasons why were also very useful to hear.

Between you and John, I have lots of great suggestions that I now need
to consider.

[Tim Mooney]

In regard to: Re: [Puppet Users] class parameters that depend on other...:

> On Tuesday, June 12, 2018 at 3:14:45 PM UTC-5, Tim.Mooney wrote:
>
>> Let's say you had *many* parameters that you wanted to set defaults for
>> (but allow overrides on an individual basis) based on a single parameter.
>>
>
> The simplest and most straightforward alternative I can think of is simply
> to reposition your classes. What I mean by that is instead of using a
> params class to set Puppet-level default parameter values, make the target
> class a wrapper that handles parameter defaults and adjustments, and passes
> them off to a private, internal class. Truly, the internal class part is
> optional, but it provides for a separation of concerns, and maps more
> cleanly onto your example.

Thanks much John for your excellent reply! Between your suggestions
and Henrik's, I have a much clearer picture of how I can modernize and
hopefully simplify several of our home-grown modules and classes. The
time you took to read and reply is greatly appreciated.