Patterns for building configs using hashes & arrays

[Jesse Cotton]

I am new to puppet and decided to really get my feet wet by writing a duplicity module (https://github.com/JCotton1123/puppet-duplicity.git). I am struggling to deal with the fact that variables are immutable and cannot be reassigned (within the same scope). This has become a real issue while trying to dynamically build hashes and arrays that reflect the options and flags that are passed to duplicity. I opted for this approach b/c it makes composition easier and it results in a very clean template definition. For example,

```

  <%- @_flags.each do |flag| -%>
  <%= flag -%> \
  <%- end -%>
  <%- @_options.each do |key,val| -%>
  <%= key -%> <%= val -%> \
  <%- end -%>
```

As opposed to checking for the existence of a dozen different variables and outputting their values if they have a "good" value.

How do others work around this issue? I am looking at this the wrong way?

[William Leese]

I've always found that when creating modules I'd focus on creating class parameters out of the most important configuration options of whatever I'm managing. After that I'd add a similar approach as yours for "everything else". This is a good approach because your module becomes usable for all the use cases you're not applying but other people might want. 

Nothing is more frustrating than finding a great module, but not being able to use it straight away because it is missing that vital configuration parameter!

So in this light, I'd say your approach is good. It might be better if you single out the most commonly used parameters and expose them directly just for clarity, but it might not always provide usability/readability benefits.

[Jesse Cotton]

I like this approach too :) I am just wondering how you build the hash or array up since variables are immutable and cannot be reassigned. For example, I want to do the below however I can't.

define duplicity::job(
  $directories        = [],
  $gpg_key_id         = undef,
  $print_stats        = false,
  $flags              = [],
  $options            = {},
  $env_vars           = {},
  $target_url         = undef,
  $full_if_older_than = '7D',
  $remove_older_than  = false,
  $hour               = '1',
  $minute             = '0',
  $pre_commands       = [],
  $post_commands      = [],
  $shell              = '/bin/bash'
) {

  if $gpg_key_id == undef {
    $flags += ['--no-encryption']
  }
  else {
    $options['--encrypt-id'] = $gpg_key_id
  }

  if ! $print_stats {
    $flags += ['--no-print-statistics']
  }
}

[William Leese]

This should help:

https://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/functions/delete_undef_values.rb

[Felix Frank]

Hi,

On 05/26/2014 03:43 PM, Jesse Cotton wrote:
> if $gpg_key_id == undef {
> $flags += ['--no-encryption']
> }

adding to arrays may actually work.

> else {
> $options['--encrypt-id'] = $gpg_key_id
> }

Yeah, I guess that's an issue.

You may find that such logic will indeed have to move to your template
code. I have used a pattern of

# Template boilerplate here
<%
- code block that munges input data here -
-%>
# template code that relies on munged data here

Hope that makes sense.

Cheers,
Felix

[jcbollinger]

Well, yes, in a sense you are looking at it the wrong way.   Regardless of your reasons for wanting to use an approach that relies on dynamically building hashes and arrays, no such approach can work altogether smoothly in Puppet.  It's like the guy who went to the doctor, complaining that his head hurt whenever he hit it with a hammer.

Puppet has good reasons for immutable variables.  If you want mutable ones, then your only alternative is to move your computation out of DSL.  The easiest avenues for that are templates and custom functions.  Of the two, I would think the latter is preferable for your case, at least if the constructed composite data objects must persist beyond the evaluation of one template.  On the other hand, if you only need them once, for creating one config file, say, then you can do that in a code block at the top of your template, and then use the result later.

There are other alternatives, too, most of them probably worse.  For example, you could use a chain of defined types or classes to build up your data objects in functional programming style.  Or you could use a template to build a string representation of your composite object, and then split() it into an array, and, if appropriate, convert it into a hash from there.  No doubt there are others.  PuppetLabs's add-in "stdlib" module has some functions that could be applied to such tasks.

Another option is to go in an altogether different direction.  Require the data to be fed to your class in a form that is easy for it to digest, and have your templates use it as the class receives it.

In between, you have the option to make your DSL code a lot cleaner and easier at the expense of making your template a little messier.  For example:

myclass.pp
----
class mymodule::myclass($param1 = 'NOT SET', $param2 = 'NOT SET', $param3 = 'NOT SET') {
  $params = {
    'param1' => $param1,
    'param2' => $param2,
    'param3' => $param3,
  }
  template('mytemplate.erb')
}


mytemplate.erb:
----
<%
  @params.each do |key, value|
    next if value == 'NOT SET'
%><%= key %> = <%= value %>  <%
  end
%>


That is, build the data object unconditionally, in such a way that it can be easily filtered when it is processed.


John