Pass parameters to the 'postgresql' module when instantiated as a dependency of the 'puppetdb' module

[comport3]

Hi All,

We are integrating a new PuppetDB role, and note that there are some key tuning parameters we'd like to modify to optimise the performance, such as 'shared_buffers' and 'work_mem'. Normally the methods to do this would be a straight parameter lookup in Hiera, however these values are nested within a Hash. No problem I hear you say, just use 'create_resources'.. well that's what I'd normally do next, however trying to declare it in a 'profile' class it complains of a 'duplicate class declaration'...

Steps to reproduce -

1. puppet module install puppetlabs-puppetdb

2. class profile::puppetdb {

 include profile::fw::puppetdb

 include ::puppetdb

 class { 'postgresql::server':

  config_hash => {

   'shared_buffers' => '512MB',

   'work_mem' => '16MB',

   }

  }

 }

Duplicate declaration, can't do..

[Chadwick Banning]

You should be able to also set them using https://forge.puppet.com/puppetlabs/postgresql#postgresql_conf.

[jcbollinger]

On Tuesday, October 16, 2018 at 3:54:13 AM UTC-5, comport3 wrote:

Hi All,

We are integrating a new PuppetDB role, and note that there are some key tuning parameters we'd like to modify to optimise the performance, such as 'shared_buffers' and 'work_mem'. Normally the methods to do this would be a straight parameter lookup in Hiera, however these values are nested within a Hash. No problem I hear you say, just use 'create_resources'..

Well no, what I say is what problem?  Hiera has no problem whatever providing hashes to Puppet, including for automatic data binding.  If the value associated with a given key in your data is a hash, then that's what Hiera provides.  That has always worked, and it is more flexible now than ever.

 

well that's what I'd normally do next, however trying to declare it in a 'profile' class it complains of a 'duplicate class declaration'...

Steps to reproduce -

1. puppet module install puppetlabs-puppetdb

2. class profile::puppetdb {

 include profile::fw::puppetdb

 include ::puppetdb

 class { 'postgresql::server':

  config_hash => {

   'shared_buffers' => '512MB',

   'work_mem' => '16MB',

   }

  }

 }

If you want the config_hash parameter of class postgresql::server to receive that hash via automated data binding, then you need two things:

1. You need appropriate data.  For example:

postgresql::server::config_hash:
  shared_buffers: '512MB'
  work_mem: '16MB'

2. You need any applicable resource-like declaration of class postgresql::server in the manifest set to not itself bind a value to the config_hash parameter.

It's not clear to me why you're shooting straight past that toward more complicated and riskier possibilities.

John

[comport3]

Hi John,

Thank you for your response. I tried putting the parameters directly into Hiera as suggested and nothing happened.

I don't fully understand Part 2: "2. You need any applicable resource-like declaration of class postgresql::server in the manifest set to not itself bind a value to the config_hash parameter."

Can you help me understand better what to do here please?

Chadwick - also, thanks, will check that out too.

[Eirik Øverby]

Hi,

> I don't fully understand Part 2: "2. You need any applicable resource-like declaration of class postgresql::server in the manifest set to not itself bind a value to the config_hash parameter."

You are probably declaring your class/resource setting default values for parameters directly in the .pp file. This will override whatever you're defining in Hiera. Just leave the declaration like

class foo:bar ( $param1, $param2='test' ) { ... }

In this way $param1 is considered mandatory, and if it's defined in hiera all is good (the hiera value will be used). Rparam2 is optional, and will be assigned the value 'test' no matter what hiera says - unless you instantiate the class with that parameter explicitly set.

/Eirik

[KevinR]

Hi Eirik,

the parameter behavior is actually the other way around:

class foo:bar(
$param1,         # This enables you to provide 'param1' when instantiating the class,
                 # and it enables auto-parameter-lookup for foo::bar::param1 in Hiera.
                 # if no value is given or found in Hiera, the value of $param1 will be undef.

$param2='test'   # This enables you to provide 'param2' when instantiating the class,
                 # and it enables auto-parameter-lookup for foo::bar::param2 in Hiera.
                 # if no value is given or found in Hiera, the value of $param2 will be 'test'.
) { ... }

In other words, specifying =<some value> after a parameter in the class parameter definition, will enable you to provide a default value that is used as a last resort.

-Kevin

[Eirik Øverby]

> class foo:bar(
> $param1, # This enables you to provide 'param1' when instantiating the class,
> # and it enables auto-parameter-lookup for foo::bar::param1 in Hiera.
> # if no value is given or found in Hiera, the value of $param1 will be undef.
>
> $param2='test' # This enables you to provide 'param2' when instantiating the class,
> # and it enables auto-parameter-lookup for foo::bar::param2 in Hiera.
> # if no value is given or found in Hiera, the value of $param2 will be 'test'.
> ) { ... }
>
> In other words, specifying =<some value> after a parameter in the class parameter definition, will enable you to provide a default value that is used as a last resort.

Really? We just had to modify about 20 classes in our end because the hiera values were *not* used when defaults were specified in the class definitions.. Is there some obscure option/setting we've inherited from puppet2/3 that we're being bitten by?

And to Chadwick - sorry for the apparently bad advice :-)

/Eirik

[KevinR]

Hi Eirik,

I can't speak for those antique 2.x/3.x versions ;-)

The automatic parameter lookup feature is available from Puppet 4.8 onwards, see the official description of it here.

As you can see from the documentation, setting parameter defaults is evaluated as Step 3, which comes after Hiera. You could only overrule Hiera if you provided parameters directly to the resource-like declaration as described in Step1.

-Kevin

[jcbollinger]

No, there is no option or setting that modulates this behavior.  It has always worked that way, from the time automated data binding was introduced in Puppet 3, and it has always been the intended (and IMO sensible) behavior.

I speculate that you've confused the semantics of class definitions with those of resource-like class declarations.  What you presented is the skeleton of a class definition, and a simple completion of it might be

class foo:bar ( $param1, $param2='test' ) {

  notify { 'foo::bar params':
    msg => "param1 = ${param1}, param2 = ${param2}"
  }
}

That simply defines what class foo::bar means.  It is roughly analogous to a function or method definition in many general-purpose programming languages.

The corresponding analogue of a function call, on the other hand, is a class declaration.  There are two kinds, "include-like" and "resource-like".  The first is performed via an include, require, or contain statement, and does not provide a means to express class parameter values in the manifest.  For example,

include foo::bar

That is preferable in many contexts, but it is also possible to declare classes as if "class" were a resource type:

class { 'foo::bar ':
  # Parameter and metaparameter bindings may appear here, such as:
  param1 => 42
}

Either way, every class parameter must be bound to a value.  Puppet determines each parameter's value by checking the following sources, in order:

1. The value specified in a resource-like class declaration has highest precedence if such a value exists.  This is what I suspect you had in mind.
2. The Puppet data lookup facility (a generalization of Hiera)
3. The default value provided by the class definition, if any.

The first source from which a value for the parameter can be obtained is used, and if none of those provides a value then the catalog builder bails.  Thus, if you want a value from Hiera to be used for parameter $foo::bar::param2 then in addition to providing such a value in the data, you must ensure that any resource-like declaration of the class that is evaluated avoids binding a value to that parameter.  And this is the same thing I was saying in my earlier response.

John