How to enable repositories for forge modules?

[Haani Niyaz]

Our SOE document states that all repositories should be disabled by default and enabled on installing a package. So when we write modules our package resource more or less looks like this:

 package { $package_name:
 ensure => "${package_version}-${release_version}",
 install_options => [ {'--disablerepo' => "*"}, { '--enablerepo' => "$releases_repo" } ],
 notify => Class['::apache::service'],
 }

In our base profile (which is included in every role) a module ensures that all repos are disabled. However this has proven to be a problem when using forge modules where it seems to expect the repositories will be enabled by default. 

One untested suggestion was to use the package resource default statement and set the install option to enable the repositories. Example shown below:

class profiles::phpwebserver {

 Package{
  install_options => [ {'--disablerepo' => "*"}, { '--enablerepo' => "*" } ],
 }
 
 $apache_template_file = hiera('apache::template_file')

 class { '::apache':
  default_vhost  => false,
  conf_template  => "apache/${apache_template_file}",
  service_name   => 'httpd',
  package_ensure => hiera('apache::version'),
  service_enable => true,
  service_ensure => 'running',
 }

 class {'::apache::mod::php':
  package_name   => 'php',
  package_ensure => hiera('apache::mod::php::version'),
  path           => "${::apache::params::lib_path}/libphp54-php5.so",

 }

 contain ::apache
 contain ::apache::mod::php

}

I'm not even sure if it  will work considering it is outside the scope of the httpd class. I can't be the only one who has run into this issue so I'm hoping someone can provide a sane solution.

[Martin Alfke]

Hi Haani,

On 15 Nov 2015, at 06:58, Haani Niyaz <haani...@gmail.com> wrote:

> Our SOE document states that all repositories should be disabled by default and enabled on installing a package. So when we write modules our package resource more or less looks like this:
>
> package { $package_name:
> ensure => "${package_version}-${release_version}",
> install_options => [ {'--disablerepo' => "*"}, { '--enablerepo' => "$releases_repo" } ],
> notify => Class['::apache::service'],
> }
>
> In our base profile (which is included in every role) a module ensures that all repos are disabled. However this has proven to be a problem when using forge modules where it seems to expect the repositories will be enabled by default.
>
> One untested suggestion was to use the package resource default statement and set the install option to enable the repositories. Example shown below:
>
>
> class profiles::phpwebserver {
>
> Package{
> install_options => [ {'--disablerepo' => "*"}, { '--enablerepo' => "*" } ],
> }

put this resource default into site.pp
Then the resource default is set globally and not only in class namespace.

Be aware that module developers might set install_options by themselves which might overwrite your default setting.

Best,
Martin

[jcbollinger]

On Sunday, November 15, 2015 at 12:58:17 AM UTC-6, Haani Niyaz wrote:

Our SOE document states that all repositories should be disabled by default and enabled on installing a package.

How odd.  Also, your two example codes don't match up: the first seems to enable only a specific repository, whereas the second enables all repositories (after uselessly disabling them all).  Some of the following comments apply more to the former approach than to the latter.

That's going to problematic in several ways, not all of them directly related to Puppet.  For example, what if a package you want to install relies on dependencies in another repository?  It happens all the time.  For example, CentOS provides separate repositories for updates and for the base packages of each release.  It can be the case that you want to install the latest version of a package (from "updates"), but it needs uninstalled packages from "base", too.

It also makes it tricky to determine when updated packages are available, because you first have to enable the repos you want to check for updates.

The bottom line is that the set of repositories on which the system relies for packages is properly a persistent property of the system, but you are making it a dynamic aspect of system management operations.

 

Resource defaults are dynamically scoped (they one of the few remaining things that are).   That means your example will work, in that the defaults declared in class profiles::phpwebserver will apply in class ::apache.

In general, dynamic scoping is a tricky beast, and I would warn that the defaults applicable when the first declaration of class ::apache is processed are the (only) ones that apply.  In this case, however, because you use a resource-like class declaration for class ::apache, if it ever is not the first processed then catalog building will fail.  I ordinarily cast that as a serious problem with the resource-like class declaration style, and recommend not using that style.  Inasmuch as you might actually want your catalog compilations to fail if the desired package installation options are not set, though, it might work for you.

Putting resource defaults at top scope or node scope instead of some class scope would work around the issue of ensuring the defaults are in scope for all declarations of a given class, but that cannot work for you (or at least, it does not afford a complete solution) because you want *different* defaults to apply to different classes.

Also, resource defaults are only defaults.  If alternative values are expressed for the parameters involved then the assigned defaults are completely ignored.  That could render your resource defaults moot in unfavorable circumstances.

I suggest an alternative approach: resource overrides.  You can perform overrides via collectors and tags to set an appropriate scope for the defaults, and to combine your installation options with any others that might be set on the packages.  For example,

class profiles::phpwebserver {

  # ... no package defaults here ...

  $apache_template_file = hiera('apache::template_file')

  # insert the needed repository names / globs here:
  $apache_repos = '*'

  class { '::apache':
    default_vhost => false,
    conf_template => "apache/${apache_template_file}",
    service_name => 'httpd',
    package_ensure => hiera('apache::version'),
    service_enable => true,
    service_ensure => 'running',
  }

  class {'::apache::mod::php':
    package_name => 'php',
    package_ensure => hiera('apache::mod::php::version'),
    path => "${::apache::params::lib_path}/libphp54-php5.so",
  }

  contain ::apache
  contain ::apache::mod::

php

  Package<| tag == 'apache' |> {
    # *append to* any install_options already set:
    install_options +> [ {'--disablerepo' => "*"}, { '--enablerepo' => "${apache_repos}" } ]
  }
}

Note, too, that that whole thing could be dramatically abbreviated if you relied exclusively on automatic data binding (via Hiera, which itself you are already using), to bind data to your class parameters.  In that case, the 'contain' statements you already have in that class would be all the class declarations you need -- you could altogether drop the resource-like declarations for a 50%+ reduction in code size.


John

[Haani Niyaz]

How odd.  Also, your two example codes don't match up: the first seems to enable only a specific repository, whereas the second enables all repositories (after uselessly disabling them all).  Some of the following comments apply more to the former approach than to the latter.

This is intentional for the reason that we build our own packages for software releases and in those modules we only enable the release repositories.

With regard to "the second enables all repositories (after uselessly disabling them all).", the disabling of all repos was required because I found that if the repository is already enabled (say by an operator) the puppet run failed.

Hiera data binding is avoided in favour of explicit hiera lookups. Sure it's a bit verbose but makes it easier to onboard new developers and debug.

Not familiar with the resource overrides but will definitely have a look!

Thanks.