ensure_packets not working

[Jochen Haeberle]

Hi,

I am using serverless puppet apply on some nodes. I prepared a set of manifests using forge plugins. I developed everything using Vagrant quite fine.

Now I put my code via git on a debian 9.7 VM with puppet 6.2 and am getting the following errors:

Error: Evaluation Error: Error while evaluating a Function Call, Duplicate declaration: Package[libapache2-mod-php7.3] is already declared at (file: /etc/puppetlabs/code/environments/production/modules/profile/manifests/software/apache.pp, line: 78); cannot redeclare (file: /etc/puppetlabs/code/environments/production/modules/profile/manifests/software/wordpress.pp, line: 24) (file: /etc/puppetlabs/code/environments/production/modules/profile/manifests/software/wordpress.pp, line: 24, column: 3)

Both instances use the stdlib function ensure_packages. If I remove one of the instances, I will get the error with another package.

puppetlabs/stdlib and all other modules are up to date, thanks to r10k, Code is working on a fresh vagrant machine.

What can be the culprit of this? I have no idea where to look for this problem.

Thanks in advance vor any hints, regards

Jochen

[Jochen Haeberle]

Hi @all,

does anyone have an idea what could be happening? I think this might be some sort of configuration error on the node, on the other hand, puppet obviously seems able to compile a catalog.

What could be the reason for such a difference in behaviour on Debian 9.7 either on Vagrant or VMware?

Thanks for any hints, regards

Jochen

[Ben Ford]

Sharing the code that you're working with will make it possible for people to help solve your problem.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/4e0cd8c4-577f-42a6-a664-e6f4015fca5d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jochen Haeberle]

Hi,

you are probably right. Here we go:

as I am using puppet apply, I have in my site.pp:

include role::fips_tool

in fips_tool there is, among other code:

...

class { profile::software::apache:

    php_version         => '7.3',

    default_redirect_to => 'http://my.host.name',

    maindir             => '/srv/labor/',

    keep_log_days       => 30,

  }

  class { profile::software::wordpress:

    php_version => '7.3',

  }

...

The offending code from profile::software::apache:

class profile::software::apache (

  String $php_version         = '7.0',

  String $default_redirect_to = 'https://my.host.name',

  String $maindir             = '/srv/www/',

  Integer $keep_log_days      = 30,

) {

...

ensure_packages([

    "php$php_version-xml",

    "php$php_version-zip",

    "php$php_version-curl",

    "php$php_version-mbstring",

    "libapache2-mod-php$php_version",

  ],

    {

      require => [Class['apt::update'], Apt::Source['sury_php'], ]

    }

  )

...}

and from profile::software::wordpress.pp:

class profile::software::wordpress (

    String $php_version = '7.0',

){

  require profile::software::mysql

  require profile::software::apache

...

ensure_packages([

    'php7.1-mysql',

    'php7.1-common',

    "libapache2-mod-php$php_version",

    "php$php_version-gd",

    "php$php_version-mysql",

  ],

    {

      require     => [Class['apt::update'], Apt::Source['sury_php']],

      notify      => Class['apache::service']

    })

...
}

On the dev Vagrant machine, I run this calling:

sudo puppet apply /etc/puppetlabs/code/environments/dev/manifests/

Code runs without a problem:

Notice: Scope(Class[Profile::Software::Tool_packages]): VirtualBox

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/dev/modules-external/ssh/manifests/hostkeys.pp, line: 39, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/dev/modules-external/ssh/manifests/hostkeys.pp, line: 32, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/dev/modules-external/ssh/manifests/hostkeys.pp, line: 32, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/dev/modules-external/ssh/manifests/hostkeys.pp, line: 32, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored (file: /etc/puppetlabs/code/environments/dev/modules-external/ssh/manifests/knownhosts.pp, line: 9, column: 7)

Notice: Scope(Class[Profile::Software::Certbot]): setting Let's encrypt staging environment!!!

Warning: Scope(Class[Apache::Mod::Status]): Class apache::mod::status: Using Allow

Warning: Scope(Class[Apache::Mod::Status]): is deprecated in Apache 2.4

Notice: Scope(Class[Java_ng]): Selected repository: ppa:oracle, selected version: 8

Notice: Compiled catalog for stretch.localdomain in environment dev in 5.74 seconds

Notice: Applied catalog in 3.20 seconds

On the live VM I checkout the exakt same code from GIT, running it with:

puppet apply /etc/puppetlabs/code/environments/production/manifests/

but the output is:

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/production/modules-external/ssh/manifests/hostkeys.pp, line: 39, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/production/modules-external/ssh/manifests/hostkeys.pp, line: 32, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/production/modules-external/ssh/manifests/hostkeys.pp, line: 32, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the export is ignored (file: /etc/puppetlabs/code/environments/production/modules-external/ssh/manifests/hostkeys.pp, line: 32, column: 7)

Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored (file: /etc/puppetlabs/code/environments/production/modules-external/ssh/manifests/knownhosts.pp, line: 9, column: 7)

Error: Evaluation Error: Error while evaluating a Function Call, Duplicate declaration: Package[libapache2-mod-php7.3] is already declared at (file: /etc/puppetlabs/code/environments/production/modules/profile/manifests/software/apache.pp, line: 78); cannot redeclare (file: /etc/puppetlabs/code/environments/production/modules/profile/manifests/software/wordpress.pp, line: 24) (file: /etc/puppetlabs/code/environments/production/modules/profile/manifests/software/wordpress.pp, line: 24, column: 3) on node my.host.name

I recreated my Vagrant machine, works like a charm. Both machines run Puppet 6.2.0 on Debian 9.7

Sorry for the extensive post, this is because I do not have any clue where to look.

Thanks, regards

Jochen

[jcbollinger]

Broadly speaking, the two possibilities are that

• stdlib's ensure_packages() does not work correctly on Puppet 6.2
  

OR

• you're using it wrongly
  

I don't see how we can discriminate between these without some code.  I suggest preparing a minimal test case that demonstrates the problem.  If the process of producing such a case does not lead you to discovering an error yourself, then we here should certainly be able to tell you whether the test case is expected to work.  If not, we'll be able to tell you how to fix it.  If so, then you will have something to submit in your bug report.

As long as I'm standing on this soapbox, though, I'd like to remark that ensure_packages() is a jerry-rig, not an actual solution to any Puppet problem.  It and its brethren, ensure_package(), ensure_resources(), and ensure_resource() should be avoided.  The correct solution in every case where one of these is used involves factoring out management of resources into a separate class.  That's not always straightforward, else these functions would not have been created, but you certainly should not design modules with the expectation of relying on them.

John

[jcbollinger]

On Thursday, January 31, 2019 at 8:00:05 AM UTC-6, jcbollinger wrote:

As long as I'm standing on this soapbox, though, I'd like to remark that ensure_packages() is a jerry-rig, not an actual solution to any Puppet problem.  It and its brethren, ensure_package(), ensure_resources(), and ensure_resource() should be avoided.

Oops: there is no ensure_package().  But the point remains.

As for the code you've presented, it indeed does reveal a problem, which is also illustrative of my soapbox position.  One of the larger issues here revolves around what to do when there are two or more declarations of a resource that specify different parameters.  The two parameter lists cannot both be satisfied (generally speaking), so which do we choose?  In fact, the functions in question make the safe choice: none of the above.  Per the docs for ensure_resource():

If the resource already exists, but does not match the specified parameters, this function attempts to recreate the resource, leading to a duplicate resource definition error.

In your case, one declaration of Package[libapache2-mod-php7.3] has a notify parameter and the other does not.

There are several ways you might refactor this to solve the specific problem, but I would suggest one of these:

• moving management of mod_php out of the apache profile
  
• My first inclination would be to put it in its own profile, or at least in its own class, but
• you could also leave it as a responsibility of the Wordpress profile
• moving management of mod_php out of the Wordpress profile
• Again, you could put it in its own class or profile, but
• you could also let the Apache profile continue to manage it, possibly under control of a class parameter

Of course, you could also just ensure that the specified parameters are identical in both places, but if you're relying on ensure_packages / _resources / _resource a lot, then you will likely find that that becomes tedious and error-prone.

John

[Jochen Haeberle]

Hi John,

thanks for your support! I see your point, especially with the different use of notify.

It’s disapointing to hear one should still keep away from ensure_packages, as it greatly helps me to install packages only when needed. This goes mostly for apache modules and php extensions in my case.

I will try to prepare a simple test which I can share.

Nevertheless the question remains why the code is working in my dev environment. Both environments are the same as much as you can get.

Regards

Jochen

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b67eecc5-1145-4eff-8d6b-fbf8808236e3%40googlegroups.com.

[jcbollinger]

On Thursday, January 31, 2019 at 8:40:40 AM UTC-6, Jochen Haeberle wrote:

 

Nevertheless the question remains why the code is working in my dev environment. Both environments are the same as much as you can get.

I hope you'll forgive me if I'm inclined to doubt that the same code is working in your dev environment.  But note that "the same code" encompasses the whole manifest set, including modules.  I can think of one or two ways that code elsewhere could affect the code you're looking at.  If you want to probe this further, then I repeat my earlier recommendation that you prepare a minimal (self-contained) test case to use for that purpose.

John