Jira (PDB-4256) Installing with PuppetDB module creates dependency cycle with latest apt module

2 views
Skip to first unread message

Maggie Dreyer (JIRA)

unread,
Jan 23, 2019, 8:17:02 PM1/23/19
to puppe...@googlegroups.com
Maggie Dreyer created an issue
 
PuppetDB / CI Blocker PDB-4256
Installing with PuppetDB module creates dependency cycle with latest apt module
Issue Type: CI Blocker CI Blocker
Assignee: Unassigned
Created: 2019/01/23 5:16 PM
Priority: Normal Normal
Reporter: Maggie Dreyer

With the latest release of the puppetlabs-apt module, beaker acceptance test presuites that install PDB via the puppetdb module have started failing with the following:

    Error: Found 1 dependency cycle:
    (Package[apt-transport-https] => Apt::Source[apt.postgresql.org] => Package[apt-transport-https])\nTry the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz

Pre-suite from puppetserver: https://github.com/puppetlabs/puppetserver/blob/master/acceptance/suites/pre_suite/foss/95_install_pdb.rb

From Charlie Sharpsteen in Slack:
Could be this line in puppetlabs-postgresql if `Package[apt-transport-https]` ended up tagged with `postgresql`:
https://github.com/puppetlabs/puppetlabs-postgresql/blob/master/manifests/repo/apt_postgresql_org.pp#L30
Hmm, https://github.com/puppetlabs/puppetlabs-apt/blob/master/manifests/source.pp#L91-L93 is causing me to give it some side-eye.
I'm not sure what the containment path for that would end up being. But if the `apt::source` in `class postgresql::repo::apt_postgresql_org` ends up being the first one evaluated, then that `ensure_package` could end up creating `Package[apt-transport-https]` that is contained within `postgresql::repo::apt_postgresql_org`
Which means it would end up with a `postgresql` tag because we take the name of the containing class, tag the resource with it, then recursively split on `::` and tag the resource with the remainder. Or something similarly ridiculous.
Could comment out apt_postgresql_org.pp#L30 and see if the problem goes away. If it does, then that's probably it.

Not sure what the correct path is to fix this, but both Server and PuppetDB's acceptance testing will fail until we do.

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Maggie Dreyer (JIRA)

unread,
Jan 23, 2019, 8:18:02 PM1/23/19
to puppe...@googlegroups.com
Maggie Dreyer updated an issue
Change By: Maggie Dreyer
With the latest release of the puppetlabs-apt module, beaker acceptance test presuites that install PDB via the puppetdb module have started failing with the following:
{code}

    Error: Found 1 dependency cycle:
    (Package[apt-transport-https] => Apt::Source[apt.postgresql.org] => Package[apt-transport-https])\nTry the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz
{code}
From [~chuck] in Slack:

Could be this line in puppetlabs-postgresql if `Package[apt-transport-https]` ended up tagged with `postgresql`:
https://github.com/puppetlabs/puppetlabs-postgresql/blob/master/manifests/repo/apt_postgresql_org.pp#L30
Hmm, https://github.com/puppetlabs/puppetlabs-apt/blob/master/manifests/source.pp#L91-L93 is causing me to give it some side-eye.
I'm not sure what the containment path for that would end up being. But if the `apt::source` in `class postgresql::repo::apt_postgresql_org` ends up being the first one evaluated, then that `ensure_package` could end up creating `Package[apt-transport-https]` that is contained within `postgresql::repo::apt_postgresql_org`
Which means it would end up with a `postgresql` tag because we take the name of the containing class, tag the resource with it, then recursively split on `::` and tag the resource with the remainder. Or something similarly ridiculous.
Could comment out apt_postgresql_org.pp#L30 and see if the problem goes away. If it does, then that's probably it.

Not sure what the correct path is to fix this, but both Server and PuppetDB's acceptance testing will fail until we do. Note that this is only failing on Debian 8 for puppetserver at least.

Austin Blatt (JIRA)

unread,
Jan 24, 2019, 4:43:03 PM1/24/19
to puppe...@googlegroups.com
Austin Blatt commented on CI Blocker PDB-4256
 
Re: Installing with PuppetDB module creates dependency cycle with latest apt module

A temporary work around until we find the cause of the issue is to install the apt module and specify the version to be 6.2.1 before installing puppetlabs-puppetdb.

puppet module install puppetlabs-apt --version 6.2.1
puppet module install puppetlabs-puppetdb

Zachary Kent (JIRA)

unread,
Jan 24, 2019, 5:41:03 PM1/24/19
to puppe...@googlegroups.com
Zachary Kent commented on CI Blocker PDB-4256

Useful comments from Charlie Sharpsteen in Slack about the possible underlying cause and potential fixes on the apt/postgres module side:

 - `puppetlabs-postgres` is using a resource collector that is selecting packages via implicit tags in order to add an Apt repo as a dependency. This is working around the fact that we never got the implementation of run stages quite right, but the tags that resources inherit implicitly from their containing class behave in surprising ways that make them unreliable.

 - `puppetlabs-apt` is using `ensure_resource()` to make sure the `apt-transport-https` package is installed without taking explicit ownership of it — and thus conflicting with other modules or user code that may be managing the package. This is working around the fact that our duplicate resource rules are very strict and we don't allow for multiple definitions of a resource in cases where there is no conflict. The downside here is that `ensure_resource` has the effect of injecting `package { 'apt-transport-https': }` into whichever scope happens to use `apt::source` first which opens that resource up to having all sorts of unwanted defaults attached to it in a way that is extremely difficult to debug.    

Some potential solutions:

 - For `puppetlabs-postgresql`, make the requirement on the apt resource explicit where it is needed, or use an explicit tag for the collector that is not "magically" generated from the class namespace. Using the module's qualified name, `puppetlabs-postgres`, would make a better tag to set explicitly and then collect.

 - For `puppetlabs-apt`, it should probably just take explicit ownership and declare `package {'apt-transport-https': }` if that is required. This may conflict with other code, but "duplicate resource" errors are usually very good at pointing to where the problem is. In contrast, errors resulting from a package being injected into some random scope are a nightmare to track down and not something that can be done quickly by someone who has never seen this sort of issue before. Alternately, if `puppetlabs-apt` wants to use `ensure_resource()`, then it should do that inside a class owned by the module instead of a defined type so that the resource is contained in a very explicit spot and stops injecting its self into whatever happens to use `apt::source` first.

 

 

 

Helen Campbell (JIRA)

unread,
Feb 1, 2019, 9:23:10 AM2/1/19
to puppe...@googlegroups.com

Hey everyone, I've made a fix which I was able to test out using Austins environment he provided (thanks Austin!). The release with the fix is now out and Postgresql 5.12.0 should work for you, would someone be able to confirm this and close out these tickets if so? Don't want to resolve anything until I know it's fixed on your side, thanks!

Austin Blatt (JIRA)

unread,
Feb 1, 2019, 10:18:10 AM2/1/19
to puppe...@googlegroups.com

Austin Blatt (JIRA)

unread,
Feb 1, 2019, 10:19:20 AM2/1/19
to puppe...@googlegroups.com

Charlie Sharpsteen (JIRA)

unread,
Feb 5, 2019, 10:42:04 AM2/5/19
to puppe...@googlegroups.com
Charlie Sharpsteen commented on CI Blocker PDB-4256
 
Re: Installing with PuppetDB module creates dependency cycle with latest apt module

I think the 5.12.0 fix turned the collection into a no-op instead of restricting it to just package resources in the puppetlabs-postgresql module — left a comment in MODULES-8553.

Claudia Petty (Jira)

unread,
Jun 21, 2023, 10:54:03 AM6/21/23
to puppe...@googlegroups.com
Claudia Petty updated an issue
 
Change By: Claudia Petty
Labels: ci-blocker
This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages