Jira (PUP-10018) The module tool won't solve some dependencies without --force or --ignore-dependencies

28 views
Skip to first unread message

John Bollinger (JIRA)

unread,
Sep 13, 2019, 12:17:03 PM9/13/19
to puppe...@googlegroups.com
John Bollinger created an issue
 
Puppet / Bug PUP-10018
The module tool won't solve some dependencies without --force or --ignore-dependencies
Issue Type: Bug Bug
Affects Versions: PUP 6.8.1
Assignee: Unassigned
Components: PMT
Created: 2019/09/13 9:16 AM
Priority: Normal Normal
Reporter: John Bollinger

Puppet Version: 6.8.1
Puppet Server Version: 6.8.1
OS Name/Version: Red Hat Enterprise Linux Server 7.4

The module tool refuses to perform valid module upgrades under some circumstances, unless the -force or ignore-dependencies option is applied.  Even -ignore-dependencies does not suffice in some cases.  Specifically, it seems unwilling to perform an upgrade across a major version of the specified module if that requires an upgrade even of the minor version of one of that module's dependencies.  Alternatively, the tool's objection might be to performing a set of upgrades that are valid as a group, but which cannot be broken down into a sequence of separate, individually-valid upgrades.

The particular case in which I experienced this can be reproduced like so:

  1. Start with a clean install of Puppet and puppetserver version 6.8.1.
  2. Install version 6.3.0 of puppetlabs-apt via the module tool
  3. Install version 2.1.2 of puppetlabs-puppet_agent via the module tool
  4. Attempt to upgrade puppetlabs-puppet_agent to version 2.2.0 via the module tool (fails on unresolvable dependencies)
  5. Attempt to upgrade puppetlabs-apt to version 7.1.0 via the module tool (fails on unresolvable dependencies)

One should not need to use -force or -ignore-dependencies to upgrade from one set of module versions in which all dependencies are satisfied to another set of module versions in which all dependencies are satisfied.

Desired Behavior:

The module tool should be willing to perform upgrades such as those described without using -force or ignore-dependencies.  In particular, it should be willing to perform any combination of minor-version upgrades in service to an upgrade of the module named on the command line, so long as it results in a final state in which all dependencies are satisfied.  Use of force or -ignore-dependencies is not desirable here because it does not automatically solve dependencies, and it can (and often does) produce a situation in which dependencies indeed are not fully satisfied.

It would be acceptable, albeit not ideal for this purpose, to require use of a new, different command-line argument.

Actual Behavior:

Here's an example of the observed behavior.  Specific module versions are expressed via --version options to ensure reproducibility.  The target versions for the upgrade commands reflect the latest versions of the modules involved as of the time of this report, and at this time, the upgrade failures can be reproduced without specifying target module versions.

Preparing the environment ...

# puppet module list
/etc/puppetlabs/code/environments/production/modules (no modules installed)
/etc/puppetlabs/code/modules (no modules installed)
/opt/puppetlabs/puppet/modules (no modules installed)

# puppet module install puppetlabs-apt --version=6.3.0
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing – do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└─┬ puppetlabs-apt (v6.3.0)
├── puppetlabs-stdlib (v5.2.0)
└── puppetlabs-translate (v1.2.0)

# puppet module install puppetlabs-puppet_agent --version=2.1.2
Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing – do not interrupt ...
/etc/puppetlabs/code/environments/production/modules
└─┬ puppetlabs-puppet_agent (v2.1.2)
├── puppetlabs-apt (v6.3.0)
├─┬ puppetlabs-inifile (v3.0.0)
│ └── puppetlabs-translate (v1.2.0)
└── puppetlabs-stdlib (v5.2.0)

Reproducing the issue ...

# puppet module upgrade puppetlabs-puppet_agent --version=2.2.0
Notice: Preparing to upgrade 'puppetlabs-puppet_agent' ...
Notice: Found 'puppetlabs-puppet_agent' (v2.1.2) in /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Error: Could not upgrade module 'puppetlabs-puppet_agent' (v2.1.2 -> v2.2.0)
No version of 'puppetlabs-puppet_agent' can satisfy all dependencies
Use `puppet module upgrade --ignore-dependencies` to upgrade only this module

# puppet module upgrade puppetlabs-apt --version=7.1.0
Notice: Preparing to upgrade 'puppetlabs-apt' ...
Notice: Found 'puppetlabs-apt' (v6.3.0) in /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Error: Could not upgrade module 'puppetlabs-apt' (v6.3.0 -> v7.1.0)
No version of 'puppetlabs-apt' can satisfy all dependencies
Use `puppet module upgrade --ignore-dependencies` to upgrade only this module

# puppet module upgrade puppetlabs-apt --version=7.1.0 --ignore-dependencies
Notice: Preparing to upgrade 'puppetlabs-apt' ...
Notice: Found 'puppetlabs-apt' (v6.3.0) in /etc/puppetlabs/code/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppet.com ...
Error: Could not upgrade module 'puppetlabs-apt' (v6.3.0 -> v7.1.0)
No version of 'puppetlabs-apt' can satisfy all dependencies
Use `puppet module upgrade --ignore-dependencies` to upgrade only this module

 
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

John Bollinger (JIRA)

unread,
Sep 13, 2019, 12:22:02 PM9/13/19
to puppe...@googlegroups.com
John Bollinger updated an issue
Change By: John Bollinger
*Puppet Version: 6.8.1*
*Puppet Server Version: 6.8.1*
*OS Name/Version: Red Hat Enterprise Linux Server 7.4*

The module tool refuses to perform valid module upgrades under some circumstances, unless the {{ \ - \ -force}} or {{ \ - \ -ignore \ -dependencies}} option is applied.  Even {{ \ - \ -ignore \ -dependencies}} does not suffice in some cases.  Specifically, it seems unwilling to perform an upgrade across a major version of the specified module if that requires an upgrade even of the minor version of one of that module's dependencies.  Alternatively, the tool's objection might be to performing a set of upgrades that are valid as a group, but which cannot be broken down into a sequence of separate, individually-valid upgrades.


The particular case in which I experienced this can be reproduced like so:
# Start with a clean install of Puppet and puppetserver version 6.8.1.
# Install version 6.3.0 of puppetlabs-apt via the module tool
# Install version 2.1.2 of puppetlabs-puppet_agent via the module tool
# Attempt to upgrade puppetlabs-puppet_agent to version 2.2.0 via the module tool (fails on unresolvable dependencies)
# Attempt to upgrade puppetlabs-apt to version 7.1.0 via the module tool (fails on unresolvable dependencies)

One should not need to use {{ \ - \ -force}} or {{ \ - \ -ignore \ -dependencies}} to upgrade from one set of module versions in which all dependencies are satisfied to another set of module versions in which all dependencies are satisfied.

*Desired Behavior:*

The module tool should be willing to perform upgrades such as those described without using {{ \ - \ -force}} or {{ \ - \ -ignore \ -dependencies}}.  In particular, it should be willing to perform any combination of minor-version upgrades in service to an upgrade of the module named on the command line, so long as it results in a final state in which all dependencies are satisfied.  Use of {{ \ - \ -force}} or {{ \ - \ -ignore \ -dependencies}} is not desirable here because it does not automatically solve dependencies, and it can (and often does) produce a situation in which dependencies indeed are not fully satisfied.


It would be acceptable, albeit not ideal for this purpose, to require use of a new, different command-line argument.

*Actual Behavior:*

Here's an example of the observed behavior.  Specific module versions are expressed via {{ \ - \ -version}} options to ensure reproducibility.  The target versions for the upgrade commands reflect the latest versions of the modules involved as of the time of this report, and at this time, the upgrade failures can be reproduced without specifying target module versions.

_*Preparing the environment ...*_

{{# puppet module list}}

{{/etc/puppetlabs/code/environments/production/modules (no modules installed)}}
{{/etc/puppetlabs/code/modules (no modules installed)}}
{{/opt/puppetlabs/puppet/modules (no modules installed)}}

{{# puppet module install puppetlabs-apt --version=6.3.0}}
{{Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...}}
{{Notice: Downloading from [https://forgeapi.puppet.com|https://forgeapi.puppet.com/] ...}}

{{Notice: Installing – do not interrupt ...}}
{{/etc/puppetlabs/code/environments/production/modules}}
{{└─┬ puppetlabs-apt (v6.3.0)}}
{{├── puppetlabs-stdlib (v5.2.0)}}
{{└── puppetlabs-translate (v1.2.0)}}

{{# puppet module install puppetlabs-puppet_agent --version=2.1.2}}
{{Notice: Preparing to install into /etc/puppetlabs/code/environments/production/modules ...}}
{{Notice: Downloading from [https://forgeapi.puppet.com|https://forgeapi.puppet.com/] ...}}

{{Notice: Installing – do not interrupt ...}}
{{/etc/puppetlabs/code/environments/production/modules}}
{{└─┬ puppetlabs-puppet_agent (v2.1.2)}}
{{├── puppetlabs-apt (v6.3.0)}}
{{├─┬ puppetlabs-inifile (v3.0.0)}}
{{│ └── puppetlabs-translate (v1.2.0)}}
{{└── puppetlabs-stdlib (v5.2.0)}}

_*Reproducing the issue ...*_

{{# puppet module upgrade puppetlabs-puppet_agent --version=2.2.0}}

{{Notice: Preparing to upgrade 'puppetlabs-puppet_agent' ...}}
{{Notice: Found 'puppetlabs-puppet_agent' (v2.1.2) in /etc/puppetlabs/code/environments/production/modules ...}}
{{Notice: Downloading from [https://forgeapi.puppet.com|https://forgeapi.puppet.com/] ...}}
{color:#ff0000}{{Error: Could not upgrade module 'puppetlabs-puppet_agent' (v2.1.2 -> v2.2.0)}}{color}
{color:#ff0000} {{No version of 'puppetlabs-puppet_agent' can satisfy all dependencies}}{color}
{color:#ff0000} {{Use `puppet module upgrade --ignore-dependencies` to upgrade only this module}}{color}

{{# puppet module upgrade puppetlabs-apt --version=7.1.0}}

{{Notice: Preparing to upgrade 'puppetlabs-apt' ...}}
{{Notice: Found 'puppetlabs-apt' (v6.3.0) in /etc/puppetlabs/code/environments/production/modules ...}}
{{Notice: Downloading from [https://forgeapi.puppet.com|https://forgeapi.puppet.com/] ...}}
{color:#ff0000}{{Error: Could not upgrade module 'puppetlabs-apt' (v6.3.0 -> v7.1.0)}}{color}
{color:#ff0000} {{No version of 'puppetlabs-apt' can satisfy all dependencies}}{color}
{color:#ff0000} {{Use `puppet module upgrade --ignore-dependencies` to upgrade only this module}}{color}

{{# puppet module upgrade puppetlabs-apt --version=7.1.0 --ignore-dependencies}}

{{Notice: Preparing to upgrade 'puppetlabs-apt' ...}}
{{Notice: Found 'puppetlabs-apt' (v6.3.0) in /etc/puppetlabs/code/environments/production/modules ...}}
{{Notice: Downloading from [https://forgeapi.puppet.com|https://forgeapi.puppet.com/] ...}}
{color:#ff0000}{{Error: Could not upgrade module 'puppetlabs-apt' (v6.3.0 -> v7.1.0)}}{color}
{color:#ff0000} {{No version of 'puppetlabs-apt' can satisfy all dependencies}}{color}
{color:#ff0000} {{Use `puppet module upgrade --ignore-dependencies` to upgrade only this module}}{color}

 

John Bollinger (JIRA)

unread,
Sep 13, 2019, 4:04:03 PM9/13/19
to puppe...@googlegroups.com
John Bollinger updated an issue
*Puppet Version: 6.8.1*
*Puppet Server Version: 6. 8 5 . 1 0 *

John Bollinger (JIRA)

unread,
Sep 13, 2019, 4:06:04 PM9/13/19
to puppe...@googlegroups.com
John Bollinger updated an issue
*Puppet Version: 6.8.1*
*Puppet Server Version: 6.5.0*

*OS Name/Version: Red Hat Enterprise Linux Server 7.4*

The module tool refuses to perform valid module upgrades under some circumstances, unless the {{\-\-force}} or {{\-\-ignore\-dependencies}} option is applied.  Even {{\-\-ignore\-dependencies}} does not suffice in some cases.  Specifically, it seems unwilling to perform an upgrade across a major version of the specified module if that requires an upgrade even of the minor version of one of that module's dependencies.  Alternatively, the tool's objection might be to performing a set of upgrades that are valid as a group, but which cannot be broken down into a sequence of separate, individually-valid upgrades.

The particular case in which I experienced this can be reproduced like so:
# Start with a clean install of Puppet and puppetserver version 6.8.1 and puppetserver 6 . 5.0.

John Bollinger (JIRA)

unread,
Sep 13, 2019, 4:08:02 PM9/13/19
to puppe...@googlegroups.com
John Bollinger updated an issue
*Puppet Version: 6.8.1*
*Puppet Server Version: 6.5.0*
*OS Name/Version: Red Hat Enterprise Linux Server 7.4*

The module tool refuses to perform valid module upgrades under some circumstances, unless the {{\-\-force}} or {{\-\-ignore\-dependencies}} option is applied.  Even {{\-\-ignore\-dependencies}} does not suffice in some cases.  Specifically, it seems unwilling to perform an upgrade across a major version of the specified module if that requires an upgrade even of the minor version of one of that module's dependencies.  Alternatively, the tool's objection might be to performing a set of upgrades that are valid as a group, but which cannot be broken down into a sequence of separate, individually-valid upgrades.

The particular case in which I experienced this can be reproduced like so:
# Start with a clean install of Puppet 6.8.1 and puppetserver 6.5.0 (I used the Puppet, Inc . RPMs, installed via yum).

John Bollinger (JIRA)

unread,
Sep 13, 2019, 4:10:02 PM9/13/19
to puppe...@googlegroups.com
John Bollinger updated an issue
*Puppet Version: 6.8.1*
*Puppet Server Version: 6.5.0*
*OS Name/Version: Red Hat Enterprise Linux Server 7.4*

The module tool refuses to perform valid module upgrades under some circumstances, unless the {{\-\-force}} or {{\-\-ignore\-dependencies}} option is applied.  Even {{\-\-ignore\-dependencies}} does not suffice in some cases.  Specifically, it seems unwilling to perform an upgrade across a major version of the specified module if that requires an upgrade even of the minor version of one of that module's dependencies.  Alternatively, the tool's objection might be to performing a set of upgrades that are valid as a group, but which cannot be broken down into a sequence of separate, individually-valid upgrades.

The particular case in which I experienced this can be reproduced like so:
# Start with a clean install of Puppet 6.8.1 and puppetserver 6.5.0 (I used the Puppet, Inc. RPMs, installed via yum).

# Install version 6.3.0 of puppetlabs-apt via the module tool
# Install version 2.1.2 of puppetlabs-puppet_agent via the module tool
# Attempt to upgrade puppetlabs-puppet_agent to version 2.2.0 via the module tool (fails on unresolvable dependencies)
# Attempt to upgrade puppetlabs-apt to version 7.1.0 via the module tool (fails on unresolvable dependencies)

One should not need to use {{\-\-force}} or {{\-\-ignore\-dependencies}} to upgrade from one set of module versions in which all dependencies are satisfied to another set of module versions in which all dependencies are satisfied.

*Desired Behavior:*

The module tool should be willing to perform upgrades such as those described without using {{\-\-force}} or {{\-\-ignore\-dependencies}}.  In particular, it should be willing to perform any combination of minor-version upgrades in service to an upgrade of the module named on the command line, so long as it results in a final state in which all dependencies are satisfied.  Use of {{\-\-force}} or {{\-\-ignore\-dependencies}} is not desirable here because it does not automatically solve dependencies, and it can (and often does) produce a situation in which dependencies indeed are not fully satisfied.

It would be acceptable, albeit not ideal for this purpose, to require use of a new, different command-line argument.

*Actual Behavior:*

Here's an example of the observed behavior.  Specific module versions are expressed via {{\-\-version}} options to ensure reproducibility.  The target versions for the upgrade commands reflect the latest versions of the modules involved as of the time of this report, and at this time, the upgrade failures can also be reproduced without specifying target module versions.

Jorie Tappa (JIRA)

unread,
Sep 23, 2019, 1:21:03 PM9/23/19
to puppe...@googlegroups.com
Jorie Tappa commented on Bug PUP-10018
 
Re: The module tool won't solve some dependencies without --force or --ignore-dependencies

Hi John Bollinger, unfortunately the Puppet Module Tool is no longer under active development, have you tried using PDK to solve this issue?

John Bollinger (JIRA)

unread,
Sep 23, 2019, 1:43:03 PM9/23/19
to puppe...@googlegroups.com

Hi Jorie Tappa, To the best of my knowledge, PDK does not address the problem of installing or upgrading modules in a working Puppet environment (the special case of the "pdk test unit" command notwithstanding). Perhaps I have used the wrong name. As the instructions for reproducing the issue show, I am talking about the "puppet module install" and especially "puppet module upgrade" commands. These do not emit deprecation warnings when run.

Rob Braden (JIRA)

unread,
Sep 30, 2019, 12:53:03 PM9/30/19
to puppe...@googlegroups.com
Rob Braden updated an issue
 
Change By: Rob Braden
Sprint: Coremunity Grooming

Rob Braden (JIRA)

unread,
Sep 30, 2019, 12:53:04 PM9/30/19
to puppe...@googlegroups.com
Rob Braden updated an issue
Change By: Rob Braden
Team: Coremunity

Josh Cooper (JIRA)

unread,
Jan 21, 2020, 12:45:04 PM1/21/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Sprint: Coremunity Grooming

Josh Cooper (Jira)

unread,
Mar 13, 2020, 1:20:04 AM3/13/20
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-10018
 
Re: The module tool won't solve some dependencies without --force or --ignore-dependencies

I think the  puppetlabs-puppet_agent upgrade from 2.1.2 to 2.2.0 fails because the latter requires apt >= 7.0.1 and the PMT doesn't know to upgrade dependencies at the same time.

I'm not sure why -ignore-dependencies doesn't work, but -force does as a workaround:

[root@soft-shipboard ~]# puppet module upgrade puppetlabs-puppet_agent --version=2.2.0 --force
 
Notice: Preparing to upgrade 'puppetlabs-puppet_agent' ...
 
Notice: Found 'puppetlabs-puppet_agent' (v2.1.2) in /etc/puppetlabs/code/environments/production/modules ...
 
Notice: Downloading from https://forgeapi.puppet.com ...
 
Notice: Upgrading -- do not interrupt ...
 
/etc/puppetlabs/code/environments/production/modules
 
└── puppetlabs-puppet_agent (v2.1.2 -> v2.2.0)

After which you can upgrade apt as expected:

[root@soft-shipboard ~]# puppet module upgrade puppetlabs-apt --version=7.1.0
 
Notice: Preparing to upgrade 'puppetlabs-apt' ...
 
Notice: Found 'puppetlabs-apt' (v6.3.0) in /etc/puppetlabs/code/environments/production/modules ...
 
Notice: Downloading from https://forgeapi.puppet.com ...
 
Notice: Upgrading -- do not interrupt ...
 
/etc/puppetlabs/code/environments/production/modules
 
└── puppetlabs-apt (v6.3.0 -> v7.1.0)

This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Josh Cooper (Jira)

unread,
Mar 13, 2020, 1:21:03 AM3/13/20
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Coremunity Puppet Developer Experience

John Bollinger (Jira)

unread,
Mar 13, 2020, 9:12:03 AM3/13/20
to puppe...@googlegroups.com
John Bollinger commented on Bug PUP-10018
 
Re: The module tool won't solve some dependencies without --force or --ignore-dependencies

Thanks for your comments, Josh Cooper.  I'm a bit distanced from this issue now, six months later, but I know that I did in the end get the module updates performed.  I forget whether I used --force for that, but I suspect that I instead uninstalled several modules and then installed new versions with their newer dependencies.  Of course, the whole point of this issue is that neither --force nor --ignore-dependencies should be required in cases such as the one presented.

Also, "the PMT doesn't know to upgrade dependencies at the same time" does seem to sum up the problem, but my vague and possibly mistaken recollection is that it's not completely accurate.  That is, I think I do see it perform multiple upgrades in the same run sometimes, just not when any of the upgrades crosses a major-version-number threshold.

Josh Cooper (Jira)

unread,
Nov 10, 2021, 11:50:02 PM11/10/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Epic Link: PUP-11215
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Josh Cooper (Jira)

unread,
Nov 10, 2021, 11:50:02 PM11/10/21
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Puppet Developer Experience
Reply all
Reply to author
Forward
0 new messages