Jira (PUP-11368) Can't install the dsc/* modules without explicitly specifying version

38 views
Skip to first unread message

Alexander Olofsson (Jira)

unread,
Nov 19, 2021, 8:25:01 AM11/19/21
to puppe...@googlegroups.com
Alexander Olofsson created an issue
 
Puppet / Bug PUP-11368
Can't install the dsc/* modules without explicitly specifying version
Issue Type: Bug Bug
Affects Versions: PUP 7.12.0, PUP 6.19.1
Assignee: Unassigned
Created: 2021/11/19 5:24 AM
Priority: Normal Normal
Reporter: Alexander Olofsson

$ puppet module install --ignore-dependencies --force --target-dir `pwd` dsc/psdscresources
Notice: Preparing to install into /tmp/tmp.F6tr8BbJhO ...
Notice: Downloading from https://forgeapi.puppet.com ...
Error: Could not install 'dsc-psdscresources' (latest)
  No releases are available from https://forgeapi.puppet.com
    Does 'dsc-psdscresources' have at least one published release?

This seems to be related to how Puppet handles module versions, as it's possible to get it to install an open-ended version definition - albeit not the correct latest one - by doing something like;

$ puppet module install --ignore-dependencies --force --target-dir `pwd` dsc/psdscresources --version '>=2.0.0-0-0'
Notice: Preparing to install into /tmp/tmp.F6tr8BbJhO ...
Notice: Downloading from https://forgeapi.puppet.com ...
Notice: Installing -- do not interrupt ...
/tmp/tmp.F6tr8BbJhO
└── dsc-psdscresources (v2.0.0-0-3)

Specifying an exact version number works as expected.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.13.2#813002-sha1:c495a97)
Atlassian logo

Josh Cooper (Jira)

unread,
Nov 19, 2021, 2:06:01 PM11/19/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11368
 
Re: Can't install the dsc/* modules without explicitly specifying version

This is a bug in semantic_puppet when comparing prerelease versions of a module against the constraint ">= 0.0.0" as can be seen from:

(byebug) SemanticPuppet::VersionRange.parse(">= 0.0.0") === SemanticPuppet::Version.parse("2.12.0-0-4")
false
(byebug) SemanticPuppet::VersionRange.parse(">= 0.0.0") === SemanticPuppet::Version.parse("2.12.0")
true

Both of those should return true.

Things go wrong here: https://github.com/puppetlabs/semantic_puppet/blob/dbb613b2124fba772861b680916e6fd19ce5f0d6/lib/semantic_puppet/dependency/graph.rb#L19

where === is aliased to include?. Since it returns false then the PMT thinks the release doesn't satisfy the constraint, leading to the error message. It works when using force, because we ignore those constraints.

We've had troubles with prereleases before, e.g. PUP-6264.

Given other priorities we're unlikely to fix the puppet module tool anytime soon, but perhaps we should stop including prerelease version in the psdscresources module that we publish to the forge. Thoughts Michael Lombardi?

Ciprian Badescu (Jira)

unread,
Nov 29, 2021, 10:16:01 AM11/29/21
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Nov 30, 2021, 11:45:02 AM11/30/21
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11368

It looks like all of the autogenerated dsc modules have prerelease versions in them: https://forge.puppet.com/modules/dsc

Ciprian Badescu (Jira)

unread,
Dec 6, 2021, 10:13:02 AM12/6/21
to puppe...@googlegroups.com

Ben Ford (Jira)

unread,
Dec 7, 2021, 5:09:02 PM12/7/21
to puppe...@googlegroups.com
Ben Ford commented on Bug PUP-11368
 
Re: Can't install the dsc/* modules without explicitly specifying version

The reason for this problem is complex. The simple version is that we don't just import the DSC resource directly from the PowerShell Gallery, we run it through a build process that repackages and vendors it into a Puppet module along with helper functions. Sometimes we have to make updates to the tool that does that build process, and when we do that, we need to rebuild a new version of the module.

We can't just republish the same version number of the module, because there be dragons, so we have to distinguish between and rank builds. We chose to do that via pre-release numbering (version-prerelease (see below), so the referenced dsc-psdscresources-2.0.0-0-3 module is the third build of the psdscresources v2.0.0 DSC module.

So why did we use prerelease instead of the build identifier (version+build)? Because semver explicitly discards the build identifier when comparing versions, so dsc-psdscresources-2.0.0+0-1 = dsc-psdscresources-2.0.0+0-2 = dsc-psdscresources-2.0.0+0-3. So this still doesn't let us have a "latest" version.

Correcting this means fixing semantic-puppet such that it compares pre-release versions properly.

The problem is in the assumption made with the first bullet point of https://github.com/puppetlabs/semantic_puppet/pull/23. If you specify the range as described, it does work according to that design requirement. I believe that is incorrect.

> SemanticPuppet::VersionRange.parse(">= 2.12.0-0") === SemanticPuppet::Version.parse("2.12.0-0-4")
 => true

Related: we also need to iterate on the release numbering anyways, since this scheme will break as soon as we get to double digits. dsc-psdscresources-2.0.0+0-2 will sort lexicographically as greater than dsc-psdscresources-2.0.0+0-12.

Tangent: During development, we also specifically chose to require version pinning, since the ecosystem was more volatile than usual. That's no longer the case though.

This message was sent by Atlassian Jira (v8.20.2#820002-sha1:829506d)
Atlassian logo

Ben Ford (Jira)

unread,
Dec 7, 2021, 5:20:02 PM12/7/21
to puppe...@googlegroups.com

Ciprian Badescu (Jira)

unread,
Dec 15, 2021, 9:34:01 AM12/15/21
to puppe...@googlegroups.com

Ciprian Badescu (Jira)

unread,
Dec 15, 2021, 9:34:01 AM12/15/21
to puppe...@googlegroups.com

Ciprian Badescu (Jira)

unread,
Dec 16, 2021, 3:19:02 AM12/16/21
to puppe...@googlegroups.com

Ciprian Badescu (Jira)

unread,
Dec 17, 2021, 3:55:02 AM12/17/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
Sprint: NW - 2021-12-17 , NW - 2022-01-04

Ciprian Badescu (Jira)

unread,
Dec 21, 2021, 7:54:02 AM12/21/21
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
Sprint: NW - 2021-12-17, NW - 2022-01- 04 12

Ciprian Badescu (Jira)

unread,
Jan 4, 2022, 4:05:02 AM1/4/22
to puppe...@googlegroups.com

Ciprian Badescu (Jira)

unread,
Jan 4, 2022, 5:58:01 AM1/4/22
to puppe...@googlegroups.com
Ciprian Badescu assigned an issue to Unassigned
Change By: Ciprian Badescu
Assignee: Yasmin Rajabi

Ciprian Badescu (Jira)

unread,
Jan 10, 2022, 7:18:03 AM1/10/22
to puppe...@googlegroups.com
Ciprian Badescu commented on Bug PUP-11368
 
Re: Can't install the dsc/* modules without explicitly specifying version

The issues is not on how semantic_puppet compares versions (as `SemanticPuppet::Version.parse("0.0.0") <= SemanticPuppet::Version.parse("2.12.0-0-4") => true` is evaluated properly), and it is more about how puppet_semantic implements VersionRange inclusion.

This seems to be inline with the Node SemVer specifications as described in the ticket https://tickets.puppetlabs.com/browse/PUP-6373. referenced with the commit (https://github.com/puppetlabs/semantic_puppet/pull/23/commits/4ced1fac803468ca22e1e9bba79d024e1ef17981 - Make VersionRange comply with Node Semver) - https://docs.npmjs.com/cli/v6/using-npm/semver while not inline with the Ruby/Gem implementation as tested bellow.

The side-effect of such change needs to be estimated and agreed before doing deciding the implementation.

2.6.4 :019 > Gem::Version.create("0.0.0") <= Gem::Version.create("2.12.0-0-4")
 => true 
2.6.4 :021 > Gem::Requirement.create(">= 0.0.0") === Gem::Version.create("2.12.0-0-4")
 => true 
2.6.4 :024 > Gem::Requirement.create(">= 2.12.0-0") === Gem::Version.create("2.12.0-0-4")
 => false (this is surprisingly) 

> semver.lt('0.0.0', '2.12.0-0-4')
true
> semver.satisfies('2.12.0-0-4', '>= 0.0.0')
false
> semver.satisfies('2.12.0-0-4', '>= 2.12.0-0')
true

2.6.4 :003 > SemanticPuppet::Version.parse("0.0.0") <= SemanticPuppet::Version.parse("2.12.0-0-4")
 => true 
2.6.4 :006 > SemanticPuppet::VersionRange.parse(">= 0.0.0") === SemanticPuppet::Version.parse("2.12.0-0-4")
 => false 
2.6.4 :005 > SemanticPuppet::VersionRange.parse(">= 2.12.0-0") === SemanticPuppet::Version.parse("2.12.0-0-4")
 => true 

 

Add Comment Add Comment
 

Ciprian Badescu (Jira)

unread,
Jan 10, 2022, 7:19:02 AM1/10/22
to puppe...@googlegroups.com
Ciprian Badescu updated an issue
Change By: Ciprian Badescu
Sprint: NW - 2021-12-17 , NW - 2022-01-12

Ben Ford (Jira)

unread,
Jan 19, 2022, 1:00:01 PM1/19/22
to puppe...@googlegroups.com

Josh Cooper (Jira)

unread,
Feb 16, 2022, 2:09:01 AM2/16/22
to puppe...@googlegroups.com
Josh Cooper commented on Bug PUP-11368

Ah ok I see now. Looking at the nodejs docs:

If a version has a prerelease tag (for example, 1.2.3-alpha.3) then it will only be allowed to satisfy comparator sets if at least one comparator with the same [major, minor, patch] tuple also has a prerelease tag.

Rubygems behaves similarly in that you have to explicitly ask for a prerelease gem, such as "gem install --pre <name>". See https://github.com/rubygems/rubygems/blob/459b12b3c3ea0890076e4cb5e1951d9169db5aca/test/rubygems/test_kernel.rb#L64

And semantic_puppet behaves similarly as seen in this test since the version "1.2.4-0" is not considered to be included in the open-ended range "> 1.2.3".

So ben.ford I think if you want to rely on prerelease numbering for DSC mopdules then we'd need an option like "puppet module install --prerelease dsc/psdscresources".

Also it looks like the semver checker you linked to has issues with prereleases: https://github.com/jubianchi/semver-check/issues/106

Reply all
Reply to author
Forward
0 new messages