Open Source 4.0 version identifier vs. very different rpm and dpkg package versions

[Vince Skahan]

I just set up a couple 4.0.0 virtual machines, trying to follow the installation instructions as-written on the website, and found some confusing things.

First - can somebody explain why you're referring to the software in docs with one version (4.0.0) and you have a very different versioning for the rpm/deb packages....and then the --version strings of the various utilities in /opt/puppetlabs/bin report something very different from either the 4.0.0 version 'or' the rpm/deb package versions.

Is there a reason you're not packaging for the various distros using the same version identifiers your code and docs seem to use ?  Seems like a gratuitous obfuscation to me.   Why would you possibly have their major versions so different from what 'foo --version' reports ?

#------ (on centos server)

# rpm -qa|grep -i pup

puppetlabs-release-pc1-0.9.2-1.el7.noarch

puppet-agent-1.0.0-1.el7.x86_64

puppetserver-2.0.0-1.el7.noarch

# puppet --version

4.0.0

 

# cfacter --version

0.4.1 (commit e9333e184e57106f7ce3fb6b44b05656f417d245)

# facter --version

2.4.3

# hiera --version

2.0.1

 

# mco --version

/opt/puppetlabs/bin/mco 2.8.1

# puppetserver --version

puppetserver version: 2.0.0 

#-------(on debian client)

# dpkg -l | grep -i pup

ii  puppet-agent                       1.0.0-1wheezy                 amd64        The Puppet Agent package contains all of the elements needed to run puppet, including ruby, facter, hiera and mcollective.

ii  puppetlabs-release-pc1             0.9.2-1wheezy                 all          Release packages for the Puppet Labs PC1 repository

# puppet --version

4.0.0

# cfacter --version

0.4.1 (commit e9333e184e57106f7ce3fb6b44b05656f417d245)

# facter --version

2.4.3

# hiera --version

2.0.1

# mco --version

/opt/puppetlabs/bin/mco 2.8.1

 

Also your 4.0 docs have lots of pointers back to 3.7 info, which is also pretty confusing, as well as having links that don't resolve.

• https://docs.puppetlabs.com/puppet/4.0/reference/system_requirements.html  (in the main body) 'pre-install tasks' doesn't resolve, but it does resolve in the menu on the left
  
• https://docs.puppetlabs.com/puppet/4.0/reference/install_linux.html points to 3.7 docs for the bottom half-dozen links
  

Lastly, when installing the agent system, it didn't register the cert request with the master until I ran agent with --test and 'then' started the agent with puppet itself.

[Felix Frank]

Hi,

I may have some of this wrong, so take this with a grain of salt, please.

On 04/20/2015 09:34 PM, Vince Skahan wrote:
> puppetlabs-release-pc1-0.9.2-1.el7.noarch

I think this package only sets up the puppetlabs repos for you. Its
versioning has literally nothing to do with *contents* of those repos.
Syncing it to current puppet stable would likely contradict semver.

> puppet-agent-1.0.0-1.el7.x86_64

I believe this is the version of the AIO agent package. This kind of
packaging is new, and apparently it was decided to start its versioning
at 1. This might be sensibly synced with the core version, yes.

> puppetserver-2.0.0-1.el7.noarch

Puppet Server is a new-ish product that just launched last year. Syncing
its version to that of core puppet would most definitely contradict
semver, unless we keep bumping the versions on both.

>
> # puppet --version
> 4.0.0

Truth.

>
>
> # cfacter --version
> 0.4.1 (commit e9333e184e57106f7ce3fb6b44b05656f417d245)
> # facter --version
> 2.4.3
> # hiera --version
> 2.0.1

While tightly interwoven with Puppet, facter and hiera have always
developed independently and are versioned independently.

In the future, it might be desirable to lift all of these to one version
that could then be applied to the All-In-One package as well, but such
measures have implications.

> # mco --version
> /opt/puppetlabs/bin/mco 2.8.1

MCO is related to Puppet, but not part of it per se.

> # puppetserver --version
> puppetserver version: 2.0.0

See above.

Hope this clears things up a little.

As for your issues with the documentation, would you raise these as bugs
in Jira?

Cheers,
Felix

[Joshua Fielden]

There is actually nothing in semver that precludes co-terming major release versions,  and less that precludes "--version"  from both displaying the local package version,  AND the release number targeted by said package if you disagreed with the first point. This is one of those circumstances where being pedantic means you reduce understanding,  not increase it.

Cheers,

jcf

Cheers,

jcf

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/55390851.1020002%40alumni.tu-berlin.de.
For more options, visit https://groups.google.com/d/optout.

[Vince Skahan]

On Thursday, April 23, 2015 at 3:44:35 PM UTC-7, jcf wrote:

There is actually nothing in semver that precludes co-terming major release versions,  and less that precludes "--version"  from both displaying the local package version,  AND the release number targeted by said package if you disagreed with the first point. This is one of those circumstances where being pedantic means you reduce understanding,  not increase it.

Agree.

What I'm looking for is clarity for the users.  What is there now is beyond unclear.

The problem as I see it is bundling.  Puppet is grouping standalone components into an aggregate 'release' of something bigger.  The pieces have versions that are far different from the aggregate.   Each piece's --version string doesn't even match the version of the rpm they were bundled into. That is just wrong.

Put yourself in the user's shoes:

• I think I'm running puppet open source 4.0.0 including both server and agent
• there is a puppetserver-2.0.0 there, and a puppet-agent-1.0.0 there   - geez, I must be YEARS behind current !
• so I look in /opt/puppetlabs/bin and see five binaries, none of which have --version = 4.0.0 or even 'close' to the rpm version they came from
• and that is clear 'how' exactly ?

My suggestion would be:

• puppetserver rpm should be 4.0.0 for puppet release 4.0.0
• puppet-agent rpm should be 4.0.0 for puppet release 4.0.0

And to me I'd add:

• everything in a typically public-interface directory (not a bundled ruby version, for example) that is therein and supports a --version should return a version reflecting the rpm version they were packaged in somehow that is user obvious.

or if you can't do that, bundle each of them into their own individually versioned rpms (ie. facter-2.4.3) and have puppet-agent require multiple rpms.

[jcbollinger]

On Friday, April 24, 2015 at 12:44:08 PM UTC-5, Vince Skahan wrote:

On Thursday, April 23, 2015 at 3:44:35 PM UTC-7, jcf wrote:

There is actually nothing in semver that precludes co-terming major release versions,  and less that precludes "--version"  from both displaying the local package version,  AND the release number targeted by said package if you disagreed with the first point. This is one of those circumstances where being pedantic means you reduce understanding,  not increase it.

Agree.

Yes, but I think there's a combination of problems here.

Let's take puppetserver first, because in some ways it's easiest.  The version of puppetserver in the puppetserver-2.0.0 packages really is 2.0.0, thus software version code and package version code are consistent.  The issue is that puppetserver 2.0.0 implements version 4.0.0 (I think) of "Puppet", where "Puppet" these days means the DSL, plugin API, and built-in type and function libraries.  This is roughly analogous to version 31 of Firefox handling version 5 of HTML, but more confusing because it's new that there is a split between software version and -- I'm not even sure what to call it, but maybe "framework version".  Inasmuch as puppetserver is taking over from the older Ruby implementation of the master, it may be that these numbers can be reconciled in the future.  (See also below.)

The puppet-agent package is where the real fun starts.  As best I can determine, the version number of the puppet-agent package is associated with the overall all-in-one installation image and packaging, as opposed to the software packaged within, the central piece of which is the Ruby implementation of the Puppet software (version 4.0.0 of that software in the puppet-agent-1.0.0 package).  I do not understand this decision, as it seems to fly in the face standard package versioning practice.  That aspect of it could certainly be fixed, however -- it should not produce any great difficulties to bump up the version number for the package to correctly agree with the version of the packaged software.

Then there's the question of standalone software packaged together with the agent, and the version numbers of those programs.  In truth, I don't think the version number mismatch is a significant issue there if you accept all-in-one packaging to begin with.  In my experience, it's pretty common for AIOs to bundle software with different version numbers, but it is unusual for such an AIO to be versioned differently than the bundle's focal software.  I think this issue would be substantially mitigated just by making the version number of the puppet-agent package in fact match the version of the puppet agent software contained within.

 

What I'm looking for is clarity for the users.  What is there now is beyond unclear.

The problem as I see it is bundling.  Puppet is grouping standalone components into an aggregate 'release' of something bigger.  The pieces have versions that are far different from the aggregate.   Each piece's --version string doesn't even match the version of the rpm they were bundled into. That is just wrong.

As you have seen, I somewhat agree.  For what it's worth, I raised a bit of a stink a few months ago in the developers group about PL's decision to make Puppet 4 depend on an AIO-style installation image, but I was unable to persuade much of anyone.  Given PL's decision to go that direction, AIO packaging is a natural result, albeit an unnecessary and unwelcome one.

 

Put yourself in the user's shoes:

• I think I'm running puppet open source 4.0.0 including both server and agent
• there is a puppetserver-2.0.0 there, and a puppet-agent-1.0.0 there   - geez, I must be YEARS behind current !
• so I look in /opt/puppetlabs/bin and see five binaries, none of which have --version = 4.0.0 or even 'close' to the rpm version they came from
• and that is clear 'how' exactly ?

The 'puppet' binary reports version 4.0.0, which is consistent with the idea that you're running that version of Puppet.  If I wanted to verify which version of puppet I was running, that's certainly the binary I'd look to.  As I said, though, the package in which it is provided should also bear that version number.

That puppetserver's version doesn't match the Puppet framework version it supports certainly seems more confusing to me.

 

My suggestion would be:

• puppetserver rpm should be 4.0.0 for puppet release 4.0.0
• puppet-agent rpm should be 4.0.0 for puppet release 4.0.0

The latter, for sure.  The former would be desirable, too, but only in conjunction with puppetserver's software version actually being bumped up to 4.0.0.

 

And to me I'd add:

• everything in a typically public-interface directory (not a bundled ruby version, for example) that is therein and supports a --version should return a version reflecting the rpm version they were packaged in somehow that is user obvious.

or if you can't do that, bundle each of them into their own individually versioned rpms (ie. facter-2.4.3) and have puppet-agent require multiple rpms.

I would be all for packaging the (semi-)independent components separately, each in a package with version number matching the packaged software's version number.  I would be even more for ensuring that none of it depended on being installed in an AIO-style image, but that's a separable issue.

If the intention is for any of the core components (agent and server, more or less) to continue to be developed on their own cycles, then it might be appropriate to add an additional version number to associate the binaries with the puppet framework version with which they are associated.  For example, one might then have

# puppetserver --version --framework-version

puppetserver version: 2.0.0
Puppet framework version: 4.0.0


John

[Thomas Müller]

I

[Thomas Müller]

I think instead of "puppet-agent" the package should be named something like "puppet-client-collection" to show that it's not just "the agent" in pre-4.0 sense.

It's bit confusing installing puppet agent 4.0 and having a package version 1.0

[Eric Sorenson]

Hi Vince and John, thanks for this feedback. There's quite a few things under discussion in this thread but I think we can unpack them.

On Monday, April 27, 2015 at 9:34:13 AM UTC-7, jcbollinger wrote:

On Friday, April 24, 2015 at 12:44:08 PM UTC-5, Vince Skahan wrote:

On Thursday, April 23, 2015 at 3:44:35 PM UTC-7, jcf wrote:

There is actually nothing in semver that precludes co-terming major release versions,  and less that precludes "--version"  from both displaying the local package version,  AND the release number targeted by said package if you disagreed with the first point. This is one of those circumstances where being pedantic means you reduce understanding,  not increase it.

Agree.

Yes, but I think there's a combination of problems here.

Let's take puppetserver first, because in some ways it's easiest.  The version of puppetserver in the puppetserver-2.0.0 packages really is 2.0.0, thus software version code and package version code are consistent.  The issue is that puppetserver 2.0.0 implements version 4.0.0 (I think) of "Puppet", where "Puppet" these days means the DSL, plugin API, and built-in type and function libraries.  This is roughly analogous to version 31 of Firefox handling version 5 of HTML, but more confusing because it's new that there is a split between software version and -- I'm not even sure what to call it, but maybe "framework version".  Inasmuch as puppetserver is taking over from the older Ruby implementation of the master, it may be that these numbers can be reconciled in the future.  (See also below.)

Puppet Server 2.0 doesn't implement any of the "DSL, plugin API, type/function libraries" at all. All of those things exist independently of puppet server, inside of the "core" Puppet codebase. You can assure yourself of that by installing only the puppet-agent-1.0 AIO (or just puppet-4.0.0.tar.gz for that matter) and running 'puppet apply', 'puppet master', etc etc, without any puppet-server whatsoever. 

But your argument is not invalid! There is a piece which ties Puppet Server 2.0 to Puppet 4, and that is the agent<->master network API. So you could totally advance the idea that there should be some version-related coupling between those two projects. This is implicit right now (or rather, it's expressed in package dependencies and text documentation) and I understand your and Vince's desires to have this made more explicit, by expressing it directly in the version numbers of each project. 

The "Framework version" you're describing is exactly the concept we're trying to express with the Collections repository (for future Googlers' context, the explanatory blog post is at: https://puppetlabs.com/blog/welcome-puppet-collections ).  PuppetDB also has some dependency tendrils, too, but its versioning is totally decoupled from Puppet's.

 

The puppet-agent package is where the real fun starts.  As best I can determine, the version number of the puppet-agent package is associated with the overall all-in-one installation image and packaging, as opposed to the software packaged within, the central piece of which is the Ruby implementation of the Puppet software (version 4.0.0 of that software in the puppet-agent-1.0.0 package).  I do not understand this decision, as it seems to fly in the face standard package versioning practice.  That aspect of it could certainly be fixed, however -- it should not produce any great difficulties to bump up the version number for the package to correctly agree with the version of the packaged software.

I think this arose from a desire to express the versioning of the AIO as a semver number which tracks the collection itself, independently from the versions of the constituent projects. Since there is ruby, facter (both flavours!), mcollective, hiera, augeas, it seemed wrong to pin the puppet-agent number to puppet-the-project. But perhaps it was an error to have a non-puppet-centric view of the world, since it's true that the purpose of all those other things is really to deliver Puppet in a batteries-included, supportable way.

 

Then there's the question of standalone software packaged together with the agent, and the version numbers of those programs.  In truth, I don't think the version number mismatch is a significant issue there if you accept all-in-one packaging to begin with.  In my experience, it's pretty common for AIOs to bundle software with different version numbers, but it is unusual for such an AIO to be versioned differently than the bundle's focal software.  I think this issue would be substantially mitigated just by making the version number of the puppet-agent package in fact match the version of the puppet agent software contained within.

I think this and Vince's example below make sense at the initial release point, but as we found when we attempt to match PE's version number to its Puppet component, it very quickly degenerates into incoherence. As an example, if it started out at puppet-agent-4.0.0 to match puppet-4.0.0, but the only change in the next AIO package is to incorporate a feature update in Facter (a hypothetical 2.5.0), we'd want to reflect that feature update in puppet-agent-4.1.0. But should that then trigger a release of a puppet-4.1.0 even though nothing has changed in puppet? 

 

What I'm looking for is clarity for the users.  What is there now is beyond unclear.

The problem as I see it is bundling.  Puppet is grouping standalone components into an aggregate 'release' of something bigger.  The pieces have versions that are far different from the aggregate.   Each piece's --version string doesn't even match the version of the rpm they were bundled into. That is just wrong.

As you have seen, I somewhat agree.  For what it's worth, I raised a bit of a stink a few months ago in the developers group about PL's decision to make Puppet 4 depend on an AIO-style installation image, but I was unable to persuade much of anyone.  Given PL's decision to go that direction, AIO packaging is a natural result, albeit an unnecessary and unwelcome one.

Well, it's more that Puppet Labs' packaging for Puppet 4 is an AIO-style installer, not that Puppet 4 depends on AIO inherently. This might be parsing too tightly, but I want to emphasise that anybody could do this to build a standalone Puppet 4 RPM:

rake package:rpm 

if the AIO is truly anathema. We're also working to open-source the packaging toolchain, so if it's not so much the idea but the particular component versions (or the installation root, as I've gotten some feedback hardcoding /opt doesn't work for some people), you could roll your own. 

  

Put yourself in the user's shoes:

• I think I'm running puppet open source 4.0.0 including both server and agent
• there is a puppetserver-2.0.0 there, and a puppet-agent-1.0.0 there   - geez, I must be YEARS behind current !
• so I look in /opt/puppetlabs/bin and see five binaries, none of which have --version = 4.0.0 or even 'close' to the rpm version they came from
• and that is clear 'how' exactly ?

The 'puppet' binary reports version 4.0.0, which is consistent with the idea that you're running that version of Puppet.  If I wanted to verify which version of puppet I was running, that's certainly the binary I'd look to.  As I said, though, the package in which it is provided should also bear that version number.

That puppetserver's version doesn't match the Puppet framework version it supports certainly seems more confusing to me.

Right, puppet --version does return 4.0 so, Vince, I'm not sure what you saw. 

If some or all of the (Collections repo, Puppet-Agent, Puppet-Server, Puppet) said "4", would things fall into place cognitively? How would you expect them to change over time? I.e should they all update in lockstep? If not, how much drift would cause the linkage between them to no longer make sense? 

  

My suggestion would be:

• puppetserver rpm should be 4.0.0 for puppet release 4.0.0
• puppet-agent rpm should be 4.0.0 for puppet release 4.0.0

The latter, for sure.  The former would be desirable, too, but only in conjunction with puppetserver's software version actually being bumped up to 4.0.0.

 

And to me I'd add:

• everything in a typically public-interface directory (not a bundled ruby version, for example) that is therein and supports a --version should return a version reflecting the rpm version they were packaged in somehow that is user obvious.

or if you can't do that, bundle each of them into their own individually versioned rpms (ie. facter-2.4.3) and have puppet-agent require multiple rpms.

 

I would be all for packaging the (semi-)independent components separately, each in a package with version number matching the packaged software's version number.  I would be even more for ensuring that none of it depended on being installed in an AIO-style image, but that's a separable issue.

If the intention is for any of the core components (agent and server, more or less) to continue to be developed on their own cycles, then it might be appropriate to add an additional version number to associate the binaries with the puppet framework version with which they are associated.  For example, one might then have

# puppetserver --version --framework-version

puppetserver version: 2.0.0
Puppet framework version: 4.0.0

We've clearly done a less-than-stellar job at either (or both!) understanding what people outside PL walls expect with regard to versioning, or communicating why the numbers landed the way they did. Let's think through changes to the current scheme in the context of the next 9-12 months of major/feature/patch releases and figure out what's going to work best.

--eric0

[Vince Skahan]

On Monday, April 27, 2015 at 2:30:00 PM UTC-7, Eric Sorenson wrote:

As an example, if it started out at puppet-agent-4.0.0 to match puppet-4.0.0, but the only change in the next AIO package is to incorporate a feature update in Facter (a hypothetical 2.5.0), we'd want to reflect that feature update in puppet-agent-4.1.0. But should that then trigger a release of a puppet-4.1.0 even though nothing has changed in puppet? 

Great example and question which I think indicates the issues I am trying to get across re: the existing versioning and bundling.

My view as a user is that puppet 4.0 is a version of an architecture and its matching DSL and API.   Now that might be totally at odds with how PL views it, but lets go down that path for discussion purposes for a bit further and see what packaging/versioning might result.

• puppetserver being versioned 2.0.0 is needlessly confusing, if it's puppet (the product) 4.0.0 that it implements.  What's 'in' it seems reasonable, it's just the server piece, but its rpm version should line up with what the binary reports when you use --version against it.
• puppet-agent is the main issue, it's really a package that could be called puppet-agent-plus-everything-else-but-the-kitchen-sink.   Too much stuff that is a little loosely related to puppet 4.0.0 (the architecture/api) is in there, and since they are so loosely related (currently) you didn't change their version strings, which differ greatly.  It also has the same mismatch of rpm version (1.0.0) vs. architecture version (4.0.0) for the rpm vs. the agent itself.

And yes, now that 4.0.0 is out, I know it'll be a joy to switch gears moving forward...

Using your example, if you needed to update facter to a hypothetical 2.5.0, what would you do ? You'd need to roll your existing puppet-agent-2.0.0 rpm to something different.  What version do you call it ?   The agent (within) is strongly linked to the server 4.0.0, the ruby interpreter and misc utils in there far less so.  Would you just call the package puppet-agent-2.1.0 hypothetically ?  The fact that puppet-agent-4.0.0 is in there (and that the cfacter change didn't alter anything 'puppet') is still confusing if you pick anything other than 4.0.x, but as you said a roll in facter that doesn't change the puppet api wouldn't normally cause a puppet-agent version roll.   Confusing no matter which way you go.

I'd suggest you need to break the core puppet stuff (server, agent) apart from the utils (mco, facter/cfacter, hiera) and common (ruby) pieces.   Version the server/agent stuff to match the puppet api version.  Version the utils any way you want if they're not linked really to the puppet version.  Same for your ruby version.

So I'd suggest repackaging ala:

puppetserver-4.0.0-1.el7.noarch

puppet-agent-4.0.0-1.el7.x86_64  <= just the agent piece

puppet-utils-4.0.0-1.el7.x86_64  <= hiera/facter/cfacter/mco 

puppet-ruby-1.9.whatever.el7.x86_64  <= just the ruby interpreter

or if you wanted to do (to me) a better componentization, I'd suggest:

puppetserver-4.0.0-1.el7.noarch

puppet-agent-4.0.0-1.el7.x86_64  <= just the agent piece

puppet-facter-2.4.3-1.el7.x86_64

puppet-mco-2.8.1-1.el7.x86_64

puppet-hiera-2.0.1.el7.x86_64

puppet-ruby-1.9.whatever-1.el7.x86_64

Your AIO installation of the multiple pieces currently in the agent rpm would still work, it would be just adding 'requires' to the agent rpm.   Of course now that 4.0.0 is in the field, switching gears is an adventure in itself I guess  and getting from 4.0.0 to 4.1.0 would be interesting to say the least.  At a minimum, it would let you roll hiera or mco or facter or ruby asynchronously at their own rates.

Sorry for the long response.  Feel free to email if I lost you someplace above.

[jcbollinger]

On Monday, April 27, 2015 at 4:30:00 PM UTC-5, Eric Sorenson wrote:

On Monday, April 27, 2015 at 9:34:13 AM UTC-7, jcbollinger wrote:

[...] I think there's a combination of problems here.

Let's take puppetserver first, because in some ways it's easiest.  The version of puppetserver in the puppetserver-2.0.0 packages really is 2.0.0, thus software version code and package version code are consistent.  The issue is that puppetserver 2.0.0 implements version 4.0.0 (I think) of "Puppet", where "Puppet" these days means the DSL, plugin API, and built-in type and function libraries.  This is roughly analogous to version 31 of Firefox handling version 5 of HTML, but more confusing because it's new that there is a split between software version and -- I'm not even sure what to call it, but maybe "framework version".  Inasmuch as puppetserver is taking over from the older Ruby implementation of the master, it may be that these numbers can be reconciled in the future.  (See also below.)

Puppet Server 2.0 doesn't implement any of the "DSL, plugin API, type/function libraries" at all. All of those things exist independently of puppet server, inside of the "core" Puppet codebase. You can assure yourself of that by installing only the puppet-agent-1.0 AIO (or just puppet-4.0.0.tar.gz for that matter) and running 'puppet apply', 'puppet master', etc etc, without any puppet-server whatsoever. 

I guess I'm not surprised to hear that you can run the Ruby puppetmaster from the code in the puppet-agent package, but it does muddy the waters even further, now casting doubt on the propriety of even the package names.  If the puppet-agent package indeed contains Puppet's full Ruby stack (which isn't so easily separable into master and agent), then why is it not named simply "puppet"?  If puppetserver does not stand on its own, then why is it the puppet-agent package on which it depends?

 

But your argument is not invalid! There is a piece which ties Puppet Server 2.0 to Puppet 4, and that is the agent<->master network API.

Ok, good point, add "network API" to the definition of "framework".

Also, I guess I misunderstood the scope of the puppetserver project.  I had thought its purpose was to replace the master altogether, which would include catalog building.  I guess, then, it substitutes only for the master's rack-based communication endpoint (which is just fine, and seems more sensible to me).

 

So you could totally advance the idea that there should be some version-related coupling between those two projects. This is implicit right now (or rather, it's expressed in package dependencies and text documentation) and I understand your and Vince's desires to have this made more explicit, by expressing it directly in the version numbers of each project. 

It's all about clarity in labeling.  The name of the "puppetserver" package suggests that you need it to run a Puppet master (that being the server component in a master/agent setup).  That is reinforced by the naming of the "puppet-agent" package, which I hope you will acknowledge does not suggest that it's a viable alternative for running a puppet master.

 

The "Framework version" you're describing is exactly the concept we're trying to express with the Collections repository (for future Googlers' context, the explanatory blog post is at: https://puppetlabs.com/blog/welcome-puppet-collections ).  PuppetDB also has some dependency tendrils, too, but its versioning is totally decoupled from Puppet's.

No, I don't think "framework version" is the same concept at all.  At least, the blog's description of collections doesn't say that to me.  My reading is that collections are about packaging and component compatibility.  Framework version is a major consideration on the compatibility end, but it is a lower-level and narrower concept than collections appear to be.

Nevertheless, Bravo! on collections.  It looks like they will be very helpful.

 

 

The puppet-agent package is where the real fun starts.  As best I can determine, the version number of the puppet-agent package is associated with the overall all-in-one installation image and packaging, as opposed to the software packaged within, the central piece of which is the Ruby implementation of the Puppet software (version 4.0.0 of that software in the puppet-agent-1.0.0 package).  I do not understand this decision, as it seems to fly in the face standard package versioning practice.  That aspect of it could certainly be fixed, however -- it should not produce any great difficulties to bump up the version number for the package to correctly agree with the version of the packaged software.

I think this arose from a desire to express the versioning of the AIO as a semver number which tracks the collection itself, independently from the versions of the constituent projects. Since there is ruby, facter (both flavours!), mcollective, hiera, augeas, it seemed wrong to pin the puppet-agent number to puppet-the-project. But perhaps it was an error to have a non-puppet-centric view of the world, since it's true that the purpose of all those other things is really to deliver Puppet in a batteries-included, supportable way.

Yes, understood, and yes.  In the RPM world, at least, the package version number describes the software in the package; it is the release code that describes the packaging and any ancillary considerations.  You could use semver for the release code if you wanted to do (e.g. puppet-4.0.0-1.0.0), though that seems like overkill to me.

 

 

Then there's the question of standalone software packaged together with the agent, and the version numbers of those programs.  In truth, I don't think the version number mismatch is a significant issue there if you accept all-in-one packaging to begin with.  In my experience, it's pretty common for AIOs to bundle software with different version numbers, but it is unusual for such an AIO to be versioned differently than the bundle's focal software.  I think this issue would be substantially mitigated just by making the version number of the puppet-agent package in fact match the version of the puppet agent software contained within.

I think this and Vince's example below make sense at the initial release point, but as we found when we attempt to match PE's version number to its Puppet component, it very quickly degenerates into incoherence. As an example, if it started out at puppet-agent-4.0.0 to match puppet-4.0.0, but the only change in the next AIO package is to incorporate a feature update in Facter (a hypothetical 2.5.0), we'd want to reflect that feature update in puppet-agent-4.1.0. But should that then trigger a release of a puppet-4.1.0 even though nothing has changed in puppet? 

No, you'd want to signal such a change via the package's release code, not its version number.  Or you could save yourself some trouble by providing the various components in separate packages, which would be a big win in my book.  In addition to avoiding package versioning headaches, package-per-component could also afford users more and easier control over the details of the combination of components in use at their sites.  And that seems like what Collections is (or should be) aiming at.  Indeed, it seems to me that there is a lot of redundancy between Collections and AIO packaging.

 

 

For what it's worth, I raised a bit of a stink a few months ago in the developers group about PL's decision to make Puppet 4 depend on an AIO-style installation image, but I was unable to persuade much of anyone.  Given PL's decision to go that direction, AIO packaging is a natural result, albeit an unnecessary and unwelcome one.

Well, it's more that Puppet Labs' packaging for Puppet 4 is an AIO-style installer, not that Puppet 4 depends on AIO inherently.

When we had this discussion on the dev list, no one from PL was prepared to promise that Puppet 4 would work correctly if not installed in an AIO-style image -- at a specific location, even.  No one was even willing to accept as a formal goal that it should do, though some agreed it would be good if that could be accomplished.  If indeed Puppet does not depend on an AIO-style installed image, then I am overjoyed!  Is it indeed time for a happy dance?

Also, your choice of wording may reflect a bit on the disconnect here.  A "package" is not at all the same thing as an "installer", even on systems that have a weak or non-existent native idea of packages.  More or less nobody but the software packager cares about the details of an installer, to the extent that a given package even contains one.  Everybody else cares mostly about what's in the package, and that's primarily who package labeling is for.

I think AIO nature (or not) of the installed image and AIO-style packaging and installation present largely orthogonal issues, however; I'd like to keep this thread's focus on the latter.

 

We're also working to open-source the packaging toolchain, so if it's not so much the idea but the particular component versions (or the installation root, as I've gotten some feedback hardcoding /opt doesn't work for some people), you could roll your own.

Open-sourcing the packaging toolchain is great, and doing so will mitigate some of the issues surrounding the AIO packaging, but I don't think it addresses the key problems that we started with, which are mostly about how PL is presenting packages to the world, not about what the world can or should do to digest them more easily.

 

We've clearly done a less-than-stellar job at either (or both!) understanding what people outside PL walls expect with regard to versioning, or communicating why the numbers landed the way they did. Let's think through changes to the current scheme in the context of the next 9-12 months of major/feature/patch releases and figure out what's going to work best.

Thanks for being receptive to these concerns.  It does appear that PL has gone a bit overboard on semver, and has perhaps focused too much on an internal view of the packaging and software delivery issue.  I hope these issues can indeed be smoothed out over the course of the next months.


John

[Corey Osman]

I am glad I am not the only one who said WTF is Puppet Agent 1.0.  If in fact this is referring to the collection repo around running puppet agent AIO than it should be called Puppet Agent Collection AIO 1.0 or something similar.

Corey

[Michael Stahnke]

On Wed, Apr 29, 2015 at 9:32 AM, Corey Osman <co...@logicminds.biz> wrote:

I am glad I am not the only one who said WTF is Puppet Agent 1.0.  If in fact this is referring to the collection repo around running puppet agent AIO than it should be called Puppet Agent Collection AIO 1.0 or something similar.

It's not just a collection for the agent though. The purpose of the collection is an ecosystem of software that works together. Server, Agent, PuppetDB are the main components, but in here eventually you could see things like lint, mco plugins, puppet-rspec, beaker, etc. The goal of the collection is compatibility. It's more like a distribution of software than something pinned to a version of any one component.  This an analogous to Fedora 22 not containing Linux Kernel 22, but just being a giant collection of lots of software. Our collection is much smaller, but serves a similar purpose. 

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/3216ad5d-e179-44ce-94d1-f2180fc3bc34%40googlegroups.com.

[Michael Stahnke]

Rather than reply and quote lots of points, I'll just bring up a few things which could help clarify our goals. I'm sure not everybody will agree or love all of them, we know there are compromises here but attempted to make choices with the user's benefit and ease of use in mind. 

* The package is called puppet-agent because Puppet is more than the tool puppet at this point. It's an ecosystem. It's a suite of tools working together to manage infrastructure and applications. The agent is the word that means "all the stuff you need to run an endpoint in this ecosystem."

* We wanted to get away from our users having to carry the cognitive load of understanding each component we have, what it does and what versions work well together or have conflicts, what dependencies are required, and where do I get those? 

* The bits inside the agent should be able to be more of a single user experience.  A key message for this point was when the head of technical docs said "Nobody wakes up in the morning and says 'I want to hiera today'".  We want the user experience to be easy to manage and work with. That doesn't mean that if somebody wants to get into the bits/packages/projects/source they shouldn't be able to -- we just don't want that to be a requirement. 

* Version numbers - they are hard. We restarted at 1.0 because it was a new thing. Eric and I were talking yesterday and maybe it would have made more sense to put it at 4; however because of the argument that he made earlier in the thread, there are cases where that could create other types of confusion.

* Somewhere in the thread somebody asked about Puppet's new filesystem layout and if things would work in another location. I've seen Fedora already pick up the puppet 4 source and package it right up for their distro, so I think that's working. [1]

* The packaging toolchain will be opened up. We're not quite there yet because there are some hard-coded assumptions that need to be abstracted away. We also need some documentation and examples -- otherwise that's going to be a difficult climb. 

* I'm not sure if introducing Collections and the AIO and Puppet 4 all at the same time helped with cognitive load. I'm sorry if that hasn't worked out. Collections from a user's perspective are mostly a package repo. The AIO is the end point. Puppet 4 is a subcomponent of that, which can be a bit confusing. We'll be working on our messaging, tooling and way we talk about this to make the largest amount of people happy. 

I do welcome questions and criticism here. We do like to hear opinions and options. I certainly can't promise we'll act on every complaint or suggestion, but I'm sure the criticisms are valid. 

[1] http://koji.fedoraproject.org/koji/buildinfo?buildID=631538

[jcbollinger]

On Thursday, April 30, 2015 at 11:32:43 AM UTC-5, Michael Stanhke wrote:

Rather than reply and quote lots of points, I'll just bring up a few things which could help clarify our goals. I'm sure not everybody will agree or love all of them, we know there are compromises here but attempted to make choices with the user's benefit and ease of use in mind. 

* The package is called puppet-agent because Puppet is more than the tool puppet at this point. It's an ecosystem. It's a suite of tools working together to manage infrastructure and applications. The agent is the word that means "all the stuff you need to run an endpoint in this ecosystem."

This conversation serves as evidence that that's not a universally accepted meaning for the word "agent", and I'd be surprised to find that more people interpreted "puppet-agent" that way than interpreted plain "puppet" that way.  Surely it's natural to interpret "puppet-agent" as something more specific than "puppet", and decidedly unnatural to interpret "puppet-agent" as more general.

 

* We wanted to get away from our users having to carry the cognitive load of understanding each component we have, what it does and what versions work well together or have conflicts, what dependencies are required, and where do I get those? 

Those are worthy goals.  Unfortunately, the present scheme isn't there yet.  Witness, for example, the misunderstanding about whether the "puppetserver" package is needed or wanted for running a master.

Moreover, the effort to lighten the cognitive load has produced an undesirable opacity to what is installed and why, and ultimately a cognitive dissonance for people who probe that, or who have already shouldered the cognitive load (i.e. many existing users).  That is essentially where this thread started, even if we've only now begun throwing around the word "cognitive".

There is a vast gulf between "it's unnecessary to know" and "it's difficult to determine or understand".

 

* The bits inside the agent should be able to be more of a single user experience.  A key message for this point was when the head of technical docs said "Nobody wakes up in the morning and says 'I want to hiera today'".  We want the user experience to be easy to manage and work with. That doesn't mean that if somebody wants to get into the bits/packages/projects/source they shouldn't be able to -- we just don't want that to be a requirement. 

The head of technical docs was wrong, at least in the sense of the implied comparison to "I want to puppet today".  People can and do use Hiera as a standalone tool, and some do care about which version is in use at their site.  Likewise for some of the other components.  It's true, however, that the "puppet" tool is the headliner in this story.  That's why it was a poor idea to put anything else's version number on the AIO package. 

 

* Version numbers - they are hard. We restarted at 1.0 because it was a new thing. Eric and I were talking yesterday and maybe it would have made more sense to put it at 4; however because of the argument that he made earlier in the thread, there are cases where that could create other types of confusion.

I don't think that reflects a firm grasp of the nature of the problem.  The issue is that the "new thing" here is not the thing that the package's version number should be describing in the first place.  I don't care about the newness of AIO layout and packaging, and I don't expect many others will either.  People don't install Puppet for its packaging.  I do care about the versions of various components of the system, but not everyone will, and anyway, we have already established that an AIO package's version number is not a good vehicle for communicating information about versions of auxilliary components.  Focus on what's important.  To your audience.

It would be ideal if every component that is developed on its own cycle and therefore has its own version number were delivered in its own package.  Package management systems and installation frameworks have long had mechanisms for dealing with the relationships among these.  Indeed, Fedora is following exactly this approach for their own Puppet packaging.  We covered it already on the dev list, but packaging the various components separately does not need to present a complicated installation experience.  Furthermore, it solves some of the other issues that have been raised, especially with respect to versioning the collection as a whole (which ceases to be relevant).
 

* Somewhere in the thread somebody asked about Puppet's new filesystem layout and if things would work in another location. I've seen Fedora already pick up the puppet 4 source and package it right up for their distro, so I think that's working. [1]

Excellent, thank you.

 

* The packaging toolchain will be opened up. We're not quite there yet because there are some hard-coded assumptions that need to be abstracted away. We also need some documentation and examples -- otherwise that's going to be a difficult climb. 

I'm pleased to hear that's on its way.

 

* I'm not sure if introducing Collections and the AIO and Puppet 4 all at the same time helped with cognitive load. I'm sorry if that hasn't worked out. Collections from a user's perspective are mostly a package repo. The AIO is the end point. Puppet 4 is a subcomponent of that, which can be a bit confusing. We'll be working on our messaging, tooling and way we talk about this to make the largest amount of people happy. 

I recognize that collections manifest to users primarily as a package repo.  I'm trying to point out that the distinction being drawn between that and AIO packaging is artificial.  Collections would easily serve as an excellent framework and vehicle for everything the AIO seems to be trying to accomplish, only better.

John

[jcf]

On May 1, 2015, at 08:14, jcbollinger <John.Bo...@stJude.org> wrote:

On Thursday, April 30, 2015 at 11:32:43 AM UTC-5, Michael Stanhke wrote:

Rather than reply and quote lots of points, I'll just bring up a few things which could help clarify our goals. I'm sure not everybody will agree or love all of them, we know there are compromises here but attempted to make choices with the user's benefit and ease of use in mind. 

* The package is called puppet-agent because Puppet is more than the tool puppet at this point. It's an ecosystem. It's a suite of tools working together to manage infrastructure and applications. The agent is the word that means "all the stuff you need to run an endpoint in this ecosystem."

This conversation serves as evidence that that's not a universally accepted meaning for the word "agent", and I'd be surprised to find that more people interpreted "puppet-agent" that way than interpreted plain "puppet" that way.  Surely it's natural to interpret "puppet-agent" as something more specific than "puppet", and decidedly unnatural to interpret "puppet-agent" as more general.

Correct. 

* We wanted to get away from our users having to carry the cognitive load of understanding each component we have, what it does and what versions work well together or have conflicts, what dependencies are required, and where do I get those? 

Those are worthy goals.  Unfortunately, the present scheme isn't there yet.  Witness, for example, the misunderstanding about whether the "puppetserver" package is needed or wanted for running a master.

The fact that component naming and versioning has suffered horribly is part of why we are here today, yes.

Moreover, the effort to lighten the cognitive load has produced an undesirable opacity to what is installed and why, and ultimately a cognitive dissonance for people who probe that, or who have already shouldered the cognitive load (i.e. many existing users).  That is essentially where this thread started, even if we've only now begun throwing around the word "cognitive”.

Isn’t Puppet a tool for carrying this cognitive load itself? Once I’ve defined an exact package version to be installed somewhere, based on research and testing, I generally “forget about” the meta details until something goes wrong that forces me to remember/re-learn these details because these sorts of Religious Wars about Versioning as well as cross-entity version tracking means that “nothing” has the same version on my system as it advertised on the tin. That the very toolset that helps me manage such things has suddenly succumbed to the very pox it helps me manage is as ironic as a Park Slope haircut.

There is a vast gulf between "it's unnecessary to know" and "it's difficult to determine or understand”.

^^ That right there.

* The bits inside the agent should be able to be more of a single user experience.  A key message for this point was when the head of technical docs said "Nobody wakes up in the morning and says 'I want to hiera today'".  We want the user experience to be easy to manage and work with. That doesn't mean that if somebody wants to get into the bits/packages/projects/source they shouldn't be able to -- we just don't want that to be a requirement. 

The head of technical docs was wrong, at least in the sense of the implied comparison to "I want to puppet today".  People can and do use Hiera as a standalone tool, and some do care about which version is in use at their site.  Likewise for some of the other components.  It's true, however, that the "puppet" tool is the headliner in this story.  That's why it was a poor idea to put anything else's version number on the AIO package. 

No one — and by this I actually mean “not a single living soul upon the Earth” — has ever woken up and said “I *want* to memorize the versioning scheme of the toolkit I use to do my job, across ${number} of different components today” unless they were studying for a certification, test, etc. People *have* woken up and said “I want to use ${toolkit_that_I_do_my_job_with} today”, even if that was not what they called the toolkit. With that said, the principle that the Hiera version number should be subordinate to the Puppet version number _on some level_ is not an unreasonable one, and happens to be the answer I advocate for personally. I do agree that the AIO/${installer} version should always correspond to the release of the nominal headlining product.

* Version numbers - they are hard. We restarted at 1.0 because it was a new thing. Eric and I were talking yesterday and maybe it would have made more sense to put it at 4; however because of the argument that he made earlier in the thread, there are cases where that could create other types of confusion.

I don't think that reflects a firm grasp of the nature of the problem.  The issue is that the "new thing" here is not the thing that the package's version number should be describing in the first place.  I don't care about the newness of AIO layout and packaging, and I don't expect many others will either.  People don't install Puppet for its packaging.  I do care about the versions of various components of the system, but not everyone will, and anyway, we have already established that an AIO package's version number is not a good vehicle for communicating information about versions of auxilliary components.  Focus on what's important.  To your audience.

I am also pretty baffled that this is considered hard, or even a matter for debate. Principle of Least Surprise, or just have the contents match the tin.

It would be ideal if every component that is developed on its own cycle and therefore has its own version number were delivered in its own package.  Package management systems and installation frameworks have long had mechanisms for dealing with the relationships among these.  Indeed, Fedora is following exactly this approach for their own Puppet packaging.  We covered it already on the dev list, but packaging the various components separately does not need to present a complicated installation experience.  Furthermore, it solves some of the other issues that have been raised, especially with respect to versioning the collection as a whole (which ceases to be relevant).

This is the ideal, IMHO — “pkg-add-command -install puppet-4-server.pkg” installs everything  one needs to run a fully-functional environment. “pkg-add-command -install puppet-4-client.pkg” successfully hides from the end-user what’s being installed, _recognizes what’s being installed and acts accordingly_ (see other thread about file permissions and puppet user active today), and is no more complicated than that if there is no need. If one cares to make sure that items that CAN be upgraded out-of-bundle “match”, the —framework-version command previously suggested (or similar) can be leveraged to make sure everything assertively and denotatively matches. At a certain level of support/upgrade planning/etc, then individual component names and numbers become interesting again, but KBs or the like should exist to assist in those circumstances.

* I'm not sure if introducing Collections and the AIO and Puppet 4 all at the same time helped with cognitive load. I'm sorry if that hasn't worked out. Collections from a user's perspective are mostly a package repo. The AIO is the end point. Puppet 4 is a subcomponent of that, which can be a bit confusing. We'll be working on our messaging, tooling and way we talk about this to make the largest amount of people happy. 

I recognize that collections manifest to users primarily as a package repo.  I'm trying to point out that the distinction being drawn between that and AIO packaging is artificial.  Collections would easily serve as an excellent framework and vehicle for everything the AIO seems to be trying to accomplish, only better.

Drawing a distinction for the user between either of these mechanisms and “Puppet”, on any level, is artificial and confusing to the end user. If you come over to my house for dinner and we’re talking about the car you came over in, it either is new, interesting, or I’m being polite — I don’t actually care if you came in a Honda or a Toyota, on most levels.

Cheers,

jcf

[jcbollinger]

On Thursday, May 7, 2015 at 2:22:24 PM UTC-5, jcf wrote:

On May 1, 2015, at 08:14, jcbollinger <John.Bo...@stJude.org> wrote:

It would be ideal if every component that is developed on its own cycle and therefore has its own version number were delivered in its own package.  Package management systems and installation frameworks have long had mechanisms for dealing with the relationships among these.  Indeed, Fedora is following exactly this approach for their own Puppet packaging.  We covered it already on the dev list, but packaging the various components separately does not need to present a complicated installation experience.  Furthermore, it solves some of the other issues that have been raised, especially with respect to versioning the collection as a whole (which ceases to be relevant).

This is the ideal, IMHO — “pkg-add-command -install puppet-4-server.pkg” installs everything  one needs to run a fully-functional environment. “pkg-add-command -install puppet-4-client.pkg” successfully hides from the end-user what’s being installed, _recognizes what’s being installed and acts accordingly_ (see other thread about file permissions and puppet user active today), and is no more complicated than that if there is no need.

This is a different facet of the problem than I was talking about, but yes, agreed.

Those two facets of ideal packaging are compatible in all the package management systems I care about, though I acknowledge that PL cares about more package management systems than I do.

John

[Eric Sorenson]

On Thu, 7 May 2015, jcf wrote:

>>
>> I don't think that reflects a firm grasp of the nature of the problem. The issue is that the "new thing" here is not the thing that the package's version number should be describing in the first place. I don't care about the newness of AIO layout and packaging, and I don't expect many others will either. People don't install Puppet for its packaging. I do care about the versions of various components of the system, but not everyone will, and anyway, we have already established that an AIO package's version number is not a good vehicle for communicating information about versions of auxilliary components. Focus on what's important. To your audience.
>
> I am also pretty baffled that this is considered hard, or even a matter for debate. Principle of Least Surprise, or just have the contents match the tin.

FWIW I find this argument pretty compelling and would like to advance the
version number of the next release of puppet-agent to '4.something'.

Our current thinking is that this will be a matched to the puppet version,
with an extra digit on the end of the version number that indicates component
revisions other than Puppet itself.

So specifically, the next release will be puppet-agent-4.2.0.0; a hypothetical
rev to include a not-very-hypothetical openssl update would be included in a
puppet-agent-4.2.0.1 package.

(We can't use the release field as suggested up-thread, because some packaging
systems don't view numbers not part of the 'version' field to be an upgrade.)

Does that align more closely with the least-surprising thing, to you?

Eric Sorenson - eric.s...@puppetlabs.com - freenode #puppet: eric0
puppet platform // coffee // techno // bicycles

[Joshua Fielden]

I'm jcf,  and I endorse this plan.

Cheers,

jcf

[jcbollinger]

I approve.  I don't consider it a complete or ideal solution, but it's much better than what is offered now.


John

[Thomas Müller]

FWIW I find this argument pretty compelling and would like to advance the
version number of the next release of puppet-agent to '4.something'.

Our current thinking is that this will be a matched to the puppet version,
with an extra digit on the end of the version number that indicates component
revisions other than Puppet itself.

So specifically, the next release will be puppet-agent-4.2.0.0; a hypothetical
rev to include a not-very-hypothetical openssl update would be included in a
puppet-agent-4.2.0.1 package.

sounds good to me. 

- Thomas 

[Ken Bowley]

This is better than what is currently being used, but I'm strongly in the AIO idea to be stupid.  Split it into multiple packages and use proper dependencies like every other sane packaging system has done for a long, long time.

If all you do is bump the version of facter, then only have me download and install the meta package that depends on the new facter, and the new facter package, not everything.

<rant>

I'm only planning on using AIO packages on the server side and stick with the gem packages on the client side.  With multiple client types (multiple versions of OS X, multiple versions and distributions of Linux, various architectures), I know the gem packages can be handled the same way on all of them.    When I'm adding a node to the network, I like being able to do type "gem install puppet", set up the key, then let puppet take care of the rest.  With the AIO packages, I have to start out doing puppets job and add a repo and refresh my package manager before I can get to the step of installing puppet.  Not to mention praying that the platform is even supported by the AIO packages.
</rant>

[priyajit nayak]

how to run puppet agent using non sudo command

[Vince Skahan]

On Thursday, June 18, 2015 at 4:18:37 PM UTC-7, Ken Bowley wrote:

This is better than what is currently being used, but I'm strongly in the AIO idea to be stupid.  Split it into multiple packages and use proper dependencies like every other sane packaging system has done for a long, long time.

If all you do is bump the version of facter, then only have me download and install the meta package that depends on the new facter, and the new facter package, not everything.

Agree.   Thought I'd chime in (late) as the original poster.

Versioning starting with 4.x is a good start, but I still think your AIO approach is wrong.

Have collector rpms that 'require' the pieces of the puzzle and package hiera/etc. in individually bundled standalone packages.  If you do that:

• you can keep versioning facter to 2.x.y if you want
• you can keep versioning puppetserver any way you want
• and just version the collection (bundle, pick a term) with the 4.x.x identifier you want to publicize as release-4.x.x

To update the client, 'yum update puppet' and have it update the sub-pieces it needs (hiera/mco/etc.)

To update the server, 'yum update puppetserver' and have it do the server piece.

Lastly, if it's me, I would not bundle the agent/client stuff 'in' the puppetserver package.  I would 'require' the client-stuff to be co-installed with the server stuff using the packaging mechanisms the os providers already give you.

(in other words, release 'empty' rpms that require x and y and z - works great if you don't cause dependency hell by getting too fancy)

[Jason Slagle]

FWIW, +1 from me too.  It seems like a lot of places that do packaging like this end up doing it this way.

If I’m only doing a security update to facter, I shouldn’t have to replace a gigantic bundle with whatever else it pulls In.  I can see you release management people hating this later, as well as security teams.

I suspect this confusion will hinder deployment – the AIO packaging is certainly in the cons category for us.

Jason

[Eric Sorenson]

On Mon, 22 Jun 2015, Jason Slagle wrote:
> On 6/22/15, 3:08 PM, "Vince Skahan" <vince...@gmail.com> wrote:
>
>> On Thursday, June 18, 2015 at 4:18:37 PM UTC-7, Ken Bowley wrote:
>>> This is better than what is currently being used, but I'm strongly in the AIO
>>> idea to be stupid. Split it into multiple packages and use proper
>>> dependencies like every other sane packaging system has done for a long, long
>>> time.
>>>
>>> If all you do is bump the version of facter, then only have me download and
>>> install the meta package that depends on the new facter, and the new facter
>>> package, not everything.
>>
>> Agree. Thought I'd chime in (late) as the original poster.
>>
>> Versioning starting with 4.x is a good start, but I still think your AIO
>> approach is wrong.
>>
>> Have collector rpms that 'require' the pieces of the puzzle and package
>> hiera/etc. in individually bundled standalone packages. If you do that:

>> * you can keep versioning facter to 2.x.y if you want
>> * you can keep versioning puppetserver any way you want
>> * and just version the collection (bundle, pick a term) with the 4.x.x

>> identifier you want to publicize as release-4.x.x

If all anybody had to deal with were $osfamily==redhat systems, I feel pretty
certain this is exactly what we'd do. But it's just not. Just to start from
first principles, the primary goals of the packaging project were:

- unify the agent across open-source and PE so testing, delivery, and upgrades
are as smooth as possible
- provide a consistently great out-of-the-box experience so you can get fresh
Puppet versions with "batteries included" on any supported OS

I love metapackages too, but short of porting yum to Windows, Mac OS X, and
Solaris I don't see how they meet those requirements.

>> To update the client, 'yum update puppet' and have it update the sub-pieces it
>> needs (hiera/mco/etc.)

So this happens today, it's just in all in one package :)

>> To update the server, 'yum update puppetserver' and have it do the server
>> piece.

And this is actually what happens today.

>> Lastly, if it's me, I would not bundle the agent/client stuff 'in' the
>> puppetserver package. I would 'require' the client-stuff to be co-installed
>> with the server stuff using the packaging mechanisms the os providers already
>> give you.

This is also what happens today; there is no agent stuff in the puppetserver
package.

>> (in other words, release 'empty' rpms that require x and y and z - works
>> great if you don't cause dependency hell by getting too fancy)

> FWIW, +1 from me too. It seems like a lot of places that do packaging like
> this end up doing it this way.

Fair enough.

> If I¹m only doing a security update to facter, I shouldn¹t have to replace a
> gigantic bundle with whatever else it pulls In. I can see you release
> management people hating this later, as well as security teams.

So the puppet-agent package is 17 megabytes on EL7, so "gigantic" is a bit of
an overstatement here. Agreed that the release pipeline is more complicated,
and I can definitely understand the desire to just update the one thing that
needs a bugfix.

> I suspect this confusion will hinder deployment ­ the AIO packaging is
> certainly in the cons category for us.

I really want to understand this, because it's a big deal. (My life goal at
this point is to get as many people as possible upgraded to Puppet 4, so
anything that gets in the way of that is a problem!) There's been a bunch of
different points in the thread, some of them about the numbering and some
about the packaging itself; what would reduce the confusion for you?

[Jason Slagle]

On 6/23/15, 9:03 PM, "Eric Sorenson" <eric.s...@puppetlabs.com> wrote:

>
>> I suspect this confusion will hinder deployment ­ the AIO packaging is
>> certainly in the cons category for us.
>
>I really want to understand this, because it's a big deal. (My life goal
>at
>this point is to get as many people as possible upgraded to Puppet 4, so
>anything that gets in the way of that is a problem!) There's been a bunch
>of
>different points in the thread, some of them about the numbering and some
>about the packaging itself; what would reduce the confusion for you?

It’s actually interesting, because it came up at a PUG meeting here
locally, and I definitely got a more negative than positive vibe from the
AIO packaging, as well as my own feelings.

In the end, it comes down to the potential security implications for some
of my clients. On the enterprise front, you provide an installer, which
you have a contractual obligation to support. When security issues arise
with say the bundled ruby, you are going to quickly act on them.

On the open source side, I’m less sure about that obligation. You guys
have been spectacular at keeping up with security patches, but when you
decide to deprecate 4.1, you’ll have people with it installed 2 years from
now. You now have a much larger software ecosystem to worry about
vulnerabilities in. Basically, it puts the open source users in a
position where they have to rely on puppetlabs for patches to upstream
projects such as the bundled ruby or openssl on the agent side.

A related concern comes with companies with infosec departments that have
to bless things. I get Ruby 2.1.0 blessed, but then the bundled ruby gets
updated to 2.1.1. Now there are a lot more compliance hoops to jump
through.

In the end, a lot of it comes down to it “not being the unix way”. I have
many of the same arguments and dislikes against systemd. I have no issue
with the AIO installer, and in fact might use it on some older
centos/rhel5 hosts where getting modern ruby is hard. My heartburn comes
from it being the only REAL way to install these packages starting with
version 4. I’d much prefer you also support a more traditional
metapackage approach for the operating systems that support it.

Finally, the AIO package creation is a lot less repeatable. If I need to
modify 3.7 locally, I modify it, change the spec to add a local component
and build a new RPM. With this AIO, I need to grab the packaging repo and
spend some amount of time trying to figure out how to navigate it.

Hope that helps.

Jason

[Miguel Di Ciurcio Filho]

On Wed, Jun 24, 2015 at 12:26 AM, Jason Slagle <rais...@tacorp.net> wrote:
>
> It’s actually interesting, because it came up at a PUG meeting here
> locally, and I definitely got a more negative than positive vibe from the
> AIO packaging, as well as my own feelings.
>

I just want to step in on this thread to highlight that there are a
lot of people, including me, that think the AIO approach is good and
offers more advantages than disadvantages.

I do too have clients and the ones that I've had the opportunity to
talk about Puppet 4, the AIO packaging is overall welcomed.

> On the open source side, I’m less sure about that obligation. You guys
> have been spectacular at keeping up with security patches, but when you
> decide to deprecate 4.1, you’ll have people with it installed 2 years from
> now. You now have a much larger software ecosystem to worry about
> vulnerabilities in. Basically, it puts the open source users in a
> position where they have to rely on puppetlabs for patches to upstream
> projects such as the bundled ruby or openssl on the agent side.
>

Open source users will always rely on a third party for security
patches anyway. Every single vendor, named CentOS, Debian and Ubuntu
drops the ball from time to time. Go and look around about the OpenSSL
problems Debian had, and the long period CentOS users were completely
in the dark for security patches.

You have to choose: best effort ou guarantee. I think this trade-off
is well understood, isn't it?

> A related concern comes with companies with infosec departments that have
> to bless things. I get Ruby 2.1.0 blessed, but then the bundled ruby gets
> updated to 2.1.1. Now there are a lot more compliance hoops to jump
> through.
>

One of the tradeoffs of using the AIO is to be able to move faster, to
get more done, instead of having to be limited by the least common
denominator. Companies with infosec departments that have to bless
things, IMHO, that is their problem. Don't want to fight the
bureaucracy? Buy the support and outsource the responsibility.

> In the end, a lot of it comes down to it “not being the unix way”. I have
> many of the same arguments and dislikes against systemd. I have no issue
> with the AIO installer, and in fact might use it on some older
> centos/rhel5 hosts where getting modern ruby is hard. My heartburn comes
> from it being the only REAL way to install these packages starting with
> version 4. I’d much prefer you also support a more traditional
> metapackage approach for the operating systems that support it.
>

The Unix way? Come on. Lets get rid of packaging then. I'm 34 and the
more senior people I've spoken told me that back in the old days they
downloaded the source code and compile it, on every machine, with that
nostalgia look in the eyes.

I think it is counter productive to stick with the metapackage
approach when there is going be a burden to maintain some operating
systems in one way, and some in another way.

[Ken Bowley]

I've already stated my opinions on the AIO packages, and as long as the gems are still available, I can maintain the systems that I currently maintain.  I don't like that the puppetserver will be special and need to use the puppetlabs packages, but I can live with it.

When you have Linux systems with CentOS 5, Debian 6 thru sid, Ubuntu 10.4-14.04, running on i386, AMD64, armv6 (RaspberryPi), and then a bunch of Mac OS workstations that range from 10.6 thru 10.11, the gem packages are the only consistent way of controlling which version is installed.  I'm a little concerned with the new C++ based facter and what headaches that's going to cause.

I do look forward to testing out puppet 4.2!  4.0 had no easy upgrade path.  The newer puppetserver helped by translating the old API calls to the new calls, but 4.1 was still not production ready with it's major issues with handling facts and puppetdb.  With all the issues that puppet 4 has had, along with the introduction of the AIO packages, upgrading from puppet 3 has been a non-starter.

[jcbollinger]

On Wednesday, June 24, 2015 at 9:03:15 AM UTC-5, miguel wrote:

On Wed, Jun 24, 2015 at 12:26 AM, Jason Slagle <rais...@tacorp.net> wrote:
>
> It’s actually interesting, because it came up at a PUG meeting here
> locally, and I definitely got a more negative than positive vibe from the
> AIO packaging, as well as my own feelings.
>

I just want to step in on this thread to highlight that there are a
lot of people, including me, that think the AIO approach is good and
offers more advantages than disadvantages.

I do too have clients and the ones that I've had the opportunity to
talk about Puppet 4, the AIO packaging is overall welcomed.

I'm curious what, specifically, you and your clients like about AIO packaging that distinguishes it from a metapackage approach.  I am looking for perceived advantages specific to the packaging, as opposed to the on-disk layout, which has an altogether different set of advantages and disadvantages.


John

[jcbollinger]

On Wednesday, June 24, 2015 at 9:03:15 AM UTC-5, miguel wrote:

On Wed, Jun 24, 2015 at 12:26 AM, Jason Slagle <rais...@tacorp.net> wrote:

 

Open source users will always rely on a third party for security
patches anyway. Every single vendor, named CentOS, Debian and Ubuntu
drops the ball from time to time. Go and look around about the OpenSSL
problems Debian had, and the long period CentOS users were completely
in the dark for security patches.

"Always" is a bit strong, but yes, it is typical these days to rely on third-party packagers for updates of all kinds, including security patches.  No one is ever locked in to that, however.  An AIO package does make it harder to apply upstream patches, though, and as the number of upstreams contributing to a single package increases, either the frequency of package updates must soar or the (average) lag between patch release and package update must rise.

Moreover, the concern about PL's commitment over time should not be discounted.  PL implements a pretty aggressive development and release cycle, and having many simultaneously supported versions will have an additional combinatorial effect on them.  Historically, they have supported their software for relatively short periods as many enterprises account things, and once a product goes out of support, you can no longer expect updates from PL for any of the pieces.
 

> A related concern comes with companies with infosec departments that have
> to bless things.  I get Ruby 2.1.0 blessed, but then the bundled ruby gets
> updated to 2.1.1.  Now there are a lot more compliance hoops to jump
> through.
>

One of the tradeoffs of using the AIO is to be able to move faster, to
get more done, instead of having to be limited by the least common
denominator.

I'm sorry, I don't follow that logic at all.  I can see the vague outlines of an argument along those lines about AIO on-disk layout and vendored components, but that's not what we're talking about.

 

Companies with infosec departments that have to bless
things, IMHO, that is their problem. Don't want to fight the
bureaucracy? Buy the support and outsource the responsibility.

That's a rather cavalier attitude, not to mention missing the point.  Even companies that pay for support may still have a requirement for the software to be blessed.  That you personally have low regard for such requirements does not make them irrelevant to the discussion.

 

> In the end, a lot of it comes down to it “not being the unix way”.  I have
> many of the same arguments and dislikes against systemd.  I have no issue
> with the AIO installer, and in fact might use it on some older
> centos/rhel5 hosts where getting modern ruby is hard.  My heartburn comes
> from it being the only REAL way to install these packages starting with
> version 4.  I’d much prefer you also support a more traditional
> metapackage approach for the operating systems that support it.
>

The Unix way? Come on. Lets get rid of packaging then. I'm 34 and the
more senior people I've spoken told me that back in the old days they
downloaded the source code and compile it, on every machine, with that
nostalgia look in the eyes.

Yes, I have no intention of going there.  Packaging is one of the significant advances of the last 30 years or so.  That's why there are so many packaging systems.  As for "the UNIX way", UNIX architecture and tradition revolves around providing many small(ish), independent components that can be combined in useful -- and often unforeseen -- ways.  To some extent, AIO packaging indeed does run against that, and to that extent I agree with Jason.

 

I think it is counter productive to stick with the metapackage
approach when there is going be a burden to maintain some operating
systems in one way, and some in another way.

That is certainly a consideration for PuppetLabs, and it seems clear that they have taken it to heart.  I am by no means convinced, however, that it would be harder to following the metapackage approach for those packaging systems that support it than not to do so.  Any way around you need different packages for different systems.  No matter what, it is not a consideration that matters to me, nor, I suspect, to most other users.


John

[Vince Skahan]

On Tuesday, June 23, 2015 at 6:03:50 PM UTC-7, Eric Sorenson wrote:

I love metapackages too, but short of porting yum to Windows, Mac OS X, and
Solaris I don't see how they meet those requirements.

That's certainly reasonable. Cross-platform is hard to do.

> If I¹m only doing a security update to facter, I shouldn¹t have to replace a
> gigantic bundle with whatever else it pulls In.  I can see you release
> management people hating this later, as well as security teams.

So the puppet-agent package is 17 megabytes on EL7, so "gigantic" is a bit of
an overstatement here. Agreed that the release pipeline is more complicated,
and I can definitely understand the desire to just update the one thing that
needs a bugfix.

 
It doesn't change too often.  That's not massive by today's measures.
Smaller than the far more frequent big firefox updates (ugh).

 

> I suspect this confusion will hinder deployment ­ the AIO packaging is
> certainly in the cons category for us.

I really want to understand this, because it's a big deal. (My life goal at
this point is to get as many people as possible upgraded to Puppet 4, so
anything that gets in the way of that is a problem!) There's been a bunch of
different points in the thread, some of them about the numbering and some
about the packaging itself; what would reduce the confusion for you?

 

What's hard is that you're trying to name a collection of differently numbered pieces as an aggregate version for that 'collection', which is why unifying your versioning will help a lot.

Release everything puppet-4.x with a 4.x.something version identifier for your AIO bundles.
That will help a lot.   And don't forget to catch up all your docs to only use the 4.x terminology :-)

 

[Miguel Di Ciurcio Filho]

>
> I'm curious what, specifically, you and your clients like about AIO
> packaging that distinguishes it from a metapackage approach. I am looking
> for perceived advantages specific to the packaging, as opposed to the
> on-disk layout, which has an altogether different set of advantages and
> disadvantages.
>

Honestly I don't care, it is not a big deal for me. I don't have any
arguments nor will try to convince you of anything.

I just configure the repository, install the package, edit some files,
get the thing working and move on. To me, in the end of the day, the
AIO doesn't change anything. If there are updates, I check the change
log, look for anything that might concern me, analise it, and do it.

Downloading 20 MB or 1 MB for the update, don't care either.

To me this metapackage VS AIO is pretty much a vim VS emacs discussion.

And my clients rely on me for the work, so they don't care either.