Puppet 4 delivery and upgrades

[Eric Sorenson]

Hi all, as some of you have noticed, our target date in JIRA for Puppet 4 is not too far off now. (We just moved it out from early December to early January based on the remaining must-have workflow). While you can poke through tickets to see exactly what's going in, it might be helpful to have a little higher level overview of what is happening. Let me try to provide that as well as ask some questions of things we're not sure about. 

1. Ruby 1.8.7 support is going away. 

2. To enable #1 but still support OSes that ship with 1.8.7, we're going to be packaging and delivering Puppet 4 an an 'all-in-one' (AIO) package bundled together with 

 - openssl

 - ruby

 - augeas

 - ruby-augeas

 - ruby-stomp

 - ruby-shadow

 - puppet

 - mcollective

 - facter

 - hiera

- + misc supporting gems/libs (deep merge, yaml, etc)

(Question: are there other *agent side* components you feel are essential to the functioning of the puppet stack?)

3. The AIO packages will have a filesystem layout that installs the programs into /opt/puppetlabs/agent and the configs into /etc/puppetlabs/agent ; the packages will be a different basename than puppet ('puppet-agent') so won't install automatically on an upgrade, but *will* obsolete the puppet packages if you decide to install them. AIO packages will be available from the Nightly repos Real Soon Now[tm] and I expect Melissa or Haus will update when they're up so everyone can poke at 'em.

4. We are planning to break cross-major-version compatibility over the network. The amount of change we need to support in order to keep moving components to the Puppet Server and out of Ruby is the main driver for this, but generally, if you can't break compatibility on a major version boundary... when CAN you? See PUP-3641 for the overview of that work.

5. The upgrade process would be as Chris described above, where you'd set up a new server running puppet 4 (hey it's an opportunity to move your puppetmasters to EL 7!) and point the new agents at it, leave the old ones running until their agents are drained.

Question: What can/should we do to make that kind of transition go more smoothly? (One idea that just struck me is to have the puppet4 agent have a different default value for $server than 'puppet', so it wouldn't need post-install configuration to point at the new server)

6. We're going to stop providing the puppetmaster/puppet-server (passenger) packaging in favour of the puppetserver packages. There will still be rack/passenger support in Puppet 4, but not in Puppet 5.

7. Umm.. I think that's all.

Eric Sorenson - eric.s...@puppetlabs.com - freenode #puppet: eric0
puppet platform // coffee // techno // bicycles

[Joshua Hoblitt]

On 11/18/2014 12:04 PM, Eric Sorenson wrote:
> 2. To enable #1 but still support OSes that ship with 1.8.7, we're
> going to be packaging and delivering Puppet 4 an an 'all-in-one' (AIO)
> package bundled together with
> - openssl
> - ruby
> - augeas
> - ruby-augeas
> - ruby-stomp
> - ruby-shadow
> - puppet
> - mcollective
> - facter
> - hiera
> - + misc supporting gems/libs (deep merge, yaml, etc)
> (Question: are there other *agent side* components you feel are
> essential to the functioning of the puppet stack?)

On EL6.x, why not use a new SCL? Or even install into the existing
ruby193 SCL?

-Josh

--

[Michael Stahnke]

Part of the goal here is to have control of the ruby version. The SCL offers one version of Ruby 1.9.3, Ubuntu offers another in their distribution, as does Mac OS X, and the next OS. Compound that with libssl fun and End of Life constraints on Ruby 1.9.3, and it's just not that fun. (SCLs also have their own lifecycle that is different than RHEL). 

We'll be picking one Ruby (2.1.z), and using that everywhere for a consistent stack.  

-Josh

--

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/546B9CA0.3060504%40cpan.org.

For more options, visit https://groups.google.com/d/optout.

[Byron Miller]

Yay!

[Erik Dalén]

On Tue Nov 18 2014 at 8:04:13 PM Eric Sorenson <eric.s...@puppetlabs.com> wrote:

Hi all, as some of you have noticed, our target date in JIRA for Puppet 4 is not too far off now. (We just moved it out from early December to early January based on the remaining must-have workflow). While you can poke through tickets to see exactly what's going in, it might be helpful to have a little higher level overview of what is happening. Let me try to provide that as well as ask some questions of things we're not sure about. 

1. Ruby 1.8.7 support is going away. 

2. To enable #1 but still support OSes that ship with 1.8.7, we're going to be packaging and delivering Puppet 4 an an 'all-in-one' (AIO) package bundled together with 

 - openssl

 - ruby

 - augeas

 - ruby-augeas

 - ruby-stomp

 - ruby-shadow

 - puppet

 - mcollective

 - facter

 - hiera

- + misc supporting gems/libs (deep merge, yaml, etc)

(Question: are there other *agent side* components you feel are essential to the functioning of the puppet stack?)

puppetdb-terminus - required for masterless setups.

Btw, is there anything stopping you from delivering this AIO package to newer systems as well? For example Ubuntu Trusty has ruby 1.9.3 by default but also has ruby 2.0.0. But an AIO package with ruby 2.1.x would be a big speed increase compared to both those other ruby versions.

 

3. The AIO packages will have a filesystem layout that installs the programs into /opt/puppetlabs/agent and the configs into /etc/puppetlabs/agent ; the packages will be a different basename than puppet ('puppet-agent') so won't install automatically on an upgrade, but *will* obsolete the puppet packages if you decide to install them. AIO packages will be available from the Nightly repos Real Soon Now[tm] and I expect Melissa or Haus will update when they're up so everyone can poke at 'em.

4. We are planning to break cross-major-version compatibility over the network. The amount of change we need to support in order to keep moving components to the Puppet Server and out of Ruby is the main driver for this, but generally, if you can't break compatibility on a major version boundary... when CAN you? See PUP-3641 for the overview of that work.

5. The upgrade process would be as Chris described above, where you'd set up a new server running puppet 4 (hey it's an opportunity to move your puppetmasters to EL 7!) and point the new agents at it, leave the old ones running until their agents are drained.

Question: What can/should we do to make that kind of transition go more smoothly? (One idea that just struck me is to have the puppet4 agent have a different default value for $server than 'puppet', so it wouldn't need post-install configuration to point at the new server)

6. We're going to stop providing the puppetmaster/puppet-server (passenger) packaging in favour of the puppetserver packages. There will still be rack/passenger support in Puppet 4, but not in Puppet 5.

7. Umm.. I think that's all.

Eric Sorenson - eric.s...@puppetlabs.com - freenode #puppet: eric0
puppet platform // coffee // techno // bicycles

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/FF17DBFD-7DB1-4241-AC3C-EE9520A2E1C9%40puppetlabs.com.

[Matthaus Owens]

Erik,
The plan is to deliver the AIO package to not just platforms with
older rubies, but to all of the deb/rpm platforms we package for
currently.

> https://groups.google.com/d/msgid/puppet-dev/CAAAzDLcqbDwMZEWZrjyZRYkDLwk6cUn%3DYjFE1YyduQuHzLucRA%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.

--
Matthaus Owens
Puppet Labs

Join us at PuppetConf 2015, October 5-9 in Portland, OR -
http://2015.puppetconf.com.
Register early to save 40%!

[Joshua Hoblitt]

On 11/18/2014 06:11 PM, Michael Stahnke wrote:

On EL6.x, why not use a new SCL?  Or even install into the existing
ruby193 SCL?

Part of the goal here is to have control of the ruby version. The SCL offers one version of Ruby 1.9.3, Ubuntu offers another in their distribution, as does Mac OS X, and the next OS. Compound that with libssl fun and End of Life constraints on Ruby 1.9.3, and it's just not that fun. (SCLs also have their own lifecycle that is different than RHEL). 

We'll be picking one Ruby (2.1.z), and using that everywhere for a consistent stack.  

I'm not unsympathetic to the desire to vendor a version of ruby but that doesn't prevent the creation of a new SCL with all PL vendored (is that a word?) packages in it.  This is a primary use case for SCLs.

I'm a bit concerned about gems and other components accumulating in the path tree of an AIO that aren't tracked; I foresee upgrade 'fun'.  I think it would be preferable to be able to package gems, report processing scripts, etc. into RPMs that install into a puppet SCL.

Would facter be part of a puppet AIO?  If so, would that mean it's no longer independently upgradable?  How would you orchestrate agents installing gems that are needed by ruby facts?

How would a puppet AIO affect mcollective?  Would it be it's own AIO with independently vendored dependencies?  How would mco agents be installed?

-Josh

--

[Joshua Hoblitt]

On 11/18/2014 12:04 PM, Eric Sorenson wrote:

> 3. The AIO packages will have a filesystem layout that installs the
> programs into /opt/puppetlabs/agent and the configs into
> /etc/puppetlabs/agent ; the packages will be a different basename than
> puppet ('puppet-agent') so won't install automatically on an upgrade,
> but *will* obsolete the puppet packages if you decide to install them.
> AIO packages will be available from the Nightly repos Real Soon
> Now[tm] and I expect Melissa or Haus will update when they're up so
> everyone can poke at 'em.

/etc/puppet/* will be the master config and /etc/puppetlabs/agent/* will
be for the agents?

/etc/<vendor name>/ feels a bit awkward -- it would be like having
/etc/oracle/ instead of /etc/java/.

-Josh

--

[Matthaus Owens]

Josh,
Facter and mcollective will both be part of the AIO. Rubygems is part
of the ruby stack that will be included in the AIO, so installing gems
shouldn't change much, you would just need to ensure that the gems are
installed using the AIO gem binary. Facter will no longer be
independently upgradable. (For testing and development, we will be
including documentation on how to run the various projects alongside
the AIO from source). Mcollective plugins will still be packaged, and
will be updated to use the new layout so they will be compatible with
the AIO package.

The AIO model is the model that our windows packages have been using
for some time now with great success.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-dev+...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/546CD8D6.8030903%40cpan.org.

>
> For more options, visit https://groups.google.com/d/optout.

[Trevor Vaughan]

Hmm...Facter and MCollective not being independently upgradable isn't super exciting.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CACD%3DwAdWbfyibjox4%2BPmj9ZKFsHBF%2Bd%3DGYG4PTCb6Q3NtihtJA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

[Eric Sorenson]

On Nov 19, 2014, at 11:20 AM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

> Hmm...Facter and MCollective not being independently upgradable isn't super exciting.

If any one of the main constituent projects having a release would cause a new AIO to come out, is that the right cycle?

(Do you currently always track the latest package of the Puppet-related projects you use?)

--eric0

[Trevor Vaughan]

I always track the latest version of the Puppet-related projects that I use that don't have exciting bugs against my code base.

I've had to patch things at random in the past to get what I needed until PRs could be pushed through and I'd like to keep that ability.

The two that generally come to mind off hand are Hiera and Facter but being able to push forward with the various independent components would be best to prevent undue wailing and gnashing of teeth.

I'm find re-rolling your RPMs when I have to, I'm less fine finding out that my re-rolled RPMs have been arbitrarily clobbered and/or it's almost impossible to re-roll your RPMs.

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/472E836F-A255-4EA7-B98E-D71DFCC73E83%40puppetlabs.com.

For more options, visit https://groups.google.com/d/optout.

[Joshua Hoblitt]

Hi Matthaus,

The mcollective plugins installed by separate packages but going into
the same path as an "AIO" sounds a bit like reinventing SCLs.

If mcollective was dependent on a ruby run time from an "AIO" puppet
package, and one tried to upgrade puppet with mco, I'd expect there to
be interesting fireworks...

-Josh

--

[R.I.Pienaar]

No because support for new features like structured facts are not in all products
at the same time - that's a recent example but that kind of thing happens. So
I prefer the ability to pick and choose what versions of things goes into the system
as a whole.

[Leonardo Rodrigues de Mello]

Thats cool a controlled and independent stack with all the dependencies. Independent of the OS is great. Scl works only on rhel 6 and 7.

Please don't forget rhel 5 packages also. I know it is outdated but it is still widely used .

And bundled software will need to receive security updates, ex ruby openssl, etc .

Best
Lmello

Best

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CACD%3DwAeOeHxhtTUBY1FDeoiOQWRzGJuhF6xWYaJp_i5As0UvYA%40mail.gmail.com.

[Martin Alfke]

+1 for RHEL/CentOS 5
and please put some love in SLES too (even though that it is only used in Germany).

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAM%3D8oNKrTp7peG3iaQuEwQb014B4L0ASmmv4FGDzqi2ecLsfPQ%40mail.gmail.com.

[Felix Frank]

On 11/18/2014 08:04 PM, Eric Sorenson wrote:
> 4. We are planning to break cross-major-version compatibility over the
> network. The amount of change we need to support in order to keep moving
> components to the Puppet Server and out of Ruby is the main driver for
> this, but generally, if you can't break compatibility on a major version
> boundary... when CAN you? See PUP-3641 for the overview of that work.

Hi,

I'm not sure that this is the question to ask, at least not the first one.

First question is, can you gracefully deprecate a line of agents, i.e.
retain 3.x agent compatibility throughout 4.x masters, and drop it at 5.0.

I shall assume that the engineering effort to go such a route would be a
magnitude or two above just dropping cross-version support, so doing the
hard cut is likely the right thing to do. But I will go on record
stating that this is not the kind of decision that should be made lightly.

Cheers,
Felix

[Daniele Sluijters]

First question is, can you gracefully deprecate a line of agents, i.e. 
retain 3.x agent compatibility throughout 4.x masters, and drop it at 5.0. 
I shall assume that the engineering effort to go such a route would be a 
magnitude or two above just dropping cross-version support, so doing the 
hard cut is likely the right thing to do. But I will go on record 
stating that this is not the kind of decision that should be made lightly. 

But why not do it in Puppet 4 but do it in Puppet 5? This seems completely arbitrary like "I would prefer if you hold off to Puppet 5", but what's to stop it from going "I would prefer to hold off until Puppet 6" by the time 5 is ready to roll out?

Now, that was your first question, what's the second?

[Leonardo Rodrigues de Mello]

Would the gems still be released ? I am concerned because we use the mcollective client gem in a web application and I also use multiple puppet and facter gems in rbenv for testing.

Best

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/472E836F-A255-4EA7-B98E-D71DFCC73E83%40puppetlabs.com.

[Spencer Krum]

There is talk of splitting out some of the types (nagios and zfs most notably) into modules, then packaging those modules. Are we thinking those module packages would be in this AIO? Is that still on target for Puppet 4?

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAM%3D8oNJ0FvT03Y-ydYPPdzCagqWuQA3v10gj1%3DRcZhKA_1%2BdGQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Spencer Krum
(619)-980-7820

[Felix Frank]

On 11/20/2014 12:17 PM, Daniele Sluijters wrote:
> First question is, can you gracefully deprecate a line of agents, i.e.
> retain 3.x agent compatibility throughout 4.x masters, and drop it
> at 5.0.
> I shall assume that the engineering effort to go such a route would
> be a
> magnitude or two above just dropping cross-version support, so doing
> the
> hard cut is likely the right thing to do. But I will go on record
> stating that this is not the kind of decision that should be made
> lightly.
>
>
> But why not do it in Puppet 4 but do it in Puppet 5? This seems
> completely arbitrary like "I would prefer if you hold off to Puppet 5",
> but what's to stop it from going "I would prefer to hold off until
> Puppet 6" by the time 5 is ready to roll out?
>
> Now, that was your first question, what's the second?

Granted, 5.0 is somewhat arbitrary. Not breaking 3.x support before the
4.1 master would also go a long way, too, for example.

The point is, do I need to introduce a turning point at which each of my
agents will either
a) talk to the obosolete 3.x master only or
b) talk to the newly added 4.x master only

With a deprecation period, I can replace the 3.x master with 4.0 after
making sure that no major breakage follows suit, without the pressure of
updating each last agent (or live with two sets of potentially diverging
masters). It is only the 4.1 master that will not be viable until agents
are upgraded.

I regard this as an advantage because such minor updates are generally
less problematic (and fear inducing to the ops team).

The second question is the one Eric posed initially: Is a major version
change an opportune time to break the network in this fashion. The
answer to this is most definitely "yes" and - again - I'm not actively
contesting the choice to do this with 4.0.

Cheers,
Felix

[Leonardo Rodrigues de Mello]

Breaking this on 4.0 is OK with semver . breaking it on 4.1 would be a mess.

Best
Lmello

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/546DD9BB.5080901%40alumni.tu-berlin.de.

[Martin Alfke]

On 20 Nov 2014, at 13:12, Leonardo Rodrigues de Mello <l...@lmello.eu.org> wrote:

> Breaking this on 4.0 is OK with semver . breaking it on 4.1 would be a mess.

ACK.

Question regarding package resource types using gem provider:

1. are we still able to install into system ruby using gem provider?
2. will there be an add-on to install gem add ons into puppet installation, too?

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAM%3D8oN%2Bc%2BxFA%2B0W014Y7OUufEuiuxw7c2sPXQZ2aRadzf-%2BGdw%40mail.gmail.com.

[Leonardo Rodrigues de Mello]

I could be wrong , but I believe this change in the way puppet open source would be distributed makes it similar  on with the way puppet enterprise is distributed.

So we should expect something like pe_gem to install gems on the puppet bundle.
And provider gem to install on system ruby.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/E1661A4A-8685-412E-A45C-AA013FE72A85%40gmail.com.

[Justin Lambert]

This again may be more effort than it's worth, but would a 3.99 agent be possible that can talk to a 3.x and 4.x master? Maybe it only has the future parser to make compatibility with the 4.0 master easier. It would break semver a bit, hence the massive version jump, but would make upgrading easier.

[Erik Dalén]

generally we don't, but almost.

Seems a bit odd to me to not be able to upgrade puppetdb, facter or mcollective independently of puppet though.

And we would likely need to install things into that ruby env anyway. So why not have the ruby env as one package and the other packages depend on that and install things into it?

 

--eric0

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/472E836F-A255-4EA7-B98E-D71DFCC73E83%40puppetlabs.com.

[John Bollinger]

On Thursday, November 20, 2014 9:38:35 AM UTC-6, Erik Dalén wrote:

On Wed Nov 19 2014 at 8:36:46 PM Eric Sorenson <eric.s...@puppetlabs.com> wrote:

On Nov 19, 2014, at 11:20 AM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

> Hmm...Facter and MCollective not being independently upgradable isn't super exciting.


If any one of the main constituent projects having a release would cause a new AIO to come out, is that the right cycle?

 

(Do you currently always track the latest package of the Puppet-related projects you use?)

generally we don't, but almost.

Seems a bit odd to me to not be able to upgrade puppetdb, facter or mcollective independently of puppet though.

And we would likely need to install things into that ruby env anyway. So why not have the ruby env as one package and the other packages depend on that and install things into it?

And instead of an AIO packages, it would be possible in such a scheme to release dependency-only packages (aka metapackages, virtual packages) that describe complete collections of components without actually containing them.  That would facilitate whole-suite installation upgrade without unduly interfering with installing or upgrading individual components.  Done right, it might also provide a mechanism to determine whether a blessed combination of components was installed, vs. a possibly-mismatched hodge podge.


John

[Corey Osman]

I can only imagine how much time has been lost over the years trying to support ruby 1.8.7.  I think this is a great move as long as we can still install puppet using gems under a newer ruby.  Will the ffi gem be part of the AIO package?  

I won't even bother creating a ticket for a bug I just stumbled upon that is only found with using puppet under ruby 1.8.7.

Corey

On Tuesday, November 18, 2014 11:04:14 AM UTC-8, Eric Sorenson wrote:

Hi all, as some of you have noticed, our target date in JIRA for Puppet 4 is not too far off now. (We just moved it out from early December to early January based on the remaining must-have workflow). While you can poke through tickets to see exactly what's going in, it might be helpful to have a little higher level overview of what is happening. Let me try to provide that as well as ask some questions of things we're not sure about. 

1. Ruby 1.8.7 support is going away. 

2. To enable #1 but still support OSes that ship with 1.8.7, we're going to be packaging and delivering Puppet 4 an an 'all-in-one' (AIO) package bundled together with 

 - openssl

 - ruby

 - augeas

 - ruby-augeas

 - ruby-stomp

 - ruby-shadow

 - puppet

 - mcollective

 - facter

 - hiera

- + misc supporting gems/libs (deep merge, yaml, etc)

(Question: are there other *agent side* components you feel are essential to the functioning of the puppet stack?)

3. The AIO packages will have a filesystem layout that installs the programs into /opt/puppetlabs/agent and the configs into /etc/puppetlabs/agent ; the packages will be a different basename than puppet ('puppet-agent') so won't install automatically on an upgrade, but *will* obsolete the puppet packages if you decide to install them. AIO packages will be available from the Nightly repos Real Soon Now[tm] and I expect Melissa or Haus will update when they're up so everyone can poke at 'em.

4. We are planning to break cross-major-version compatibility over the network. The amount of change we need to support in order to keep moving components to the Puppet Server and out of Ruby is the main driver for this, but generally, if you can't break compatibility on a major version boundary... when CAN you? See PUP-3641 for the overview of that work.

[Joshua Hoblitt]

On 11/19/2014 12:36 PM, Eric Sorenson wrote:
> If any one of the main constituent projects having a release would cause a new AIO to come out, is that the right cycle?

Yes, no, maybe, it depends?

Is the proposed plan to roll out a new AIO if PL product versions are
unchanged but a vendored product needs to be updated? For example, a
Ruby GC fix or an openssl security issue. Things start to get tricky
under that scenario if you need to update the AIO to get the vendored
product fix but your production env has drifted several PL product
versions behind. Doing AIO "bug fix only releases" with the same
featured product versions for N-X (where X is at least 2) is a solution
but that incurs significant additional release engineering.

There is also some dependence on the rate at which hypothetical AIOs
would be released. If "things I need to update" was > 2 per month, I'd
likely need to get much more formalized about planning roll outs and
start skipping over releases. That in and of itself would increase the
required amount of pre-production testing because installing a new AIO
would mean multiple components are changing at the same time. Better
test automation on my part would help (ie, standing up every role type
at the same time and for multiple agent runs) but I'd still have
hesitation around disturbing the puppet package. My experience has been
that there's always glitches with a small % of nodes where agents hang
forever, leave uncleaned up lock files, etc. when rolling out new puppet
releases that often require mco/pssh intervention. We have a lot of
baremetal "special snowflakes" with lots of state so re-provisioning to
do orchestration roll outs is a no-go.

I've made a histogram of the release timestamps of the product &
dependencies RPMs in the EL6 yum repos. The goal was try to get a sense
of how many AIO releases there might be in the time frame of a month.

https://github.com/jhoblitt/plot-puppet-pkg-releases

>
> (Do you currently always track the latest package of the Puppet-related projects you use?)

Historically, I've been more or less able to keep up with the ~monthly
puppet releases but some other components, especially mcollective, tend
to be updated much less frequently. Others, such as theforeman, tend to
get "bursty" updates when tracking new/desired features.

-Josh

--

[Eric Sorenson]

On Nov 18, 2014, at 11:04 AM, Eric Sorenson <eric.s...@puppetlabs.com> wrote:

> 7. Umm.. I think that's all.

Wow thanks for all the responses, there are a ton of very good questions in here -- some of them got answered inline and I think we rolled up the rest into this little FAQ. which I''m sure will generate a new set of questions :)

0 - Which platforms will get FOSS AIO packages?

RHEL/Centos 5, 6, 7
Fedora 20, 21
Ubuntu 10.04, 12.04, 14.04
Debian Squeeze, Wheezy
Windows x86, x64

Commercial OS'es (Windows excepted) will get PE-only AIO
SLES 10, 11
Solaris 10,11
AIX
Mac OS X


1 - How do I manage gems with AIO ruby?

Use the AIO included gem binary. ('/opt/puppet/agent/bin/gem install somerandomgem') To make this easier, we can either fix PUP-3688 or add a module with a minimal 'puppet_gem' provider using the AIO path.

2 - Will there be security updates, etc?

Yes, absolutely.

3 - Will it include puppetdb-terminus?

Yes.

4 - What will the release cadence look like? What if I want to manage component upgrades on a different cadence than upstream, e.g. just facter, or just hiera?

We're planning to provide three trains of packaging:
• Nightlies, that get built whenever there's a change in one of the component repositories that passes through CI. These will essentially track the latest, like the nightly repos do now.
• Roll-up releases that would track tagged versions of the component repositories. This would work like having 'ensure => latest' pointed at the current 'products' repository.
• PE releases that track stable releases and are coordinated with the rest of the PE ecosystem, are QA tested, and have some set of backports / commercial enhancements.

For power users who want more control than that, we’ll have documentation on how to roll your own. The tooling to create AIO will itself be open-sourced so you could build a package that picks the versions you want to include.


5 - Why not use a metapackage which has no content of its own, but has requirements on specific versions of the other 'real' packages that contain the actual programs?
• Avoiding a coordinated major version ship across all projects… We're changing the layout on the filesystem (moving to opt) for a few reasons, including shipping our own stack without conflicting with system libraries. This kind of change shouldn't be done in a "y" release (of x.y.z). Components that have been updated can’t mix/match with components that haven’t. Example: A user has facter ensure => latest, but not puppet. They’ll get facter with the new layout (and maybe the new PL-provided ruby, if its a dependency of the new facter), but not puppet, and have a broken install. If the meta-package provides facter, then users with ensure => latest on facter but not puppet will have a completely new install of their whole stack. Neither of these are ideal. To avoid any automatic installs, we could have facter *not* provide itself, which is also probably not the route to go down. The AIO ensures users have everything they need at the same time, and can just continue using the agent after upgrade.
• We want to simplify the endpoint install experience for users. A single endpoint puppet agent package is arguably among the simplest possible deployment mechanisms.
• Meta-package only solves the problem for agents with a package manager that can do remote dependency resolution. We want the install experience for agents to be excellent/seamless regardless of the platform you are running, and be the same experience across the board. Users installing a Solaris 10 agent package shouldn’t have a different experience than users installing a Debian 7 agent package.

6 - Will using AIO puppet/mco to upgrade AIO work?

Once we have AIO packages ready, we’ll be doing some testing of upgrade scenarios.

7 - Why are you dropping 3.x agent compat with 4.0 master?

We didn’t take this decision lightly, and it came down to getting 4.0 features out into people’s hands sooner rather than later.

Backward compatibility on the network in this case is a fair bit of work, which we haven’t fully scoped, so in 4.0 we opted for the current tradeoff.

We recommend users take a “migrate, not update” approach to 4.0 upgrades, and people taking this approach probably won’t much care about this change.


--eric0

[Nan Liu]

On Fri, Nov 21, 2014 at 12:46 PM, Eric Sorenson <eric.s...@puppetlabs.com> wrote:

On Nov 18, 2014, at 11:04 AM, Eric Sorenson <eric.s...@puppetlabs.com> wrote:

> 7. Umm.. I think that's all.

Wow thanks for all the responses, there are a ton of very good questions in here -- some of them got answered inline and I think we rolled up the rest into this little FAQ. which I''m sure will generate a new set of questions :)

0 - Which platforms will get FOSS AIO packages?

RHEL/Centos 5, 6, 7
Fedora 20, 21
Ubuntu 10.04, 12.04, 14.04
Debian Squeeze, Wheezy
Windows x86, x64

Windows 7/8/2008/12? 

 

Commercial OS'es (Windows excepted) will get PE-only AIO
SLES 10, 11
Solaris 10,11
AIX
Mac OS X


1 - How do I manage gems with AIO ruby?

Use the AIO included gem binary. ('/opt/puppet/agent/bin/gem install somerandomgem') To make this easier, we can either fix PUP-3688 or add a module with a minimal 'puppet_gem' provider using the AIO path.

Maybe just patch up pe_gem module to be smarter? Otherwise a new puppet_gem module requires a lot of module updates.

2 - Will there be security updates, etc?

Yes, absolutely.

3 - Will it include puppetdb-terminus?

Yes.

4 - What will the release cadence look like? What if I want to manage component upgrades on a different cadence than upstream, e.g. just facter, or just hiera?

We're planning to provide three trains of packaging:
        • Nightlies, that get built whenever there's a change in one of the component repositories that passes through CI. These will essentially track the latest, like the nightly repos do now.
        • Roll-up releases that would track tagged versions of the component repositories. This would work like having 'ensure => latest' pointed at the current 'products' repository.
        • PE releases that track stable releases and are coordinated with the rest of the PE ecosystem, are QA tested, and have some set of backports / commercial enhancements.

For power users who want more control than that, we’ll have documentation on how to roll your own. The tooling to create AIO will itself be open-sourced so you could build a package that picks the versions you want to include.

That's really helpful, so essentially puppet omnibus. 

5 - Why not use a metapackage which has no content of its own, but has requirements on specific versions of the other 'real' packages that contain the actual programs?
        • Avoiding a coordinated major version ship across all projects… We're changing the layout on the filesystem (moving to opt) for a few reasons, including shipping our own stack without conflicting with system libraries. This kind of change shouldn't be done in a "y" release (of x.y.z). Components that have been updated can’t mix/match with components that haven’t. Example: A user has facter ensure => latest, but not puppet. They’ll get facter with the new layout (and maybe the new PL-provided ruby, if its a dependency of the new facter), but not puppet, and have a broken install. If the meta-package provides facter, then users with ensure => latest on facter but not puppet will have a completely new install of their whole stack. Neither of these are ideal. To avoid any automatic installs, we could have facter *not* provide itself, which is also probably not the route to go down. The AIO ensures users have everything they need at the same time, and can just continue using the agent after upgrade.
        • We want to simplify the endpoint install experience for users. A single endpoint puppet agent package is arguably among the simplest possible deployment mechanisms.
        • Meta-package only solves the problem for agents with a package manager that can do remote dependency resolution. We want the install experience for agents to be excellent/seamless regardless of the platform you are running, and be the same experience across the board. Users installing a Solaris 10 agent package shouldn’t have a different experience than users installing a Debian 7 agent package.

6 - Will using AIO puppet/mco to upgrade AIO work?

Once we have AIO packages ready, we’ll be doing some testing of upgrade scenarios.

7 - Why are you dropping 3.x agent compat with 4.0 master?

We didn’t take this decision lightly, and it came down to getting 4.0 features out into people’s hands sooner rather than later.

Backward compatibility on the network in this case is a fair bit of work, which we haven’t fully scoped, so in 4.0 we opted for the current tradeoff.

We recommend users take a “migrate, not update” approach to 4.0 upgrades, and people taking this approach probably won’t much care about this change.

Thanks,

Nan

[Rob Reynolds]

On Fri, Nov 21, 2014 at 2:59 PM, Nan Liu <nan...@gmail.com> wrote:

On Fri, Nov 21, 2014 at 12:46 PM, Eric Sorenson <eric.s...@puppetlabs.com> wrote:

On Nov 18, 2014, at 11:04 AM, Eric Sorenson <eric.s...@puppetlabs.com> wrote:

> 7. Umm.. I think that's all.

Wow thanks for all the responses, there are a ton of very good questions in here -- some of them got answered inline and I think we rolled up the rest into this little FAQ. which I''m sure will generate a new set of questions :)

0 - Which platforms will get FOSS AIO packages?

RHEL/Centos 5, 6, 7
Fedora 20, 21
Ubuntu 10.04, 12.04, 14.04
Debian Squeeze, Wheezy
Windows x86, x64

Windows 7/8/2008/12? 

All of the currently supported platforms for Windows, yes. 

 

--

You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CACqVBqDKb8pV5zT8ER%2Bk3hiE6Om2mPs6yA6FnF6iNtvWxxkg0g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Rob Reynolds

Developer, Puppet Labs

Join us at PuppetConf 2015, October 5-9 in Portland, OR - http://2015.puppetconf.com/

[Daniel De Marco]

* Eric Sorenson <eric.s...@puppetlabs.com> [11/21/2014 21:47]:

> Commercial OS'es (Windows excepted) will get PE-only AIO

> Mac OS X

will there be a FOSS puppet 4 for OSX ? It will still be available as a
gem or as a single package, right?

Thanks, Daniel.

[Martin Alfke]

On 21 Nov 2014, at 21:46, Eric Sorenson <eric.s...@puppetlabs.com> wrote:

>
> 0 - Which platforms will get FOSS AIO packages?
>
> RHEL/Centos 5, 6, 7
> Fedora 20, 21
> Ubuntu 10.04, 12.04, 14.04
> Debian Squeeze, Wheezy
> Windows x86, x64

OpenSuSE?
Debian Jessie?

>
> 1 - How do I manage gems with AIO ruby?
>
> Use the AIO included gem binary. ('/opt/puppet/agent/bin/gem install somerandomgem') To make this easier, we can either fix PUP-3688 or add a module with a minimal 'puppet_gem' provider using the AIO path.

That would be very ugly (e.g. for installing hiera add ons).
Why can we not adopt the behaviour of pe_gem provider?

>
> 3 - Will it include puppetdb-terminus?
>
> Yes.

Can we choose from multiple versions?
e.g. we have new puppet and older puppet (migration process) running in parallel and we want both masters to make use of the same puppetdb.

>
> 4 - What will the release cadence look like? What if I want to manage component upgrades on a different cadence than upstream, e.g. just facter, or just hiera?
>

> For power users who want more control than that, we’ll have documentation on how to roll your own. The tooling to create AIO will itself be open-sourced so you could build a package that picks the versions you want to include.

+1

[Erik Dalén]

Will there be a version of this AIO package for 3.7.4 (or even 3.7.3) so we can try out the packaging without trying out all the stuff in puppet 4.0 at the same time?

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/08BAFC24-F5DB-401B-99FD-046B1AAB3917%40gmail.com.

[Martin Alfke]

On Nov 24, 2014 11:33 AM, "Erik Dalén" <erik.gus...@gmail.com> wrote:
>
> Will there be a version of this AIO package for 3.7.4 (or even 3.7.3) so we can try out the packaging without trying out all the stuff in puppet 4.0 at the same time?

+1

>
> On Sat Nov 22 2014 at 11:46:16 AM Martin Alfke <tux...@gmail.com> wrote:
>>
>>
>> On 21 Nov 2014, at 21:46, Eric Sorenson <eric.s...@puppetlabs.com> wrote:
>>
>> >
>> > 0 - Which platforms will get FOSS AIO packages?
>> >
>> > RHEL/Centos 5, 6, 7
>> > Fedora 20, 21
>> > Ubuntu 10.04, 12.04, 14.04
>> > Debian Squeeze, Wheezy
>> > Windows x86, x64
>>
>> OpenSuSE?
>> Debian Jessie?
>>
>> >
>> > 1 - How do I manage gems with AIO ruby?
>> >
>> > Use the AIO included gem binary. ('/opt/puppet/agent/bin/gem install somerandomgem') To make this easier, we can either fix PUP-3688 or add a module with a minimal 'puppet_gem' provider using the AIO path.
>>
>> That would be very ugly (e.g. for installing hiera add ons).
>> Why can we not adopt the behaviour of pe_gem provider?
>>
>> >
>> > 3 - Will it include puppetdb-terminus?
>> >
>> > Yes.
>> Can we choose from multiple versions?
>> e.g. we have new puppet and older puppet (migration process) running in parallel and we want both masters to make use of the same puppetdb.
>>
>> >
>> > 4 - What will the release cadence look like? What if I want to manage component upgrades on a different cadence than upstream, e.g. just facter, or just hiera?
>> >
>> > For power users who want more control than that, we’ll have documentation on how to roll your own. The tooling to create AIO will itself be open-sourced so you could build a package that picks the versions you want to include.
>>
>> +1
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

>> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/08BAFC24-F5DB-401B-99FD-046B1AAB3917%40gmail.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Developers" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAAAzDLe72HCy8OBoi1s18eOb1bYOoTVE2VVL%3DhbyVY3LpCdVuA%40mail.gmail.com.

[Trevor Vaughan]

+2

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CAH_Azc2p_3MuwcUS_zQHTA1_CVpw2r5nuVUQJ-eRjsD%2Bs2Xmzg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

[John Bollinger]

On Friday, November 21, 2014 2:47:05 PM UTC-6, Eric Sorenson wrote:

 

5 - Why not use a metapackage which has no content of its own, but has requirements on specific versions of the other 'real' packages that contain the actual programs?
        • Avoiding a coordinated major version ship across all projects… We're changing the layout on the filesystem (moving to opt) for a few reasons, including shipping our own stack without conflicting with system libraries.

Not buying it.  An AIO package does not inherently avoid these sorts of problems (i.e. what if I have both 'facter' and 'puppet-aio' packages installed at the same time?), and a metapackage approach is not inherently any more susceptible to them (it's all about which packages are managed).

        • We want to simplify the endpoint install experience for users. A single endpoint puppet agent package is arguably among the simplest possible deployment mechanisms.

Yay for simple installation.  A metapackage achieves that, too.

 

        • Meta-package only solves the problem for agents with a package manager that can do remote dependency resolution. We want the install experience for agents to be excellent/seamless regardless of the platform you are running, and be the same experience across the board. Users installing a Solaris 10 agent package shouldn’t have a different experience than users installing a Debian 7 agent package.

So?  You are committed to separate native packaging for a wide variety of systems anyway.  Why not make use of the features of those systems that have more capable package managers?

I really, really dislike AIO packages.  In fact, I tend to break them up when I can.  They do confer a greater level of control on packagers, but along with that comes considerably more responsibility and exposure, and considerably less user flexibility.  If anything needs changing in any component, the whole AIO needs to be updated.  Any flaw or vulnerability in any component is a flaw in the whole AIO.  The more components you bundle together, the worse it is, and it is particularly bad when you bundle third-party components.

It just doesn't have to be that way on systems that can support metapackages.  You can have your cake and eat it too: a metapackage provides for easy installation and update, and done right it provides for ensuring that a given installation has a tested and approved combination of components, but you still have the flexibility of managing components individually (possibly requiring the removing the metapackage but retaiing the component packages).

In the end, you'll of course do what you think best.  If it ends up being AIO packages across the board, then I simply won't use your packages.  What a shame.


John

[Trevor Vaughan]

I'm in 100% agreement with John here.

Right now, I have a forked Hiera that I'm using based off of some patches that I have PRs in for. Once this gets resolved, I'll merge everything back to the mainline, but I *must* have the ability to deviate when necessary for the best results of my users.

As with John, if this is not the case, I'll just end up re-packaging everything myself and forgo the AIO packages (and not be thrilled about it).

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/50155310-cda5-4384-b7d8-45ebe729449c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Trevor Vaughan]

Hi All,

I'm curious as to what decisions (if any) have been made here.

Thanks,

Trevor

[Justin Lambert]

I was also curious where (and if there was a guess when) to look for nightly builds so I can start testing the decisions made. 

Thanks,

jl

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUSjRqZUv29Ofu%3DpiHz3Y2Nk9Lwu3oT1qXDf53bCnbchg%40mail.gmail.com.

[Eric Sorenson]

We're not going to ship deb and rpm metapackages, to address the last points on the thread. We _will_ make it as easy as possible to build AIO with different combinations of component versions so you can compose your own but still retain up- and down-grade compatibility with PL supplied packages. I'm sorry you and John aren't convinced by the all-in-one plan; I see your point and totally agree that as an advanced Linux+FOSS Puppet user, it's not the optimum solution. 

--eric0

On Dec 7, 2014, at 3:33 PM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

Hi All,

I'm curious as to what decisions (if any) have been made here.

Thanks,

Trevor

On Mon, Nov 24, 2014 at 12:40 PM, Trevor Vaughan <tvau...@onyxpoint.com> wrote:

I'm in 100% agreement with John here.

Right now, I have a forked Hiera that I'm using based off of some patches that I have PRs in for. Once this gets resolved, I'll merge everything back to the mainline, but I *must* have the ability to deviate when necessary for the best results of my users.

As with John, if this is not the case, I'll just end up re-packaging everything myself and forgo the AIO packages (and not be thrilled about it).

Thanks,

Trevor

On Mon, Nov 24, 2014 at 11:12 AM, John Bollinger <john.bo...@stjude.org> wrote:

On Friday, November 21, 2014 2:47:05 PM UTC-6, Eric Sorenson wrote:

 

5 - Why not use a metapackage which has no content of its own, but has requirements on specific versions of the other 'real' packages that contain the actual programs? 
        • Avoiding a coordinated major version ship across all projects… We're changing the layout on the filesystem (moving to opt) for a few reasons, including shipping our own stack without conflicting with system libraries.

Not buying it.  An AIO package does not inherently avoid these sorts of problems (i.e. what if I have both 'facter' and 'puppet-aio' packages installed at the same time?), and a metapackage approach is not inherently any more susceptible to them (it's all about which packages are managed).

        • We want to simplify the endpoint install experience for users. A single endpoint puppet agent package is arguably among the simplest possible deployment mechanisms. 

Yay for simple installation.  A metapackage achieves that, too.

 

        • Meta-package only solves the problem for agents with a package manager that can do remote dependency resolution. We want the install experience for agents to be excellent/seamless regardless of the platform you are running, and be the same experience across the board. Users installing a Solaris 10 agent package shouldn’t have a different experience than users installing a Debian 7 agent package.

So?  You are committed to separate native packaging for a wide variety of systems anyway.  Why not make use of the features of those systems that have more capable package managers?

I really, really dislike AIO packages.  In fact, I tend to break them up when I can.  They do confer a greater level of control on packagers, but along with that comes considerably more responsibility and exposure, and considerably less user flexibility.  If anything needs changing in any component, the whole AIO needs to be updated.  Any flaw or vulnerability in any component is a flaw in the whole AIO.  The more components you bundle together, the worse it is, and it is particularly bad when you bundle third-party components.

It just doesn't have to be that way on systems that can support metapackages.  You can have your cake and eat it too: a metapackage provides for easy installation and update, and done right it provides for ensuring that a given installation has a tested and approved combination of components, but you still have the flexibility of managing components individually (possibly requiring the removing the metapackage but retaiing the component packages).

In the end, you'll of course do what you think best.  If it ends up being AIO packages across the board, then I simply won't use your packages.  What a shame.

-- 

Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

-- 

Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699
tvau...@onyxpoint.com

-- This account not approved for unencrypted proprietary information --

-- 
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUSjRqZUv29Ofu%3DpiHz3Y2Nk9Lwu3oT1qXDf53bCnbchg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[John Bollinger]

On Tuesday, December 9, 2014 12:58:01 PM UTC-6, Eric Sorenson wrote:

We're not going to ship deb and rpm metapackages, to address the last points on the thread. We _will_ make it as easy as possible to build AIO with different combinations of component versions so you can compose your own but still retain up- and down-grade compatibility with PL supplied packages. I'm sorry you and John aren't convinced by the all-in-one plan; I see your point and totally agree that as an advanced Linux+FOSS Puppet user, it's not the optimum solution. 

I am disappointed, and I don't see how that's an optimum solution for anybody.  It has drawbacks even for PL (which is not to say that a metapackage approach may not have different ones).  Still, the decision has been made.  It's good to at least know what's coming.

Just so we're clear, please understand that I will not be building or using AIO packages with any combination of components.  I can't speak for anyone else, but the best way to support my needs within this scheme is to continue to provide a separate source downloads for each PL component involved, and to provide good documentation of external dependencies and how everything fits together.  Additionally, avoiding hardcoded paths and avoiding the assumption that Puppet has a private Ruby to play around in will be helpful (and wise in any case).


John

[Jeff McCune]

On Wed, Dec 10, 2014 at 10:27 AM, John Bollinger <john.bo...@stjude.org> wrote:

Additionally, avoiding hardcoded paths and avoiding the assumption that Puppet has a private Ruby to play around in will be helpful (and wise in any case).

If the best predictor of future behavior is past behavior then our components will make assumptions about hardcoded paths and the location of specific dependencies like Ruby, Java, etc...  I don't think it's possible to avoid this outcome given the number of people and teams working on the various projects and the desire to reduce the complexity of the system.

Do you have a sense of how much impact this will be on you John?

-Jeff

[John Bollinger]

On Wednesday, December 10, 2014 1:00:14 PM UTC-6, Jeff McCune wrote:

On Wed, Dec 10, 2014 at 10:27 AM, John Bollinger <john.bo...@stjude.org> wrote:

Additionally, avoiding hardcoded paths and avoiding the assumption that Puppet has a private Ruby to play around in will be helpful (and wise in any case).

If the best predictor of future behavior is past behavior then our components will make assumptions about hardcoded paths and the location of specific dependencies like Ruby, Java, etc...  I don't think it's possible to avoid this outcome given the number of people and teams working on the various projects and the desire to reduce the complexity of the system.

I don't accept that it's impossible, but I would believe that it would require effort, resources, and discipline that PL is unwilling to invest.  I think you are making a poor decision, at least from a design perspective, but it's your call to make.

 

Do you have a sense of how much impact this will be on you John?

If I cannot install Puppet into a Ruby installation of my choice (of sufficiently recent version) and have it work correctly, or if its installation alters the behavior of other software relying on that Ruby, then I do not foresee updating until I or someone else can fix those problems.  I am unlikely to be able to devote any effort to such an endeavor any time soon, so my adoption of the new version would be forestalled indefinitely.


John

[John Bollinger]

On Thursday, December 11, 2014 3:34:17 PM UTC-6, John Bollinger wrote:

If I cannot install Puppet into a Ruby installation of my choice (of sufficiently recent version) and have it work correctly, or if its installation alters the behavior of other software relying on that Ruby, then I do not foresee updating until I or someone else can fix those problems.  I am unlikely to be able to devote any effort to such an endeavor any time soon, so my adoption of the new version would be forestalled indefinitely.

Similarly, as long as Puppet is unable to share general-purpose third-party components, such as Ruby modules or a Java runtime, with other subsystems, I do not anticipate updating.

Although I am disappointed that PL intends to offer only AIO packages, that decision will be only an inconvenience for me as long as the software does not inherently depend on AIO structure and layout.  You may make suggestions, but you may not make system configuration decisions for me.


John

[Byron Miller]

I don't see these negatives at all. I think having puppet decoupled is a good thing - for reliability, management and freeing us from "ruby hell".  I can't see an AIO package altering the native behavior of the OS or system rubbies either..

Since it is OSS as well, is there a reason you wouldn't be able to pull down the git repo and update ruby according to your own needs? my assumption is that its a LOT easier to upgrade puppet ruby if it is indeed decoupled from system ruby or ruby used by apps on the platform your trying to manage.

-byron

[David Schmitt]

Hi,



The usual argument at this point in the discussion is that AIO packages
of any vendor will - by definition - have worse security support than
the system versions.

People who have to certify/verify/validate/audit all binaries that are
running on a given system will have additional high costs from bundled
components that may render using those packages infeasible.

Adding more binaries to a system will mean that they will require
additional non-dedup-able hardware resources.


Independently of those technical arguments, "freeing puppetlabs from the
ruby hell" will be read by free software proponents(sic!) as
"unwillingness to support the diversity of the free software ecosystem".
See also http://islinuxaboutchoice.com/ and so on. My own opinion is
split between the economical realities of vendors and the fact that
everything outside Debian main (+ backports) is not *Debian*.



Regards, David

> as the software does not inherently /depend/ on AIO structure and

> layout. You may make suggestions, but you may not make system
> configuration decisions for me.
>
>
> John
>

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-dev+...@googlegroups.com

> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/052267c9-5504-4587-84d5-a2f5442065e5%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-dev/052267c9-5504-4587-84d5-a2f5442065e5%40googlegroups.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

--
* Always looking for people I can help with awesome projects *
Twitter: @dev_el_ops G+: https://plus.google.com/+DavidSchmitt
Blog: http://club.black.co.at/log/
LinkedIn: http://at.linkedin.com/in/davidschmitt

[Eric Shamow]

I think it’s worth asking the question of whether you want PL to spend their time building good CM software or dealing with and QA’ing the myriad combinations of Rubies and gems that they are currently forced to support due to the sloppiness and inconsistency of distro packaging and the gem ecosystem.

Personally, I’d rather allow PL to vendor the Ruby and gems required to keep Puppet itself working properly so that I can rely on it to perform basic system tasks. Since the vendored Ruby and gems are far away from system Ruby, I view this as a *freedom* in my ability to manage a system. I now don’t have to worry about keeping my custom or third-party Ruby tool code at the same version and gemset as a single tool across the system.

This then switches the burden to them to keep up with security and bugfix issues in the Rubies and gems they bundle, but frankly I have a greater belief that PL will manage to do this successfully - in aggregate - than the distributions and hand-rolled gemsets that people are using now.

-- 
Eric Shamow
Sent with Airmail

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/548DFC3B.2070604%40dasz.at.

[John Bollinger]

On Sunday, December 14, 2014 4:25:51 PM UTC-6, Eric Shamow wrote:

I think it’s worth asking the question of whether you want PL to spend their time building good CM software or dealing with and QA’ing the myriad combinations of Rubies and gems that they are currently forced to support due to the sloppiness and inconsistency of distro packaging and the gem ecosystem.

I'm not asking PL to QA my particular combination of supporting components, nor any particular set at all.  Rather, I am asking that Puppet itself be a proper component, that I can integrate into my systems in a manner I choose, subject to reasonable constraints.  That is in no way incompatible with PL offering AIO packages, by the way -- in fact, it should simplify that process.

Moreover, it is my firm belief that maintaining Puppet's position as an independent component will be better for Puppet's intermediate- to long-term health as a software project, as well as better for PL as a business.  PL seems to disagree, which of course is its prerogative.

Personally, I’d rather allow PL to vendor the Ruby and gems required to keep Puppet itself working properly so that I can rely on it to perform basic system tasks. Since the vendored Ruby and gems are far away from system Ruby, I view this as a *freedom* in my ability to manage a system. I now don’t have to worry about keeping my custom or third-party Ruby tool code at the same version and gemset as a single tool across the system.

You can always install current Puppet in its own Ruby if you wish.  I am completely in favor of supporting that alternative going forward.  It's about to switch from being the harder path to being the easier path (provided that you don't care which particular Ruby hosts Puppet).  That in itself will be inconvenient for me, but no more than that.  On the other hand, I am asking and hoping some particular implementation of that alternative not become the only viable path.  That is, I don't want your freedom to come at the expense mine.

 

This then switches the burden to them to keep up with security and bugfix issues in the Rubies and gems they bundle, but frankly I have a greater belief that PL will manage to do this successfully - in aggregate - than the distributions and hand-rolled gemsets that people are using now.

And that's a large reason why I don't think it's a particularly wise business decision to provide AIO packages: the fewer CVEs have Puppet's name on them, the better.  But that's really beside the point, which is not so much about PL offering AIO packages as about future Puppet being dependent on AIO form, and on a particular AIO form at that.  I don't have to use PL's packages, but I do have to deal with Puppet's built-in requirements and limitations, and to reconcile those with my system management policies and practices.

As an armchair IP quarterback, I'd also be concerned that if Puppet works only in AIO form then it will be the AIO form that must then be considered "Puppet".  The AIO will incorporate a great deal of IP that does not belong to PL.  In the past, PL has preferred a conservative approach to IP, so I'll suppose that either this isn't an issue of genuine concern or that PL has not considered it.


John

[Eric Shamow]

On December 15, 2014 at 7:51:58 AM, John Bollinger (john.bo...@stjude.org) wrote:

This then switches the burden to them to keep up with security and bugfix issues in the Rubies and gems they bundle, but frankly I have a greater belief that PL will manage to do this successfully - in aggregate - than the distributions and hand-rolled gemsets that people are using now.

And that's a large reason why I don't think it's a particularly wise business decision to provide AIO packages: the fewer CVEs have Puppet's name on them, the better.  But that's really beside the point, which is not so much about PL offering AIO packages as about future Puppet being dependent on AIO form, and on a particular AIO form at that.  I don't have to use PL's packages, but I do have to deal with Puppet's built-in requirements and limitations, and to reconcile those with my system management policies and practices.

As an armchair IP quarterback, I'd also be concerned that if Puppet works only in AIO form then it will be the AIO form that must then be considered "Puppet".  The AIO will incorporate a great deal of IP that does not belong to PL.  In the past, PL has preferred a conservative approach to IP, so I'll suppose that either this isn't an issue of genuine concern or that PL has not considered it.

Without speaking for PL, I’d suspect that’s *exactly* how they’d like users to see it, as with Omnibus packaging. You get Puppet, and Puppet can assist in troubleshooting for Puppet only as delivered.

Don’t get me wrong John - I totally agree that having proper paths and improving coding discipline so that the project is properly composable is not only the right thing, I’ve been irritated in ways you can’t even imagine by the fact that it’s not. But the reality is that this is the state the codebase is in and it’s a project we’ve all invested a lot of time in…I’m hoping PL’s engineering team is putting hard thought into the amount of hard-coded paths and infrastructure assumptions that are made in the code, but for right here and right now, I’m not sure there’s a better fix.

[Byron Miller]

One could ague puppetlabs is better off sharing the package specs for included rubies so people can keep them current themselves, security updates directly from ruby vs having to wait for downstream backports and vendor repos sounds better to me long term. Not only can puppetlabs push updates quicker but the community can embrace them in the puppetworld without having to wait for impact analysis to their "Shared" dependencies.

-byron