Announce: Puppet 5 Platform Available!

[Eric Sorenson]

I have run out of superlatives to try to express how excited I am for this release: the Puppet 5 Platform is available for download now.

The primary goals of this release are to harmonize numbering across the major components (Puppet Agent, PuppetDB, Puppet Server) to "5", as a first step towards delivering these components as a unified platform; include Hiera 5 with eyaml as a built-in capability; provide clean UTF-8 support; move network comms to fast, interoperable JSON. Our current Ruby versions are EOL'ed, so we're moving to MRI Ruby 2.4 on the agent and (opt-in) jruby9k on the server. The PE-only puppet-server metrics service is now open-sourced. 

In addition to the features, there are some substantial performance boosts waiting for you. According to our perf testing (thanks Doug!):

• Puppet 5 Agent run-times were 30% lower at equivalent loads. (Average of 8 seconds vs 5.5 seconds)

• Puppet 5 Server CPU utilization was at least 20% lower than Puppet 4 in all scenarios.

• CPU utilization for Puppet 5 PuppetDB and PostgreSQL were also lower in all scenarios.

• Puppet 5 catalog compile times reported by Puppet Server were between 7-10% lower than Puppet 4.

• Puppet 5 scaled to an additional 40% increase in the number of agents while Puppet 4 agent run-times became dangerously high.

This is a "semver major" with some backwards incompatibilities, but we have worked very hard to retain module compatibility with Puppet 4.x modules. With a few careful (and hopefully rarely used) exceptions, module code that works under Puppet 4 should not need revision to work under Puppet 5. 

For a full list of changes and download instructions, check out the full release notes: https://docs.puppet.com/puppet/5.0/release_notes.html

I'd like to send out huge thanks to the Puppet teams who worked on this release and to community members who provided feedback on both the design discussions and early preview releases — extra special thanks to Josh Cooper for shepherding this out the door. It has a special significance for me since it's version five and (by total coincidence!) yesterday was my five year anniversary at Puppet :) I think it's going to be a great release series.

Eric Sorenson - er...@puppet.com 
director of product, ecosystem and platform

[Trevor Vaughan]

Awesome! We'll start working on testing with this as soon as we can.

First issue found: When using rspec-puppet, we need to test with 'server_facts' but now can no longer set those since they are 'reserved' but appear to be unset by default. Any idea how to get around this?

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/C17DC8B4-D7D3-417D-A3D5-43066C5BCB3F%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc

(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --

[R.I.Pienaar]

People upgrading should note the PuppetDB dependency is now postgres
postgresql96, this is not stated on the package dependency metadata so
upgrade works fine but puppetdb fails to start until you update

> > modules. With a few careful (and hopefully rarely used) exceptions, *module

> > code that works under Puppet 4 should not need revision to work under

> > Puppet 5*.

> >
> > For a full list of changes and download instructions, check out the full
> > release notes: https://docs.puppet.com/puppet/5.0/release_notes.html
> >
> > I'd like to send out huge thanks to the Puppet teams who worked on this
> > release and to community members who provided feedback on both the design
> > discussions and early preview releases — extra special thanks to Josh
> > Cooper for shepherding this out the door. It has a special significance for
> > me since it's version five and (by total coincidence!) yesterday was my
> > five year anniversary at Puppet :) I think it's going to be a great release
> > series.
> >
> > Eric Sorenson - er...@puppet.com
> > director of product, ecosystem and platform
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Puppet Developers" group.
> > To unsubscribe from this group and stop receiving emails from it, send an

> > email to puppet-dev+...@googlegroups.com.

> > To view this discussion on the web visit https://groups.google.com/d/
> > msgid/puppet-dev/C17DC8B4-D7D3-417D-A3D5-43066C5BCB3F%40puppet.com

> > <https://groups.google.com/d/msgid/puppet-dev/C17DC8B4-D7D3-417D-A3D5-43066C5BCB3F%40puppet.com?utm_medium=email&utm_source=footer>
> > .

> > For more options, visit https://groups.google.com/d/optout.
> >
>
>
>
> --
> Trevor Vaughan
> Vice President, Onyx Point, Inc
> (410) 541-6699 x788
>
> -- This account not approved for unencrypted proprietary information --
>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to puppet-dev+...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWD%2BN9B%3DHAGxs7DtOdydzDN2SNVE4miZfeeEToOMy5kZQ%40mail.gmail.com.

> For more options, visit https://groups.google.com/d/optout.


--

R.I.Pienaar / www.devco.net / @ripienaar

[Trevor Vaughan]

I'm seeing a lot of Warning: Defining "data_provider": "hiera" in metadata.json is deprecated. How do we successfully support both Puppet 4.7 (LTS) and Puppet 5 (Today) without this garbage popping up at every run?

I don't want to kill any *real* deprecation warnings, just ones that conflict between LTS and New Shiny.

Thanks,

Trevor

[Henrik Lindberg]

On 28/06/17 16:31, Trevor Vaughan wrote:
> I'm seeing a lot of Warning: Defining "data_provider": "hiera" in
> metadata.json is deprecated. How do we successfully support both Puppet
> 4.7 (LTS) and Puppet 5 (Today) without this garbage popping up at every run?
>
> I don't want to kill any *real* deprecation warnings, just ones that
> conflict between LTS and New Shiny.
>

You should be able to suppress the deprecation warnings. However, there
were bugs in several places that logged deprecations so they may not all
be suppressed by that action alone. You may also need to set strict=off.

- henrik

> used) exceptions, /module code that works under Puppet 4 should
> not need revision to work under Puppet 5/.

>
> For a full list of changes and download instructions, check out
> the full release notes:
> https://docs.puppet.com/puppet/5.0/release_notes.html
> <https://docs.puppet.com/puppet/5.0/release_notes.html>
>
> I'd like to send out huge thanks to the Puppet teams who worked
> on this release and to community members who provided feedback
> on both the design discussions and early preview releases —
> extra special thanks to Josh Cooper for shepherding this out the
> door. It has a special significance for me since it's version
> five and (by total coincidence!) yesterday was my five year
> anniversary at Puppet :) I think it's going to be a great
> release series.
>

> Eric Sorenson - er...@puppet.com <mailto:er...@puppet.com>

> director of product, ecosystem and platform
>
> --
> You received this message because you are subscribed to the
> Google Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from

> it, send an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/C17DC8B4-D7D3-417D-A3D5-43066C5BCB3F%40puppet.com

> <https://groups.google.com/d/msgid/puppet-dev/C17DC8B4-D7D3-417D-A3D5-43066C5BCB3F%40puppet.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
>
>
>
> --
> Trevor Vaughan
> Vice President, Onyx Point, Inc

> (410) 541-6699 x788 <tel:(410)%20541-6699>

>
> -- This account not approved for unencrypted proprietary information --
>
>
>
>
> --
> Trevor Vaughan
> Vice President, Onyx Point, Inc
> (410) 541-6699 x788
>
> -- This account not approved for unencrypted proprietary information --
>

> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to puppet-dev+...@googlegroups.com
> <mailto:puppet-dev+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUtm8k6iHXcpC5qZPps%3DMGjZTNR972aGUra8aD6gXS5yQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUtm8k6iHXcpC5qZPps%3DMGjZTNR972aGUra8aD6gXS5yQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.


--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[R.I. Pienaar]

Also if you are a mcollective user it might be worth waiting a bit. Many deprecation warnings will show on your client and server logs which are really annoying

I made PRs to fix them so maybe next release

---
R.I.Pienaar

[Eric Sorenson]

On Jun 28, 2017, at 9:28 AM, R.I.Pienaar <r...@devco.net> wrote:

People upgrading should note the PuppetDB dependency is now postgres
postgresql96, this is not stated on the package dependency metadata so
upgrade works fine but puppetdb fails to start until you update

Hm, as I understand it there isn't a package-level dependency on postgresql at all because you can install puppetdb and postgres on different hosts. 

The version bump is mentioned in the puppetdb 5 docs but it's probably worth bringing that into the overall platform release notes  https://docs.puppet.com/puppetdb/5.0/#postgresql-96

[R.I. Pienaar]

On 30 Jun 2017, at 01:22, Eric Sorenson <er...@puppet.com> wrote:

On Jun 28, 2017, at 9:28 AM, R.I.Pienaar <r...@devco.net> wrote:

People upgrading should note the PuppetDB dependency is now postgres
postgresql96, this is not stated on the package dependency metadata so
upgrade works fine but puppetdb fails to start until you update

Hm, as I understand it there isn't a package-level dependency on postgresql at all because you can install puppetdb and postgres on different hosts. 

That's a good point. Sounds legit :)

The version bump is mentioned in the puppetdb 5 docs but it's probably worth bringing that into the overall platform release notes  https://docs.puppet.com/puppetdb/5.0/#postgresql-96

Yeah - I think though since it's now like "5 Platform" probably some kind of holistic upgrade guide is needed. 

I have never updated PostgreSQL for puppetdb so it took a while to figure out all the incantations needed to do that and there is definitely an order in which this has to be done between the various components. Get it wrong and you're running stuff with puppet apply on the master since you broke it like I did :P

I am still unsure how to do this easily since puppetdb 5 requires puppetserver 5 I can only think of nasty break ways to do that update. How did others do it? You will always end up with a completely down puppetserver and now using the puppet module to manage puppetdb etc will require apply. 

Eric Sorenson - er...@puppet.com 
director of product, ecosystem and platform

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/81F9258B-89EF-464C-9B1C-FDC677DE800C%40puppet.com.

[Erik Dalén]

On Fri, 30 Jun 2017 at 07:47 R.I. Pienaar <r...@devco.net> wrote:

On 30 Jun 2017, at 01:22, Eric Sorenson <er...@puppet.com> wrote:

On Jun 28, 2017, at 9:28 AM, R.I.Pienaar <r...@devco.net> wrote:

People upgrading should note the PuppetDB dependency is now postgres
postgresql96, this is not stated on the package dependency metadata so
upgrade works fine but puppetdb fails to start until you update

Hm, as I understand it there isn't a package-level dependency on postgresql at all because you can install puppetdb and postgres on different hosts. 

That's a good point. Sounds legit :)

The version bump is mentioned in the puppetdb 5 docs but it's probably worth bringing that into the overall platform release notes  https://docs.puppet.com/puppetdb/5.0/#postgresql-96

Yeah - I think though since it's now like "5 Platform" probably some kind of holistic upgrade guide is needed. 

I have never updated PostgreSQL for puppetdb so it took a while to figure out all the incantations needed to do that and there is definitely an order in which this has to be done between the various components. Get it wrong and you're running stuff with puppet apply on the master since you broke it like I did :P

I am still unsure how to do this easily since puppetdb 5 requires puppetserver 5 I can only think of nasty break ways to do that update. How did others do it? You will always end up with a completely down puppetserver and now using the puppet module to manage puppetdb etc will require apply. 

Version 4.x of PuppetDB works with Postgres 9.6.x, so it should be possible to upgrade Postgres first and then do the rest of the upgrade.

According to documentation PuppetDB 5.x should  be able to process commands from the PuppetDB 4.x terminus as well, so upgrading puppetdb before puppet server should work I guess. Or do you mean there is a package level dependency on puppetserver 5?

I've always been running them on separate hosts, so a dependency on having them on the same host sounds annoying.

Eric Sorenson - er...@puppet.com 
director of product, ecosystem and platform

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/81F9258B-89EF-464C-9B1C-FDC677DE800C%40puppet.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/E7AB3172-D214-4F8A-A335-C6526D93A316%40devco.net.

[R.I.Pienaar]

ah yeah that's a good idea.

>
> According to documentation PuppetDB 5.x should be able to process
> commands from the PuppetDB 4.x terminus as well, so upgrading puppetdb before
> puppet server should work I guess. Or do you mean there is a package level
> dependency on puppetserver 5?

The doco for system requirements says:

Your site’s Puppet masters must be running Puppet Server 5.0.0 or later.
You will need to connect your Puppet masters to PuppetDB after
installing it. If you wish to use PuppetDB with standalone nodes that
are running puppet apply, every node must be running 5.0.0 or later.

[Thomas Müller]

Am Mittwoch, 28. Juni 2017 04:13:24 UTC+2 schrieb Eric Sorenson:

I have run out of superlatives to try to express how excited I am for this release: the Puppet 5 Platform is available for download now.

great news ! just finished upgrading to Puppet 4 with puppetserver which already dropped compile times from 20s to 4s. :) Looking forward to puppet 5 now.

[Peter Meier]

> Our current
> Ruby versions are EOL'ed, so we're moving to MRI Ruby 2.4 on the agent
> and (opt-in) jruby9k on the server.

So what's the current status on that opt-in?

I'm running into a nasty bug with Jruby 1.7 [1] and while I read the
configure guide [2] I'm still a bit unsure whether I should switch over
to 9k (which would fix my encoding issue) or I should try to work around
(which I mostly can ftm).

So anyone running it? What are the main issues with it? When are you hit
by the performance and memory leak issues that are mentioned in the
jruby github report and linked in documention [3].

Reading SERVER-1586 [4] & SERVER-1122 [5] and seeing both closed and not
any further relevant tickets in the backlog regarding 9k, I assume the
work on Puppet's side is mostly done and it's all about the upstream
jruby issues?

Also I assume I need to re-install all my gems for puppetserver that I
installed with jruby 1.7, right? The docs don't mention anything like that.

best

~pete

[1] https://github.com/jruby/jruby/issues/4767
[2]
https://docs.puppet.com/puppetserver/5.0/configuration.html#configuring-the-jruby-version
[3] https://github.com/jruby/jruby/issues/4112#issuecomment-242504130 and
[4] https://tickets.puppetlabs.com/browse/SERVER-1586
[5] https://tickets.puppetlabs.com/browse/

[Peter Meier]

On 08/31/2017 01:21 AM, Peter Meier wrote:
>> Our current
>> Ruby versions are EOL'ed, so we're moving to MRI Ruby 2.4 on the agent
>> and (opt-in) jruby9k on the server.
>
> So what's the current status on that opt-in?
>
> I'm running into a nasty bug with Jruby 1.7 [1] and while I read the
> configure guide [2] I'm still a bit unsure whether I should switch over
> to 9k (which would fix my encoding issue) or I should try to work around
> (which I mostly can ftm).
>
> So anyone running it? What are the main issues with it? When are you hit
> by the performance and memory leak issues that are mentioned in the
> jruby github report and linked in documention [3].
>
> Reading SERVER-1586 [4] & SERVER-1122 [5] and seeing both closed and not
> any further relevant tickets in the backlog regarding 9k, I assume the
> work on Puppet's side is mostly done and it's all about the upstream
> jruby issues?

So to report back after running it for >2 weeks: 0 problems so far and I
don't really feel a difference to 1.7

AND JRuby devs really urge you to move to 9k, 1.7 is a dead end:

https://github.com/jruby/jruby/issues/4767#issuecomment-327939620

So when will we officially ove to 9k?

> Also I assume I need to re-install all my gems for puppetserver that I
> installed with jruby 1.7, right? The docs don't mention anything like that.

I did do that, without testing whether it actually works or not.

best ~pete

[Deepak Giridharagopal]

On Mon, Sep 18, 2017 at 4:33 AM, Peter Meier <peter...@immerda.ch> wrote:

On 08/31/2017 01:21 AM, Peter Meier wrote:

So to report back after running it for >2 weeks: 0 problems so far and I
don't really feel a difference to 1.7

Thanks for trying it out! Is there a performance difference if you change server's "compile-mode" option? I'm mostly curious about running with JIT compilation on.

To other folks reading this: have you tried out 9k? Are your observations similar to Peter's?

 

AND JRuby devs really urge you to move to 9k, 1.7 is a dead end:

https://github.com/jruby/jruby/issues/4767#issuecomment-327939620

So when will we officially ove to 9k?

I don't think we'll make it the default until the next major, but we can definitely make it non-experimental in 5.x timeframe. 

deepak

[Peter Meier]

> So to report back after running it for >2 weeks: 0 problems so far and I
> don't really feel a difference to 1.7
>
>
> Thanks for trying it out! Is there a performance difference if you
> change server's "compile-mode" option? I'm mostly curious about running
> with JIT compilation on.
>
> To other folks reading this: have you tried out 9k? Are your
> observations similar to Peter's?

I was running too quickly into the UTF-8 problems to be able to make a
proper statement whether there is a performance difference. Sorry :/

But at least memory usage seems stable, hence likely no leaks
encountered yet.

best

~pete