[Puppet-Users] Puppet Platform 6 Update
Josh Cooper
I wanted to share some significant developments as we progress towards a Puppet Platform 6 release. I encourage you to try out nightly builds available in the puppet6 repos:
http://nightlies.puppet.com/yum/puppet6-nightly/
http://nightlies.puppet.com/apt/puppet6-nightly/
http://nightlies.puppet.com/downloads/{mac,windows}/puppet6-nightly/
1. Unvendoring Semantic Puppet
Previously, the puppet repo, puppet-agent and puppetserver vendored/packaged different versions of the semantic_puppet gem. We've untangled that mess so that in Platform 6:
* puppet has a runtime gem dependency on the semantic_puppet gem
* puppet-agent bundles the semantic_puppet 1.0.2 gem
* puppetserver no longer knows about puppet's transitive gem dependencies
* we can bump the semantic_puppet version in puppet-agent in the future without breaking puppetserver running on the same host. The same is true for other puppet runtime gem dependencies like fast_gettext and multi_json.
See https://tickets.puppetlabs.com/browse/PA-1880 for more details.
2. Puppet Platform 6 requires Ruby 2.3
Puppet Platform 6 requires Ruby 2.3 or up, so we can now use modern syntax such as keyword arguments, dig, squiggly heredocs, etc. Puppet will error when running on unsupported ruby versions such as 2.2, which went EOL on March 31, 2018.
Since puppetserver runs puppet code in a JRuby interpreter and JRuby 1.7 conforms to the 1.9.3 Ruby language, we first had to move puppetserver from JRuby 1.7 to 9K. In Platform 5, we made it possible to opt into using JRuby 9K. In Platform 6, we will drop JRuby 1.7 and only support JRuby 9.1.x.x, which conforms to Ruby 2.3.
To ensure puppet code does not break puppetserver/JRuby, we've started running puppet PRs against JRuby 9K in TravisCI.
See https://tickets.puppetlabs.com/browse/PUP-6893 and https://tickets.puppetlabs.com/browse/SERVER-2155 for more details.
3. Intermediate CA improvements
Currently, customers can set up Puppet to use an intermediate CA by manually generating and distributing certificates and keys, installing them in the proper locations on disk, for both the master and agent. This is time intensive, error prone, and even once these certs have been put in place, full validation using CRL chains was not possible.
For Puppet 6, we we are making both tooling and functionality improvements to this process. In this increment, we have implemented full validation with chained certificates and CRLs, and we have changed the agent-side SSL bootstrapping to automatically download these full chains from the master and store and use them appropriately. It is now no longer necessary for intermediate CA users to manually distribute SSL files to their agents. On the server side, we are working to create a puppetserver CLI for setting up and interacting with the CA. See https://tickets.puppetlabs.com/browse/SERVER-2171.
4. Server-stack containerization
We’ve been working primarily on the automation and tooling to improve building and shipping updated containers for the Puppet Platform server components (puppetserver, puppetdb, and r10k). The build tooling for these containers has moved into the individual project repos, and we’re getting very close to having containers that will auto-publish to dockerhub.
We also have a number of workflow improvements planned for running the server stack in a containerized environment. That work will be beginning in the near future.
See https://tickets.puppetlabs.com/browse/CPR-560 and https://tickets.puppetlabs.com/browse/CPR-592 for more details on the ongoing and upcoming work.
5. MCollective has been removed
For Puppet Enterprise users, we’ve already been recommending the new orchestrator for some time. Last summer, we introduced Bolt and Tasks. We feel these technologies solve most of the problems MCollective did, and are removing it from the puppet-agent so that we can focus on other engineering efforts.
While we’d obviously love to see everyone move to Tasks, if you depend strongly on MCollective then it is still maintained by R.I.Pienaar at https://choria.io.
6. Includes the Resource API
The Resource API provides a simple way to create new native resources in the form of types and providers for Puppet. Using a little bit of ruby, you can finally get rid of that brittle exec, or manage that one API that eluded you until now.
See https://github.com/puppetlabs/puppet-resource_api and https://github.com/puppetlabs/puppet-specifications for more details on how to use it.
Eric Sorenson
--
You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CA%2Bu97u%3D75niK-2BgNanor9p6tHmHOhV1p%2BepLGOvs0rNPaf0Sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
R.I.Pienaar
On Tue, 17 Jul 2018, at 02:40, Eric Sorenson wrote:
> Another effort that's underway but not yet complete is the extraction of
> non-core types/providers into modules. This addresses some long-standing
> requests to, for example, be able to change the nagios types and OS-
> specific resources without needing to get a full agent release out. The
> extracted types will be available in a modulepath structure in the
> puppet agent package, so (with a few targeted exceptions) there won't be
> any user-visible changes to what's available when you get the package,
> but an implication that hasn't really come up is around using Puppet in
> rubygem format. The extracted types are available on github and on the
> forge as separate modules, so if you currently use some of these
> extracted types, you'd need a way to get them installed locally.
>
> So my question is -
> - do you current use/rely on 'gem install puppet' for your workflows? If
> so, what do you do with it? (does anybody use a 'gem install puppet' as
> their production "puppet agent" daemon?)
with FPM but its the same outcome really. We need things in custom paths
and puppet-agent isn't relocatable so thats the path of least resistance.
Regardless we probably could not use puppet-agent even if relocatable as
different teams do different things
> - given the above, what would be the easiest/most intuitive way to get
> those extracted types into your puppet installation? some ideas we've
> kicked around are
> * a puppet type 'meta module' that, akin to a rpm/deb metapackage,
> doesn't have content, just dependencies on the actual modules at
> particular pinned versions that match the agent package versions
> * a Puppetfile that you could point r10k at to get the modules
> installed
> * individual gems for each of the extracted modules with Gemfile
> dependencies (note: this is a Bad Idea™)
Martin Alfke
> On 17. Jul 2018, at 01:40, Eric Sorenson <er...@puppet.com> wrote:
>
> Another effort that's underway but not yet complete is the extraction of non-core types/providers into modules. This addresses some long-standing requests to, for example, be able to change the nagios types and OS-specific resources without needing to get a full agent release out. The extracted types will be available in a modulepath structure in the puppet agent package, so (with a few targeted exceptions) there won't be any user-visible changes to what's available when you get the package, but an implication that hasn't really come up is around using Puppet in rubygem format. The extracted types are available on github and on the forge as separate modules, so if you currently use some of these extracted types, you'd need a way to get them installed locally.
>
> So my question is -
> - do you current use/rely on 'gem install puppet' for your workflows? If so, what do you do with it? (does anybody use a 'gem install puppet' as their production "puppet agent" daemon?)
> - given the above, what would be the easiest/most intuitive way to get those extracted types into your puppet installation? some ideas we've kicked around are
> * a puppet type 'meta module' that, akin to a rpm/deb metapackage, doesn't have content, just dependencies on the actual modules at particular pinned versions that match the agent package versions
> * a Puppetfile that you could point r10k at to get the modules installed
> * individual gems for each of the extracted modules with Gemfile dependencies (note: this is a Bad Idea™)
Eric Sorenson
On 17. Jul 2018, at 01:40, Eric Sorenson <er...@puppet.com> wrote:
So my question is -
- do you current use/rely on 'gem install puppet' for your workflows? If so, what do you do with it? (does anybody use a 'gem install puppet' as their production "puppet agent" daemon?)
We install puppet as a gem in CI/CD unit testing.
- given the above, what would be the easiest/most intuitive way to get those extracted types into your puppet installation? some ideas we've kicked around are
* a puppet type 'meta module' that, akin to a rpm/deb metapackage, doesn't have content, just dependencies on the actual modules at particular pinned versions that match the agent package versions
* a Puppetfile that you could point r10k at to get the modules installed
* individual gems for each of the extracted modules with Gemfile dependencies (note: this is a Bad Idea™)
We need at least a note how we have to add the module with the separated types/providers.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/8C2CE062-F810-42FA-BC0A-590B2FF3BB46%40gmail.com.
Martin Alfke
> On 18. Jul 2018, at 00:43, Eric Sorenson <er...@puppet.com> wrote:
>
> On Jul 17, 2018, at 2:26 AM, Martin Alfke <tux...@gmail.com> wrote:
>>
>>> On 17. Jul 2018, at 01:40, Eric Sorenson <er...@puppet.com> wrote:
>>>
>>> So my question is -
>>> - do you current use/rely on 'gem install puppet' for your workflows? If so, what do you do with it? (does anybody use a 'gem install puppet' as their production "puppet agent" daemon?)
>>
>> We install puppet as a gem in CI/CD unit testing.
>>
>
> Hi Martin! Does this use depend on types and providers in puppet's lib/ directory? Or is it just having the core puppet code available?
On acceptance testing (with beaker) we install the OS puppet-agent package.
Eric Sorenson
On Jul 16, 2018, at 10:52 PM, R.I.Pienaar <r...@devco.net> wrote:
On Tue, 17 Jul 2018, at 02:40, Eric Sorenson wrote:Another effort that's underway but not yet complete is the extraction of
non-core types/providers into modules. This addresses some long-standing
requests to, for example, be able to change the nagios types and OS-
specific resources without needing to get a full agent release out. The
extracted types will be available in a modulepath structure in the
puppet agent package, so (with a few targeted exceptions) there won't be
any user-visible changes to what's available when you get the package,
but an implication that hasn't really come up is around using Puppet in
rubygem format. The extracted types are available on github and on the
forge as separate modules, so if you currently use some of these
extracted types, you'd need a way to get them installed locally.
So my question is -
- do you current use/rely on 'gem install puppet' for your workflows? If
so, what do you do with it? (does anybody use a 'gem install puppet' as
their production "puppet agent" daemon?)
we use it to get apply on machines - actually we package the gem into a rpm
with FPM but its the same outcome really. We need things in custom paths
and puppet-agent isn't relocatable so thats the path of least resistance.
Regardless we probably could not use puppet-agent even if relocatable as
different teams do different things
- given the above, what would be the easiest/most intuitive way to get
those extracted types into your puppet installation? some ideas we've
kicked around are
* a puppet type 'meta module' that, akin to a rpm/deb metapackage,
doesn't have content, just dependencies on the actual modules at
particular pinned versions that match the agent package versions
sounds good, I do similar with Choria* a Puppetfile that you could point r10k at to get the modules
installed
handy* individual gems for each of the extracted modules with Gemfile
dependencies (note: this is a Bad Idea™)
yes probably a bad idea
R.I.Pienaar
But it would also probably be quite a surprise when `gem install` doesn't yield a working setup for the average person who does not go and read install docs or something, we had exactly this situation with the mcollective gem. You can install it but its kind of pointless since you have no connectivity plugins or security plugins - or anything that makes it work and this really caused a lot of unhappy users. I still don't know what would have been a better path and it seems to me this is more or less the same story
Josh Cooper
versioned based on the major puppet version, e.g.
https://puppet.com/puppet6/modules.html?
Another option could be to provide instructions for installing a
metamodule? Could be generic puppetlabs-core module, or one based on
platform? We basically have that already for
https://forge.puppet.com/puppetlabs/windows.
$ puppet module install puppetlabs-linux
$ puppet module install puppetlabs-windows
$ puppet module install puppetlabs-osx
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+...@googlegroups.com.