Puppet 4 Patterns and Best Practices

[Lee Lowder]

All,

I recently upgraded my Puppet (open source) install to Puppet 4.3.0.

I'm currently in the process of updating internal documentation and training the rest of my teams, but I was wondering if anyone had any patterns or best practices that were specific to puppet 4? Specifically in cases where you no longer need to maintain compatibility with older versions.

Thanks.

 - Lee "FriedBob" Lowder

[Martin Alfke]

Hi Lee,

I did a talk regarding new functionality and deprecations in Puppet 4: https://puppetlabs.com/presentations/power-puppet-4
Besides this I encourage customers to at least make use of the type system.
For upgrading an existing platform I recommend to have a new Puppet 4 Master in place and make it the CA server (easy if you did the dns_alt_name configuration when you set up the first master).
Then switch nodes by nodes.
At my actual project we make use of the future parser setting in environment.conf for a special puppet 4 upgrade environment.

Best,
Martin

>
> Thanks.
>
> - Lee "FriedBob" Lowder
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c41cd42f-6158-4903-b187-0b24cc88155c%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Henrik Lindberg]

On 2015-09-12 10:19, Martin Alfke wrote:
> Hi Lee,
> On 08 Dec 2015, at 20:12, Lee Lowder <thefr...@gmail.com> wrote:
>
>> All,
>>
>> I recently upgraded my Puppet (open source) install to Puppet 4.3.0.
>>
>> I'm currently in the process of updating internal documentation and training the rest of my teams, but I was wondering if anyone had any patterns or best practices that were specific to puppet 4? Specifically in cases where you no longer need to maintain compatibility with older versions.
>
> I did a talk regarding new functionality and deprecations in Puppet 4: https://puppetlabs.com/presentations/power-puppet-4
> Besides this I encourage customers to at least make use of the type system.
> For upgrading an existing platform I recommend to have a new Puppet 4 Master in place and make it the CA server (easy if you did the dns_alt_name configuration when you set up the first master).
> Then switch nodes by nodes.
> At my actual project we make use of the future parser setting in environment.conf for a special puppet 4 upgrade environment.
>

Good advice from Martin. This inspired me to write a blog post, in short:

- use type system
- use EPP instead of ERB
- transition from using hira functions to using lookup (and next step to
use 'data in environments and modules').
- namespace all functions you create
- whenever possible write functions in puppet instead of ruby
- iterate and use regular resource syntax instead of create_resources
- do not iterate by creating resources recursively
- avoid depending on resource defaults expressions (murky area still)
- avoid depending on automatic tagging (murky area still)
- document your manifests using Puppet Strings yard compatible syntax
- avoid exposing temporary, "private" variables from classes (e.g. use
'with' function for creating temporary local scope).

Just some things from the top of my mind...

- henrik

--

Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/

[Christopher Wood]

(inline)

On Wed, Dec 09, 2015 at 09:19:13AM +0000, Martin Alfke wrote:
> Hi Lee,
> On 08 Dec 2015, at 20:12, Lee Lowder <thefr...@gmail.com> wrote:
>
> > All,
> >
> > I recently upgraded my Puppet (open source) install to Puppet 4.3.0.
> >
> > I'm currently in the process of updating internal documentation and training the rest of my teams, but I was wondering if anyone had any patterns or best practices that were specific to puppet 4? Specifically in cases where you no longer need to maintain compatibility with older versions.
>
> I did a talk regarding new functionality and deprecations in Puppet 4: https://puppetlabs.com/presentations/power-puppet-4
> Besides this I encourage customers to at least make use of the type system.
> For upgrading an existing platform I recommend to have a new Puppet 4 Master in place and make it the CA server (easy if you did the dns_alt_name configuration when you set up the first master).

How long does this generally take for medium sized deployments? I'm really wincing at the difficulty level of getting everybody in the company on the same page about moving hosts between CAs. If there's a fairly misconfiguration-proof way to go 3.8->future parser->4 without CA changes I'd rather do that instead.

> Then switch nodes by nodes.
> At my actual project we make use of the future parser setting in environment.conf for a special puppet 4 upgrade environment.
>
> Best,
> Martin
>
>
> >
> > Thanks.
> >
> > - Lee "FriedBob" Lowder
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c41cd42f-6158-4903-b187-0b24cc88155c%40googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/66326082-3541-4884-9CE0-B4FC3205827B%40gmail.com.

[Martin Alfke]

(inline)

On 09 Dec 2015, at 14:30, Christopher Wood <christop...@pobox.com> wrote:

> (inline)
>
> On Wed, Dec 09, 2015 at 09:19:13AM +0000, Martin Alfke wrote:
>> Hi Lee,
>> On 08 Dec 2015, at 20:12, Lee Lowder <thefr...@gmail.com> wrote:
>>
>>> All,
>>>
>>> I recently upgraded my Puppet (open source) install to Puppet 4.3.0.
>>>
>>> I'm currently in the process of updating internal documentation and training the rest of my teams, but I was wondering if anyone had any patterns or best practices that were specific to puppet 4? Specifically in cases where you no longer need to maintain compatibility with older versions.
>>
>> I did a talk regarding new functionality and deprecations in Puppet 4: https://puppetlabs.com/presentations/power-puppet-4
>> Besides this I encourage customers to at least make use of the type system.
>> For upgrading an existing platform I recommend to have a new Puppet 4 Master in place and make it the CA server (easy if you did the dns_alt_name configuration when you set up the first master).
>
> How long does this generally take for medium sized deployments? I'm really wincing at the difficulty level of getting everybody in the company on the same page about moving hosts between CAs. If there's a fairly misconfiguration-proof way to go 3.8->future parser->4 without CA changes I'd rather do that instead.

There is a puppet rollover module (need to goggle for it) which ensures agents get a certificate for a new puppet server and have the puppet master signing it.
The problem is mostly related to the CA where people missed the dns_alt_names configuration in the first place.
Without dns_alt_names the CA only has the fqdn of the puppet master inside.
But you might want to have another fqdn on the new puppet master.

Most easy way I can think of:
- new ca on new puppet master (think of dns_alt_names)
- enable autosign on new puppet master
- remove certificates form node, re-run puppet agent against new master

>
>> Then switch nodes by nodes.
>> At my actual project we make use of the future parser setting in environment.conf for a special puppet 4 upgrade environment.
>>
>> Best,
>> Martin
>>
>>
>>>
>>> Thanks.
>>>
>>> - Lee "FriedBob" Lowder
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/c41cd42f-6158-4903-b187-0b24cc88155c%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/66326082-3541-4884-9CE0-B4FC3205827B%40gmail.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/20151209143023.GA26225%40iniquitous.heresiarch.ca.

[Christopher Wood]

On Wed, Dec 09, 2015 at 03:04:31PM +0000, Martin Alfke wrote:
> (inline)
> On 09 Dec 2015, at 14:30, Christopher Wood <christop...@pobox.com> wrote:
>
> > (inline)
> >
> > On Wed, Dec 09, 2015 at 09:19:13AM +0000, Martin Alfke wrote:
> >> Hi Lee,
> >> On 08 Dec 2015, at 20:12, Lee Lowder <thefr...@gmail.com> wrote:
> >>
> >>> All,
> >>>
> >>> I recently upgraded my Puppet (open source) install to Puppet 4.3.0.
> >>>
> >>> I'm currently in the process of updating internal documentation and training the rest of my teams, but I was wondering if anyone had any patterns or best practices that were specific to puppet 4? Specifically in cases where you no longer need to maintain compatibility with older versions.
> >>
> >> I did a talk regarding new functionality and deprecations in Puppet 4: https://puppetlabs.com/presentations/power-puppet-4
> >> Besides this I encourage customers to at least make use of the type system.
> >> For upgrading an existing platform I recommend to have a new Puppet 4 Master in place and make it the CA server (easy if you did the dns_alt_name configuration when you set up the first master).
> >
> > How long does this generally take for medium sized deployments? I'm really wincing at the difficulty level of getting everybody in the company on the same page about moving hosts between CAs. If there's a fairly misconfiguration-proof way to go 3.8->future parser->4 without CA changes I'd rather do that instead.
>
> There is a puppet rollover module (need to goggle for it) which ensures agents get a certificate for a new puppet server and have the puppet master signing it.

I think it's this one:

https://github.com/reidmv/puppet-module-rekey

I added that to my internal puppet 4 upgrade task to look at "eventually", thank you.

> To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/F8E9963D-9C30-43AE-8CF5-A18B93022FA5%40gmail.com.

[Alessandro Franceschi]

Very interesting points, Henrik, could you elaborate a little about the few ones I quote below:

On Wednesday, December 9, 2015 at 1:38:48 PM UTC+1, Henrik Lindberg wrote:

- use EPP instead of ERB

Besides params validating and eventual affinity with Puppet language are there other (performance?) reasons for preferring epp templates?

 

- whenever possible write functions in puppet instead of ruby

Same as on epp templates

Thank you

al

[R.I.Pienaar]

hello,

----- Original Message -----
> From: "Henrik Lindberg" <henrik....@cloudsmith.com>

> - use type system
> - use EPP instead of ERB
> - transition from using hira functions to using lookup (and next step to
> use 'data in environments and modules').
> - namespace all functions you create
> - whenever possible write functions in puppet instead of ruby
> - iterate and use regular resource syntax instead of create_resources
> - do not iterate by creating resources recursively

What do you mean by this? avoid resource{[array,of,names]: } ?

> - avoid depending on resource defaults expressions (murky area still)

But the new syntax works yes?

file {
default:
mode => '0600',
owner => 'root',
group => 'root',
ensure => file,
'/etc/ssh_host_dsa_key.pub':
mode => '0644',

}

> - avoid depending on automatic tagging (murky area still)
> - document your manifests using Puppet Strings yard compatible syntax
> - avoid exposing temporary, "private" variables from classes (e.g. use
> 'with' function for creating temporary local scope).

good stuff

[R.I.Pienaar]

----- Original Message -----
> From: "R. I. Pienaar" <r...@devco.net>
> To: "puppet-users" <puppet...@googlegroups.com>
> Sent: Wednesday, December 16, 2015 2:18:49 PM
> Subject: Re: [Puppet Users] Puppet 4 Patterns and Best Practices

> hello,
>
> ----- Original Message -----
>> From: "Henrik Lindberg" <henrik....@cloudsmith.com>
>
>> - use type system
>> - use EPP instead of ERB
>> - transition from using hira functions to using lookup (and next step to
>> use 'data in environments and modules').
>> - namespace all functions you create
>> - whenever possible write functions in puppet instead of ruby
>> - iterate and use regular resource syntax instead of create_resources

Seeing this reminded me I wanted to blog about iteration in 3 and 4, see:

https://www.devco.net/archives/2015/12/16/iterating-in-puppet.php

[Henrik Lindberg]

On 2015-16-12 14:12, Alessandro Franceschi wrote:
> Very interesting points, Henrik, could you elaborate a little about the
> few ones I quote below:
>
> On Wednesday, December 9, 2015 at 1:38:48 PM UTC+1, Henrik Lindberg wrote:
>
> - use EPP instead of ERB
>
>

The main reason is that we in the long term are moving away from Ruby.
Calling Ruby templates will continue to work, but with a performance
penalty. I also imagine that open ended "see all variables in scope"
will not be supported at that point as the plan is to support logic in
Ruby over RPC to separate process and that requires serialization.

> Besides params validating and eventual affinity with Puppet language are
> there other (performance?) reasons for preferring epp templates?
>
> - whenever possible write functions in puppet instead of ruby
>
>
> Same as on epp templates
>

Same rationale as for EPP templates.
- henrik


> Thank you
> al

>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to puppet-users...@googlegroups.com

> <mailto:puppet-users...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/puppet-users/343744e7-6b3b-457f-a91e-23da9fb819e3%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/343744e7-6b3b-457f-a91e-23da9fb819e3%40googlegroups..com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

[Henrik Lindberg]

On 2015-16-12 14:18, R.I.Pienaar wrote:
> ----- Original Message -----
>> From: "Henrik Lindberg" <henrik....@cloudsmith.com>
>
>> - use type system
>> - use EPP instead of ERB
>> - transition from using hira functions to using lookup (and next step to
>> use 'data in environments and modules').
>> - namespace all functions you create
>> - whenever possible write functions in puppet instead of ruby
>> - iterate and use regular resource syntax instead of create_resources
>> - do not iterate by creating resources recursively
>
> What do you mean by this? avoid resource{[array,of,names]: } ?
>

Nah, that is fine. There is nothing to gain by turning that into an
iteration. I meant when it is done recursively. (The only way you could
do iteration in 3.x).

This area is not fully explored; there is no real reason why resource
evaluation is lazy, and if we are changing the style to be production
based (i.e. like Prolog) and allowing specification of the same resource
multiple times and allowing them to merge (with given precedence of
rules), I am not sure if the exact behavior as we have to today can be
retained (the lazy evaluation that ensures that one resource is
completely evaluated before the next (even when done recursively).

Also, it is harder for users to read and understand such code than
explicit iteration, it creates resources in the catalog that are only
there for the sake of iteration, etc.

>> - avoid depending on resource defaults expressions (murky area still)
>
> But the new syntax works yes?
>

Yes, that will continue to work for sure

> file {
> default:
> mode => '0600',
> owner => 'root',
> group => 'root',
> ensure => file,
> '/etc/ssh_host_dsa_key.pub':
> mode => '0644',
> }
>
>> - avoid depending on automatic tagging (murky area still)
>> - document your manifests using Puppet Strings yard compatible syntax
>> - avoid exposing temporary, "private" variables from classes (e.g. use
>> 'with' function for creating temporary local scope).
>
> good stuff
>

[Henrik Lindberg]

On 2015-16-12 14:58, R.I.Pienaar wrote:
>> ----- Original Message -----
>>> From: "Henrik Lindberg" <henrik....@cloudsmith.com>
>>
>>> - use type system
>>> - use EPP instead of ERB
>>> - transition from using hira functions to using lookup (and next step to
>>> use 'data in environments and modules').
>>> - namespace all functions you create
>>> - whenever possible write functions in puppet instead of ruby
>>> - iterate and use regular resource syntax instead of create_resources
>
> Seeing this reminded me I wanted to blog about iteration in 3 and 4, see:
>
> https://www.devco.net/archives/2015/12/16/iterating-in-puppet.php
>

That sums it up nicely.