Puppet MultiMaster open source

[Geoff Galitz]

Hi...

Is it possible to get compile master, master of master and code manager (file sync) working in the open source versions?  Specifically:

puppetserver-2.3.1-1.el7.noarch

puppetlabs-release-pc1-1.0.0-1.el7.noarch

Or are there pointers to get similar functionality via the open source packages?

-G

--

Geoff Galitz, Systems Engineer

Shutterstock GmbH

Greifswalder Strasse 212

Aufgang F, 2 Hof

10405 Berlin

[Christopher Wood]

Yes it is.

We have one bigip virtual server for the CA (with a single host behind it) and one virtual server per datacenter location with some compile-only masters. Each agent has both ca_server and server set in their configuration at build time.

We use the puppetserver and puppet-agent rpms from the PC1 repositories. Still haven't upgraded most of the agents to puppet4 though.

http://yum.puppetlabs.com/el/6/PC1
http://yum.puppetlabs.com/el/7/PC1

There's a cron job every minute to do an r10k run, r10k pulls from a local gitlab community edition. The git repositories themselves are backed up off site but that came with using a virtual machine for gitlab.

https://about.gitlab.com/features/

We have a custom r10k plugin (evolved from https://github.com/acidprime/r10k/blob/master/files/agent/r10k.ddl) which both deploys environments and checks them by git hash. The deployment part is a thin wrapper around the r10k command itself.

The question is not really if it's possible, but if the tradeoffs work for you r organization.

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [1]puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> [2]https://groups.google.com/d/msgid/puppet-users/CABiayTAACGByEk9kEzCddLdWHNkBzdhcLr%2Btg32ATCD4%3D9OruQ%40mail.gmail.com.
> For more options, visit [3]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:puppet-users...@googlegroups.com
> 2. https://groups.google.com/d/msgid/puppet-users/CABiayTAACGByEk9kEzCddLdWHNkBzdhcLr%2Btg32ATCD4%3D9OruQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> 3. https://groups.google.com/d/optout

[Kevin Corcoran]

On Wed, Apr 20, 2016 at 5:58 AM, Geoff Galitz <gga...@shutterstock.com> wrote:

Is it possible to get compile master, master of master and code manager (file sync) working in the open source versions?

Code manager and file sync are only available as part of Puppet Enterprise.  I believe the standard alternative is to run r10k directly on each master.

[Geoff Galitz]

Thanks for the replies... any pointers on setting up puppet servers as compile masters?

-G

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAF-bMBeJ8%3DJ3tP7uBP4C3sugoEr6hdhugHmkzHUN_g5oFTkvxQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Christopher Wood]

https://docs.puppet.com/puppetserver/2.2/external_ca_configuration.html#disabling-the-internal-puppet-ca-service

I use that line and have puppetized builds for puppetmasters anyway. Albeit that setting the Subject Alternative Name for the outside of the load balancer takes slightly more effort than signing a cert for a standard puppetmaster.

https://docs.puppet.com/guides/scaling_multiple_masters.html#before-running-puppet-agent-or-puppet-master

On Wed, Apr 20, 2016 at 07:50:04PM +0200, Geoff Galitz wrote:
> Thanks for the replies... any pointers on setting up puppet servers as
> compile masters?
> -G
> On Wed, Apr 20, 2016 at 5:12 PM, Kevin Corcoran
> <[1]kevin.c...@puppet.com> wrote:
>
> On Wed, Apr 20, 2016 at 5:58 AM, Geoff Galitz
> <[2]gga...@shutterstock.com> wrote:
>
> Is it possible to get compile master, master of master and code
> manager (file sync) working in the open source versions?
>
> Code manager and file sync are only available as part of Puppet
> Enterprise.  I believe the standard alternative is to run r10k directly
> on each master.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to [3]puppet-users...@googlegroups.com.

> To view this discussion on the web visit

> [4]https://groups.google.com/d/msgid/puppet-users/CAF-bMBeJ8%3DJ3tP7uBP4C3sugoEr6hdhugHmkzHUN_g5oFTkvxQ%40mail.gmail.com.
> For more options, visit [5]https://groups.google.com/d/optout.

>
> --
> Geoff Galitz, Systems Engineer
> Shutterstock GmbH
> Greifswalder Strasse 212
> Aufgang F, 2 Hof
> 10405 Berlin
>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to [6]puppet-users...@googlegroups.com.

> To view this discussion on the web visit

> [7]https://groups.google.com/d/msgid/puppet-users/CABiayTDyM%3DMzsbc1HAmKqzy12-eqmuxfH%2B-VjK_JxBNCLi%3DrDA%40mail.gmail.com.
> For more options, visit [8]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. mailto:kevin.c...@puppet.com
> 2. mailto:gga...@shutterstock.com
> 3. mailto:puppet-users...@googlegroups.com
> 4. https://groups.google.com/d/msgid/puppet-users/CAF-bMBeJ8%3DJ3tP7uBP4C3sugoEr6hdhugHmkzHUN_g5oFTkvxQ%40mail.gmail.com?utm_medium=email&utm_source=footer
> 5. https://groups.google.com/d/optout
> 6. mailto:puppet-users...@googlegroups.com
> 7. https://groups.google.com/d/msgid/puppet-users/CABiayTDyM%3DMzsbc1HAmKqzy12-eqmuxfH%2B-VjK_JxBNCLi%3DrDA%40mail.gmail.com?utm_medium=email&utm_source=footer
> 8. https://groups.google.com/d/optout

[Ryan Anderson]

This is possible, and it is not documented well. Different parts of getting it to work are in different sections of the online documentation. I got this working recently with these conditions:

• One puppet server is the CA master as well as a normal compile master
• Puppet masters exist at other data centers and are compile masters for agents at their sites, with all other masters/agents using the CA master for SSL
• Each puppet master has a cron to regularly do a 'git pull' of the puppet environment git repos to /etc/puppetlabs/code/environments so all agents use the same code. Recommendation: GitLab CE (free) is amazing.
• A separate server hosts puppetdb, and all masters send agent reports to it. To make this work, all agents/masters need to use the same CA

On the first puppetmaster that will also be the CA master:

• Install puppetserver
• In its /etc/puppetlabs/puppet/puppet.conf [main] section:
• server = <its FQDN>
• ca_server = <its FQDN>
• Startup puppetserver on it
• Put your code into /etc/puppetlabs/code/environments
• Configure it to be an agent to itself and ensure it successfully runs before proceeding

On other masters:

• Install puppetserver
• Setup the git repos into /etc/puppetlabs/code/environments as you did for the CA master
• Modify /etc/puppetlabs/puppetserver/bootstrap.cfg and follow the comment's instruction on disabling the CA service
• Modify /etc/puppetlabs/puppetserver/conf.d/webserver.conf per https://docs.puppet.com/puppetserver/2.2/external_ca_configuration.html
  
• In its /etc/puppetlabs/puppet/puppet.conf [main] section:
• server = <its FQDN>
• ca_server = <FQDN of CA master>
• In its [agent] section, make its 'server' be the CA master and set the correct environment
• Run the agent successfully (against the CA master), then make 'server' be its own FQDN, then run it successfully again

Configure all masters to send reports to puppetdb:

• Under [master], add:

• storeconfigs = true
  storeconfigs_backend = puppetdb
  reports = puppetdb

On agents:

• In the [main] section, have 'ca_server = <FQDN of CA master>'
• In the [agent] section, have 'server = <FQDN of the master for its site>'
• Send agent reports, in the [agent] section, add: report = true

On the separate PuppetDB server:

• Configure it as an agent and successfully run the puppet agent
• Install the excellent puppetlabs-puppetdb module from puppetforge into your environment(s)
• In your site.pp node section, configure your masters and puppetdb server as such with the module:

• # Puppet masters send reports to puppetdb server
      if $::hostname =~ /(camaster|master1|master2|master3)/ {
        class { 'puppetdb::master::config':
          puppetdb_server => 'puppetdbserver.example.com',
        }
      }
  
      # This is the puppetdb server
      if $::hostname == 'puppetdbserver' {
        class { 'puppetdb':
          listen_address        => '0.0.0.0',
          open_listen_port      => true,
          open_ssl_listen_port  => true,
          java_args => { '-Xmx' => '2g', },
        }
      }

• I highly recommend using puppetboard with puppetdb: https://github.com/voxpupuli/puppetboard

[Geoff Galitz]

Thanks for your replies.. much appreciated.

-G

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/592c18b2-b20a-44a2-90c8-86365a1ee542%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[sadusey]

Which version & edition of puppet u used for this.

I could find any bootstrap.cfg in puppet server.

Thanks,

y

[Walid]

for the code manager replacement we use r10k, nfs share, ssync, and gitlab-ce with webhooks. we stopped using mcollective

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/592c18b2-b20a-44a2-90c8-86365a1ee542%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Geoff Galitz, Systems Engineer

Shutterstock GmbH

Greifswalder Strasse 212

Aufgang F, 2 Hof

10405 Berlin

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CABiayTDeRB93btfVjf01VLoVLtmDpRHz8KpQJgwd82nMmrNnxw%40mail.gmail.com.