Scaling of Puppet masters

[Peter Berghold]

Folks,

I am in the midst of creating a deployment guide for a client that I am working with for the use of Puppet in their environment. The company is big enough that even though the company as a whole uses Puppet already (it's even on the Puppet web site) the local teams I am working with have no knowledge of this.

One of the perennial concerns that has been raised is about scaleability.  How many clients can a single Puppet master handle using the following as a baseline:

processor0 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor1 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor2 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor3 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor4 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor5 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor6 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

processor7 => Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

Second:  is there a way to use a load balancer in front of a bank of Puppet masters and keep the reports in sync?  I am considering proposing Puppet Dashboard as an ENC.

That's all I can think of right now but I'll probably have more questions along these lines as I develop my document and strategy. 

--

Peter L. Berghold                       Salty....@gmail.com

http://blog.berghold.net

[Jerald Sheets]

On May 19, 2014, at 3:40 PM, Peter Berghold <salty....@gmail.com> wrote:

> Second: is there a way to use a load balancer in front of a bank of Puppet masters and keep the reports in sync?

PuppetDB eliminates this need for you, and you can scale at the PostGres layer instead. Have a peek at PuppetDB. It’s a default install with PE, and easy to install with OSS Puppet.

> I am considering proposing Puppet Dashboard as an ENC.

According to my communications with various folks doing this for a living, Foreman seems to be the favored ENC to recommend given that Dashboard is being deprecated. Not totally, as it’s been released to the community, but it’s not the “go-to” any more, necessarily.

—jms

[Cory Stoker]

I can weigh in a little on this...

As far as scalability of the puppet masters, it can be variable
depending on what kind of puppet code you are running. The largest
item that can ruin scalability is long catalog compilation time.
Another thing is run interval.

Current at my workplace we run a pretty comprehensive puppet code base
(100% provisioned and deployed via puppet) over 10,000+ puppet agents.
We service the nodes through an LVS load balancer pair and have 3
puppet masters with 8 cores running Nginx with Passenger. We have a 6
hour run interval and also perform ad hoc runs. We have had 0 issues
with this setup so far in regard to load however the puppet masters
are consistently busy on all cores. PuppetDB with Postgres is a must.

I know a lot of people are now turning to Hiera backends to do their
ENC as well as Dashboard and Foreman.

Thanks,
Cory