Re: [Puppet Users] Pros and Cons of a Decentralized Puppet Architecture

[Gabriel Filion]

On 12-07-17 09:11 AM, JMeterX wrote:
> We have around 300 RHEL servers that are currently connecting to a
> Puppetmaster server. However, we have noticed some performance
> bottlenecks and it is the point of failure in our system. I am fairly
> new to puppet in general and I am considering creating a decentralized
> puppet architecture instead of having Puppet clients connect to the
> Puppetmaster server. Aside from what I would suspect to happen such as
> performance gain and lack of signing and exchanging SSL certs for new
> machines, what are other pros and cons to setting up a decentralized
> architecture?

here's an incomplete list that comes from the top of my head.

pros:

* you move the single point of failure security-wise to a repository
from which your nodes pull the manifests, which is probably easier to
secure (less moving parts).
* the setup scales much better than with a centralised puppet master.
the performance that you'll need to be verifying is wheter too much
nodes pull the manifests simultaneously. everything else is done by each
node.

cons:

* passing resources and files from one node to another (e.g. collecting
resources) is tougher. you pretty much have to reinvent a method of
doing it.
* you really have to trust users with root access on each node since
manifests are pulled directly on the server (and can thus be modified
locally).

--
Gabriel Filion

[R.I.Pienaar]

not really the case, you still have a single place where manifests live
that nodes pull the manifests from using git or whatever, thats still
a single point of attack.

the key exchange problem is generally harder since now you need to figure
out how to get git (or whatever) credentials on the node, often that means
a ssh private key and you need to either manage a per node public key on
the git server or share among your ndoes the same one thats tightly controlled
via say git-shell and force commands.

worse now you have all the code on all the nodes unless you invest heavily
into a setup that can output build artefacts of just the data relevant to
that node this is in most circumstances very undesirable.

> * the setup scales much better than with a centralised puppet
> master. the performance that you'll need to be verifying is wheter too much
> nodes pull the manifests simultaneously. everything else is done by
> each node.
>
> cons:
>
> * passing resources and files from one node to another (e.g.
> collecting resources) is tougher. you pretty much have to reinvent a method of
> doing it.

exported resources work, puppetdb will mostly work even in a masterless
environment, the 'mostly' bit is being worked on.

> * you really have to trust users with root access on each node since
> manifests are pulled directly on the server (and can thus be modified
> locally).

if they have root they can just amend the machine direct in ways you dont
manage, the problem of the code on the node is its much easier to determine
how to work around puppet and do things that wont be managed etc since its
all clear - similarly easy if u get your hands on a compiled catalog though.

Bigger problem is any kind of secrets, keys, certs etc for different hosts
that you might have in your code is shared.

A big con is also that 'apply' is very much a 2nd class citizen, there's a
number of things it cant do - write classes.txt, resources.txt etc, pluginsync
is broken etc. Some of these are slated for improvement in puppet 3 but mostly
its a pain.

That said, I run masterless now and have been much happier since, warts and all.