Remote site puppet infrastructure

[Jake - USPS]

I'm trying to figure out if there is a common/working setup that people are using to host puppet in remote sites.

At first our setup was that we have several puppetmasters (PM) in site A and a couple in our smaller site B.  In site A we are also hosting Foreman and our mysql DB for storeconfigs.  site A and B are connected via a WAN that is fairly slow.

With this setup the issue I ran into was a lot of puppet agent runs in site B would timeout.  They connect to their PM in site B just fine, but then the site B PM needs to connect to the storeconfigs DB in site A.  This connection usually takes too long.  So we end up with about 50% error rate in site B while having a 0% error rate in site A.

I've tried to address this by using async_storeconfigs which seemed to help a lot but ran into https://projects.puppetlabs.com/issues/12140 which was a dealbreaker.

Something else I've tried is just having agents in site B use the PMs in site A (so no more PM in site B).  This helps a lot, but we still get errors.  The 50% error rate drops down to about 1% error rate.  It seems the traffic from agent to PM is less then the traffic from PM to DB?  Not only did the error rate go down, but config retrievals sped up.

One thing I'm wondering is if there is a way to setup a mysql storeconfigs DB in site B to reduce latency.  Either only nodes in site B will use storeconfigs in site B or maybe site A and B are 'clustered' somehow over WAN?  I'm not very strong with mysql skills though so I'm not sure what all can be done with it to address the issues I'm facing ...

So I was wondering does anyone else manage remote sites?  Is there a way to do it while still maintaining a central storeconfigs DB and 'dashboard/ENC' (don't want to have 'login here to view these nodes, login there to view these other nodes)?

Thanks for all comments!

Jake

[Ohad Levy]

I think thats one of the known shortcoming of storeconfigs, and one of the main reasons I've never used them personally in the past.

What is your usage case fore storeconfigs? I always found ways to avoid using it (e.g. by storing the data i need somewhere else).

I had (job N-1) about 15 sites (each with PM's) working really nice, but without storeconfigs..

Ohad

Jake

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/Wb3Mae8Kd3YJ.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Jake - USPS]

Ohad,

Thanks very much for the reply.  Functionally we do not use storeconfigs (no exported resources).  We just like having all of the facts available to query/search against (CMDBish).  I guess since we use Foreman this means we really don't need to use storeconfigs to do what we want, but can instead push facts from our PMs to our Foreman DB which I see now after reading some more from the Foreman FAQ.

Thanks!

Jake