HA puppetmaster in AWS

[Dejan Golja]

Hello guys,

so puppet community I seek some guidance. I am rebuilding our company Puppet 3 AWS infrastructure from scratch. Right now the design is to have a multiple availability ELB balancer and behind 2-4 puppet masters, so in case one AZ fails we still have a running puppet environment and also at the same time we can distribute the load.

For module/environment synchronization we are using  r10 + mcollective + post-commit git hooks, however we have one main issue and that is how to properly share the /var/lib/puppet/ssl folder. The thing is with Amazon ELB you have limited control over the load balance policy, so we need to make sure that SSL certs are in sync all the time.

We tried with yas3fs, but we abandoned that solution because was just not reliable enough. Also we considered GlusterFS, but again on some other projects the experience wasn't great.

So my question is how you guys manage that ?

I know we could run an external PuppetCA, however we would still need to share the SSL certs and for as is really important that we have the HA between different zones.

So any experience to share ?

regards,
Dejan

[Ankush Grover]

Hi Dejan,

You can try using S3 for this purpose.  So keep all the data like SSL or CA on S3 and ask all the puppet masters to pickup the ssl or any other data from S3.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/efbc0980-c0a0-44e4-a1eb-ac0743a2b5b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dejan Golja]

yas3fs is using s3 as a backend, but unfortunately did not work out.

[Juan Sierra Pons]

Hi

What about this approach? [1] Sync Puppet Certs between EC2 regions

It seems very easy to implement: unison + incron + scripts

Disclaimer: not tested yet. Hope to have a prof of concept next week.

Best regards

[1] http://blog.mague.com/?p=468

--------------------------------------------------------------------------------------
Juan Sierra Pons ju...@elsotanillo.net
Linux User Registered: #257202
Web: http://www.elsotanillo.net Git: http://www.github.com/juasiepo
GPG key = 0xA110F4FE
Key Fingerprint = DF53 7415 0936 244E 9B00 6E66 E934 3406 A110 F4FE
--------------------------------------------------------------------------------------

[Dejan Golja]

Not sure if it would work always, because if using unison you can get conflicts on files such as serial, inventory.txt ,ca_crl.pem, etc and then you need to merge them manually.

Quoting:

Unlike simple mirroring or backup utilities, Unison can deal with updates to both replicas of a distributed directory structure. Updates that do not conflict are propagated automatically. Conflicting updates are detected and displayed.

[Matt Zagrabelny]

On Fri, Jul 18, 2014 at 1:37 AM, Dejan Golja <dejan...@gmail.com> wrote:

> We tried with yas3fs, but we abandoned that solution because was just not
> reliable enough. Also we considered GlusterFS, but again on some other
> projects the experience wasn't great.
>
> So my question is how you guys manage that ?

DRBD?

-mz

[chris mague]

I used this approach detailed below in production for a large-ish environment.

When used in conjunction with Nginx load balancing in AWS (http://blog.mague.com/?p=286) it worked very well.

1) Route all certificate requests (explained above) to a pair of boxes that have the sync setup listed below

2) Route specific environments to specific puppet backends

One further refinement is to set up a puppet master running in debug mode and create a debug environment which is useful for troubleshooting.

-c

[Sher Chowdhury]

Hi Dejan,

Would you be able describe your setup for trying to get yas3fs to work. In particular what settings did you specify in your puppetmaster's conf file. I am interested in having a go at this as well. I'll feedback details of my attempts too. 

Kind regards,

Sher

[Dejan Golja]

Hey Sher,

at the end I didn't use yas3fs, because was not stable enough. Also since then a lot of things changed in the puppet land. Look for master compilers, masters of masters approach.