Puppet Server on public host/IP address?

[Matthias Pigulla]

Dear Puppet Users,

until now, I have been using Puppet in firewalled environments only, where agents were on the same trusted network as the server or connected through VPN tunnels.

Now there seem to be some good reaons for switching to a "perimeterless security" approach, which would mean to drop the VPN and put the Puppet Server on the public internet. In my special case, I could not even do any IP-based filtering.

I could not really find any good material or recommendations on this. Is this a discouraged/dangerous practice, or is it more common than I was assuming?

The basic approach of mutual, certificate-based authentication in Puppet seems to perfectly support this scenario, and comes with encryption built in. And yes, of course I would _not_ enable certificate autosigning.

Are there other risks to be aware of? Any recommendations on hardening the setup?

Maybe I am a bit sceptical because a component like Puppet Server has not received the scrutinity as e. g. an Apache or Ngnix webserver regarding potential attack surfaces and security issues. The sensitive information a compromised Puppet Server might leak cannot be ignored.

Would it make sense to place the Puppet Server behind a major webserver/proxy (Apache, Varnish etc.)? Would it be possible to reject all connections that do not provide client certificates and use some out-of-band process for signing new client certs?

Thank you for all suggestions!

-mp.

[David Sandilands]

Our support for TLS termination is messy and you need to use a reverse proxy in front of PE,  We have a couple of Patterns surrounding multi region solutions using proxies and compilers which may be helpful

https://puppet.com/docs/pe/2021.7/installing_compilers.html#multi-region-load-balancing
AWS Multi-region architectures for Puppet Enterprise
PE Multi-region reference architectures (puppet.com)

[Matthias Pigulla]

Thank you for these suggestions.

I might be missing something, but these patterns seem to link parts of the infrastructure at different locations/availablility zones through virtual private cloud links. I did not see if/how a Puppet Master is exposed to the public internet there?

Any suggestions for Open Source Puppet as well?

Thanks

Matthias

[David Sandilands]

Hi Matthias,

Yeah it was more as closest documentation I could find. I got some advice that the main thing people looked for when publically exposed was to prevent anyone being able to make CSR requests to the server which would be on the configured in /etc/puppetlabs/puppetserver/conf.d/auth.conf

       {
            "allow-unauthenticated": true,
            "match-request": {
                "method": [
                    "get",
                    "put"
                ],
                "path": "/puppet-ca/v1/certificate_request",
                "query-params": {},
                "type": "path"
            },
            "name": "puppetlabs csr",
            "sort-order": 500
        }

This should be disabled and something like  puppetlabs/puppet-agent-bootstrap (github.com) used to generate the certificates for communication. I'm afraid that I don't think we have much as we generally avoid having publically exposed servers.

Thanks

David

[Karsten Heymann]

Hi Matthias,

I considered this myself some time ago in a project which was later aborted, so here are some unfinished thoughts:

* You put all your trust into the cryptography implementation code of the puppetserver process, so that server should always be kept up to date, including the kernel.

* Definitely do not allow any external connections against the server holding the puppet CA private key. This means having a puppet CA and (one or more) servers providing environments to agents (we call them "compile masters") and only allow connections against the latter

* If you have internal and external clients and those two groups don't share the same catalogs, I would configure separate puppet compile masters for internal and external agents, no need to have internal configuration details on a system which is more likely to be hacked.

* Directly on the internet does not mean unprotected. You should at least set up fail2ban on the puppet port in order to protect you from CSR generation or other DOS attacks, better some kind of firewall that can analyze the characteristics of tls traffic and detect anomalies.

* Harden the puppet server as much as possible. Prepare for it to be hacked. What damage could an attacker do from that system? Configure selinux, only allow connections into the rest of your network that are absolutely needed. Push your puppet environments onto your servers, don't pull them from another system. Set up external logging that cannot be modified from the server itself.

* Consider geoip-based firewalling if your clients are not completely global.

HTH,

Karsten

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/5b48116f-4831-4244-a90b-c7dc89a5f3a3n%40googlegroups.com.