Client self-deregistration from PuppetDB

[Matt Wise]

We boot up/shut-down 50-100 hosts a day on average... we're exploring PuppetDB, but I'm concerned about the model of just 'waiting' for hosts to be purged based on some checkin time. Is there any way to have our hosts send a signal through the puppet-masters (or directly to puppetdb?) to purge themselves when they're being terminated?

Matt Wise

Sr. Systems Architect

Nextdoor.com

[Ryan Anderson]

You could try a hack like having the system going away call a cgi on the puppet master (via wget) that in-turn does the 'puppet node deactivate' command.

[Daniele Sluijters]

There's no need for CGI magic through the Puppet Master, you can talk to PuppetDB directly. See the PuppetDB API documentation: https://docs.puppetlabs.com/puppetdb/latest/api/commands.html#examples-using-curl

[Martin Alfke]

Hi Matt,

On 09 Dec 2014, at 19:58, Matt Wise <ma...@nextdoor.com> wrote:

> We boot up/shut-down 50-100 hosts a day on average... we're exploring PuppetDB, but I'm concerned about the model of just 'waiting' for hosts to be purged based on some checkin time. Is there any way to have our hosts send a signal through the puppet-masters (or directly to puppetdb?) to purge themselves when they're being terminated?

You can use the puppetdb rest api:
https://docs.puppetlabs.com/puppetdb/2.2/api/index.html

In my actual project we disable hosts via VM management system using this API.
Works like a charm.

hth,

Martin

[Wyatt Alt]

To follow up on what others have said, the method described in the
command example

https://docs.puppetlabs.com/puppetdb/2.2/api/commands.html#examples-using-curl

will deactivate your node, which is different than what we call
"purging", whether or not you intended.

Deactivating a node will not purge it from the database, it will simply
mark it deactivated and cause it to be filtered from the output of most
queries. The node is actually purged from the database when it has been
deactivated for the length of time specified by the setting
node-purge-ttl, which is documented here:

https://docs.puppetlabs.com/puppetdb/latest/configure.html#node-purge-ttl

If you set this to 0m, nodes will be purged on the next garbage
collection (configurable by the gc-interval setting, 60 minutes by default.)

Wyatt

[Matt W]

Thanks... I got a few private responses as well that all seemed to be in-line with what I figured we needed to do. Its entirely reasonable for us to have our clients 'curl ...' out to some endpoint to remove themselves at shutdown time. The concern I have is that I'd like to keep our clients from being able to do any other damage to the PuppetDB database while they're at it. We obviously want to use the Puppet CertName Whitelist in PuppetDB so that only our Puppet servers can send reports/connect to PuppetDB, and none of our clients can.

So that said ... I think I may end up going the 'CGI script' route. We already have what we call a 'cert-api' endpoint on our Puppet servers that allows our puppet clients to re-up their SSL certs every 15 days (we expire them very quickly). Its not unreasonable to add functionality to this endpoint allowing a client to request that its own node be destroyed.

That said, I have one question. We don't match our puppet 'node_name' to our puppet 'cert_name's. That is, our certnames are real FQDNs ... but our node names are kind of a combination of an arbitrary node name (like "web_server") and the certname. They look something like this "web_proxy_thingy|my.fqdn.her.com". In an ideal world, I would be able to tell PuppetDB that the true identifier that I care about is the 'certname' not the 'nodename'. That said, I think in our case we're going to have to do some hackery to figure this out.

Thanks again for the suggestions though.

[Martijn]

Matt, I'd be very interested in that 'cert-api' endpoint code once you've had a chance to work on this. Is there a change you could open-source that? I think it would be very useful to the community, even if it is imperfect.

Hope you'll consider it,

Martijn

Op vrijdag 12 december 2014 18:18:47 UTC+1 schreef Matt W:

[Matt Wise]

Martijn,

   Sorry for the delay ... but yes, in the future we don't mind sharing this 'cert-api' code. Unfortunately today its not in a terribly share-able state. It was quite literally my 2nd python-program ever, written 3+ years ago, and written in a short-term hacky way because we naively believed that the PuppetLabs folks would ultimately solve the whole 're-signing certificates' problem (seriously ... 2011 ... http://projects.puppetlabs.com/issues/7272).

  At this point, we're in the midst of a full puppet-server-redesign, and part of that is going to include a ground-up fresh cert-api daemon. Its simple code, so we should get it done within a few days of beginning, but we just havn't quite started yet. When we do, though, it'll definitely be opensourced.

  The basic model is that we set our SSL certs to expire after 30 days. Our clients run a little cron job daily that says "is the cert expiring in the next 5 days?", and if that is true, it reaches out to our puppet masters and gets its cert renewed. We've been doing this for years now, with over 20,000 hosts (not simultaneously of course... just the number of hosts we've launched in 3 years), and had no problems with the model.

  We'll be adding some additional features to the API to support things like automatic node deregistration in PuppetDB as well.

Matt Wise

Sr. Systems Architect

Nextdoor.com

--
You received this message because you are subscribed to a topic in the Google Groups "Puppet Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/puppet-users/o-X54IznCD8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/2f343d00-13dd-451e-8b91-4ef0c18afcaa%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.