Can Puppet alert about dead nodes?

[Sandra Schlichting]

Hi all =)

Can the puppet master tell me which nodes it haven't heard from in X days?

Hugs,

Sandra =)

[Juan Sierra Pons]

Hi,

If you have dashboard installed unresponsive nodes can be seen in the
left panel.

Hope it helps

Best regards
--------------------------------------------------------------------------------------
Juan Sierra Pons ju...@elsotanillo.net
Linux User Registered: #257202 http://www.elsotanillo.net
GPG key = 0xA110F4FE
Key Fingerprint = DF53 7415 0936 244E 9B00 6E66 E934 3406 A110 F4FE
--------------------------------------------------------------------------------------


2012/10/31 Sandra Schlichting <littles...@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/IJO5Osd3rVoJ.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.

[Sandra Schlichting]

If you have dashboard installed unresponsive nodes can be seen in the
left panel.

The reason I am interested in this information is that I would like to have the puppet master tricker Nagios to send an alert. =)

[Maxence Dunnewind]

Hi,

> The reason I am interested in this information is that I would like to have
> the puppet master tricker Nagios to send an alert. =)

You can do it with a nagios check in at least 2 ways :
- Check reported_at/updated_at fields in the nodes table of the dashboard
database.
- Check the state/last_run_summary.yaml date/content on each node

I use the 2nd solution with nagios checks to detect when a puppet run has
failed.

Maxence
--
Maxence Dunnewind

[Bernd Adamowicz]

I provided some posts about the health of Puppet agents some time ago. They might help too. The first one is found here: http://www.bernd-adamowicz.de/80/monitoring-puppet-part-1/. The others are linked.

Bernd

[C R Ritson]

My solution to this is a script that enumerates the signed keys that the master server knows about and compares this with which clients have been seen in the past 24 hours. There's probably also a way to do it with some of puppet's reporting facilities. My script is appended if it is of any use. You will need to customise file names and the email address.

#!/bin/bash

TMP=/tmp/awol.$$

PATH=/usr/local/bin:/usr/bin:/bin export PATH
TODAY=$(date '+%b %e')

puppet cert --list --all | cut '-d ' -f2 >${TMP}.1

grep "^${TODAY} .*: Compiled catalog for [^ ]* in environment production" /var/log/daemon.log \
| awk '{print $9}' \
| sort -uo ${TMP}.2

diff ${TMP}.1 ${TMP}.2 \
| grep '^[<>]' \
| tr '<>' '-+' >${TMP}.3

puppet cert --list | grep -v '^$' >${TMP}.4
if [ -s "${TMP}.4" ] ; then
cat - ${TMP}.4 >>${TMP}.3 <<-EOF

The following machines have certificates awaiting signature:-
EOF
fi

MISSING=$(wc -l <${TMP}.3)

if [ "${MISSING}" -gt 0 ] ; then
cat - ${TMP}.3 <<-EOF \
| mail -s "Some of our puppets are missing" pupp...@universtiy.ac.uk
There are some puppets missing (-) or interlopers (+).
EOF
fi

rm -f ${TMP}.?

Chris Ritson (Computing Officer and School Safety Officer)

[Sandra Schlichting]

I provided some posts about the health of Puppet agents some time ago. They might help too. The first one is found here: http://www.bernd-adamowicz.de/80/monitoring-puppet-part-1/. The others are linked.

That is exactly what I was looking for =)

Very interesting blog, btw. Lots of useful puppet stuff =)

 

[Sandra Schlichting]

My solution to this is a script that enumerates the signed keys that the master server knows about and compares this with which clients have been seen in the past 24 hours. There's probably also a way to do it with some of puppet's reporting facilities. My script is appended if it is of any use. You will need to customise file names and the email address.

Wow! This is also what I am looking for! Bernd posted a similar solution. I really like the nodes are not asked if they are dead.

Now I have two scripts to try =)

Thanks Bernd and Chris =)

[Lucas Vickers]

Interesting stuff.

I am currently using the open source edition so I have no dashboard.  The idea of an agent based log analysis makes sense, however checking each agent for logs will become tedious.  My obvious solution would be to write a simple client/server tool.  The agents check in on status and I keep track of the last time each agent sent in a heartbeat.

I'm writing this because I want to be sure I'm not missing an open source feature, or module someone wrote, that does a lot of this work already.  Have any of you heard of any?

thanks!

[Greg Sutcliffe]

On 1 November 2012 17:54, Lucas Vickers <lucasv...@gmail.com> wrote:

I am currently using the open source edition so I have no dashboard. 

Puppet Dashboard is opensource, so you could use it, but it's being deprecated (last I heard, anyway). You could also checkout The Foreman, which will display nodes out-of-sync (and email you about it, if you like).

Greg

[Martijn]

Op donderdag 1 november 2012 19:03:20 UTC+1 schreef Greg Sutcliffe het volgende:

On 1 November 2012 17:54, Lucas Vickers <lucasv...@gmail.com> wrote:

I am currently using the open source edition so I have no dashboard. 

Puppet Dashboard is opensource, so you could use it, but it's being deprecated (last I heard, anyway). You could also checkout The Foreman, which will display nodes out-of-sync (and email you about it, if you like).

We also use Puppet Dashboard for reporting, and it visually shows changed nodes and shows the nodes that have not reported recently. We use the open source Puppet stack and the Dashboard nicely complements it.

I've not heard anything about it being deprecated and have to assume you're mistaken since a quick Google could find no mention of it. Dashboard is still actively updated, and since it's the main interface to Puppet Enterprise as well I don't see it going away soon. The Foreman is also an impressive dashboard with more features than Puppet Dashboard but a bit more complexity as well.

Regards, Martijn Heemels

[Greg Sutcliffe]

This post to the list concerns PuppetLabs ceasing development on Dashboard - https://groups.google.com/forum/#!msg/puppet-users/j44EbTJY7HI/EQSJgh-g1fgJ

A few people have expressed interest in taking Dashboard on, but I've seen no announcement that this has officially happened. Perhaps I missed it though... I'd love to be corrected if so.

Greg

[Walter Heck]

The open source dashboard is being deprecated indeed, as announced on this list by puppet Labs last month.
The enterprise dashboard is a different story.

Over time there will be a replacement for the open source dashboard, but it might be a while and it might not be a one to one replacement.

Be advised :-)

sent from my mobile

--

You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/jkwbFc8FL6kJ.

[Martijn Heemels]

Thanks for the info, Greg and Walter. I missed that announcement. We can keep using the current Dashboard until something else comes along. I have no problems with the parts that we use.

The deprecation actually makes sense, now that I read it. I was never happy with the ENC functionality, which is why we're just using it for reporting. I'm glad Puppetlabs seems committed to provide some sort of replacement, in the form of more specialized tools for ENC and reporting, integrated with PuppetDB, which we're already using.

[Lucas Vickers]

I enabled the dashboard for my project.  We update 180 nodes in rapid succession.  I found that the delayed dashboard workers were slowing my computer down even more than the puppet workers themselves.  Very odd, but I did not put the time into seeing what the problem was.

Given my specific situation I'll probably just add a script at the end of the update that reports back to a server on my computer.  The server then tells me which nodes have not completed updates.  I guess the big question is now, how do I make that script dependent on a 100% successful puppet update.

L