A little script to generate drbd monitoring with nagios .
Sp4rKY
http://groups.google.com/group/ganeti/browse_thread/thread/719c17ff03de3e46
), I've just wrote a little script to monitor drbd devices with
nagios. It is based on
http://www.monitoringexchange.org/attachment/download/Check-Plugins/Operating-Systems/Linux/check_drbd/check_drbd
and provided a simple way to generate the config for nagios (2 or 3) .
* install *
on all ganeti nodes, just install check_drbd (http://
www.monitoringexchange.org/attachment/download/Check-Plugins/Operating-Systems/Linux/check_drbd/check_drbd)
somewhere in the path.
on ganeti master, put this script somewhere in the path (/usr/local/
bin for ex)
Important : This script only define drbd checks, so you already need
to have the hosts configured in your nagios.
* use *
To generate the config, after a new instance has been added, run :
drbd_gen_nagios.sh > drbd.cfg
on ganeti master, then just copy the generated file into your nagios
conf.d directory, and check the config / restart your nagios.
The script :
#!/bin/bash
#
# Copyright (C) 2009 Maxence Dunnewind <max...@dunnewind.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/
>.
gnt-instance list -o name|grep -v Instance|while read vm;do
gnt-instance info -s $vm > /tmp/$vm
NODE_A=$(grep "nodeA" /tmp/$vm|grep -o "[^ ]\+host.gnt"|head
-1)
NODE_B=$(grep "nodeB" /tmp/$vm|grep -o "[^ ]\+host.gnt"|head
-1)
DEV_A=$(grep "nodeA" /tmp/$vm|cut -d "=" -f 2)
DEV_B=$(grep "nodeB" /tmp/$vm|cut -d "=" -f 2)
for disk in $DEV_A;do
echo "define service {"
echo " host_name $NODE_A"
echo " service_description DRBD instance $vm /
device n° $disk"
echo " check_command check_drbd!$disk"
echo " use generic-service"
echo "}"
echo ""
done
for disk in $DEV_B;do
echo "define service {"
echo " host_name $NODE_B"
echo " service_description DRBD instance $vm /
device n° $disk"
echo " check_command check_drbd!$disk"
echo " use generic-service"
echo "}"
echo ""
done
done
Maxence
Iustin Pop
> Hi, after the kvm plugin for munin (
> http://groups.google.com/group/ganeti/browse_thread/thread/719c17ff03de3e46
> ), I've just wrote a little script to monitor drbd devices with
> nagios. It is based on
> http://www.monitoringexchange.org/attachment/download/Check-Plugins/Operating-Systems/Linux/check_drbd/check_drbd
> and provided a simple way to generate the config for nagios (2 or 3) .
>
>
> * install *
> on all ganeti nodes, just install check_drbd (http://
> www.monitoringexchange.org/attachment/download/Check-Plugins/Operating-Systems/Linux/check_drbd/check_drbd)
> somewhere in the path.
>
> on ganeti master, put this script somewhere in the path (/usr/local/
> bin for ex)
>
> Important : This script only define drbd checks, so you already need
> to have the hosts configured in your nagios.
>
> * use *
>
> To generate the config, after a new instance has been added, run :
>
> drbd_gen_nagios.sh > drbd.cfg
just curious, does this means that a gnt-instance replace-disks will
need this to be re-run?
> on ganeti master, then just copy the generated file into your nagios
> conf.d directory, and check the config / restart your nagios.
Interesting. Two comments below:
> The script :
> …
> gnt-instance list -o name|grep -v Instance|while read vm;do
The list of instance names is available (since ganeti 2.0) in the file
/var/lib/ganeti/ssconf_instance_names, which will be faster (if you run
this often) than gnt-instance list. In any case, the gnt-instance info
is the slow one.
> gnt-instance info -s $vm > /tmp/$vm
/tmp/$vm is not a good file name for temporary files (symlink attacks,
etc.). I would suggest to use mktemp to generate a temporary file (e.g.
TMPF=`mktemp`; gnt-instance -info -s $vm > $TMPF, etc.).
thanks!
iustin
Michael Hanselmann
> gnt-instance list -o name|grep -v Instance|while read vm;do
Instead of “| grep -v Instance” you should use --no-header for
“gnt-instance info”. I also recommend you to use a separator,
otherwise you get unnecessary spaces before linebreaks.
$ gnt-instance info --no-header --separator=: -o name
Regards,
Michael
Sp4rKY
thanks for your answers.
> just curious, does this means that a gnt-instance replace-disks will
> need this to be re-run?
Yes because the host (and maybe drbd device) will change. Anyway, you
can just update the entry in previously generated file, without
regenerating the whole file.
> he list of instance names is available (since ganeti 2.0) in the file
> /var/lib/ganeti/ssconf_instance_names,
I thought to use this file, but, as you said, the time of gnt-instance
list is not really important when we do many gnt-instance info :)
Moreover, it implies that ganeti was installed with --localstatedir=/
var. I think gnt-instance is more generic.
> /tmp/$vm is not a good file name for temporary files (symlink attacks,
> etc.). I would suggest to use mktemp to generate a temporary file (e.g.
> TMPF=`mktemp`; gnt-instance -info -s $vm > $TMPF, etc.).
indeed, good tip, I updated the script.
> Instead of “| grep -v Instance” you should use --no-header for
> “gnt-instance info”. I also recommend you to use a separator,
> otherwise you get unnecessary spaces before linebreaks.
Thanks for these tips, updated :)
> $ gnt-instance info --no-header --separator=: -o name
gnt-instance list :)
I updated the "howto", with nagios config part I forgot :
====================================================
= install =
== on all ganeti nodes ==
just install check_drbd (http://www.monitoringexchange.org/attachment/
download/Check-Plugins/Operating-Systems/Linux/check_drbd/check_drbd)
somewhere in the path.
== on ganeti master ==
put this script somewhere in the path (/usr/local/bin for ex):
{{{
#!sh
#!/bin/bash
#
# Copyright (C) 2009 Maxence Dunnewind <max...@dunnewind.net>
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/
>.
gnt-instance list --no-header --separator=: -o name|while read vm;do
TMP_FILE=$(mktemp)
gnt-instance info -s $vm > $TMP_FILE
NODE_A=$(grep "nodeA" $TMP_FILE|grep -o "[^ ]\+,"|head -1|sed
's/,//')
NODE_B=$(grep "nodeB" $TMP_FILE|grep -o "[^ ]\+,"|head -1|sed
's/,//')
DEV_A=$(grep "nodeA" $TMP_FILE|cut -d "=" -f 2)
DEV_B=$(grep "nodeB" $TMP_FILE|cut -d "=" -f 2)
rm -rf $TMP_FILE
for disk in $DEV_A;do
echo "define service {"
echo " host_name $NODE_A"
echo " service_description DRBD instance $vm /
device n° $disk"
echo " check_command check_drbd!$disk"
echo " use generic-service"
echo "}"
echo ""
done
for disk in $DEV_B;do
echo "define service {"
echo " host_name $NODE_B"
echo " service_description DRBD instance $vm /
device n° $disk"
echo " check_command check_drbd!$disk"
echo " use generic-service"
echo "}"
echo ""
done
done
}}}
*Important* : This script only define drbd checks, so you already need
to have the hosts configured in your nagios.
== on nagios ==
Define check_drbd, which will use check_by_ssh to call check_drbd
script on the nodes.
{{{
define command{
command_name check_drbd
command_line /usr/lib/nagios/plugins/check_by_ssh -H
$HOSTADDRESS$ -l root -C "/usr/local/bin/check_drbd -d $ARG1$"
}
}}}
for example in /etc/nagiosX/conf.d/command.cfg
Also, be sure nagios user can connect using ssh without password.
= Use =
To generate the config, after a new instance has been added, run :
{{{
drbd_gen_nagios.sh > drbd.cfg
}}}
on ganeti master, then just copy the generated file into your nagios
conf.d directory, and check the config / restart your nagios.
====================================================
Maxence
Iustin Pop
> > The list of instance names is available (since ganeti 2.0) in the file
> > /var/lib/ganeti/ssconf_instance_names,
>
> I thought to use this file, but, as you said, the time of gnt-instance
> list is not really important when we do many gnt-instance info :)
> Moreover, it implies that ganeti was installed with --localstatedir=/
> var. I think gnt-instance is more generic.
Ah, good point, didn't think of that. Hmm, this gives me the idea of
adding the ssconf file path to the output of gnt-cluster info or
--version or something like that.
iustin