How to call my custom bash scripts in stackdriver monitoring and send alert based on script output

69 views
Skip to first unread message

Vipul Dixit

unread,
Jul 29, 2019, 2:19:18 PM7/29/19
to Google Stackdriver Discussion Forum
Need help here. I have written custom bash scripts to check if any process is running or not.
Please suggest the steps to use this in stackdriver monitoring and send alert based on script result.

Igor Peshansky

unread,
Jul 29, 2019, 2:31:53 PM7/29/19
to Vipul Dixit, Google Stackdriver Discussion Forum
Can you please elaborate on your use case? The Stackdriver monitoring agent will ingest process metrics by default once installed, so you should be able to set alerts on those (e.g., metric absence of a process with a specific command). Do you plan to use your script to send in a custom metric? How would it be different from what the monitoring agent does by default?
        Igor

On Mon, Jul 29, 2019 at 2:19 PM Vipul Dixit <vipul...@gmail.com> wrote:
Need help here. I have written custom bash scripts to check if any process is running or not.
Please suggest the steps to use this in stackdriver monitoring and send alert based on script result.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.
---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/4de025f5-542b-4457-aa7e-e1031ded4a8d%40googlegroups.com.

Rory Petty

unread,
Jul 29, 2019, 6:12:54 PM7/29/19
to Igor Peshansky, Vipul Dixit, Google Stackdriver Discussion Forum
You may not even need a custom script. As Igor mentioned, if you have the SD Monitoring agent installed, you can use "Process Health" alert conditions to check if certain processes are running:

Screen Shot 2019-07-29 at 6.12.20 PM.png

To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/CAN%3DUupc72fEEb%3DRAVkwV6n3bVH2t5KBsPWc9C4916w7UL9v-kQ%40mail.gmail.com.

Vipul Dixit

unread,
Jul 30, 2019, 12:46:36 AM7/30/19
to Google Stackdriver Discussion Forum
Ok Let me explain my use case in following bullet points -
1. Currently i am using nagios core to monitoring host/applications/components.
2. For default host resources like - cpu, disk, memory, total process i use nagios agent (nrpe client) and nagios provided binaries to monitor same
3. In addition to above we validate few application related specific services like  - to monitor count of specific files, to monitor specific folder mounted or unmounted, to monitor if any specific application logs is created or not. For this we written bash scripts - here is one example (to montior count of specific files in folder) ->

****************** snapshot of custom script written in nagios server ******************************
Filecount=`ssh -i /home/nagios/key.pem -o StrictHostKeyChecking=no ubuntu@$1 "sudo su aeris  -c 'ls /vol1/aaa/stop/emit | wc -l'"`

if [ $Filecount -le 500 ]
then
echo "Recordcount for $2 is less then or equal to 500.Current Recordcount is $Filecount"
exit 0
else
echo "Recordcount for $2 is  more than 500.Current usage is $Filecount"
exit 2
fi
****************************************xxxxx***********************************************************************************

4. Based on output if nagios server send alert.
5. Now we plan to migrate to stackdriver. So for default host resourced disk, cpu, memory, etc.. we know to configure monitoring and alert policy.
6. But now for custom scripts as i explained in step 3, how can we use these in stackdriver and configure alert policy. Is custom metrics a way, if yes, then need all steps to configure same. or anyother easy way. Also please tell me if bash scripts are supported in stackdriver or we need to use python.
Please help here.


On Tuesday, July 30, 2019 at 12:01:53 AM UTC+5:30, Igor Peshansky wrote:
Can you please elaborate on your use case? The Stackdriver monitoring agent will ingest process metrics by default once installed, so you should be able to set alerts on those (e.g., metric absence of a process with a specific command). Do you plan to use your script to send in a custom metric? How would it be different from what the monitoring agent does by default?
        Igor

On Mon, Jul 29, 2019 at 2:19 PM Vipul Dixit <vipul...@gmail.com> wrote:
Need help here. I have written custom bash scripts to check if any process is running or not.
Please suggest the steps to use this in stackdriver monitoring and send alert based on script result.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdriver-discu...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-discussion+unsub...@googlegroups.com.

Jorge González

unread,
Jul 30, 2019, 7:17:01 AM7/30/19
to Vipul Dixit, Google Stackdriver Discussion Forum
Hi,

I tried to do something very similar some time ago, using SD agent, and the help pages. The help pages are not very useful for this, got a promise of improvement though.
For a test, I ended up talking directly to the API and it was way simpler. 

Just in case it helps.

BR.

Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/70e0da4e-9f5a-4dba-bade-5a35e9360027%40googlegroups.com.


--
Jorge Gonzalez
DevOps Engineer | Endocode AG
jorge.g...@endocode.com


Endocode AG, Brückenstraße 5A, 10179 Berlin
+49 30 1206 4472 | in...@endocode.com | www.endocode.com

Vorstandsvorsitzender: Mirko Boehm
Vorstände: Lisa Nöth, Dr. Thomas Fricke, Sebastian Sucker
Aufsichtsratsvorsitzende: Alexandra Boehm

Registergericht: Amtsgericht Charlottenburg - HRB 150748 B

Rory Petty

unread,
Jul 30, 2019, 1:14:20 PM7/30/19
to Jorge González, Vipul Dixit, Google Stackdriver Discussion Forum
For your application logs, if you send them to Stackdriver Logging you could just use logs-based metrics directly.

Here are docs on writing custom metrics via the Monitoring agent:

You can also use OpenCensus or the SD python library to write custom metrics from your script (might be easier if it was Python vs. Bash, but still possible).



To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/CAFZNyuBs3wUYqY55ESBMV54ZdVeYZtbhpmXpxvvwAFY9joFNqA%40mail.gmail.com.

Jorge González

unread,
Jul 31, 2019, 7:58:47 AM7/31/19
to Rory Petty, Vipul Dixit, Google Stackdriver Discussion Forum
If you make it work with that documentation you'll be my hero.

BR.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/CAJnOKH0j29O%2BWc20N%2BNMt4UEv3cKRsKG0-eRzHccamG%2B9J28Eg%40mail.gmail.com.

Vipul Dixit

unread,
Aug 1, 2019, 7:19:35 AM8/1/19
to Google Stackdriver Discussion Forum
I will try this link and log-metrics as well. Hope i could succeed, will update. Meantime if anyone get any documented procedure step by step then please share.


On Wednesday, July 31, 2019 at 5:28:47 PM UTC+5:30, Jorge González wrote:
If you make it work with that documentation you'll be my hero.

BR.

Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdriver-discu...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-discussion+unsub...@googlegroups.com.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdriver-discu...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-discussion+unsub...@googlegroups.com.


--
Jorge Gonzalez
DevOps Engineer | Endocode AG


Endocode AG, Brückenstraße 5A, 10179 Berlin
+49 30 1206 4472 | in...@endocode.com | www.endocode.com

Vorstandsvorsitzender: Mirko Boehm
Vorstände: Lisa Nöth, Dr. Thomas Fricke, Sebastian Sucker
Aufsichtsratsvorsitzende: Alexandra Boehm

Registergericht: Amtsgericht Charlottenburg - HRB 150748 B

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdriver-discu...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-discussion+unsub...@googlegroups.com.

--
© 2016 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdriver-discu...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.

---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-discussion+unsub...@googlegroups.com.


--
Jorge Gonzalez
DevOps Engineer | Endocode AG
Reply all
Reply to author
Forward
0 new messages