Mock Escalation Policy

215 views
Skip to first unread message

he...@andrewhowden.com

unread,
Mar 24, 2017, 6:17:23 AM3/24/17
to Prometheus Users
Hi All, 

Looking to mock an escalation policy in Alert manager. Has anyone done this before? Something like:

- Alert@1min in slack for relevant team
- Alert@3min in general slack
- Alert@5min via custom webhook (Twilio)

Cheers!

conor.b...@robustperception.io

unread,
Mar 24, 2017, 7:35:08 AM3/24/17
to Prometheus Users, he...@andrewhowden.com
Hi,

The timing features you're looking for here are out of scope for the Alertmanager.

What you could do is feed in the alerts coming from the Alertmanager into another service like PagerDuty or OpsGenie and from there set up the escalation policy you're after with webhooks to Slack and Twilio.

Conor

conor.b...@robustperception.io

unread,
Mar 24, 2017, 7:44:31 AM3/24/17
to Prometheus Users, he...@andrewhowden.com
Here are two potentially helpful articles for hooking up the Alertmanger with OpsGenie and PagerDuty



On Friday, 24 March 2017 10:17:23 UTC, he...@andrewhowden.com wrote:

Andrew Howden

unread,
Mar 24, 2017, 8:19:12 AM3/24/17
to conor.b...@robustperception.io, Prometheus Users

Hi Conor,

Thanks for your help! However, I was hoping to keep this within the scope of alert manager if possible, so as to remove the need for a third party service (beyond Twilio).

Cheers!

--
Andrew Howden
Careful, well crafted web development.
--
W: https://andrewhowden.com/
PGP: https://pgp.andrewhowden.com (79BAC08A6ED1FF1EABE350A7587D3B3A961D2D2D)
--

Julius Volz

unread,
Mar 24, 2017, 8:46:14 AM3/24/17
to Andrew Howden, conor.b...@robustperception.io, Prometheus Users
On Fri, Mar 24, 2017 at 1:19 PM, Andrew Howden <he...@andrewhowden.com> wrote:

Hi Conor,

Thanks for your help! However, I was hoping to keep this within the scope of alert manager if possible, so as to remove the need for a third party service (beyond Twilio).

Unfortunately, there is pretty wide (unanimous?) agreement within the Prometheus maintainers that that would be indeed out of scope for Alertmanager, sorry. There are related things out of scope for it as well: managing people and their contact info, their oncall schedules, service mappings, etc. All the things that e.g. PagerDuty would usually do for you, we don't want to replicate.

Cheers!


On Fri, 24 Mar 2017, 22:44 <conor.broderick@robustperception.io> wrote:
Here are two potentially helpful articles for hooking up the Alertmanger with OpsGenie and PagerDuty



On Friday, 24 March 2017 10:17:23 UTC, he...@andrewhowden.com wrote:
Hi All, 

Looking to mock an escalation policy in Alert manager. Has anyone done this before? Something like:

- Alert@1min in slack for relevant team
- Alert@3min in general slack
- Alert@5min via custom webhook (Twilio)

Cheers!
--
Andrew Howden
Careful, well crafted web development.
--
W: https://andrewhowden.com/
PGP: https://pgp.andrewhowden.com (79BAC08A6ED1FF1EABE350A7587D3B3A961D2D2D)
--

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CA%2BFfCg6tE5a%3DKJ%3DuphUC06%3Dk%3DvHTQQXLao6F_ZkwGJ-XOnuZ3w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Andrew Howden

unread,
Mar 24, 2017, 8:47:52 AM3/24/17
to Julius Volz, conor.b...@robustperception.io, Prometheus Users

Ah, perfect. Not a problem at all; putting certain things out of scope gives me somewhat more confidence in the project, rather than finding it annoying.

Thanks for getting back to me Julius!


To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

To post to this group, send email to promethe...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages