Multipe routes in alertmanager

33 views
Skip to first unread message

Thomas Rodriguez

unread,
Mar 17, 2021, 6:58:27 PM3/17/21
to Prometheus Users

Hello,

I want to send alert in a different way for a different time period.
For that I installed 2 alertmanager (alertmanager-ho and alertmanager-hno).
I put inhibitor for alertmanager-ho during night and inhibitor for alertmanager-hno during day.

If I put theses 2 alertmanager in cluster mode. What will hapend? How the alertmanager will be chosen?  Do we have an other way for sending I silence in 2 alertmanager ?

I am following the right way for what I want to do?

Thanks,
Thomas

Julien Pivotto

unread,
Mar 17, 2021, 7:02:51 PM3/17/21
to Thomas Rodriguez, Prometheus Users
Hello,

The best way to achieve what you want is to use continue: true in your
routing tree.

>
> Thanks,
> Thomas
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/e5e2d182-a715-4ce7-a488-5882a09c2244n%40googlegroups.com.


--
Julien Pivotto
@roidelapluie

Thomas Rodriguez

unread,
Mar 17, 2021, 7:11:36 PM3/17/21
to Thomas Rodriguez, Prometheus Users
I don't think it's possible because the same alert can be send to different  people  depending on the time. For that I use inhibition.

For example the alert "apache down" can be inhabited  on the first alert manager during day and the second during night. With the same route and continue the alert will never ring

Thomas

Julien Pivotto

unread,
Mar 17, 2021, 7:20:51 PM3/17/21
to Thomas Rodriguez, Prometheus Users
On 18 Mar 00:11, Thomas Rodriguez wrote:
> I don't think it's possible because the same alert can be send to
> different people depending on the time. For that I use inhibition.
>
> For example the alert "apache down" can be inhabited on the first alert
> manager during day and the second during night. With the same route and
> continue the alert will never ring
>
> Thomas

We have time-based silencing coming to alertmanager, hopefully later
this month.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAP8gRqEHLmFSmaz6DVUocvEXrin8-WppqbRZfmHfYa_5c7PYrQ%40mail.gmail.com.

--
Julien Pivotto
@roidelapluie

Thomas Rodriguez

unread,
Mar 18, 2021, 3:10:14 AM3/18/21
to Thomas Rodriguez, Prometheus Users
If I use time based silencing I have to create  the same alert 2 time for my different time period.  Actually  on my "apache  down" alert I have 2 labels.
sms_day : true => label inhibited during night on my second alertmanager
sms_night: true => label inhibited during day on my first alertmanager 

It's working very well but I have to silence  2 times the same alert

I guess  the only way is creating a script for silencing my 2 alertmanagers at the same time. 
Reply all
Reply to author
Forward
0 new messages