Alertmanager's wierd behaviour - Firing -> Resolved -> Firing -> Resolved.

yagyans...@gmail.com

unread,

Jan 19, 2021, 2:44:58 PM1/19/21

to Prometheus Users

Hi. I am using Prometheus version 2.12.0 and a HA Alertmanager setup with Alertmanager version 0.21.0. Problem is whenever I receive a resolved notification, I receive an extra Firing + Resolved notification pair for every alert.

For example, if an Alert A1 is firing, when this alert resolves I'll receive 3 notifications - Resolved, Firing, Resolved i.e for every resolved alert I am receiving an extra pair of (Firing,Resolved) notification. And these 3 notifications(Actual resolved + Extra Pair) arrive at exactly the same time.

Attaching the snapshot for one such alert.

Please help.

Thanks in advance!

ezgif.com-gif-maker.png

yagyans...@gmail.com

unread,

Jan 22, 2021, 7:34:50 AM1/22/21

to Prometheus Users

Can someone please help!

Yingtao Wang

unread,

Jan 28, 2021, 4:23:02 AM1/28/21

to Prometheus Users

may be post your config yaml?

I think this may be a issue related to your alert group setting

Nolan Crooks

unread,

Mar 23, 2021, 6:06:30 PM3/23/21

to Prometheus Users

Are you using the --cluster.peer flag on your Alertmanagers?

Reply all

Reply to author

Forward