Alertmanager: resolved message received immediately after warning message even resolve_timeout is 5m

21 views
Skip to first unread message

leiwa...@gmail.com

unread,
Aug 5, 2020, 4:21:59 AM8/5/20
to Prometheus Users
rules.yml
groups:
- name: network-delay
  rules:
  - alert: "network delay"
    expr: probe_duration_seconds * 1000 > 3000
    for: 1s
    labels:
      severity: warning
      team: ops
    annotations:
      description: "{{$labels.instance}} : {{ $value }}"

alertmanager resolve_timeout is 5m

So after a warning is fired, in my opinion at least 5 mins later the resolved messge can be sent. But the resolved msg is sent immediately after the warning msg. why?

Thanks,
Lei

Christian Hoffmann

unread,
Aug 5, 2020, 6:31:20 AM8/5/20
to leiwa...@gmail.com, Prometheus Users
resolve_timeout is only relevant in very rare cases, at least when
Alertmanager is used together with Prometheus.

What's your reason for asking? Are you maybe looking for group_wait and
group_interval instead?

Kind regards,
Christian

leiwa...@gmail.com

unread,
Aug 5, 2020, 6:44:18 AM8/5/20
to Prometheus Users
Yes,my AlertManager is used together with Prometheus. 
In prometheus.yml: 

alerting:
  alertmanagers:
  - static_configs:
    - targets: ["x.x.x.x:9093"]


For all the warning message i received,  then immediately  a resolved message received.  This is not i want. 

Thanks,
Lei
Reply all
Reply to author
Forward
0 new messages