Alertmanager route match on mute_time_interval

420 views
Skip to first unread message

xkilian

unread,
Jun 29, 2021, 2:14:42 PM6/29/21
to Prometheus Users

Can a route use a mute_time_interval as a route matcher criteria?

  routes:

  # Select receiver based on time of day
    - match:
          severity: 5
      receiver: daygroup
      routes:
          - match:
              mute_time_interval: offhours
              receiver: offhoursgroup


(Not sure the indentation is perfect)

Thank you,

xkilian

xkilian

unread,
Jun 29, 2021, 3:42:01 PM6/29/21
to Prometheus Users
Ok, I will answer myself.

The way to do this is with continue, continue will permit firing to more than one receiver. I had mistakenly thought that continue would simply continue without firing at first match. Alertmanager is deceptively powerful. The only missing piece is a piece of UI to manage the list of pager numbers for on-call rotation that can now be defined in the alertmanager (offhoursgroup).

# Select receiver based on time of day
    - match:
          severity: 5
      receiver: daygroup
      mute_time_interval: offhours
      continue
    - match:
          severity: 5
      receiver: offhoursgroup   
      mute_time_interval: workhours

Then I just need to manage/automate who is in the offhoursgroup based on the rotation schedule.

This would work for my purposes.

Benjamin Ridley

unread,
Jun 29, 2021, 10:02:21 PM6/29/21
to Prometheus Users
Glad you managed to resolve your issue xkilian!

Just for some context, the mute_time_intervals mechanism is entirely separate to routes and route matching. All it does is 'silence' the route from sending out alerts in the specified periods.

The pattern you've discovered of using multiple receivers with the continue option is, I believe, the simplest way to achieve what you wanted.

Cheers,
Ben

ihor.pi...@gmail.com

unread,
Jul 21, 2021, 9:28:21 AM7/21/21
to Prometheus Users
That's a wonderful find,  but still it is very much difficult to do a proper routing when you need to put f.x. watch schedule  for 3 people to alert manager. Slightly more complex if not a binary choice...  
Reply all
Reply to author
Forward
0 new messages