Alertmanager:- mute_time_intervals help with routing

[pandrian]

Hey guys, Im trying to figure out how to send to mute notifications for a specific instance which is doing snapshots, so would like to avoid the spam notifications if possible. 

But I'm failing to understand how to do the routing for that specific instance to be muted on specific time intervals, anybody would be kind enough to guide me?

I tried the below but it seems its disregarding the matcher.

Thank you very much.

global:
      resolve_timeout: 30s
    route:
      group_by: [cluster, alertname, severity]
      group_wait: 1m
      group_interval: 5m
      repeat_interval: 30m
      receiver: 'slack-email-notifications'
      routes:
      - receiver: 'slack-email-notifications'
        matchers:
        - severity="warning"
      - receiver: 'slack-email-pagerduty-notifications'
        matchers:
        - severity="critical"
      - receiver: 'slack-email-notifications'
        matchers:
        - severity="warning"
        - instance_name="dr-00"
        mute_time_intervals:
        - 'dr-downtime-hours'
    receivers:
    - name: slack-email-notifications
      slack_configs:
      .......
      email_configs:

      ........

    - name: slack-email-pagerduty-notifications
      slack_configs:
      ..........
      email_configs:
      ..........
      pagerduty_configs:
       ......

    mute_time_intervals:
      - name: dr-downtime-hours
        time_intervals:
        - times:
          - start_time: '18:00'
            end_time: '18:30'

[Brian Candler]

The routes are tested in turn, and the first one which matches prevents any further rules from being examined (unless you set "continue: true" in which case multiple receivers may be activated)

Therefore, the more specific match

      - receiver: 'slack-email-notifications'
        matchers:
        - severity="warning"
        - instance_name="dr-00"

needs to be moved earlier in the routes, before

      - receiver: 'slack-email-notifications'
        matchers:
        - severity="warning"

which catches *everything* with severity="warning"

Incidentally, there's a tool for trying out your routes here:

https://prometheus.io/webtools/alerting/routing-tree-editor/

[pandrian]

Thanks a bunch Brian! 

Was not aware that the matching is sequential, all sorted now i think.