Alertmanager Labels with Opsgenie integration - Alert routing to multiple teams

[vikram yerneni]

Fellas,

Anyone setup Alertmanager with OpsGenie integration "by using labels within alerts" for Opsgenie to route the alerts to different teams based on the labels set in the alerts?

I am trying to read it on this section but I cannot find any sort of documentation online on how to achieve this configuration.

Any tips here fellas...

Thanks

Vikram 

[Sally Lehman]

You might be missing a couple keywords:

1. In your Alertmanager configuration you need to set up opsgenie as a receiver - https://prometheus.io/docs/alerting/configuration/#receiver. 

2. Then set a route to that receiver: https://prometheus.io/docs/alerting/configuration/#route, the 'match' configuration within route has a label

3. That allows you to match the label from an alert rule you set up inside the prometheus configuration: https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/#defining-alerting-rules

[vikram yerneni]

Thanks Sally for the reply here... Yes, I did setup the OpsGenie as a receiver in my configuration and added the route. I did added a dummy label in the "Alerting rules" (please see the test configuration I am using below):

alertmanagerFiles:

  alertmanager.yml:
    global:
      resolve_timeout: 1m
      opsgenie_api_key: APIKEY
      opsgenie_api_url: "https://api.opsgenie.com/"
    route:
      repeat_interval: 1h
      group_wait: 10s
      group_interval: 5m
      receiver: 'saas-ops'

    receivers:
    - name: 'saas-ops'
      opsgenie_configs:
        - api_key: "API KEY"
          api_url: "https://api.opsgenie.com/"
          responders:
            - name: "Vikram-test"
              type: team

## Prometheus server ConfigMap entries
##
serverFiles:

  ## Alerts configuration
  ## Ref: https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/
  alerting_rules.yml:
   groups:
     - name: AllInstances
       rules:
         - alert: InstanceDown1
           expr: up == 1
           for: 1m
           annotations:
             title: 'Instance {{ $labels.instance }} down'
             description: '{{ $labels.instance }} of job {{ $labels.job }} has been down for more than 1 minute.'
           labels:
             severity: 'critical'

However, I am not sure how the "labels" here in Prometheus gets tagged into as "tags" in OpsGenie? This part, I am mot sure.

Thanks

[vikram yerneni]

What I am trying to achieve from this is, have the alerts to get routed to different teams based on labels. In one of the GitHub issues, one of the user mentioned that its possible and I am trying to research on this piece.