Question regarding templates

40 views
Skip to first unread message

Rob

unread,
Jul 20, 2021, 4:54:50 PM7/20/21
to Prometheus Users
All,

I am trying to configure Prometheus and the way it creates the alert that is provided to our interaction with pagerduty. 

For the Summary and Description I thought I could do something like this . 

# Prometheus DeadManSwitch is an always-firing alert. It's used as an end-to-end test of Prometheus through the Alertmanager.
    - alert: DeadMan Switch
      expr: vector(1)
      for: 5m
      labels:
        severity: critical
        team: xxx
      annotations:
        summary: "Prometheus AlertManager E2E dead man switch {{ $labels.severity }} {{ $labels.environment }} {{ $labels.customer }} triggered"
        description: "Prometheus DeadManSwitch is an always-firing alert. {{ $labels.severity }} {{ $labels.environment }} {{ $labels.customer }}"

however those labels I added don't seem to be picked up. what shows up in Pager duty

{ "client": "AlertManager", "client_url": "http://xxxxxxxxx", "description": "[FIRING:1] (DeadMan Switch xxx gcp dev false xxx monitoring/prometheus-kube-prometheus-prometheus critical acc)", "event_type": "trigger", "incident_key": "xxxxxxxx", "service_key": "xxxxxxxx", "details": { "firing": "Labels: - alertname = DeadMan Switch - customer = xxxx - datacenter = gcp - environment = dev - is_eu = false - platform = ems - prometheus = monitoring/prometheus-kube-prometheus-prometheus - severity = critical - team = xxx Annotations: - description = Prometheus DeadManSwitch is an always-firing alert. - summary = Prometheus AlertManager E2E dead man switch triggered Source: http://xxxxxx ", "num_firing": "1", "num_resolved": "0", "resolved": "" } }


Is there a way I can setup the template to when an trigger alerts I set certain to then use to create the alert? 

thanks in advance 

Reply all
Reply to author
Forward
0 new messages