All,
I am trying to configure Prometheus and the way it creates the alert that is provided to our interaction with pagerduty.
For the Summary and Description I thought I could do something like this .
# Prometheus DeadManSwitch is an always-firing alert. It's used as an end-to-end test of Prometheus through the Alertmanager.
- alert: DeadMan Switch
expr: vector(1)
for: 5m
labels:
severity: critical
team: xxx
annotations:
summary: "Prometheus AlertManager E2E dead man switch {{ $labels.severity }} {{ $labels.environment }} {{ $labels.customer }} triggered"
description: "Prometheus DeadManSwitch is an always-firing alert. {{ $labels.severity }} {{ $labels.environment }} {{ $labels.customer }}"
however those labels I added don't seem to be picked up. what shows up in Pager duty
{
"client": "AlertManager",
"client_url": "
http://xxxxxxxxx",
"description": "[FIRING:1] (DeadMan Switch xxx gcp dev false xxx monitoring/prometheus-kube-prometheus-prometheus critical acc)",
"event_type": "trigger",
"incident_key": "xxxxxxxx",
"service_key": "xxxxxxxx",
"details": {
"firing": "Labels:
- alertname = DeadMan Switch
- customer = xxxx - datacenter = gcp
- environment = dev
- is_eu = false
- platform = ems
- prometheus = monitoring/prometheus-kube-prometheus-prometheus
- severity = critical
- team = xxx Annotations:
- description = Prometheus DeadManSwitch is an always-firing alert.
- summary = Prometheus AlertManager E2E dead man switch triggered
Source:
http://xxxxxx ",
"num_firing": "1",
"num_resolved": "0",
"resolved": ""
}
}
Is there a way I can setup the template to when an trigger alerts I set certain to then use to create the alert?
thanks in advance