I have a prometheus-alertmanager-Opsgenie and prometheus-alertmanager-Slack integrations . Both are working correctly but I am missing labels in both, but in opsgenie i get an elaborated section called labels and i will be able to find which is the cluster and instances etc, but in slack those labels are missing, Anyone have idea how can i tackle this situation?
Below is my sample alert and attaching the alerts received via opsgenie and directly to slack.
- alert: KubePodCrashLooping for pod {{`{{`}} $labels.pod {{`}}`}} in {{ .Values.alertmanager.clusterName }} cluster
annotations:
message: Pod {{`{{`}} $labels.namespace {{`}}`}}/{{`{{`}} $labels.pod {{`}}`}} ({{`{{`}} $labels.container {{`}}`}}) is restarting {{`{{`}} printf "%.2f" $value {{`}}`}} times / 5 minutes.
expr: max_over_time(kube_pod_container_status_waiting_reason{reason="CrashLoopBackOff", job="kube-state-metrics"}[5m]) >= 1
for: 1h
labels:
severity: critical
ClusterName: {{ .Values.alertmanager.clusterName }}
resource: "{{`{{`}} $labels.namespace {{`}}`}}/{{`{{`}} $labels.pod {{`}}`}}"