templating not working in alertmanager

[kapil kinha]

I am trying to use templating to define ops-genie priority. Here is my config

global:

  resolve_timeout: 5m

route:

  group_by: [alertname]

  group_wait: 20s

  group_interval: 30s

  repeat_interval: 3h

  receiver: 'genie_default'

receivers:

- name: 'genie_default'

  opsgenie_configs:

  - api_key: '5da5482c-d266-4b1a-b875-5d49d081e36c'

    teams: 'zPOC - TEST - OPS Team'

    #priority: '{{ range .Alerts }}{{ if eq .Labels.severity "critical" }}P1{{ end }}{{ if eq .Labels.severity "warning" }}P2{{ end }}{{ if eq .Labels.severity "moderate" }}P3{{ end }}{{ if eq .Labels.severity "low" }}P5{{ end }}{{end}}'

    #priority: '{{ range .Alerts }} {{ if eq .Labels.severity "critical" }}P1{{ else }}{{if eq .Labels.severity "warning" }}P2{{ end }}{{ end }} {{ end }}'

    priority: '{{range .Alerts}}{{if eq .Labels.severity "critical"}} P1 {{else}} P5 {{end}}{{end}}'

    send_resolved: true

I have tried three different templates but none of them is working

    priority: '{{ range .Alerts }}{{ if eq .Labels.severity "critical" }}P1{{ end }}{{ if eq .Labels.severity "warning" }}P2{{ end }}{{ if eq .Labels.severity "moderate" }}P3{{ end }}{{ if eq .Labels.severity "low" }}P5{{ end }}{{end}}'

    priority: '{{ range .Alerts }} {{ if eq .Labels.severity "critical" }}P1{{ else }}{{if eq .Labels.severity "warning" }}P2{{ end }}{{ end }} {{ end }}'

    priority: '{{range .Alerts}}{{if eq .Labels.severity "critical"}} P1 {{else}} P5 {{end}}{{end}}'

I get below error in alertmanager logs when i use templates

level=debug ts=2019-07-19T03:43:18.181361245Z caller=notify.go:632 component=dispatcher msg="Notify attempt failed" attempt=1 integration=opsgenie receiver=genie_default err="unexpected status code 422"

level=error ts=2019-07-19T03:43:18.181554006Z caller=notify.go:339 component=dispatcher msg="Error on notify" err="cancelling notify retry for \"opsgenie\" due to unrecoverable error: unexpected status code 422"

level=error ts=2019-07-19T03:43:18.18159778Z caller=dispatch.go:264 component=dispatcher msg="Notify for alerts failed" num_alerts=27 err="cancelling notify retry for \"opsgenie\" due to unrecoverable error: unexpected status code 422"

When I do not use templates, I am successfully able to send alerts to ops-genie.

Please advise.

[Simon Pasquier]

The logs show that there are 27 alerts in the group so the priority
field would end up like "P1P2P1...".
You need to add the severity label to the group_by clause and then use
".CommonLabels.severity".

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/766adf83-cc82-4aaf-97d4-0068409a1af5%40googlegroups.com.

[kapil kinha]

Thanks Simon!!
It worked