Hi,
We are running 2 node alertmanager cluster. Its working fine in all the cases except for couple of alerts and instead I see this in the logs.
Apr 04 16:00:51 prom0012 alertmanager[231282]: level=error ts=2022-04-``` ```04T20:00:51.937Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="techops/opsgenie[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 422: {\"message\":\"Request body is not processable. Please check the errors.\",\"errors\":{\"message\":\"Message can not be empty.\"},\"took\":0.001,\"requestId\":\"0ae8692e-9d18-4ea1-855d-c6d4d6a3bde7\"}"```
expr: sum by(instance, device) (increase(node_network_receive_errs_total{device=~"eth"}[1m])) > 0
for: 0s
Evaluation interval on prometheus is set to 2s
Group interval on alert manager is set to 0s.
I dont see any errors on Opsgenie.