Slack Notification Problem

[Jeffrey Ollie]

I'm trying to set up Prometheus 1.0.2 and AlertManager 0.4.0 to send
alerts to Slack. My config is as follows, however I keep getting
"Notify attempt 11 failed: unexpected status code 404" in the logs.
I've double and triple checked the Slack webhook URL, and it's exactly
as it appears on the Slack website. Can anyone tell me what I'm doing
wrong?

global:
slack_api_url:
https://hooks.slack.com/services/XXXXXXXX/XXXXXXXXX/xxxxxxxxxxxxxxxxxxxxxxxx

route:
receiver: slack
group_wait: 30s
group_interval: 5m
repeat_interval: 1h

receivers:
- name: slack
slack_configs:
- channel: '#notifications'


--
Jeff Ollie
The majestik møøse is one of the mäni interesting furry animals in Sweden.

[Jeffrey Ollie]

I'm back trying to figure this out. I've quadruple-checked the Slack API url in the config and it's correct. A test using cURL from the command line works just fine:

$ curl -X POST -d 'payload={"text":"test"}' https://hooks.slack.com/services/XXXXXXXX/XXXXXXXXX/xxxxxxxxxxxxxxxxxxxxxxxx
ok$

It seems as if Slack notifications are working for other people, so I appear to be doing something wrong but for the life of me I can't figure out what...

[Fabian Reinartz]

Did you do it from the command line on the node Alertmanager is running on?

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeffrey Ollie]

Yes, did the command line from the alertmanager server as well as from another server, same result.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Fabian Reinartz]

It indeed looks all correct. Can you maybe tcpdump the request AM is making and check whether if differentiates from the one you do with curl?

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tristan Colgate]

If you are getting a 404 make sure you a posting to a channel that exists

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jeff Ollie
The majestik møøse is one of the mäni interesting furry animals in Sweden.

--

You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.

[Jeffrey Ollie]

DOH! Of course, the channel name on Slack was misspelled. It would be useful in the future if the error message that Slack returns with the 404 ("channel_not_found") was included in the log, I would have looked much sooner at the channel name if it had. I never noticed this before because all of the other integrations I've set up used the default channel assigned to the webhook, which of course was chosen from a list, rather than being typed in. I think another useful change to the slack notifcation would be the ability to use the default channel assigned to the webhook.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Jeff Ollie
The majestik møøse is one of the mäni interesting furry animals in Sweden.

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.