Alertmanager is not accepting labels containing a "-"

871 views
Skip to first unread message

maximi...@gmail.com

unread,
Feb 15, 2018, 7:54:19 AM2/15/18
to Prometheus Users
Hi there,

i used to have labels that contained a "-". But once a alert fired and was send to Alertmanager I got the following Alertmanager error:

level=debug ts=2018-02-15T12:33:37.664709239Z caller=api.go:791 msg="Decoding request failed" err="\"service-environment\" is not a valid label name"
level=error ts=2018-02-15T12:33:37.664824413Z caller=api.go:780 msg="API error" err="bad_data: \"service-environment\" is not a valid label name"
level=debug ts=2018-02-15T12:33:37.678279848Z caller=api.go:791 msg="Decoding request failed" err="\"service-nodeexporter\" is not a valid label name"
level=error ts=2018-02-15T12:33:37.678361568Z caller=api.go:780 msg="API error" err="bad_data: \"service-nodeexporter\" is not a valid label name"

I replaced the "-" with a "_" now. Looks like a bug to me. Because prometheus itself had no problem with these labels. 

Anyone knows about that?

Max


Brian Brazil

unread,
Feb 15, 2018, 7:57:57 AM2/15/18
to maximi...@gmail.com, Prometheus Users
service-nodeexporter is not a valid labelname, it is however a valid label value.


--

maximi...@gmail.com

unread,
Feb 15, 2018, 8:02:45 AM2/15/18
to Prometheus Users
Thanks for your reply. Strange is that Prometheus seemed to have accepted these label names...

mtanatwine

unread,
Feb 15, 2018, 8:24:47 AM2/15/18
to Prometheus Users
I believe it's converted to an underscore in Prometheus. I discovered the same thing when using dashes in an ec2 tag - it was converted to an underscore before being registered as a tag.

maximi...@gmail.com

unread,
Feb 15, 2018, 8:55:32 AM2/15/18
to Prometheus Users

can't confirm that


Brian Brazil

unread,
Feb 15, 2018, 9:07:41 AM2/15/18
to maximi...@gmail.com, Prometheus Users
On 15 February 2018 at 13:55, <maximi...@gmail.com> wrote:

can't confirm that



What version of Prometheus is this, and where are you seeing that?

Brian
 

Am Donnerstag, 15. Februar 2018 14:24:47 UTC+1 schrieb mtanatwine:
I believe it's converted to an underscore in Prometheus. I discovered the same thing when using dashes in an ec2 tag - it was converted to an underscore before being registered as a tag.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/68a5fe10-6e92-426b-8af1-834f938c83e4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

maximi...@gmail.com

unread,
Feb 15, 2018, 9:12:13 AM2/15/18
to Prometheus Users
v2.0.0 running in the official docker container. The screenshot was taken in the graph tab, i believe. (Don't be confused - the colors are inverted)


Am Donnerstag, 15. Februar 2018 15:07:41 UTC+1 schrieb Brian Brazil:
On 15 February 2018 at 13:55, <maximi...@gmail.com> wrote:

can't confirm that



What version of Prometheus is this, and where are you seeing that?

Brian
 

Am Donnerstag, 15. Februar 2018 14:24:47 UTC+1 schrieb mtanatwine:
I believe it's converted to an underscore in Prometheus. I discovered the same thing when using dashes in an ec2 tag - it was converted to an underscore before being registered as a tag.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.



--

Brian Brazil

unread,
Feb 15, 2018, 9:31:49 AM2/15/18
to maximi...@gmail.com, Prometheus Users
What does the /targets page show for that target?

Brian

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/36e001bd-29d1-4aab-8935-437159b79b52%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Brian Brazil

unread,
Feb 15, 2018, 9:32:29 AM2/15/18
to maximi...@gmail.com, Prometheus Users
Also, can you share your prometheus.yml? This is a bug.

Brian
--

maximi...@gmail.com

unread,
Feb 19, 2018, 11:15:31 AM2/19/18
to Prometheus Users
see my answer in the github issue



--



--
Reply all
Reply to author
Forward
0 new messages