Prometheus Newbie questions on notification data GroupLabels, CommonLabels and CommonAnnotations

1,327 views
Skip to first unread message

Stephen Eaton

unread,
Nov 27, 2017, 11:48:53 AM11/27/17
to Prometheus Users
Hi all,

As per the subject line I'm a prometheus newb for just over a week and trying to wrap my head around the alerting and notifications, specifically how I can use notification data: GroupLabels, CommonLabels, CommonAnnotation as per https://prometheus.io/docs/alerting/notifications/#data-structures

Where do these different KV pairs get defined?

Are there already standard KV pairs?  if so where can I see what these are? 

or do I have to set these?  

Is there documentation somewhere outlining best practice around this?

I feel that I'm missing something simple and when its explained to me the Doh! light will go on.

regards,

Stephen.

Brian Brazil

unread,
Nov 28, 2017, 7:38:23 AM11/28/17
to Stephen Eaton, Prometheus Users
On 27 November 2017 at 16:48, Stephen Eaton <sea...@strobotics.com.au> wrote:
Hi all,

As per the subject line I'm a prometheus newb for just over a week and trying to wrap my head around the alerting and notifications, specifically how I can use notification data: GroupLabels, CommonLabels, CommonAnnotation as per https://prometheus.io/docs/alerting/notifications/#data-structures

Where do these different KV pairs get defined?

Are there already standard KV pairs?  if so where can I see what these are? 

or do I have to set these?  

The documentation you linked to is the KV pairs that you'll have access to in notification templates.

 

Is there documentation somewhere outlining best practice around this?

I feel that I'm missing something simple and when its explained to me the Doh! light will go on.

regards,

Stephen.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b12afb86-cf75-4866-aa77-ba3c4eac55ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

Stephen Eaton

unread,
Nov 28, 2017, 8:48:24 AM11/28/17
to Prometheus Users
Thanks Brian,

What I'm after is what defined KV pairs are already avail?  Also or how/where do I set them myself?  i.e. are these KV pairs the ones that can be set in the Rules?  or are they set elsewhere?


On Tuesday, 28 November 2017 20:38:23 UTC+8, Brian Brazil wrote:
On 27 November 2017 at 16:48, Stephen Eaton <sea...@strobotics.com.au> wrote:
Hi all,

As per the subject line I'm a prometheus newb for just over a week and trying to wrap my head around the alerting and notifications, specifically how I can use notification data: GroupLabels, CommonLabels, CommonAnnotation as per https://prometheus.io/docs/alerting/notifications/#data-structures

Where do these different KV pairs get defined?

Are there already standard KV pairs?  if so where can I see what these are? 

or do I have to set these?  

The documentation you linked to is the KV pairs that you'll have access to in notification templates.

 

Is there documentation somewhere outlining best practice around this?

I feel that I'm missing something simple and when its explained to me the Doh! light will go on.

regards,

Stephen.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.



--

Brian Brazil

unread,
Nov 28, 2017, 8:58:43 AM11/28/17
to Stephen Eaton, Prometheus Users
On 28 November 2017 at 13:48, Stephen Eaton <sea...@strobotics.com.au> wrote:
Thanks Brian,

What I'm after is what defined KV pairs are already avail?  Also or how/where do I set them myself?  i.e. are these KV pairs the ones that can be set in the Rules?  or are they set elsewhere?

They're determined automatically based on the alerts in the group. If you have a label set in an alerting rule, it may appear here depending on the route's group_by and what other alerts and labels are in use.

Brian
 

On Tuesday, 28 November 2017 20:38:23 UTC+8, Brian Brazil wrote:
On 27 November 2017 at 16:48, Stephen Eaton <sea...@strobotics.com.au> wrote:
Hi all,

As per the subject line I'm a prometheus newb for just over a week and trying to wrap my head around the alerting and notifications, specifically how I can use notification data: GroupLabels, CommonLabels, CommonAnnotation as per https://prometheus.io/docs/alerting/notifications/#data-structures

Where do these different KV pairs get defined?

Are there already standard KV pairs?  if so where can I see what these are? 

or do I have to set these?  

The documentation you linked to is the KV pairs that you'll have access to in notification templates.

 

Is there documentation somewhere outlining best practice around this?

I feel that I'm missing something simple and when its explained to me the Doh! light will go on.

regards,

Stephen.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
 To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b12afb86-cf75-4866-aa77-ba3c4eac55ea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/0b47bca5-ab13-4243-ba26-197c53d4f7d2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Message has been deleted

Stephen Eaton

unread,
Nov 29, 2017, 3:35:25 AM11/29/17
to Prometheus Users
Hi Brian and thanks for the reply,

So basically all of these labels / annotations are set within the individual rules or alert group.

In examples online and around the place I see a label named 'env' being used in notifications is this one that I need to set somewhere?



--

Brian Brazil

unread,
Nov 29, 2017, 3:47:33 AM11/29/17
to Stephen Eaton, Prometheus Users
On 29 November 2017 at 08:35, Stephen Eaton <sea...@strobotics.com.au> wrote:
Hi Brian and thanks for the reply,

So basically all of these labels / annotations are set within the individual rules or alert group.

In examples online and around the place I see a label named 'env' being used in notifications is this one that I need to set somewhere?

The only labels that Prometheus has built in in some way are job, instance and alertname. Everything else is going to come from either instrumentation, external labels or target labels via SD/relabelling.

Brian
 
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/bb7366a0-d772-4ab2-aa1b-c5f15a6e964f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Stephen Eaton

unread,
Nov 30, 2017, 7:41:24 AM11/30/17
to Prometheus Users
Thanks for your help Brian,

Ive been playing around with it all today and now have more of a handle on it and how group labels are set.
..I'm on my way now :)



--
Reply all
Reply to author
Forward
0 new messages