split by labels values

817 views
Skip to first unread message

Javier Mendiara Cañardo

unread,
May 26, 2022, 8:42:57 AM5/26/22
to Prometheus Users
Hi, 

I have an info metric with all possible tag values
tag_info{tag="small"} 1 
tag_info{tag="medium"} 1 
tag_info{tag="big"} 1 
tag_info{tag="azure"} 1 
tag_info{tag="aws"} 1 

I have an info metric with a multi-value label called  "tags", as a csv with commas envelope
vm_info{name="AAAAA" tags=",big,azure,"} 1
vm_info{name="BBBBB" tags=",big,aws,"} 1
vm_info{name="CCCCC" tags=",small,aws,"} 1

I want to get how many machines have that specific tag:
{tag="big"} 2
{tag="small"} 1
{tag="azure"} 1
{tag="aws"} 2

The tags values are not known when coding the query

Any hints? It's a kind-of label_split missing function.

Thanks!

PD: I have under the control the exporter to make a metrics refactor.


Brian Brazil

unread,
May 26, 2022, 8:51:43 AM5/26/22
to Javier Mendiara Cañardo, Prometheus Users
On Thu, 26 May 2022 at 13:42, Javier Mendiara Cañardo <jmen...@gmail.com> wrote:
Hi, 

I have an info metric with all possible tag values
tag_info{tag="small"} 1 
tag_info{tag="medium"} 1 
tag_info{tag="big"} 1 
tag_info{tag="azure"} 1 
tag_info{tag="aws"} 1 

I have an info metric with a multi-value label called  "tags", as a csv with commas envelope
vm_info{name="AAAAA" tags=",big,azure,"} 1
vm_info{name="BBBBB" tags=",big,aws,"} 1
vm_info{name="CCCCC" tags=",small,aws,"} 1

I want to get how many machines have that specific tag:
{tag="big"} 2
{tag="small"} 1
{tag="azure"} 1
{tag="aws"} 2

The tags values are not known when coding the query

Any hints? It's a kind-of label_split missing function.

I don't believe this is possible with PromQL in a general form. You can figure out the value for a single known tag with a regex matcher, but not in the general case. Such a function would also be a footgun, as it could greatly increase the cardinality of the output relative to the input. As it stands you can only increase input to output cardinality via absent/absent_over_time, and that's only from 0 to 1.

Thanks!

PD: I have under the control the exporter to make a metrics refactor.

This is what you'd need to do. You want a separate gauge metric with a sample per tag present.

Brian
 


--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/885bc38a-9950-4100-a247-68d1e52f17e7n%40googlegroups.com.


--

Brian Candler

unread,
May 26, 2022, 11:57:41 AM5/26/22
to Prometheus Users
Or separate the tags into individual labels:

vm_info{name="AAAAA",tag_big="1",tag_azure="1"} 1

Brian Candler

unread,
May 26, 2022, 12:08:01 PM5/26/22
to Prometheus Users
... although that doesn't help you with your count-by-arbitrary-tag problem, sorry.  For that as Brian B says, you need separate series:

vm_tag{name="AAAAA",tag="big"} 1
vm_tag{name="AAAAA",tag="azure"} 1

Javier Mendiara Cañardo

unread,
May 27, 2022, 12:44:15 AM5/27/22
to Prometheus Users
Thank you Brians for your responses!
I was hopping this problem could be solved with some dark magic promQL queries. I'd probably go with the vm_tag solution, as it is a general purpose metric that allows queries with group_right vector matching, solving more use cases around "tagging".


FYI:
To give you some context about why I chose

vm_info{name="AAAAA" tags=",big,azure,"} 1

is inspiration in prometheus code itself, in the ec2 service discovery code (and others)

So it seems this "tagging" pattern is already in the community, but unfortunately does not solves all use cases by itself.

Best!
Javier

Brian Brazil

unread,
May 27, 2022, 2:21:19 AM5/27/22
to Javier Mendiara Cañardo, Prometheus Users
On Fri, 27 May 2022 at 05:44, Javier Mendiara Cañardo <jmen...@gmail.com> wrote:
Thank you Brians for your responses!
I was hopping this problem could be solved with some dark magic promQL queries. I'd probably go with the vm_tag solution, as it is a general purpose metric that allows queries with group_right vector matching, solving more use cases around "tagging".


FYI:
To give you some context about why I chose
vm_info{name="AAAAA" tags=",big,azure,"} 1

is inspiration in prometheus code itself, in the ec2 service discovery code (and others)

So it seems this "tagging" pattern is already in the community, but unfortunately does not solves all use cases by itself.

That's designed for relabelling when you're matching against one known tag for service discovery, but not really beyond that.

Brian


 
Reply all
Reply to author
Forward
0 new messages