Regexp re-writes on Legend Format -grafana

[goel....@gmail.com]

I'm trying to shorten my legends a bit, and the usual searches aren't helping me with applying regular expressions. For a graph/metric showing bits transmitted, my current setting for the "Legend Format" setting is:  

{{name}}

This results in legends that look like this: 

bots_languagedetection.1.u13kbsn40s1gsnnf10fbvh4f5

 

My wish is to have legends that look like this: 

 bots_languagedetection.1

Can you please help me in this?

[Brian Candler]

Does your metric actually look like this?

mymetric{name="bots_languagedetection.1.u13kbsn40s1gsnnf10fbvh4f5"} 1

I don't know if grafana can rewrite labels.

In PromQL queries you can use label_replace:

https://prometheus.io/docs/prometheus/latest/querying/functions/#label_replace

However, it might be worth doing metric relabelling at ingestion time to get more meaningful labels, even if it's only as simple as

mymetric{name="bots_languagedetection.1",id="u13kbsn40s1gsnnf10fbvh4f5"}

[goel....@gmail.com]

Thanks Brian, 

Below is the metrics which i am using, but i am not able to utilize label_replace, can you please help me how to use it.

avg by(name, container_label_com_docker_swarm_node_id) (rate(container_cpu_usage_seconds_total{container_label_com_docker_swarm_task_name=~".+"}[30s])) * 100

[Brian Candler]

Show some output from the query (including full label sets) and how you've tried using label_replace.

[goel....@gmail.com]

Yes Brian, I tried using label_replace but every time i get error below is my query which i tried.

avg by(name, container_label_com_docker_swarm_node_id)(rate(label_replace(container_cpu_usage_seconds_total{container_label_com_docker_swarm_task_name=~".+", name=~".+"}, "name", "$1", "name", "(.*)"[30s]))) * 100

Normal Query without label_replace with output is below:

avg by(name, container_label_com_docker_swarm_node_id) (rate(container_cpu_usage_seconds_total{container_label_com_docker_swarm_task_name=~".+"}[30s])) * 100

{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="telegraf.1.l0gqe5th54e1ko8f2kdynl5q9"}     0.12159615838850078
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="prometheus.1.14sp9pvx6y8mb5ymq7ev60fns"}   0.2783589968152833
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="cadvisor.z2cjapl00v69kmfjpmsin79lc.hwiz9dknbul892jo0jcq4h829"} 1.215465595861067  

Output i am looking for is:(there are 2 dots in the name and i am looking the value before the second dot as below)

{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="telegraf.1"}                                                                0.12159615838850078
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="prometheus.1"}                                                        0.2783589968152833
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="cadvisor.z2cjapl00v69kmfjpmsin79lc"}               1.215465595861067 

[Brian Candler]

On Friday, 21 August 2020 05:21:45 UTC+1, goel....@gmail.com wrote:

Normal Query without label_replace with output is below:

avg by(name, container_label_com_docker_swarm_node_id) (rate(container_cpu_usage_seconds_total{container_label_com_docker_swarm_task_name=~".+"}[30s])) * 100

{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="telegraf.1.l0gqe5th54e1ko8f2kdynl5q9"}     0.12159615838850078
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="prometheus.1.14sp9pvx6y8mb5ymq7ev60fns"}   0.2783589968152833
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="cadvisor.z2cjapl00v69kmfjpmsin79lc.hwiz9dknbul892jo0jcq4h829"} 1.215465595861067  

Output i am looking for is:(there are 2 dots in the name and i am looking the value before the second dot as below)

{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="telegraf.1"}                                                                0.12159615838850078
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="prometheus.1"}                                                        0.2783589968152833
{container_label_com_docker_swarm_node_id="z2cjapl00v69kmfjpmsin79lc",name="cadvisor.z2cjapl00v69kmfjpmsin79lc"}               1.215465595861067 

I'd suggest using your existing working expression, and wrapping a label_replace(...) around it.

label_replace(

   ... avg by expression as before ..., "name", "$1", "name", "(.*)[.]"

)

.* matches any sequence of characters

[.] matches a dot

So this is any series of characters, followed by a dot.  The part inside the round brackets is captured as $1.

Regular expressions are "greedy" by default, so this will capture all the text up to but not including the last dot in the value.

[goel....@gmail.com]

Thanks Brian, 

Now the expression is not giving any error but it is still not working as expected, I am still getting the same previous result. Below are the expression and the output:

Exp:

label_replace(avg by(name, container_label_com_docker_swarm_node_id) (rate(container_cpu_usage_seconds_total{container_label_com_docker_swarm_task_name=~".+"}[30s])) * 100, "name", "$1", "name", "(.*)[.]" )

Output

{container_label_com_docker_swarm_node_id="abc",name="monitor_telegraf.1.l0gqe5th54e1ko8f2kdynl5q9"}                     0.0981453569681763
{container_label_com_docker_swarm_node_id="abc",name="monitor_alertmanager.1.kd3rgqmo8ufhvnpilhj30090j"}            0.07092644961033587
{container_label_com_docker_swarm_node_id="abc",name="monitor_node-exporter.abc.nruc17pnvjzyiw3485f51dopk"}      0.13081823778053095  

[Brian Candler]

My mistake, you also have to match the trailing junk after the last dot.  Try:

"name", "$1", "name", "(.*)[.].*"

[goel....@gmail.com]

Thanks a lot Brian this works, 

Can you please suggest me how can i get more info on this type of issue so that it can be tackled be me in future.

[Brian Candler]

On Monday, 24 August 2020 10:37:22 UTC+1, goel....@gmail.com wrote:

Thanks a lot Brian this works, 

Can you please suggest me how can i get more info on this type of issue so that it can be tackled be me in future.

I'm not sure what you mean.

There are lots of good blog posts on prometheus: the ones especially worth reading are those by Brian Brazil at https://www.robustperception.io/blog

including the older ones which cover fundamentals.  Here are some I have bookmarked:

https://www.robustperception.io/using-time-series-as-alert-thresholds

https://www.robustperception.io/target-labels-are-for-life-not-just-for-christmas/

https://www.robustperception.io/controlling-the-instance-label

https://www.robustperception.io/on-the-naming-of-things/

https://www.robustperception.io/running-into-burning-buildings-because-the-fire-alarm-stopped

https://www.robustperception.io/how-to-have-labels-for-machine-roles

https://www.robustperception.io/exposing-the-software-version-to-prometheus

For label_replace, I just read the prometheus documentation on functions:

https://prometheus.io/docs/prometheus/latest/querying/functions/

It does assume you know about regular expressions.  There is much generic information about regular expressions on the Internet.  The regular expressions used by Prometheus come from the Go regexp library which implements a flavour called RE2:

https://github.com/google/re2/wiki/Syntax

[goel....@gmail.com]

Thanks a lot Brian, all this really helps very much.