Storing Query as a variable

[Sandesh Shivakumar]

Hello All,

I wanted to store the below Prometheus query into an variable.

(node_memory_MemTotal_bytes{job="kubernetes-service-endpoints",cluster="$cluster",instance=~"$node"} * on(instance) group_left(nodename) group(node_uname_info{})by(instance,nodename)-(node_memory_MemFree_bytes{job="kubernetes-service-endpoints",cluster="$cluster",instance=~"$node"} * on(instance) group_left(nodename) group(node_uname_info{})by(instance,nodename) ))

and use it in second query ,

z = (sum(rate(metric)[1m]) - sum(avg_over_time(metric[1w])) / sum(stddev_over_time(metric[1w])) 

Is there any way to achieve this.

Thankyou

[Julius Volz]

Hi Sandesh,

There's unfortunately no way to create variables directly in a PromQL query itself, but you could use a recording rule (see https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/) to pre-record the long expression into a shorter new metric name that you can then use in your second query.

Regards,

Julius

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/f8887aef-98dc-4207-b353-77247fcaf453n%40googlegroups.com.

--

Julius Volz

PromLabs - promlabs.com

[Brian Candler]

If you want to do this inside prometheus, then you want recording rules.

Note that with recording rules you're not really creating a new "variable", you're creating a whole new timeseries. However this is what you need, since your second query wants to perform aggregation_over_time() on these precomputed values.

It'll take a week until you have enough data from your recording rule to use, unless you backfill.

Your other option is to substitute the first expression into the two places where you want to use it in the second expression.  But in this case, you'll need to use a subquery, e.g.

    (some - long - expression)[1w:1h]

rather than

    metric[1w]

The subquery shown evaluates (some - long - expression) over a 1w period at 1h intervals to create a range vector.  Note that unlike a simple metric range vector, you need to give it the sampling interval.  Or you can write

   (some - long - expression[1w:]

but this will just use whatever global default evaluation interval you have configured (which might be, say, 15 seconds or 1 minute)

P.S. Are you sure that query you showed is valid?  The first part is fine, but the end doesn't make sense to me, and I can't see this syntax permitted either by operators or functions.

    node_memory_MemTotal_bytes{job="kubernetes-service-endpoints",cluster="$cluster",instance=~"$node"} * on(instance) group_left(nodename) group(node_uname_info{})by(instance,nodename)

I would write this as simply:

    node_memory_MemTotal_bytes{job="kubernetes-service-endpoints",cluster="$cluster",instance=~"$node"} * on(instance) group_left(nodename) node_uname_info

assuming that:

- N instances of the LHS match to 1 instance of the RHS with a corresponding "instance" label

- you want to add the "nodename" label from the RHS to all the timeseries returned by the LHS

[Brian Candler]

Also, the whole expression can be simplified to:

(node_memory_MemTotal_bytes{job="kubernetes-service-endpoints",cluster="$cluster",instance=~"$node"} - node_memory_MemFree_bytes) * on(instance) group_left(nodename) node_uname_info

This is because you don't need to filter both node_memory_MemTotal_bytes and node_memory_MemFree_bytes.  When you combine them with a binary operator (in this case subtraction), the result vector will only have values where both LHS and RHS have exactly matching labels.  So any values from the RHS which don't match the LHS are discarded automatically, without having to explicitly filter them..

[Julius Volz]

On Tue, Dec 13, 2022 at 10:07 AM Brian Candler <b.ca...@pobox.com> wrote:

Also, the whole expression can be simplified to:

(node_memory_MemTotal_bytes{job="kubernetes-service-endpoints",cluster="$cluster",instance=~"$node"} - node_memory_MemFree_bytes) * on(instance) group_left(nodename) node_uname_info

This is because you don't need to filter both node_memory_MemTotal_bytes and node_memory_MemFree_bytes.  When you combine them with a binary operator (in this case subtraction), the result vector will only have values where both LHS and RHS have exactly matching labels.  So any values from the RHS which don't match the LHS are discarded automatically, without having to explicitly filter them..

While that is semantically true, it can have performance implications because the PromQL engine will first need to load *all* selected time series (in this case, as many series as you have hosts) and only then throw away the unmatched series. So if you have many hosts, this could get inefficient.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/63be3ffe-9427-4558-9e5e-bcdab9ffaa5fn%40googlegroups.com.