Default value of zero when metric is null in a formula

[keith...@crunchydata.com]

I'm trying to use the following formula for a stats panel display in grafana

pg_up{cluster="[[pgcluster]]"} / (ccp_is_in_recovery_status{cluster="[[pgcluster]]"})

This can return multiple values if there are multiple clusters, which is expected. However if one of the clusters goes down, the right hand side metric goes away completely while the left side just changes from 1 to 0. As it's written above, that makes the metric completely disappear which is not what I want because then the panel completely goes away in Grafana vs being able to show a "down" status.

I've been trying to see about how to handle the right side going null but still returning a value. I found the vector(0) trick, but it still doesn't work as expected. For example, for a single cluster:

pg_up{cluster="alpha"}  /  ccp_is_in_recovery_status{cluster="alpha"} OR vector(0)

If all nodes are up, it still always returns an extra vector(0) in addition to the normal node values. And if more than one node is down, it only returns one empty value vs having multiples returned.

So, basically I'm just trying to have a default value for the right hand side if that expression happens to go null. Or I guess the more general question is how do you allow a default value anywhere in a PromQL forumla when that particular value may go NULL but you still want a metric returned so you can make use of that.

Thanks!

Keith

[David Hopkins]

Hi Keith, facing similar issue. Did you ever manage to resolve this?

Cheers

[Julius Volz]

Hi Keith and David,

You should be able to solve that like this:

-----------------

  pg_up{cluster="[[pgcluster]]"} / ccp_is_in_recovery_status{cluster="[[pgcluster]]"}

or

  pg_up{cluster="[[pgcluster]]"} * 0

-----------------

So basically you take advantage of a metric (in this case "pg_up") that is always present with all the label values that you expect (no matter if a PG cluster is up or not), and then just force to a particular value (in this case, 0).

Cheers,

Julius

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/8c15b3e5-68c4-4604-8dd8-c408fa887e96n%40googlegroups.com.

--

Julius Volz

PromLabs - promlabs.com