Is there something akin to ifnull() function in Prometheus PromQL ?

[Peter Zaitsev]

Hi,

I'm still dealing with my issue of dealing with different resolution data and zoom and Prometheus data source in Grafana

For example I capture amount of Queries  with 1 second resolution as I want to be able to see if there are some stalls couple of seconds in length,  while   when I'm looking at things like table sizes or access to individual tables I capture it with 1min resolution because capturing it with 1 second resolution would cause unreasonable overhead.

This causes to the bad problem with "zoomable graphs" where if I plot these values on the same graph together with 1 hour step over week  I have both data shown,  as I zoom in to the high level of details  so  go over 15min with 10sec step the   low resolution data dissapears

I'm using rate(metric[$interval])   function to plot the data to make sure I'm computing the average over the proper  resolution interval to avoid noise until I have zoomed in.   

If I would use  irate(metric[5m]) instead I can get both data points however it would be very volatile showing me for queries the QPS for random second every minute rather then average QPS for a given minute I would like to see. 

The solution which looks like it would work is using rate if data is available with required resolution but using irate() when it is not.  It is not mathematically perfect but  much better than simply low resolution data disappearing,

So something like this would work:

ifnull(rate(metric[$interval]),irate(metric[5m]))    which would  use rate  when it is available but irate if it does not.

Is there something like ifnull() function available in Prometheus or other syntax to achieve what I'm trying to do ? 

Thanks!

--

Peter Zaitsev, CEO, Percona
Tel: +1 888 401 3401 ext 7360   Skype:  peter_zaitsev

[Julius Volz]

It sounds like you could use the "or" operator:

rate(metric[$interval]) or irate(metric[5m])

See: http://prometheus.io/docs/querying/operators/#logical/set-binary-operators

"vector1 or vector2 results in a vector that contains all original elements (label sets + values) of vector1 and additionally all elements of vector2 which do not have matching label sets in vector1."

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Zaitsev]

Julius,

Thank you. You made my day.  Now I can get the graph to show reasonable data even when the Interval in resolution is smaller than the data capture interval:

This is the full formula I'm using:

  topk(5,(rate(mysql_info_schema_table_statistics_rows_read_total{alias="$host"}[$interval]) or irate(mysql_info_schema_table_statistics_rows_read_total{alias="$host"}[5m]))>0)

The only unpleasantness is now I will have to use much more complicated formulas in my dashboards.  I believe it is common behavior to want and it would be nice if there would be rate function variant to provide it :)

[Julius Volz]

On Sat, Feb 27, 2016 at 5:23 PM, Peter Zaitsev <p...@percona.com> wrote:

Julius,

Thank you. You made my day.  Now I can get the graph to show reasonable data even when the Interval in resolution is smaller than the data capture interval:

This is the full formula I'm using:

  topk(5,(rate(mysql_info_schema_table_statistics_rows_read_total{alias="$host"}[$interval]) or irate(mysql_info_schema_table_statistics_rows_read_total{alias="$host"}[5m]))>0)

Yay!

 

The only unpleasantness is now I will have to use much more complicated formulas in my dashboards.  I believe it is common behavior to want and it would be nice if there would be rate function variant to provide it :)

Well :) Let's say, Percona is the first use case I've heard about that needs this, and I would consider what you folks are doing quite extreme. To run into this, you need to have both a dynamic rate() window based on the graphing range, as well as have a graph zoom smaller than your scrape rate. And then also *care* about the fact that rates disappear under those circumstances. If this starts becoming a pattern with other people, we can think about what to do.