How to query to know if a metric has been available for an hour

62 views
Skip to first unread message

Arnav Bose

unread,
Apr 30, 2020, 4:51:31 PM4/30/20
to Prometheus Users
Hi,

I know I can check in the graph how long the metric has been available. In my case I want to create a query which will list down the data for a metric which has been available for an hour, excluding the ones which at least went down/missing during the same period. 

Here is my metric - CMSummary{ipAddr="$$$"}. There are at least 20000 different IP sources with this metric. I want to know which ones have been available for the last 1 hr. 


Thanks,
Arnav


Julius Volz

unread,
May 1, 2020, 4:55:56 AM5/1/20
to Arnav Bose, Prometheus Users
Do you also need to exclude IPs that are present at the beginning of the hour, go missing briefly in between (as in, the time series becomes fully absent), but are present again at the end of the hour? We don't have a way in PromQL to check whether a series has been absent just briefly within an interval, but is there most of the time. We have absent_over_time(), but that only checks whether a series has been fully absent over a given time range.

But if your IPs behave in such a way that they don't appear / disappear that rapidly, you could check which ones were there both at the beginning and the end of the interval:

  CMSummary offset 1h and CMSummary

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d21d5adf-0307-4367-b050-ec81cf0ce8b2%40googlegroups.com.

Julius Volz

unread,
May 1, 2020, 4:59:17 AM5/1/20
to Arnav Bose, Prometheus Users
Btw. you could even modify this expression to check manually every e.g. 15 minutes within the last hour, whether an IP was present at that time increment within the last hour:

    CMSummary offset 1h
  and
    CMSummary offset 45m
  and
    CMSummary offset 30m
  and
    CMSummary offset 15m
  and
    CMSummary

So you see, theoretically you could even check every minute or so for the presence, but that would become a long query...

Aliaksandr Valialkin

unread,
May 1, 2020, 7:34:47 AM5/1/20
to Julius Volz, Arnav Bose, Prometheus Users
Try the following Prometheus subquery

count_over_time((CMSummary)[1h:1m]) == 60

It should return only time series, which were available every minute during the last hour.




--
Best Regards,

Aliaksandr

Julius Volz

unread,
May 1, 2020, 7:37:01 AM5/1/20
to Aliaksandr Valialkin, Arnav Bose, Prometheus Users
Oh yeah, smart idea for doing the same much easier, thanks :)
Reply all
Reply to author
Forward
0 new messages