PromQL: show graph for all up metrics where a job+instance has not been up in the given time range

19 views
Skip to first unread message

Felix

unread,
Mar 11, 2020, 6:26:32 AM3/11/20
to Prometheus Users
Hi,

I have now searched for a solution for over two days and I dont know how to continue. I am using Prometheus with Grafana and the Graph I want to achieve is actually pretty simple:

I want to have a graph that shows the availability of all scrape jobs that had issues in the given time range by grafana.

So basically lets say in the last 24 hours - if the scrape job by instance and job was 0 for one or more times, I want to show the full graph with all values for those 24 hours for that instance + job.

I tried it with that:

up and up == 0

and also

up + on(instance,job) group_right() up == 0


but unfortunately this compares the values as well and then only shows zero-values for the given instances and jobs but not the availability. Does anyone have an idea regarding this? It seems so simple, but I guess there's just a knot in my head somehwere.

Best Regards, Felix

Julien Pivotto

unread,
Mar 11, 2020, 6:33:30 AM3/11/20
to Felix, Prometheus Users
On 11 Mar 03:26, Felix wrote:
> Hi,
>
> I have now searched for a solution for over two days and I dont know how to
> continue. I am using Prometheus with Grafana and the Graph I want to
> achieve is actually pretty simple:
>
> *I want to have a graph that shows the availability of all scrape jobs that
> had issues in the given time range by grafana.*
>
> So basically lets say in the last 24 hours - if the scrape job by instance
> and job was 0 for one or more times, I want to show the full graph with all
> values for those 24 hours for that instance + job.
>
> I tried it with that:
>
> up and up == 0

you can try up and min_over_time(up[24h]) == 0, but it will only show
the values from the first failure onwards.

>
> and also
>
> up + on(instance,job) group_right() up == 0
>
>
> but unfortunately this compares the values as well and then only shows
> zero-values for the given instances and jobs but not the availability. Does
> anyone have an idea regarding this? It seems so simple, but I guess there's
> just a knot in my head somehwere.
>
> Best Regards, Felix
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/db7971b5-0df8-40ac-9e3b-8838ebe451ed%40googlegroups.com.


--
(o- Julien Pivotto
//\ Open-Source Consultant
V_/_ Inuits - https://www.inuits.eu
signature.asc

Felix

unread,
Mar 11, 2020, 7:00:24 AM3/11/20
to Prometheus Users
Hello Julien Pivotto,

thank you very much! I have now done the following query:

up and min_over_time(up[$__range]) == 0

and this query gives me my expected result listing all values with 1s and 0s of all series that have been 0 in the currently selected range in Grafana. I've got the variable from https://grafana.com/docs/grafana/latest/reference/templating/#time-range-variables and the $__range variable is currently only available for prometheus - I guess for exactly that purpose.

Thank you very much, topic solved :)


Am Mittwoch, 11. März 2020 11:33:30 UTC+1 schrieb Julien Pivotto:
On 11 Mar 03:26, Felix wrote:
> Hi,
>
> I have now searched for a solution for over two days and I dont know how to
> continue. I am using Prometheus with Grafana and the Graph I want to
> achieve is actually pretty simple:
>
> *I want to have a graph that shows the availability of all scrape jobs that
> had issues in the given time range by grafana.*
>
> So basically lets say in the last 24 hours - if the scrape job by instance
> and job was 0 for one or more times, I want to show the full graph with all
> values for those 24 hours for that instance + job.
>
> I tried it with that:
>
> up and up == 0

you can try up and min_over_time(up[24h]) == 0, but it will only show
the values from the first failure onwards.

>
> and also
>
> up + on(instance,job) group_right() up == 0
>
>
> but unfortunately this compares the values as well and then only shows
> zero-values for the given instances and jobs but not the availability. Does
> anyone have an idea regarding this? It seems so simple, but I guess there's
> just a knot in my head somehwere.
>
> Best Regards, Felix
>
> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages