Prometheus query double rate

[meher...@gmail.com]

hi

In prometheus there is a rate function which "calculates the per-second average rate of increase of the time series in the range vector" as per the docs

Now I am interested in finding how much this rate value itself is changing over time?

Using the example in your docs

rate(http_requests_total{job="api-server"}[5m])

The above "returns the per-second rate of HTTP requests as measured over the last 5 minutes"

One hour has 12 5 min slots, so the above rate function can potentially give 12 rate values assuming I do rate query every 5 mins.

I was wondering is it possible to do a rate on a rate

i.e

x = rate(http_requests_total{job="api-server"}[5m])

y = rate(x[1h])

Then in alert rules :

If y is between 1 - 10 : low severity alarm

If y above 10 or below -10 : high severity alarm

Can prometheus support querying for y and if so any pointers to achieve that would be great?

[Ben Kochie]

In order to do a second derivative, you will need to record the rate in a rules file.

http_requests_total:rate5m = rate(http_requests_total[5m])

Then you can apply the deriv function[0].

deriv(http_requests_total:rate5m[5m])

You need to use deriv() and not rate() because rate() is used for counters that reset and so it can't handle negative rates.

[0]: https://prometheus.io/docs/querying/functions/#deriv()

--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[meher...@gmail.com]

Hi Ben

This is great thank you

The docs for deriv() state "deriv should only be used with gauges."

But a lot of the metrics are counters , case in point http_requests_total.

How would i do second deriv on counters?

Am I missing something here?

[Tobias Schmidt]

The rate of a counter produces a gauge which can go up and down.

[meher...@gmail.com]

Thanx Tobias and Ben