Any solutions to achieve "avg_over_time(max(http_requests_total)[5m])"?

[zichen chuh]

avg_over_time(max(http_requests_total)[5m])

max(http_requests_total)[5m]

produced following error messages:

Error executing query: invalid parameter 'query': parse error at char 42: range specification must be preceded by a metric selector, but follows a *promql.AggregateExpr instead

Then I tried the following promql, still failed!

vector(scalar(max(http_requests_total)))[5m]

Error executing query: invalid parameter 'query': parse error at char 44: range specification must be preceded by a metric selector, but follows a *promql.Call instead

Any solutions?

Thanks in advance.

[Brian Candler]

In which dimension are you trying to do the "max" - the maximum across timeseries, or the maximum over time?

http_requests_total could be several timeseries each with their own labels.  Think of it in two dimensions:

http_requests_total{instance="A"}  a1 a2 a3 a4 a5

http_requests_total{instance="B"}  b1 b2 b3 b4 b5 

http_requests_total{instance="C"}  c1 c2 c3 c4 c5
                                   ---------------> time

If you want the maximum over time, see https://prometheus.io/docs/prometheus/latest/querying/functions/#aggregation_over_time

    max_over_time(http_requests_total[5m])

This will return the same set of timeseries as you had before, but for each one giving the maximum over the 5 minute window.

http_requests_total{instance="A"}  max(a1,a2,a3,a4,a5)

http_requests_total{instance="B"}  max(b1,b2,b3,b4,b5) 

http_requests_total{instance="C"}  max(c1,c2,c3,c4,c5)

Aside: this is probably *not* what you want to do with a counter - you might want to use rate() on it first.  But it makes sense with a gauge.

In the other dimension:

    max(http_requests_total)

gives you a single value which is the maximum across all the available timeseries: see https://prometheus.io/docs/prometheus/latest/querying/operators/#aggregation-operators

As an instant query, you'll get just the most recent time:

http_requests_total   max(a5,b5,c5)

but if you graph this, the instant time will be swept over the graph window, so you'll see the max across timeseries at each point in time.  Again, this probably doesn't make sense with counters as they have arbitrary offsets, but maximum across gauges makes sense (e.g. show me the maximum temperature).

If you want to do aggregation over time on this expression, you can turn it back into a range vector using a subquery: e.g.

(max(http_requests_total))[5m:1m]

That calculates the max(http_requests_total) at the current time and at 1 minute intervals over the previous 5 minutes.  You then have a range vector that you can aggregate over time.

This is more useful with expressions like:

max_over_time (rate(http_requests_total[5m])[60m:1m])

i.e. calculate the 5-minute average rate at 1 minute intervals, and give me the maximum rate seen over the last 60 minutes.

[zichen chuh]

Hi Brian,

Thanks for your prompt response.

The dimension I was trying to do the "max" was both - the maximum across timeseries, and the maximum over time.

So here is what I need: avg_over_time(max(<metric name>)[5m:1m]) .

在 2020年5月20日星期三 UTC+8下午7:31:54，Brian Candler写道：