Question about "rate" & Performance in Prometheus

[Geoffrey Graveaud]

Hello everyone.
How are you ?

 Please, I have a question regarding the "rate" in prometheus metrics.

 Does it affect performance?  maybe thoughtput or ingestion ?  
 I think so but I am having difficulty identifying it clearly.

 For example, "rate_1m" vs "rate_5mn".
If you have some answers from experience, I'm a taker .

Regards,
Geoffrey Graveaud

[Matthias Rampke]

What is "rate_1m" in this case? Is this a metric that has rate in the name, or are you asking about the rate query function?

Ingestion is mostly independent from query evaluation. It is in principle possible to overload the server with queries, and thus impact ingestion, but you'd have to try quite hard.

For any functions that take a range vector, the time range does have an impact, but the difference between 1m and 5m is small. There will be more samples to iterate over, but in my experience CPU is almost never the limiting factor. What is more important is how much data there is to load. For the most part, samples within a few minutes of each other will be stored together. If you loaded 1m worth of data you probably loaded the other 4 minutes as well. It gets more tricky if you are trying to use 30+ minute ranges.

Also keep in mind that the number of time series plays a role – you can get away with ranges over several hours or even days on a single time series, but trying to query a day's worth of a few thousand time series is going to be painful.

The best way around this is recording rules: say, you want to graph a service's HTTP request rate over a long period of time; you can make the rate calculation immediately after ingestion, when the data is still in memory; this is cheap even over many many time series. At the same time, you can aggregate into a small number of series: when graphing a long time, you are probably not interested in a breakdown by ephemeral instances; sum that away and you reduce the data needs at query time by a lot.

All in all, I would not worry about it until you have a problem though.

/MR

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/4d13fc24-fd29-45b7-8578-690718407711n%40googlegroups.com.

[Geoffrey Graveaud]

Hello Matthias,

Yes, it was about rate query function.

Thank you very much.