What does prometheus do when querying with min step in 10s but the scrape interval is 15s?

[Ray Wu]

I use grafana to plot some system metrics from node_exporter. I set my prometheus scrape interval to 15s and grafana's default min step is 10s.

Does that make sense to plot in the finer granularity than the sampling rate at all?

Is there any doc explain how does Prometheus calculate this?

[Ben Kochie]

Generally, if you're using $__interval in Grafana, you want to have a min step that's 3x or 4x your scrape interval. This allows the rate() function to handle missed scrapes and counter resets better.

It definitely doesn't make sense to have the min step finer than your interval. You need 2x the interval at a minimum for rate() to even function properly.

If you need more resolution than 1m, I would suggest a scrape interval of 5s.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/12858dde-c547-454c-80a6-87f9e67096b8o%40googlegroups.com.

[Ray Wu]

To be clear, I'm talking about the "Min step" in here https://grafana.com/docs/grafana/latest/features/datasources/prometheus/#prometheus-query-editor

which maps to "step: Query resolution step width in duration format or float number of seconds." in range query API.

The 3x, 4x sounds like applying to the range query selector? I saw the similar suggestion at robust perception as well.

I didn't know how to use  $__interval though, let me check how that works. Thanks!

On Sunday, June 14, 2020 at 12:09:08 PM UTC-7, Ben Kochie wrote:

Generally, if you're using $__interval in Grafana, you want to have a min step that's 3x or 4x your scrape interval. This allows the rate() function to handle missed scrapes and counter resets better.

It definitely doesn't make sense to have the min step finer than your interval. You need 2x the interval at a minimum for rate() to even function properly.

If you need more resolution than 1m, I would suggest a scrape interval of 5s.

On Sun, Jun 14, 2020 at 7:21 PM Ray Wu <ray...@gmail.com> wrote:

I use grafana to plot some system metrics from node_exporter. I set my prometheus scrape interval to 15s and grafana's default min step is 10s.

Does that make sense to plot in the finer granularity than the sampling rate at all?

Is there any doc explain how does Prometheus calculate this?

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.