/metrics lifecycle

[Vasiliy B]

Folks,
  Am researching a use case where we collect /metrics data with Prometheus only when needed to do some investigation. During normal operating hours, we would flush the /metrics endpoint on a time schedule which is greater than the scrape_interval ?

From the documentation, it is clear that Prometheus server will honor the scrape_interval config setting. But what about on the service side?  Do the metrics have the ability to reset to zero after a predefined time, i.e. 1 minute?

Looking for feedback if this is a feasible, any gotchas, or clarification on how metrics are stored on the client side.

Thanks!

[Ben Kochie]

Yes, you can reset as long as it's longer than the interval. But, why? There's no specific need in Prometheus to do this.

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/663233c1-7bdf-4903-9aa2-d7dce327a27bn%40googlegroups.com.

[Stuart Clark]

A call to /metrics should always return the "current" value of every
metric. For counters (which are generally recommended) they always
increase and therefore only reset to zero if the application itself is
restarted. For gauges the ideal situation is that the scraping request
returns the live value of those metrics. For some metrics (such as where
a call to an external system is needed to generate) it can be quite
resource intensive to produce, and therefore that process wants to be
limited in frequency. In that case the common method is to have a
separate timed process which updates the metric, with the call to
/metrics just returning the latest values calculated. While this can
reduce the impact of such "heavy" metrics generation processes it does
mean that you may be getting old values (depending on the frequency of
the metric generation process) and there is a risk of that process
breaking without you noticing.

So in general, no there isn't any idea of "flushing" or resetting metrics.

Would you be able to give a bit more detail about what you are trying to
do, and why you think resetting metrics is important / needed?

--
Stuart Clark