Hello Prometheus Community!
There have been many discussions and issues related to the problem of new timeseries and the increase() (example:
here, or
here) and even the official documentation of Prometheus mentions to "
avoid missing metrics".
The workaround consist in manually initializing our metrics to 0. This can become complex with vectors when we don't know in advance the combination of label values, moreover we need to be sure the first scrape from Prometheus include this 0 value.
At Goto.com we implemented
a solution to this problem that has been running in production for more than 1 year and we decided to open it to the community.
It's golang library extending the official client that ensure the first collection(s) of counters always return 0 instead of their actual value. This works for counters/histogram/summaries and the corresponding vectors.
This is done in the background without anything to add in your application code.
We hope this will help the community, try it!
feedback are welcome.
https://github.com/goto-opensource/smart-prometheus-client Have a nice day!
Bruno