Counters become zero when exporter restarted (how to deal with that?)

[Fabian Stäber]

Hi,

this is probably a newbie question, I'd be happy for any ideas or pointers to documentation:

I learned from the best practices section that:

As a general rule, for every line of logging code you should also have a counter that is incremented.

I experimented with my own log file exporter, and I run into problems when the exporter is restarted. After the restart, counters start from zero. I am trying to show graphs with information on the number of events in a given time interval, but I see negative values for that. This makes the information on my Grafana dashboard quite useless.

What is a good way to deal with restarts in exporters that analyze logging output?

Thanks for your help!

Fabian

[Brian Brazil]

I'm not sure how you're getting negative values, but you should use the rate() function which automatically handles counter resets.

--

Brian Brazil

www.robustperception.io

[Julius Volz]

Or increase(), depending on your needs.

http://prometheus.io/docs/querying/functions/#rate

http://prometheus.io/docs/querying/functions/#increase 

[Fabian Stäber]

Ok, thanks a lot. I used sum(delta(...[1d])), but maybe this was just the wrong thing. Seemed good before I restarted the exporter.

[Julius Volz]

Yes, delta() is exactly for the case where you want a raw delta without any counter resets :)

Use delta() for gauges (temperatures, free disk space, etc.) that can go up or down.

Use rate() / irate() / increase() for counters (number of requests etc.) that can only go up, modulo counter resets.

--

You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Julius Volz]

s/without any counter resets/without compensating for any counter resets/