Timestamp value with custom metrics

[Pramod Kumar]

Hi ,

I want to add timestamp with the custom metric. For example : I have a counter as total_orders_placed.

And I need to see the timestamp value when the counter got incremented. e.g.

1. total_orders_placed : 100 at 2020-01-23 11:03:02.543

2. total_orders_placed : 200 at 2020-01-23 11:04:00.123

So, I am looking for the counter with the count value as 100 with time 2020-01-23 11:03:02.543.

Please help me out. 

Thanks.

[Stuart Clark]

You will get timing information from Prometheus automatically. Every time your application is scraped the current value of the counter is stored against the scrape time, so if the previous scrape had a value of 100 and it is now 200, you know the increase happened between the two scrapes.

For counters the rate() functions are useful to show how quickly things are changing.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Pramod Kumar]

Hi Clark,

Thank you for quick reply.

Actually, in my case, the application is generating some numbers (with timestamp) and pushing inside file. Now, this number has to be consider as counters and shown in prometheus along with the below timestamp. e.g. 

2020-11-25 20:32:19.991(timestamp) 608(value)
2020-11-25 20:32:29.991(timestamp)1529(value)  

I want to show this timestamp insted of prometheus scrape time along with the counter value.

Please provide your inputs.

Thanks

[Stuart Clark]

On 23/12/2020 14:59, Pramod Kumar wrote:
> Hi Clark,
>
> Thank you for quick reply.
> Actually, in my case, the application is generating some numbers (with
> timestamp) and pushing inside file. Now, this number has to be
> consider as counters and shown in prometheus along with the below
> timestamp. e.g.
> 2020-11-25 20:32:19.991(timestamp) 608(value)
> 2020-11-25 20:32:29.991(timestamp)1529(value)
> I want to show this timestamp insted of prometheus scrape time along
> with the counter value.

That is not how Prometheus works. Prometheus is a scrape based metrics
system which records a new value at each scrape for all metrics, rather
than an event system where changes to values are made at times other
than the scrape time. If you want tighter timings around when changes
happened you should look to increase the scraping frequency.

It would be possible to add an additional metric which gives the
timestamp of the last change (and it is quite common to have metrics of
that type).

While the Prometheus text formal does include an optional timestamp
field it should not be used. It is there for a very specific edge case
where metrics are coming from another metrics system (such as AWS
CloudWatch) with some form of delay. Even in that situation the
timestamp should be increasing over scrapes.

[Pramod Kumar]

Thank you so much Clark.