Problem with using custom timestamp

409 views
Skip to first unread message

hai huang

unread,
Mar 6, 2020, 9:31:18 AM3/6/20
to Prometheus Users
I am getting the error message "Error on ingesting samples that are too old or are too far into the future" when trying to put my own custom timestamp into the scrape data. According to this issue (https://github.com/prometheus/prometheus/issues/4911), this problem problem has been resolved. However, in that issue, the reporter only tried to put a custom timestamp that is 30 minutes back. In my test, custom timestamp works fine unless it is older than 1 hour. Is there a configuration that one could tweak? Thanks!

sayf eddine Hammemi

unread,
Mar 6, 2020, 10:16:50 AM3/6/20
to hai huang, Prometheus Users
hmmm this maybe linked to the WAL size ?

On Fri, Mar 6, 2020 at 3:31 PM hai huang <huan...@gmail.com> wrote:
I am getting the error message "Error on ingesting samples that are too old or are too far into the future" when trying to put my own custom timestamp into the scrape data. According to this issue (https://github.com/prometheus/prometheus/issues/4911), this problem problem has been resolved. However, in that issue, the reporter only tried to put a custom timestamp that is 30 minutes back. In my test, custom timestamp works fine unless it is older than 1 hour. Is there a configuration that one could tweak? Thanks!

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/d92ae30c-dabb-4082-b7ee-8a6e8059451f%40googlegroups.com.

hai huang

unread,
Mar 6, 2020, 11:13:47 AM3/6/20
to Prometheus Users
I don't see anything that allows me to change its behavior. I've increased --storage-tsdb.retention.time and --query.lookback-delta, and it didn't change anything.


On Friday, March 6, 2020 at 10:16:50 AM UTC-5, sayf eddine Hammemi wrote:
hmmm this maybe linked to the WAL size ?

On Fri, Mar 6, 2020 at 3:31 PM hai huang <huan...@gmail.com> wrote:
I am getting the error message "Error on ingesting samples that are too old or are too far into the future" when trying to put my own custom timestamp into the scrape data. According to this issue (https://github.com/prometheus/prometheus/issues/4911), this problem problem has been resolved. However, in that issue, the reporter only tried to put a custom timestamp that is 30 minutes back. In my test, custom timestamp works fine unless it is older than 1 hour. Is there a configuration that one could tweak? Thanks!

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to promethe...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages