About prometheus backup and restore snapshot documentation

5,914 views
Skip to first unread message

suyog.k...@gslab.com

unread,
Apr 30, 2018, 5:01:58 AM4/30/18
to Prometheus Users
Hello ,

This is a request for general guidance or at least improved documentation in the backup/restore area and the use of snapshots.

kevinbla...@gmail.com

unread,
Apr 30, 2018, 9:52:51 AM4/30/18
to Prometheus Users
First, for the snapshot when you launch prometheus, you'll need to add
--web.enable-admin-api

Then after that, you can trigger a snapshot by hitting:
curl -XPOST http://<promethues IP:Port>/api/v2/admin/tsdb/snapshot

This will respond with something like:

{"name":"2018-04-30T21:45:35Z-6e4d24be51aa32e3a"}


From the prometheus server, look in <value of --storage.tsdb.path>/snapshots and you'll see a dir with the name matching the response above. 

If you need to recover from snapshot, you can just stop prometheus, delete the items in the storage.tsdb.path directory, and put the contents of the snapshot directory in its place.  Or you can update the actual value of the storage.tsdb.path to point to the snapshot, but I prefer the first way.


The snapshot command can be ran from a cron job or similar to trigger regular cleanups.
Since it's creating hard links, space usage and time to initially create is minimal.  But if you move the files, and over time you'll see the space needs increase. 

You can just rm contents of the snapshot folder when they're no longer needed, or copy them off to a different location after the snapshot is generated  The snapshot can not just target a different location initially. 

suyog.k...@gslab.com

unread,
May 2, 2018, 1:03:20 AM5/2/18
to Prometheus Users
Hi Thanks for your reply.. i just want to know how do i take a snapshots of prometheus in Container environment. How do i pass the --web.enable-admin-api to running container.

Sutirtha Das

unread,
Jan 7, 2019, 8:30:15 PM1/7/19
to Prometheus Users
Thank you very much. I've been looking for a complete step guide for a while. Worked like a charm.

bp3...@gmail.com

unread,
Apr 23, 2019, 12:00:24 PM4/23/19
to Prometheus Users
How do you stop prometheus? Is it a simple kill -SIGHUP?

Ben Kochie

unread,
Apr 23, 2019, 1:23:59 PM4/23/19
to bp3...@gmail.com, Prometheus Users
SIGTERM or use the lifecycle API[0] (not enabled by default)


--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/23547fbb-febe-46b2-97a3-4012ffb7f58f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages