Migrate existing data from prometheus storage to victoria single node

[pmsd...@gmail.com]

Hi, I have nearly 1.5 T of prometheus data stored on the local-storage of my prometheus server  (it handles 3.3 million metrics, it ingests 50.5Kops metrics) and I need to migrate to victorametrics

so that Grafana can send heavy requests to it, and prevent crashing my prometheus.

Can you provide on the procedure to migrate/replicate the data from prometheus storage to victoria storage, and a

Guide to  dimensioning victorametrics node / cluster ?

 Thanks in advance, 

[Aliaksandr Valialkin]

Hi,

On Thu, Jul 11, 2019 at 5:29 PM <pmsd...@gmail.com> wrote:

Hi, I have nearly 1.5 T of prometheus data stored on the local-storage of my prometheus server  (it handles 3.3 million metrics, it ingests 50.5Kops metrics) and I need to migrate to victorametrics

so that Grafana can send heavy requests to it, and prevent crashing my prometheus.

Can you provide on the procedure to migrate/replicate the data from prometheus storage to victoria storage

There is no a tool for migrating historical data from Prometheus to VictoriaMetrics yet, but such a tool should be quite easy to create. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/93 for details.

 

Guide to  dimensioning victorametrics node / cluster ?

See https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/README.md#capacity-planning for basic information on capacity planning. It covers mostly data ingestion part and doesn't cover query path, because hardware resources for query path heavily depend on query workload. It is better to start with a certain resources for query path and adjust them according to the actual load.

Best Regards,

Aliaksandr

[pmsd...@gmail.com]

Thanks for the prompt reply,

now main reason to migrate is that queries hitting prometheus are bringing it down, Is the ingestion controlled in a separated way from the reading of the metrics?

Can I somehow dedicate a box to "reads" so that I do an in-place replacement for grafana; and another box for ingestion (remote writes)?

is so, is there a way to dimension it? 

[Aliaksandr Valialkin]

On Thu, Jul 11, 2019 at 6:40 PM <pmsd...@gmail.com> wrote:

Thanks for the prompt reply,

now main reason to migrate is that queries hitting prometheus are bringing it down, Is the ingestion controlled in a separated way from the reading of the metrics?

Single-node VictoriaMetrics runs both ingestion code and query code in a single operating system process, so the ingestion and query paths may contend for the available resources - RAM, CPU, network, storage.

Cluster VictoriaMetrics splits data ingestion and query paths - the ingestion mostly consumes resources of vminsert layer, while the query path mostly consumes the resources of vmselect layer. These layers usually run on distinct machines, so they don't interfere with each other. To be clear, these layers still share resources on the vmstorage layer, but the majority of resources are used on vminsert and vmselect layers. See cluster overview for understanding better how these layers work and interact with each other.

 

Can I somehow dedicate a box to "reads" so that I do an in-place replacement for grafana; and another box for ingestion (remote writes)? 

It is impossible to do with single-node version, but this is how cluster version actually works - "reads" consume mostly resources on vmselect layer, while "writes" consume vminsert resources.

 

is so, is there a way to dimension it? 

All the layers of the cluster version - vminsert, vmselect and vmstorage - may independently scale to multiple nodes and may run on the most suitable hardware configuration.

 

On Thursday, 11 July 2019 16:29:40 UTC+2, pmsd...@gmail.com wrote:

Hi, I have nearly 1.5 T of prometheus data stored on the local-storage of my prometheus server  (it handles 3.3 million metrics, it ingests 50.5Kops metrics) and I need to migrate to victorametrics

so that Grafana can send heavy requests to it, and prevent crashing my prometheus.

Can you provide on the procedure to migrate/replicate the data from prometheus storage to victoria storage, and a

Guide to  dimensioning victorametrics node / cluster ?

 Thanks in advance, 

--
You received this message because you are subscribed to the Google Groups "victorametrics-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to victorametrics-u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/victorametrics-users/5c23b738-5926-4ef6-adf1-4adb3978cb86%40googlegroups.com.

--

Best Regards,

Aliaksandr