Import static CSV file to Prometheus server

[Sandosh Kumar P]

Hi,

I need help on how to import the below sample csv file every 30 mins into prometheus server in order to see the trend of down services. Can you help me with an example & documentation if any?

Example CSV file: 

testserver1,service1,stopped

testserver1,service2,running

testserver2,service1,running

testserver2,service2,stopped

Prometheus URL: http://localhost:9090

Thanks

Sandosh

[Brian Candler]

Prometheus isn't designed to be used this way. The normal operating mode of prometheus is "scraping", where it makes a periodic outbound HTTP call (say once per minute) to fetch the *current* version of each metric, and ingest it into its database.

In particular you should note:

1. There is a "backfill" capability but the data has to be in Openmetrics format, not CSV.  And there are a number of restrictions, like not being able to import data from the last 3 hours. It's really just for importing historical data, with new data coming from regular scrapes.

2. All metrics in Prometheus are numeric; you can't have values like "stopped" or "running".  To model that, you'd normally make an enumerated set of timeseries:

foo_state{instance="testserver1",service="service1",state="stopped"} 1

foo_state{instance="testserver1",service="service1",state="running"} 0

3. You *could* write a program which reads your CSV every half hour, and then uses the remote_write capability to post the data to Prometheus.  But you're going to have to do all that integration work yourself, i.e. it involves programming against the remote write API.  I don't see any existing integration for CSV.

In short: you're using Prometheus the wrong way, and if you need to work this way you're probably better off using a different database.

[Brian Candler]

On Monday, 30 January 2023 at 21:46:39 UTC sando...@gmail.com wrote:

Example CSV file: 

testserver1,service1,stopped

testserver1,service2,running

testserver2,service1,running

testserver2,service2,stopped

I should add: I notice that the CSV sample data isn't timestamped, so it seems that every 30 minutes you just want to ingest the *current* versions of those values, not any previous history of what's been going on over the last 30 minutes.  In that case, it is pretty straightforward:

* On a host of your choice (which could be the prometheus server itself), run node_exporter with the textfile collector enabled:   --collector.textfile.directory=/var/lib/node_exporter

* Whenever you get a new CSV file, run it through a simple script (which you'll have to write yourself) to convert it into openmetrics format.  In this case, the output would be:

foo_status{instance="testserver1",service="service1",state="stopped"} 1

foo_status{instance="testserver1",service="service1",state="running"} 0

foo_status{instance="testserver1",service="service2",state="stopped"} 0

foo_status{instance="testserver1",service="service2",state="running"} 1

foo_status{instance="testserver2",service="service1",state="stopped"} 0

foo_status{instance="testserver2",service="service1",state="running"} 1

foo_status{instance="testserver2",service="service2",state="stopped"} 1

foo_status{instance="testserver2",service="service2",state="running"} 0

* Write this to /var/lib/node_exporter/my_status.prom.new, and then rename /var/lib/node_exporter/my_status.prom.new to /var/lib/node_exporter/my_status.prom

* Configure prometheus to scrape node_exporter at least once every 2 minutes - I'd recommend once per minute or more.

These metrics will then appear in prometheus, and will be updated whenever you decide to update the CSV file.

Longer term, you can consider changing the process for collecting this data so that it is exposed directly in openmetrics format and can be scraped directly.

[Sandosh Kumar P]

Hi Brian,

Thank you so much. I will take a look on both these suggestions.

Thanks

Sandosh