[Pg_Prometheus] Remote read doesn't seem to work

[imra...@gmail.com]

Hi,

I am trying to set up postgresql as a remote storage for prometheus. I have the following configuration.
Looks like the data from scrapes is written to postgresql db. However, when I inserted data into the database, prometheus is not picking it up.

It seems to only show the data scraped. How can I enable remote read from the database?

thanks,
Imran

INSERT INTO metrics VALUES ('cpu_usage{service="nginx",host="machine1"} 34.6 1494595898000');

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).


  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'zodiac-external'
remote_write:
  - url: "http://prometheus_postgresql_adapter:9201/write"
remote_read:
  - url: "http://prometheus_postgresql_adapter:9201/read"


# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'zodiac'
    honor_labels: true
    scrape_interval: 5s
    metrics_path: '/metrics'


    static_configs:
      - targets:
         - 'docker.for.mac.localhost:9091'

[imra...@gmail.com]

Hi,

Any ideas what is wrong with my config?

thanks,

Imran

[imra...@gmail.com]

I should have done some more investigation on this. Figured out, after digging a bit more.

1. For some reason, Prometheus UI only shows scraped metrics in the drop down. That made me believe that the metric I added directly into remote database is not being read.
2. I integrated this prometheus instance with Grafana. Also, Grafana UI doesn't show the new metric I added in the database directly. But when queried, returns results.

How does prometheus expose these metrics (names)?

thanks,
Imran

On Tuesday, December 5, 2017 at 5:20:44 PM UTC-8, imra...@gmail.com wrote:

[Tom Wilkie]

You are correct, remote read doesn’t implement the right methods to make this work. Please file an issue for it, it is something we want to support at some point. I suspect we will want to move to gRPC for remote read first though.

Thanks

Tom

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/92d52a2e-5b1c-4a2f-87a0-a85d92379a97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Imran]

Thank you for the reply, Tom.

Here is the issue: https://github.com/prometheus/prometheus/issues/3560

thanks,
Imran

To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.

[sesha...@gmail.com]

hi

I am doing similar to what you have done. postgresql as remote storage for prometheus.  prometheus-remote-storage-adapter working fine

But I do not see any data. Which table I have to query ?  I. have metrics view in Postgres db and metrics_values etc.  

You have written that data from scrapes  is written to postgresql.  Where it is there ?

thanks

Krish

On Tuesday, December 5, 2017 at 5:20:44 PM UTC-8, imra...@gmail.com wrote: