Prometheus remote write to InfluxDB

[dzh...@contextlogic.com]

Hello all,

I am trying to set up a remote write path from our Prometheus server to a remote InfluxDB database. I keep getting errors like these whenever I start Prometheus:

WARN[0097] Error sending 61 samples to remote storage: server returned HTTP status 400 Bad Request: {"error":"unable to parse '\ufffd@\ufffdF': missing fields\nunable to parse 'r': missing fields\nunable to parse '8': missing fields\nunable to parse '\u0008__name__\u0012,prometheus_sd_file_scan_duration_seconds_sum': missing tag value\nunable to parse '\  source="queue_manager.go:500"

prometheus.yml:

global:
   scrape_interval: 15s

remote_write:
   - url: "http://<IP of InfluxDB instance>:8086/write?db=Prometheus"

scrape_configs:
  - job_name: prometheus
    static_configs:
    - targets: ['localhost:9090']

Any help would be greatly appreciated. Thanks a lot in advance.

DZ

[Ce D. Hardaway]

I'm facing a mistake which really looks like this one. Did you find a way to get threw it?
Thanks in advance for your reply.

Regards.

[Benjamin Ritcey]

That looks like you're trying to write directly to InfluxDB, which won't work;  if you're using a nightly build of Influx, the Prometheus write endpoint is something like

- url: "http://<IP of InfluxDB>:8086/api/v1/prom/write?u=paul&p=foo&db=prometheus"

per https://www.influxdata.com/blog/influxdb-now-supports-prometheus-remote-read-write-natively/ - but, again, that's not available in the 1.3 release, it'll be there in 1.4 when it's released.

Until 1.4 is released, I believe you need

https://github.com/prometheus/prometheus/tree/master/documentation/examples/remote_storage/remote_storage_adapter

[Ce D. Hardaway]

Hello Benjamin, thanks a lot for your feedback,

I didn't manage to go threw the installation of that package:
$go build

./main.go:168: cannot use c (type *graphite.Client) as type writer in append:

        *graphite.Client does not implement writer (wrong type for Write method)

                have Write("github.com/prometheus/prometheus/vendor/github.com/prometheus/common/model".Samples) error

                want Write("github.com/prometheus/common/model".Samples) error

./main.go:176: cannot use c (type *opentsdb.Client) as type writer in append:

        *opentsdb.Client does not implement writer (wrong type for Write method)

                have Write("github.com/prometheus/prometheus/vendor/github.com/prometheus/common/model".Samples) error

                want Write("github.com/prometheus/common/model".Samples) error

./main.go:191: cannot use conf (type "github.com/influxdata/influxdb/client/v2".HTTPConfig) as type "github.com/prometheus/prometheus/vendor/github.com/influxdata/influxdb/client/v2".HTTPConfig in argument to influxdb.NewClient

./main.go:196: cannot use c (type *influxdb.Client) as type "github.com/prometheus/client_golang/prometheus".Collector in argument to "github.com/prometheus/client_golang/prometheus".MustRegister:

        *influxdb.Client does not implement "github.com/prometheus/client_golang/prometheus".Collector (wrong type for Collect method)

                have Collect(chan<- "github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus".Metric)

                want Collect(chan<- "github.com/prometheus/client_golang/prometheus".Metric)

./main.go:197: cannot use c (type *influxdb.Client) as type writer in append:

        *influxdb.Client does not implement writer (wrong type for Write method)

                have Write("github.com/prometheus/prometheus/vendor/github.com/prometheus/common/model".Samples) error

                want Write("github.com/prometheus/common/model".Samples) error

Could you provide a little bit more of your help?
Thanks in advance.
Regards.

[Benjamin Ritcey]

Not sure I can help at this point; just a wild guess, are you using a version of go older than 1.8?

It build ok for me w/ go 1.9