What is the best and correct way to push metrics to Prometheus from json file?
4,392 views
Skip to first unread message
epich...@gmail.com
Aug 13, 2018, 4:09:57 PM8/13/18
to Prometheus Developers
Hi,
I'd like not to pull the metrics but push my own metrics to Prometheus after running some tests and recording the results to a json file. I am thinking to create grafana dashboard (based on Prometheus datasource) with green/red gauge for each test.
Example:
{
"servers": {
"server1": {
"ping": 1,
"auth": 1,
"push": 0
}
}
}
Is it possible?
Thank you
Matt Doughty
Aug 13, 2018, 4:30:30 PM8/13/18
to epich...@gmail.com, Prometheus Developers
All things are possible, but the closest thing to a paved path is probably using the push gatway:
https://github.com/prometheus/pushgateway
https://github.com/prometheus/pushgateway
--Matt
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Matt
Julius Volz
Aug 13, 2018, 4:31:44 PM8/13/18
to Matt Doughty, Katya Pichugin, Prometheus Developers
...or if you cannot change the service that runs the tests and produces JSON to use the Pushgateway, you could also write an exporter (https://prometheus.io/docs/instrumenting/writing_exporters/) that reads the JSON file and translates it into Prometheus metrics (that can be pulled).
On Mon, Aug 13, 2018 at 10:30 PM, Matt Doughty <mdou...@gmail.com> wrote:
All things are possible, but the closest thing to a paved path is probably using the push gatway:
https://github.com/prometheus/pushgateway--Matt
On Mon, Aug 13, 2018 at 4:09 PM, <epich...@gmail.com> wrote:
Hi,
I'd like not to pull the metrics but push my own metrics to Prometheus after running some tests and recording the results to a json file. I am thinking to create grafana dashboard (based on Prometheus datasource) with green/red gauge for each test.
Example:
{
"servers": {
"server1": {
"ping": 1,
"auth": 1,
"push": 0
}
}
}
Is it possible?
Thank you
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscri...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
----Matt
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAGyBzcw9CX_9BJfrpw1AkigWSOosGcXj-GDjx6p50f%3DyYFGxQg%40mail.gmail.com.
epich...@gmail.com
Aug 13, 2018, 4:33:46 PM8/13/18
to Prometheus Developers
Thank you for a quick reply, I've tried that and trying to deal with "bad gateway" error when creating a prometheus datasource in grafana.
On Monday, 13 August 2018 16:30:30 UTC-4, Matt Doughty wrote:
> All things are possible, but the closest thing to a paved path is probably using the push gatway:
> https://github.com/prometheus/pushgateway
>
>
>
> --Matt
>
>
> On Mon, Aug 13, 2018 at 4:09 PM, <epich...@gmail.com> wrote:
> Hi,
>
>
>
> I'd like not to pull the metrics but push my own metrics to Prometheus after running some tests and recording the results to a json file. I am thinking to create grafana dashboard (based on Prometheus datasource) with green/red gauge for each test.
>
>
>
> Example:
>
> {
>
> "servers": {
>
>
>
> "server1": {
>
> "ping": 1,
>
> "auth": 1,
>
> "push": 0
>
> }
>
> }
>
> }
>
>
>
> Is it possible?
>
>
>
> Thank you
>
>
>
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
>
> To post to this group, send email to prometheus...@googlegroups.com.
On Monday, 13 August 2018 16:30:30 UTC-4, Matt Doughty wrote:
> All things are possible, but the closest thing to a paved path is probably using the push gatway:
> https://github.com/prometheus/pushgateway
>
>
>
> --Matt
>
>
> On Mon, Aug 13, 2018 at 4:09 PM, <epich...@gmail.com> wrote:
> Hi,
>
>
>
> I'd like not to pull the metrics but push my own metrics to Prometheus after running some tests and recording the results to a json file. I am thinking to create grafana dashboard (based on Prometheus datasource) with green/red gauge for each test.
>
>
>
> Example:
>
> {
>
> "servers": {
>
>
>
> "server1": {
>
> "ping": 1,
>
> "auth": 1,
>
> "push": 0
>
> }
>
> }
>
> }
>
>
>
> Is it possible?
>
>
>
> Thank you
>
>
>
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
>
> To post to this group, send email to prometheus...@googlegroups.com.
Matt Doughty
Aug 13, 2018, 4:43:52 PM8/13/18
to epich...@gmail.com, Prometheus Developers
I'm not sure I understand. Do you have the pushgateway setup with prometheus scraping it? Where are you seeing the bad gateway errors?
--Matt
On Mon, Aug 13, 2018 at 4:33 PM, <epich...@gmail.com> wrote:
Thank you for a quick reply, I've tried that and trying to deal with "bad gateway" error when creating a prometheus datasource in grafana.
On Monday, 13 August 2018 16:30:30 UTC-4, Matt Doughty wrote:
> All things are possible, but the closest thing to a paved path is probably using the push gatway:
> https://github.com/prometheus/pushgateway
>
>
>
> --Matt
>
>
> On Mon, Aug 13, 2018 at 4:09 PM, <epich...@gmail.com> wrote:
> Hi,
>
>
>
> I'd like not to pull the metrics but push my own metrics to Prometheus after running some tests and recording the results to a json file. I am thinking to create grafana dashboard (based on Prometheus datasource) with green/red gauge for each test.
>
>
>
> Example:
>
> {
>
> "servers": {
>
>
>
> "server1": {
>
> "ping": 1,
>
> "auth": 1,
>
> "push": 0
>
> }
>
> }
>
> }
>
>
>
> Is it possible?
>
>
>
> Thank you
>
>
>
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
>
> To post to this group, send email to prometheus...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
> --
>
> --Matt
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/075e6f98-9a2a-437a-ae64-f5f8df2cf9a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Matt
Julius Volz
Aug 13, 2018, 5:17:02 PM8/13/18
to Matt Doughty, Katya Pichugin, Prometheus Developers
The architecture you would be going for should be this:
application ---[PUSH]---> pushgateway <--[SCRAPE]--- Prometheus <---[QUERY]--- Grafana
On Mon, Aug 13, 2018 at 10:43 PM, Matt Doughty <mdou...@gmail.com> wrote:
I'm not sure I understand. Do you have the pushgateway setup with prometheus scraping it? Where are you seeing the bad gateway errors?--Matt
On Mon, Aug 13, 2018 at 4:33 PM, <epich...@gmail.com> wrote:
Thank you for a quick reply, I've tried that and trying to deal with "bad gateway" error when creating a prometheus datasource in grafana.
On Monday, 13 August 2018 16:30:30 UTC-4, Matt Doughty wrote:
> All things are possible, but the closest thing to a paved path is probably using the push gatway:
> https://github.com/prometheus/pushgateway
>
>
>
> --Matt
>
>
> On Mon, Aug 13, 2018 at 4:09 PM, <epich...@gmail.com> wrote:
> Hi,
>
>
>
> I'd like not to pull the metrics but push my own metrics to Prometheus after running some tests and recording the results to a json file. I am thinking to create grafana dashboard (based on Prometheus datasource) with green/red gauge for each test.
>
>
>
> Example:
>
> {
>
> "servers": {
>
>
>
> "server1": {
>
> "ping": 1,
>
> "auth": 1,
>
> "push": 0
>
> }
>
> }
>
> }
>
>
>
> Is it possible?
>
>
>
> Thank you
>
>
>
>
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscri...@googlegroups.com.
>
> To post to this group, send email to prometheus...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
> --
>
> --Matt
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscri...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/075e6f98-9a2a-437a-ae64-f5f8df2cf9a5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
----Matt
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAGyBzcxV4TJJf6iL8S-un8EYg4YUy1Tp%3DREgyuj-%2BCRSO3qPQQ%40mail.gmail.com.
epich...@gmail.com
Aug 13, 2018, 7:03:05 PM8/13/18
to Prometheus Developers
Right. I am stuck at Query step I think.
Steps done:
1. Running Prometheus
2. Running Pushgateway:
docker pull prom/pushgateway
docker run -d -p 9091:9091 prom/pushgateway
3. Updated prometheus.yml by adding honer_labels: true and target with pushgateway:
- job_name: node
honor_labels: true
scrape_interval: 5s
scrape_timeout: 5s
metrics_path: /metrics
scheme: http
static_configs:
- targets:
- aaaa:9100
- targets:
- bbbb:9182
- targets:
- xxxx:9091
4. Restarted Prometheus container.
5. Ran
echo "some_metric 3.14" | curl --data-binary @- http://xxxx:9091/metrics/job/some_job
Returned the page with metrics and status. Looks good.
6. In Grafana trying to add a new datasource with URL http://xxxx:9091/ and it gives me "HTTP Error Not Found" when trying to save and test a datasource.
And from docker logs <grafana_container>:
2018/08/13 22:53:52 http: proxy error: dial tcp 127.0.0.1:9091: connect: connection refused
I do use --web.external-url="http://xxxx:3000" for alertmanager and prometheus. Grafana doesn't support that.
Googling now a solution.
Any ideas? Thank you!
> > To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
Steps done:
1. Running Prometheus
2. Running Pushgateway:
docker pull prom/pushgateway
docker run -d -p 9091:9091 prom/pushgateway
3. Updated prometheus.yml by adding honer_labels: true and target with pushgateway:
- job_name: node
honor_labels: true
scrape_interval: 5s
scrape_timeout: 5s
metrics_path: /metrics
scheme: http
static_configs:
- targets:
- aaaa:9100
- targets:
- bbbb:9182
- targets:
- xxxx:9091
4. Restarted Prometheus container.
5. Ran
echo "some_metric 3.14" | curl --data-binary @- http://xxxx:9091/metrics/job/some_job
Returned the page with metrics and status. Looks good.
6. In Grafana trying to add a new datasource with URL http://xxxx:9091/ and it gives me "HTTP Error Not Found" when trying to save and test a datasource.
And from docker logs <grafana_container>:
2018/08/13 22:53:52 http: proxy error: dial tcp 127.0.0.1:9091: connect: connection refused
I do use --web.external-url="http://xxxx:3000" for alertmanager and prometheus. Grafana doesn't support that.
Googling now a solution.
Any ideas? Thank you!
>
> >
>
> > To post to this group, send email to prometheus...@googlegroups.com.
>
> >
>
> > To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
>
> >
>
> > For more options, visit https://groups.google.com/d/optout.
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > --
>
> >
>
> > --Matt
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
> >
>
> > To post to this group, send email to prometheus...@googlegroups.com.
>
> >
>
> > To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
>
> >
>
> > For more options, visit https://groups.google.com/d/optout.
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > --
>
> >
>
> > --Matt
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/075e6f98-9a2a-437a-ae64-f5f8df2cf9a5%40googlegroups.com.
>
>
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
> --
>
> --Matt
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-devel...@googlegroups.com.
>
>
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
> --
>
> --Matt
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
epich...@gmail.com
Aug 13, 2018, 7:06:58 PM8/13/18
to Prometheus Developers
To avoid confusion, 3000 is my Prometheus port, port 3002 is Grafana.
Paul Traylor
Aug 13, 2018, 7:40:41 PM8/13/18
to epich...@gmail.com, prometheus...@googlegroups.com
If your Prometheus port is 3000, then you should add http://xxxx:3000
in Grafana so that Grafana can talk to your Prometheus instance.
Grafana does not know how to read from pushgateway
2018年8月14日(火) 8:07 <epich...@gmail.com>:
in Grafana so that Grafana can talk to your Prometheus instance.
Grafana does not know how to read from pushgateway
2018年8月14日(火) 8:07 <epich...@gmail.com>:
epich...@gmail.com
Aug 13, 2018, 11:04:30 PM8/13/18
to Prometheus Developers
Thank you.
At first I thought that http://xxxx:3000 wouldn't show my added metrics, but now I see it. I guess the configuration I added to prometheus.yml worked properly.
I will try to create green/red dashboard now.
Thanks again for a quick replies.
At first I thought that http://xxxx:3000 wouldn't show my added metrics, but now I see it. I guess the configuration I added to prometheus.yml worked properly.
I will try to create green/red dashboard now.
Thanks again for a quick replies.
Julius Volz
Aug 14, 2018, 6:20:18 AM8/14/18
to Katya Pichugin, Prometheus Developers
You are adding the Pushgateway's port here (which Grafana cannot query), you probably want 9090 to query Prometheus.
> > To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
>
> >
>
> > To post to this group, send email to prometheus...@googlegroups.com.
>
> >
>
> > To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/bdb05b24-618d-4856-b53a-619221d2f88c%40googlegroups.com.
>
> >
>
> > For more options, visit https://groups.google.com/d/optout.
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > --
>
> >
>
> > --Matt
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/075e6f98-9a2a-437a-ae64-f5f8df2cf9a5%40googlegroups.com.
>
>
>
> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
>
> --
>
> --Matt
>
>
>
>
>
> --
>
> You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
>
> To post to this group, send email to prometheus...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAGyBzcxV4TJJf6iL8S-un8EYg4YUy1Tp%3DREgyuj-%2BCRSO3qPQQ%40mail.gmail.com.
>
>
>
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsub...@googlegroups.com.
To post to this group, send email to prometheus-developers@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/84303f79-c7fd-4d58-a06b-7736f1557c5c%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages