How to push Hystrix metrics to Prometheus

[charmin...@gmail.com]

Hi,
I am new to both Hystrix and Prometheus. I understand that Prometheus_Hystrix is not part of the development handled by Prometheus developers. But, I'm trying my luck here and see if my question can be answered.
I wanted to know what are the steps on showing the Hystrix metrics to Prometheus. I have integrated this code https://github.com/soundcloud/prometheus-hystrix to my Java code.
I only installed the precompiled binary for prometheus (https://prometheus.io/download/) which is now running at http://localhost:9090. I can see both the standard metrics and graph. After this, do I need to configure the prometheus.yml file to push the hystrix metrics to Prometheus? My Java code runs on wildfly with this URL http://localhost:8080/hystrix/test.

Can someone help on what I should do further?

Thanks a lot in advance.

Kind Regards,
Charmine

[Björn Rabenstein]

On 19 July 2016 at 01:29, <charmin...@gmail.com> wrote:
>
> After this, do I need to configure the prometheus.yml file to push
> the hystrix metrics to Prometheus? My Java code runs on wildfly
> with this URL http://localhost:8080/hystrix/test

Prometheus will pull in the metrics. You have to expose the metrics on
an HTTP endpoint in your Java code (usually /metrics) and then
configure that endpoint in your prometheus.yml.

Look at the javadoc of the Prometheus Java client to get an idea how it works.

--
Björn Rabenstein, Engineer
http://soundcloud.com/brabenstein

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany
Managing Director: Alexander Ljung | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG Charlottenburg |
HRB 110657B

[charmin...@gmail.com]

On Tuesday, 19 July 2016 19:54:39 UTC+10, Björn Rabenstein wrote:

Thank you Björn for your quick reply.
How to expose the wildfly endpoint? My metrics now appears at http://localhost:8080/hystrix/metrics --> JBOSS Wildfly 10 server
I have the following in my prometheus.yml file. It seems not showing the metrics to
http://localhost:9090/metrics --> prometheus server

---------
prometheus.yml file
global:
scrape_interval: 15s
evaluation_interval: 15s

external_labels:
monitor: 'codelab-monitor'
scrape_configs:
- job_name: "hystrix"

scrape_interval: 5s

metrics_path: /hystrix/metrics

static_configs:
- targets: ['localhost:8080']
------------

Thanks again for your help.

[Björn Rabenstein]

On 20 July 2016 at 08:39, <charmin...@gmail.com> wrote:
> How to expose the wildfly endpoint? My metrics now appears at http://localhost:8080/hystrix/metrics --> JBOSS Wildfly 10 server
> I have the following in my prometheus.yml file. It seems not showing the metrics to
> http://localhost:9090/metrics --> prometheus server

Your config looks good. If you see metrics at
http://localhost:8080/hystrix/metrics, you made it.
http://localhost:9090/metrics will only show metrics about the
Prometheus server itself, not the metrics the Prometheus server has
ingested.

You should be able to see the ingested hystrix metrics in the
expression browser now ( http://localhost:9090/graph ). If not, check
out http://localhost:9090/targets , where you can see if the scrape
succeeded.

[charmin...@gmail.com]

On Wednesday, 20 July 2016 21:19:01 UTC+10, Björn Rabenstein wrote:

Thank you very much Björn. I can now see the hystrix metrics in Prometheus.

[sushan...@gmail.com]

Hey, how did you get the hystrix metrics ? when I'm hitting /metrics endpoint I'm getting some data but that data seems to be actually spring actuator metrics and it is not in prometheus format! So it is throwing "text format parsing error... " can you please help?

[Björn Rabenstein]

On 8 November 2017 at 17:16, <sushan...@gmail.com> wrote:
>
> Hey, how did you get the hystrix metrics ? when I'm hitting /metrics endpoint I'm getting some data but that data seems to be actually spring actuator metrics and it is not in prometheus format! So it is throwing "text format parsing error... " can you please help?

Could you post the metrics output you see?

[bsus...@athenahealth.com]

I'll explain in detail what I've done:
I've this code >> http://jsoneditoronline.org/?id=44919d5633d2613e0931753e35e51af5
which is exposing spring boot metrics(I guess) to prometheus endpoint.
Then before actually starting the spring application I add this line >>
HystrixPrometheusMetricsPublisher.register("Gateway");
For these to work I've prometheus-hystrix library added + simpleclient_spring_boot library

Now I can see some data without any error on /prometheus endpoint but I'm not sure whether this is actually hystrix data. here>> http://jsoneditoronline.org/?id=44919d5633d2613e0931753e35e62e7e

When I point my prometheus.yml file >>

global:
scrape_interval: 15s
evaluation_interval: 15s

scrape_configs:
- job_name: 'gateway'
scrape_interval: 90s
scrape_timeout: 90s
metrics_path: '/prometheus'
static_configs:
- targets: ['localhost:18888']

like above then I get status as UP but in graph section it says no data points found.

[bsus...@athenahealth.com]

Now after some tweaking I'm getting everything related to spring-boot-metrics in prometheus but still not able to get the hystrix metrics? can you help?

[bsus...@athenahealth.com]

Does anybody has a sample java code for getting hystrix metrics ?