job/task level metrics via jmx

[Marcin]

Hi,

it looks like a limited set of metrics is exposed via JMX, namely only the ones related to jvm stuff (such as gc, memory, threads) whereas the job/task level metrics are skipped. Is it done like that on purpose, or we're just missing something?

We use Gobblin for Kafka -> HDFS/HBase ingestion and would like to make use of some of its metrics in our monitoring infrastructure (Prometheus + Kibana).

Cheers,

Marcin

[Abhishek Tiwari]

Hi Marcin,

We are not big on exposing metrics via JMX, but checkout: http://gobblin.readthedocs.io/en/latest/metrics/Gobblin-Metrics/ 

This outlines various mechanisms you can use to capture metrics being emitted by Gobblin. 

Regards, 

Abhishek

--
You received this message because you are subscribed to the Google Groups "gobblin-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gobblin-users+unsubscribe@googlegroups.com.
To post to this group, send email to gobbli...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gobblin-users/194a688d-d028-48bc-804e-e3c8c5bedc94%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marcin]

ok; thanks Abhishek!

Cheers,

Marcin

On Tuesday, April 4, 2017 at 8:04:35 PM UTC+2, Abhishek Tiwari wrote:

Hi Marcin,

We are not big on exposing metrics via JMX, but checkout: http://gobblin.readthedocs.io/en/latest/metrics/Gobblin-Metrics/ 

This outlines various mechanisms you can use to capture metrics being emitted by Gobblin. 

Regards, 

Abhishek

On Tue, Apr 4, 2017 at 5:32 AM, Marcin <marcin.s...@gmail.com> wrote:

Hi,

it looks like a limited set of metrics is exposed via JMX, namely only the ones related to jvm stuff (such as gc, memory, threads) whereas the job/task level metrics are skipped. Is it done like that on purpose, or we're just missing something?

We use Gobblin for Kafka -> HDFS/HBase ingestion and would like to make use of some of its metrics in our monitoring infrastructure (Prometheus + Kibana).

Cheers,

Marcin

--
You received this message because you are subscribed to the Google Groups "gobblin-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to gobblin-user...@googlegroups.com.

[lbe...@gmail.com]

Hi Marcin,

Did you try to use the GraphiteReporter along with Prometheus + https://github.com/prometheus/graphite_exporter ?

You only need to map the flat graphite metrics to the multi-dimensional Prometheus ones and scrape from the graphite endpoint.

Thanks,

Lorand