Confluent Replicator monitoring - using JMX /Prometheus

[karan alang]

Hi All - 

have a question regd. confluent replicator monitoring..

I'm trying to use JMX_PORT to monitor Replicator Producers/Consumer

(ref - http://docs.confluent.io/current/multi-dc/replicator-tuning.html)

Here is what i've done -

1) set JMX_PORT(=12345) & re-started Kafka Connect

2)

Then, i've configured Prometheus yaml to add the Connect  

- job_name: 'connect'
    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    static_configs:
      - targets: ['<host where kafka-connect runs>:12345']
 

 

3) started the replicator & the producer/consumer

Producer ->

$CONFLUENT1/bin/kafka-producer-perf-test --topic repStr3 --num-records 10000000 --record-size 1000 --throughput 10000 --producer-props bootstrap.servers=hostname:8092 interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor acks=all

Consumer ->

$CONFLUENT/bin/kafka-console-consumer --topic repStr3.replica --new-consumer --bootstrap-server hostname:9092 --consumer.config $CONFLUENT/consumer3.config

However, i don't see the confluent producer/consumer metrics in Prometheus.

so, do i need to do any additional steps to enable monitoring of the Replicator producer/consumer ?

Pls .let me know.

Pls note -> i plan on displaying the metrics on Grafana dashboard (by linking Prometheus + Grafana)

thanks! 

[Brian Brazil]

On 24 August 2017 at 23:02, karan alang <karan...@gmail.com> wrote:

Hi All - 

have a question regd. confluent replicator monitoring..

I'm trying to use JMX_PORT to monitor Replicator Producers/Consumer

(ref - http://docs.confluent.io/current/multi-dc/replicator-tuning.html)

Here is what i've done -

1) set JMX_PORT(=12345) & re-started Kafka Connect

Prometheus doesn't talk JMX directly, you should use the jmx_exporter as a java agent as an intermediary.

Brian

 

2)

Then, i've configured Prometheus yaml to add the Connect  

- job_name: 'connect'
    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    static_configs:
      - targets: ['<host where kafka-connect runs>:12345']
 

 

3) started the replicator & the producer/consumer

Producer ->

$CONFLUENT1/bin/kafka-producer-perf-test --topic repStr3 --num-records 10000000 --record-size 1000 --throughput 10000 --producer-props bootstrap.servers=hostname:8092 interceptor.classes=io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor acks=all

Consumer ->

$CONFLUENT/bin/kafka-console-consumer --topic repStr3.replica --new-consumer --bootstrap-server hostname:9092 --consumer.config $CONFLUENT/consumer3.config

However, i don't see the confluent producer/consumer metrics in Prometheus.

so, do i need to do any additional steps to enable monitoring of the Replicator producer/consumer ?

Pls .let me know.

Pls note -> i plan on displaying the metrics on Grafana dashboard (by linking Prometheus + Grafana)

thanks! 

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/00fe4d56-42b4-4dbf-b9f8-2b47e62d7034%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Brian Brazil

www.robustperception.io