Problem collecting metrics from Kafka Connect

[ralph.g...@gmail.com]

Hi all,

With v 1.0.0, Kafka Connect has dramatically improved its monitoring capabilities. We are currently trying to get the Kafka Connect metrics into Prometheus via the JMX exporter. Most of it is working fine but the status metric is failing:

Mar 12, 2018 4:14:12 PM io.prometheus.jmx.JmxCollector$Receiver recordBean FINE: Ignoring unsupported bean: kafka.connect<type=connector-task-metrics, connector=foo, task=0><>status: running




It seems that the JMX exporter does not support Strings (JmxCollector.java line 375ff). I'm not too familiar with Prometheus internals, therefore I'm not sure whether that is by design or not. The status of a Connect Task is quite interesting so we are investigating solutions on either the Prometheus or the Kafka side of things.

Is failing on String valued metrics intended behavior or not?

Thanks a lot in advance!

Best,

Ralph

[Brian Brazil]

Strings are supported, however there's extra config required as strings are not metrics. What you'd want is something like:

rules:

  - pattern: '....status: running'

    name: kafka_connector_running

    value: 1

  - pattern: '....status: .*'

    name: kafka_connector_running

    value: 0

So you'd end up with a gauge indicating whether it is running. 

Brian

 

Thanks a lot in advance!

Best,

Ralph

--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscribe@googlegroups.com.
To post to this group, send email to prometheus-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/0dc1eeaa-4f97-40a0-9cba-394df5a43ddb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Brian Brazil

www.robustperception.io

[clayt...@gmail.com]

Can you post the .yml configuration file that you are using with jmx_prometheus_javaagent for Kafka Connect?

There is an official configuration file for Kafka itself:

https://raw.githubusercontent.com/prometheus/jmx_exporter/parent-0.3.0/example_configs/kafka-0-8-2.yml

But, AFAIK, there is no official configuration file for Kafka Connect. Can you provide what you are using?

[Ralph Guderlei]

Sorry for the late reply. We were able to monitor our Kafka Connect infrastructure using the following jmx exporter config:

startDelaySeconds: 2

#hostPort: "kafka-connect:9999"

jmxUrl: 'service:jmx:rmi:///jndi/rmi://kafka-connect:9999/jmxrmi'

ssl: false

whitelistObjectNames: ["kafka.connect:*"]

lowercaseOutputName: false

lowercaseOutputLabelNames: false

rules:

  - pattern: 'status: running'

    value: 1

  - pattern: 'status: *'

    value: 0

  - pattern: ".*"

Rewriting the status strings using rules did the trick. Thank you a lot for your support!

Best,
Ralph

--
You received this message because you are subscribed to a topic in the Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/prometheus-users/WmCRQFFUpVY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to prometheus-use...@googlegroups.com.
To post to this group, send email to promethe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ce014a4e-eacd-4676-bce5-dc9b56e05c17%40googlegroups.com.

[Daniel Bryła]

Hello, 

I'm sorry for refreshing the old topic, but I have problem that JMX is only reporting "running" tasks, I can't get any metrics for failed ones. Is there anything else which I need to add to configuration?

Best regards,

To unsubscribe from this group and all its topics, send an email to promethe...@googlegroups.com.

[Julien Pivotto]

We use

https://github.com/zenreach/kafka-connect-exporter


for this. it uses http.

> >> promethe...@googlegroups.com <javascript:>.

> >> To post to this group, send email to promethe...@googlegroups.com

> >> <javascript:>.

> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/prometheus-users/ce014a4e-eacd-4676-bce5-dc9b56e05c17%40googlegroups.com

> >> <https://groups.google.com/d/msgid/prometheus-users/ce014a4e-eacd-4676-bce5-dc9b56e05c17%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
>

> --
> You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/34286084-52a6-47b2-bf2e-30a0eca7cfeb%40googlegroups.com.


--
(o- Julien Pivotto
//\ Open-Source Consultant
V_/_ Inuits - https://www.inuits.eu

[Daniel Bryła]

It's great in terms providing information about connectors, but I'm losing all other information from Prometheus Exporter. :(