Custom metric reporters in kafka streams

[Vinay Ramkrishnan]

Hi , 

I am trying to configure a custom metrics reporter for my stream-apps. I am using the following configuration to enable this.

config.put(METRIC_REPORTER_CLASSES_CONFIG, PrometheusReporter.class.getCanonicalName());

config.put(StreamsConfig.METRICS_RECORDING_LEVEL_CONFIG, "DEBUG");

I am using kafka-streams-2.2.2 library.
But my reported metrics (for all thread, task, processor-node metrics) are always 0.0 or NaN for the mix, max, avg ones. 

Is there any configuration that I am missing ?
The streams app uses Stream DSL and not the processor API. Does that matter?

Any help appreciated,

Thanks,

Vinay

[Bruno Cadonna]

Hi Vinay,

Could you check the values reported to JMX?

For that you need to connect jconsole to your Streams app and verify the
MBeans in jconsole that contain the metrics.

If they show other values, the issue is likely in your custom metrics
reporter.

Using the DSL or the processor API should not matter. The number of
metrics will probably differ, but for example processor node metrics
like create-latency-avg should show up in both.

Best,
Bruno

On 21.04.21 01:20, Vinay Ramkrishnan wrote:
> Hi ,
>
> I am trying to configure a custom metrics reporter for my stream-apps. I
> am using the following configuration to enable this.
>

> /config.put(METRIC_REPORTER_CLASSES_CONFIG,
> PrometheusReporter.class.getCanonicalName());/
> /config.put(StreamsConfig.METRICS_RECORDING_LEVEL_CONFIG, "DEBUG");/

>
> I am using kafka-streams-2.2.2 library.
> But my reported metrics (for all thread, task, processor-node metrics)
> are always 0.0 or NaN for the mix, max, avg ones.
> Is there any configuration that I am missing ?
> The streams app uses Stream DSL and not the processor API. Does that matter?
>
> Any help appreciated,
> Thanks,
> Vinay
>

> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platf...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/2bd430a5-5e50-470c-9f61-77eb9ff98a1an%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/2bd430a5-5e50-470c-9f61-77eb9ff98a1an%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Vinay Ramkrishnan]

Hey Bruno,

Thanks for the response. Yeah the bug was indeed in my prometheus exporter configuration.

I have it fixed up and it seems to work now.

I have 2 follow-up questions:

1. Is there anyway to find the list of metrics that won't be available with the Streams DSL API ?

2. I also noticed in my metrics that each time I restart the app the client-id on the metrics is different, so they wouldn't match up across deploys and restarts. 

    I realize that the client-id by default is <appID-Random-UUID> and I can set it to be deterministic by using StreamConfig.CLIENT_ID_CONFIG and setting it to <appID-server-host>

    Is there any concern or downside to changing it for an existing deployed app. I don't think there is but wanted to confirm anyway.

-Vinay

[Bruno Cadonna]

Hi Vinay,

Good to hear that you were able to fix it!

1. All metrics will be available with the Streams DSL. With the
Processor API it could be that some are not exposed or are constant
zero, because some metrics are only used and recorded in the DSL. For
example suppression-emit-rate is specific to the supression operator
which is part of the DSL and you will not find this metric in a Streams
application that only uses the Processor API. There are not many of such
metrics, though.

2. I do not think that there is an issue by setting
StreamConfig.CLIENT_ID_CONFIG for an existing application. However,
please test this in a test environment to be sure.

Best,
Bruno

> <https://groups.google.com/d/msgid/confluent-platform/2bd430a5-5e50-470c-9f61-77eb9ff98a1an%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/confluent-platform/2bd430a5-5e50-470c-9f61-77eb9ff98a1an%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platf...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/confluent-platform/f38e7c6f-4375-4a4f-b6dd-88993e1e9d01n%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/f38e7c6f-4375-4a4f-b6dd-88993e1e9d01n%40googlegroups.com?utm_medium=email&utm_source=footer>.