Extend Kafka Connect for Debezium with OpenTracing

[Jan Baudisch]

Hello,

are there instructions on how to make distributed tracing running with Debezium (Outbox Event Router) and Confluent Kafka Connect? The Debezium documentation only says:

Neither the specification JAR files nor the Jaeger client are part of the Debezium Kafka Connect container image. The user either needs to extend the image with them or can use the Strimzi Kafka image.

Using the Strimzi Docker Image which has set up distributed tracing out of the box, everything works fine. However, we want to use the Confluent Docker image.

Our approach is:

- copy all OpenTracing,and Jaeger .jars to /kafka/libs folder. There is still the NoopTracer configured (verified with remote debugging)

- now we would create and deploy a custom .jar which registers the Jaeger Tracer.

We think, this is quite inconvenient. Anybody knows an easier approach, e.g. by setting some property?

Best regards,

Jan

[Jan Baudisch]

we solved by adding a custom .jar with a class implementing the ConfigProvider interface (SPI), like described here:

 

https://docs.confluent.io/kafka-connectors/self-managed/userguide.html#configprovider-interface

 

public class TracingConfigProvider implements ConfigProvider {

 

    @Override

    public void configure(Map<String, ?> map) {

        final JaegerTracer tracer = Configuration.fromEnv().getTracer();

        GlobalTracer.registerIfAbsent(tracer);

    }

 

 

    @Override

    public ConfigData get(String s) {

        return null;

    }

 

    @Override

    public ConfigData get(String s, Set<String> set) {

        return null;

    }

 

    @Override

    public void close(){}

 

}

 

 

And setting these Environment Variables in Kafka Connect

 

- CONNECT_CONFIG_PROVIDERS=tracing

- CONNECT_CONFIG_PROVIDERS_TRACING_CLASS=org.example.TracingConfigProvider

Another solution could be to use a Java agent to register the tracer:
https://github.com/strimzi/strimzi-kafka-operator/tree/main/tracing-agent/