Kafka Streams and slf4j

[Francesco Nobilia]

Dear all,

I'm developing an application that uses slf4j instead of log4j. My problem is that exceptions raised by Kafka Stream are not printed into stdout. I guess this issue is related to the hard log4j dependency inside zkclient that conflicts with my slf4j dependency. Is there any workaround to solve my issue or I have to with to log4j?

 

[Guozhang Wang]

Hello Francesco,

Which Kafka version are you using? Since 0.10.0.1 the hard dependency of log4j introduced by zkClient is removed I'd suspect it is the source code.

Copy-pasting from build.gradle:

"

// this dependency should be removed after KIP-4
compile (libs.zkclient) {
  // users should be able to choose the logging implementation (and slf4j bridge)
  exclude module: 'slf4j-log4j12'
  exclude module: 'log4j'
  exclude module: 'jline'
  exclude module: 'netty'
}

// zkclient has a hard log4j dependency, we set it here as well to avoid copying an older
// version to the binary package; the following two dependencies should be removed after KIP-4
// (along with the zkclient dependency)
compile libs.log4j
compile libs.jacksonDatabind

"

Guozhang

[Francesco Nobilia]

I'm using 0.10.0.1. My gradle has these dependencies

dependencies {
    // The production code uses the SLF4J logging API at compile time
    compile group: 'org.slf4j', name:'slf4j-api', version: slf4jVersion

    // to implement producers and consumers
    compile group: 'org.apache.kafka', name: 'kafka-clients', version: '0.10.0.1'
    compile group: 'org.apache.kafka', name: 'kafka-streams', version: '0.10.0.1'
    compile group: 'io.confluent', name: 'kafka-avro-serializer', version: '3.0.1'

    ...

}

[Guozhang Wang]

Have you configured your slf4j logger in the log4j.properties file? Otherwise you may see sth. like:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

Guozhang

[Zlatko Sisic]

1. configure your logging implementation for slf4j ( this example uses logback by including logback-classic)
2. exclude log4j AND slf4j-log4j bridge as transitive dependency being pulled in
3. provide a log4j bridge to write back to slf4j to ensure compatibility with libs using log4j
   

compile (group: 'org.apache.kafka', name: 'kafka-streams', version: project.kafkaVersion ) {
    exclude group: 'log4j'
    exclude group: 'org.slf4j'
}
compile group: 'org.slf4j', name: 'slf4j-api', version: project.slf4jVersion
compile group: 'org.slf4j', name: 'log4j-over-slf4j', version: project.slf4jVersion
compile group: 'ch.qos.logback', name: 'logback-core', version: project.logbackVersion
compile group: 'ch.qos.logback', name: 'logback-classic', version: project.logbackVersion