After upgrading our app to Kafka 0.10.2.0, we began seeing the following warnings:
18:27:44.065 [main] WARN k.server.BrokerMetadataCheckpoint - No meta.properties file under dir /tmp/kafka-log-0/meta.properties
18:27:44.779 [main] WARN k.server.BrokerMetadataCheckpoint - No meta.properties file under dir /tmp/kafka-log-0/meta.properties
18:27:44.864 [Controller-0-to-broker-0-send-thread] WARN kafka.controller.RequestSendThread - [Controller-0-to-broker-0-send-thread], Controller 0's connection to broker 192.168.10.103:9092 (id: 0 rack: null) was unsuccessful
java.io.IOException: Connection to 192.168.10.103:9092 (id: 0 rack: null) failed
at kafka.utils.NetworkClientBlockingOps$.awaitReady$1(NetworkClientBlockingOps.scala:84)
at kafka.utils.NetworkClientBlockingOps$.blockingReady$extension(NetworkClientBlockingOps.scala:94)
at kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:232)
at kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:185)
at kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:184)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)
...
This continues indefinitely until we manually stop with `^C`.
What's going on here? I'm guessing it's a client-server version mismatch, although we should be using 0.10.1.x or 0.10.2.0 among our many libraries, which should be compatible.