Error while setting up a multi-broker cluster on Local Machine

[rajat...@m.darcl.com]

Using Kafka on Windows:

Zookeeper started

Kafka started

Topic Created

Producer started

Consumer started

Messages are delivering fine from producer to consumer

I created 2 new server.properties files as:

server-1.properties as:

broker.id=1

listeners=PLAINTEXT://:9093

log.dir=C:\kafka\kafka-logs-1

server-2.properties as:

broker.id=2

listeners=PLAINTEXT://:9094

log.dir=C:\kafka\kafka-logs-2

When i started the new broker as:

bin\windows\kafka-server-start.bat config\server-1.properties &

Got Error as:

.
.
[2017-03-03 11:21:58,416] INFO Client environment:java.io.tmpdir=C:\Users\RAJAT~1.YAD\AppData\Local\Temp\ (org.apache.zookeeper.ZooKeeper)
[2017-03-03 11:21:58,431] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,447] INFO Client environment:os.name=Windows 10 (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,447] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,447] INFO Client environment:os.version=10.0 (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,463] INFO Client environment:user.name=rajat.yadav (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,463] INFO Client environment:user.home=C:\Users\Rajat.Yadav (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,463] INFO Client environment:user.dir=C:\kafka (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,478] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@c267ef4 (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:58,556] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient)

[2017-03-03 11:21:58,556] INFO Opening socket connection to server RajatYadav.DARCL.CORP/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)

[2017-03-03 11:21:58,572] INFO Socket connection established to RajatYadav.DARCL.CORP/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)

[2017-03-03 11:21:58,634] INFO Session establishment complete on server RajatYadav.DARCL.CORP/0:0:0:0:0:0:0:1:2181, sessionid = 0x15a92b7fb510001, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)

[2017-03-03 11:21:58,650] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)

[2017-03-03 11:21:58,853] INFO Cluster ID = fmnc7b71S0SaWXH489NL2g (kafka.server.KafkaServer)

[2017-03-03 11:21:58,869] WARN No meta.properties file under dir c:\kafka\kafka-logs-1\meta.properties (kafka.server.BrokerMetadataCheckpoint)

[2017-03-03 11:21:58,900] INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)

[2017-03-03 11:21:58,916] INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)

[2017-03-03 11:21:58,947] INFO Log directory 'c:\kafka\kafka-logs-1' not found, creating it. (kafka.log.LogManager)

[2017-03-03 11:21:58,947] INFO Loading logs. (kafka.log.LogManager)

[2017-03-03 11:21:58,963] INFO Logs loading complete in 16 ms. (kafka.log.LogManager)

[2017-03-03 11:21:59,025] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)

[2017-03-03 11:21:59,041] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)

[2017-03-03 11:21:59,088] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)

kafka.common.KafkaException: Socket server failed to bind to 0.0.0.0:9092: Address already in use: bind.

        at kafka.network.Acceptor.openServerSocket(SocketServer.scala:330)

        at kafka.network.Acceptor.<init>(SocketServer.scala:255)

        at kafka.network.SocketServer$$anonfun$startup$1.apply(SocketServer.scala:98)

        at kafka.network.SocketServer$$anonfun$startup$1.apply(SocketServer.scala:90)

        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)

        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)

        at kafka.network.SocketServer.startup(SocketServer.scala:90)

        at kafka.server.KafkaServer.startup(KafkaServer.scala:215)

        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)

        at kafka.Kafka$.main(Kafka.scala:67)

        at kafka.Kafka.main(Kafka.scala)

Caused by: java.net.BindException: Address already in use: bind

        at sun.nio.ch.Net.bind0(Native Method)

        at sun.nio.ch.Net.bind(Unknown Source)

        at sun.nio.ch.Net.bind(Unknown Source)

        at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)

        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)

        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)

        at kafka.network.Acceptor.openServerSocket(SocketServer.scala:326)

        ... 10 more

[2017-03-03 11:21:59,134] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)

[2017-03-03 11:21:59,166] INFO [Socket Server on Broker 1], Shutting down (kafka.network.SocketServer)

[2017-03-03 11:21:59,181] INFO [Socket Server on Broker 1], Shutdown completed (kafka.network.SocketServer)

[2017-03-03 11:21:59,181] INFO Shutting down. (kafka.log.LogManager)

[2017-03-03 11:21:59,213] INFO Shutdown complete. (kafka.log.LogManager)

[2017-03-03 11:21:59,228] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)

[2017-03-03 11:21:59,244] INFO Session: 0x15a92b7fb510001 closed (org.apache.zookeeper.ZooKeeper)

[2017-03-03 11:21:59,259] INFO EventThread shut down for session: 0x15a92b7fb510001 (org.apache.zookeeper.ClientCnxn)

[2017-03-03 11:21:59,275] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)

[2017-03-03 11:21:59,275] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)

kafka.common.KafkaException: Socket server failed to bind to 0.0.0.0:9092: Address already in use: bind.

        at kafka.network.Acceptor.openServerSocket(SocketServer.scala:330)

        at kafka.network.Acceptor.<init>(SocketServer.scala:255)

        at kafka.network.SocketServer$$anonfun$startup$1.apply(SocketServer.scala:98)

        at kafka.network.SocketServer$$anonfun$startup$1.apply(SocketServer.scala:90)

        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)

        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)

        at kafka.network.SocketServer.startup(SocketServer.scala:90)

        at kafka.server.KafkaServer.startup(KafkaServer.scala:215)

        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)

        at kafka.Kafka$.main(Kafka.scala:67)

        at kafka.Kafka.main(Kafka.scala)

Caused by: java.net.BindException: Address already in use: bind

        at sun.nio.ch.Net.bind0(Native Method)

        at sun.nio.ch.Net.bind(Unknown Source)

        at sun.nio.ch.Net.bind(Unknown Source)

        at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)

        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)

        at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)

        at kafka.network.Acceptor.openServerSocket(SocketServer.scala:326)

        ... 10 more

[2017-03-03 11:21:59,353] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)

Please help me ASAP.

[Eno Thereska]

I posted answer here https://groups.google.com/forum/#!topic/confluent-platform/vhq5xYcDJz0

[Rahul Sharma]

Hi Rajat,

Did you find the solution? I am also facing same issue. Please help!

Thanks!