java.net.ConnectException: Connection refused

[David Montgomery]

Hi,

I am trying to ingest data into kafka.  Whilel tailing the realtime node log file I am getting the below error...

Here is my realtime config file

-server
-Xmx256m
-Duser.timezone=UTC
-Dfile.encoding=UTF-8

druid.host=127.0.0.1
druid.service=realtime
druid.port=8082

druid.extensions.coordinates=["io.druid.extensions:druid-kafka-eight:0.6.52"]


druid.zk.service.host=1.dzk.development.sf.test.com:2181

# The realtime config file.
druid.realtime.specFile=/home/ubuntu/druid-services-0.6.52/config/realtime/realtime_druid.json

# Choices: db (hand off segments), noop (do not hand off segments).
druid.publish.type=db

druid.db.connector.connectURI=jdbc\:mysql\://druid.mysql.development.sf.test.com\:3306/dry
druid.db.connector.user=root
druid.db.connector.password=test

druid.processing.buffer.sizeBytes=100000000






java.net.ConnectException: Connection refused
    at sun.nio.ch.Net.connect0(Native Method)
    at sun.nio.ch.Net.connect(Net.java:465)
    at sun.nio.ch.Net.connect(Net.java:457)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
    at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
    at kafka.producer.SyncProducer.connect(SyncProducer.scala:146)
    at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161)
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88)
    at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:65)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
2014-02-08 05:42:03,345 WARN [topic-pixel_do-druidrealtime-sf-development-20140204220702-1391670173115-69294c26-leader-finder-thread] kafka.client.ClientUtils$ - Fetching topic metadata with correlation id 954 for topics [Set(topic-pixel)] from broker [id:1391209402,host:127.0.0.1,port:9092] failed
java.net.ConnectException: Connection refused
    at sun.nio.ch.Net.connect0(Native Method)
    at sun.nio.ch.Net.connect(Net.java:465)
    at sun.nio.ch.Net.connect(Net.java:457)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
    at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
    at kafka.producer.SyncProducer.connect(SyncProducer.scala:146)
    at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161)
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88)
    at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:65)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
2014-02-08 05:42:03,346 WARN [topic-pixel_do-druidrealtime-sf-development-20140204220702-1391670173115-69294c26-leader-finder-thread] kafka.consumer.ConsumerFetcherManager$LeaderFinderThread - [topic-pixel_do-druidrealtime-sf-development-20140204220702-1391670173115-69294c26-leader-finder-thread], Failed to find leader for Set([topic-pixel,1], [topic-pixel,0])
kafka.common.KafkaException: fetching topic metadata for topics [Set(topic-pixel)] from broker [ArrayBuffer(id:1391209402,host:127.0.0.1,port:9092)] failed
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:67)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:88)
    at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:65)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.Net.connect0(Native Method)
    at sun.nio.ch.Net.connect(Net.java:465)
    at sun.nio.ch.Net.connect(Net.java:457)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
    at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
    at kafka.producer.SyncProducer.connect(SyncProducer.scala:146)
    at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161)
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
    ... 3 more

[Gian Merlino]

I would guess that "Connection refused" exception is the firehose trying to talk to your kafka broker. You are trying to connect to a broker at 127.0.0.1:9092; is your broker running on the same machine as your realtime node? If not, then you probably need to figure out how to get your broker to listen on and advertise the correct network interface.