Kafka Warns.

[Vicky Kak]

We are getting the error which is printed as the WARN from this code
https://apache.googlesource.com/kafka/+/0.8.2.2/clients/src/main/java/org/apache/kafka/common/network/Selector.java

I have spend some time seeing around but could not find its cause, any help will be be appreciated.

Here is the gobblin log
**************************************************************************************************************************************
2017-01-24 07:04:05 UTC INFO  [TaskExecutor-0] gobblin.runtime.Task  176 - Extracted 77 data records
2017-01-24 07:04:05 UTC INFO  [TaskExecutor-0] gobblin.runtime.Task  177 - Row quality checker finished with results:
2017-01-24 07:04:05 UTC INFO  [ForkExecutor-0] gobblin.runtime.Fork  376 - Wrapping writer gobblin.writer.PartitionedDataWriter@31b456f1
2017-01-24 07:04:05 UTC INFO  [ForkExecutor-1] gobblin.runtime.Fork  376 - Wrapping writer gobblin.writer.PartitionedDataWriter@2d0e4287
2017-01-24 07:04:05 UTC WARN  [kafka-producer-network-thread | gobblin] org.apache.kafka.common.network.Selector  276 - Error in I/O with localhost/127.0.0.1
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
        at java.lang.Thread.run(Thread.java:745)
2017-01-24 07:04:05 UTC WARN  [kafka-producer-network-thread | gobblin] org.apache.kafka.common.network.Selector  276 - Error in I/O with localhost/127.0.0.1
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
        at java.lang.Thread.run(Thread.java:745)
2017-01-24 07:04:05 UTC WARN  [kafka-producer-network-thread | gobblin] org.apache.kafka.common.network.Selector  276 - Error in I/O with localhost/127.0.0.1
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
        at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
        at java.lang.Thread.run(Thread.java:745)

**************************************************************************************************************************************

The WARN is getting printed continuously.

The ports are bound as you can see

root@ubuntu:~# lsof -i :9092
COMMAND  PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
java    5069 root  164u  IPv6 18818602      0t0  TCP ubuntu:9092 (LISTEN)
java    5069 root  168u  IPv6 18818604      0t0  TCP ubuntu:54186->ubuntu:9092 (CLOSE_WAIT)
root@ubuntu:~# lsof -i :2181
COMMAND  PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
java    4837 root   94u  IPv6 18818501      0t0  TCP *:2181 (LISTEN)
java    4837 root   95u  IPv6 18817707      0t0  TCP ubuntu:2181->223.227.0.30:18679 (ESTABLISHED)
java    4837 root   96u  IPv6 18818595      0t0  TCP localhost:2181->localhost:33134 (ESTABLISHED)
java    5069 root   93u  IPv6 18818594      0t0  TCP localhost:33134->localhost:2181 (ESTABLISHED)

The host name ubuntu is mapped to 127.0.0.1 in the /etc/hosts....

Thanks,
Vicky

[Vicky Kak]

I don't see an issue on my local machine, I see the similar issue from the gobblin when Kafka broker is not running. This confirms that the gobblin is not able to connect to the Kafka Broker which is obvious from the "Connection Refused" message in the earlier logs.

Thanks,
Vicky

[Vicky Kak]

I provided the IP address instead of the localhost, don't know why it was not getting resolved by DNS. Sorry for making a noise here, I got it sorted.