Kafka- "Received -1 from channel reading from source socket has likely been closed"

[Ashish Dutt]

Dear all,

I first create a topic test using the command 

/opt/cloudera/parcels/KAFKA/bin/kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Then when I try to insert a new message to a topic in Kafka using the command 

/opt/cloudera/parcels/KAFKA/bin/kafka-console-producer --broker-list localhost:2181 --topic test

Then i enter a message like "name":"Ashish"

I get the error "received -1 from channel reading from source socket has likely been closed"

What am I doing wrong here?

Any pointers to the solution or help will be appreciated.

Thank you,

Ashish

[Patrick D'Souza]

What are you using to identify that new messages are not being appended to the topic?

Do you have a consumer reading off the topic? The consumer will display the messages from the Kafka topic that you subscribe to. If you're using the consumer script kafka-console-consumer.sh bundled with Kafka, you might want to use the switch --from-beginning to fetch the messages from the smallest offset.

- Patrick

On Thu, Jul 23, 2015 at 12:19 AM, Ashish Dutt <ashish...@gmail.com> wrote:

Patrick,

Thank you for your detailed response.

It worked..

Now the problem is that new messages do not get appended at all after I issue this command /opt/cloudera/parcels/KAFKA/bin/kafka-console-producer --broker-list localhost:9092 --topic test

Would you know how to fix this problem?

Thanks,

Ashish

On Wed, Jul 22, 2015 at 7:19 PM, Patrick D'Souza <patrick...@gmail.com> wrote:

Ashish,

The broker port is 9092. This line "/opt/cloudera/parcels/KAFKA/bin/kafka-console-producer --broker-list localhost:2181 --topic test" should read as "/opt/cloudera/parcels/KAFKA/bin/kafka-console-producer --broker-list localhost:9092 --topic test"

Zookeeper by default runs on port 2181 while the broker runs on the default port 9092. Let me know if that works.

- Patrick

--
You received this message because you are subscribed to the Google Groups "Camus - Kafka ETL for Hadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camus_etl+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ashish Dutt]

Many thanks Patrick for your prompt response. Much appreciated.

My environment: Using CDH5.4 with parcels, Hue 3.7.0, Kafka version 0.8.2.0

So this is what I am doing;

Step 1: create a new topic test using command 

                       /usr/bin/kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Step 2: Next, I list topics using command 

                      /usr/bin/kafka-topics --list --zookeeper localhost:2181 # I see the topic test listed

Step 3: Now, I insert a message into the topic test using the command 

                      /usr/bin/kafka-console-producer --broker-list localhost:9092 --topic test  # able to insert a message

Step 4: Finally, I execute the command 

                      /usr/bin//kafka-console-consumer -zookeeper localhost:2181 -topic test -from-beginning  # but nothing happens, the cursor just blinks on a new line and then I have to press Ctrl+Z to kill the operation?

Step 5: I try then the command kafka-topics --list --zookeeper localhost:2181 --topic test # To Read data from a Kafka topic and write it to standard output but nothing happens, the cursor just keeps blinking on the next line (I followed this link here )

Executing step 5, leads me to conclude that the message is not being written to the topic? Why it is not being written? 

Question 1: How do I see if the message was inserted successfully? what command is it 

Question 2: Why cannot I see the topic test in Hue (I'm looking in "File browser- /user/kafka/" and i see camus and exec folder )

What am I doing wrong here. Please suggest

Thank you,

 Ashish

Sincerely,

Ashish Dutt

PhD Candidate

Department of Information Systems

University of Malaya, Lembah Pantai,

50603 Kuala Lumpur, Malaysia