0.9.0.1 Kafka Java Client - KafkaProducer async send hangs indefinitely

[Muthukumaran Kothandaraman]

Hi, 

Trying a very simple Producer with following code. "producer.send" hangs indefinitely. Attaching thread-dump snippet so that I can get some advice if something is wrong with my code or configuration. 

Kafka runs in a VM and producer runs in host - it's a single-broker setup for basic testing. Kafka version used is is 0.9.0.1

    public static void main(String args[]) throws InterruptedException, ExecutionException {

        

        

        Properties props = new Properties();

        props.put("bootstrap.servers", "<REMOTE-IP-REACHABLE-FROM-LOCAL-MC>:9092");

        //props.put("acks", "all");

        props.put("retries", 0);

        props.put("batch.size", 16384);

        props.put("linger.ms", 1);

        props.put("buffer.memory", 33554432);

        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        Producer<String, String> producer = new KafkaProducer<>(props);

        System.out.println("Connected ...");

        for(int i = 0; i < 100; i++){

            producer.send(new ProducerRecord<String, String>("consumer-tutorial", Integer.toString(i), Integer.toString(i)));

            System.out.println("Sent " + i);

        }

        producer.close();

    }

Thread Dump snippet :

 

"main" prio=6 tid=0x0000000002238000 nid=0x1390 in Object.wait() [0x00000000025bf000]

   java.lang.Thread.State: TIMED_WAITING (on object monitor)

at java.lang.Object.wait(Native Method)

- waiting on <0x00000007aecacea0> (a org.apache.kafka.clients.Metadata)

at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:121)

- locked <0x00000007aecacea0> (a org.apache.kafka.clients.Metadata)

at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:483)

at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:412)

at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:339)

at kafkaexperiments.ProducerDriver.main(ProducerDriver.java:41)

Regards

Muthu

[Muthukumaran Kothandaraman]

Additional info and checks done 

1)  bin/kafka-topics.sh --list --zookeeper <VM-IP>:2181  - this clearly shows that the topic "consumer-tutorial" is present 

2)  to check if there are issues with kafka itself, I locally (within VM where Kafka and ZK runs) checked with kafka-console-producer.sh and kafka-console-consumer.sh with a different topic and could verify that there is no issue with Kafka setup. It worked fine

3) I am using following imports 

import org.apache.kafka.clients.producer.KafkaProducer;

import org.apache.kafka.clients.producer.Producer;

import org.apache.kafka.clients.producer.ProducerRecord;

[Matt Gunter]

I see this was about 1 year ago, but I am facing a similar problem. Are there any other details that you can share?