KafkaProducer BUG?
1,050 views
Skip to first unread message
Marek Svitok
Nov 21, 2016, 11:39:33 AM11/21/16
to Confluent Platform
Hi all.
I have this kind of problem. I'm sending a message to some topic and once I call producer send I get a lot of these messages:
17:36:17.619 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[,cc-notification1]} to node 0
and the message is not sent at all. As you can see, it tries to get metadata for an empty topic. I don't know where it gets from.
Please help.
Thanks,
Marek.
I have this kind of problem. I'm sending a message to some topic and once I call producer send I get a lot of these messages:
17:36:17.619 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[,cc-notification1]} to node 0
and the message is not sent at all. As you can see, it tries to get metadata for an empty topic. I don't know where it gets from.
Please help.
Thanks,
Marek.
Dustin Cote
Nov 21, 2016, 4:18:35 PM11/21/16
to confluent...@googlegroups.com
Are you seeing this with the console producer? If you aren't getting metadata back, it's likely you have an error in the `bootstrap.servers` configuration and you can use the console producer to verify your settings. Additionally, there might be more information prior to this message that can help you figure out what's happening. The most likely culprit from the information presented is a problem with the `bootstrap.servers` configuration.
--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/96c6c674-e9e0-4c89-8e7c-6ba103b45593%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Marek Svitok
Nov 22, 2016, 3:09:20 AM11/22/16
to Confluent Platform
Thank you Dustin. I'll take a look at it. Once I find out something more, I'll get back to you.
Marek.
Dňa pondelok, 21. novembra 2016 22:18:35 UTC+1 Dustin Cote napísal(-a):
Marek.
Dňa pondelok, 21. novembra 2016 22:18:35 UTC+1 Dustin Cote napísal(-a):
Are you seeing this with the console producer? If you aren't getting metadata back, it's likely you have an error in the `bootstrap.servers` configuration and you can use the console producer to verify your settings. Additionally, there might be more information prior to this message that can help you figure out what's happening. The most likely culprit from the information presented is a problem with the `bootstrap.servers` configuration.
On Mon, Nov 21, 2016 at 11:39 AM, Marek Svitok <marek....@gmail.com> wrote:
Hi all.
I have this kind of problem. I'm sending a message to some topic and once I call producer send I get a lot of these messages:
17:36:17.619 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[,cc-notification1]} to node 0
and the message is not sent at all. As you can see, it tries to get metadata for an empty topic. I don't know where it gets from.
Please help.
Thanks,
Marek.
--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/96c6c674-e9e0-4c89-8e7c-6ba103b45593%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Message has been deleted
Marek Svitok
Nov 22, 2016, 3:36:41 AM11/22/16
to Confluent Platform
Hi Dustin.
This is what I've found so far:
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node -1 for sending metadata request
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -1
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[]} to node -1
09:27:40.226 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 0 : {=INVALID_TOPIC_EXCEPTION}
09:27:40.226 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = sFYBPokrQHOEhsifK88vjA, nodes = [localhost:9092 (id: 0 rack: null)], partitions = [])
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node 0 for sending metadata request
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 0 at localhost:9092.
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-sent
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-received
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.latency
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node 0
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 0
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[]} to node 0
09:27:40.330 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 1 : {=INVALID_TOPIC_EXCEPTION}
09:27:40.330 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 3 to Cluster(id = sFYBPokrQHOEhsifK88vjA, nodes = [localhost:9092 (id: 0 rack: null)], partitions = [])
09:27:40.430 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[]} to node 0
09:27:40.432 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 2 : {=INVALID_TOPIC_EXCEPTION}
09:27:40.433 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 4 to Cluster(id = sFYBPokrQHOEhsifK88vjA, nodes = [localhost:9092 (id: 0 rack: null)], partitions = [])
As you can see it tries to get the metadata from node -1 which is obviously wrong. Then from node 0 which does exist, but an empy list/set of partition which is wrong.
Marek.
This is what I've found so far:
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node -1 for sending metadata request
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
09:27:40.222 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -1
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1
09:27:40.223 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[]} to node -1
09:27:40.226 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 0 : {=INVALID_TOPIC_EXCEPTION}
09:27:40.226 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = sFYBPokrQHOEhsifK88vjA, nodes = [localhost:9092 (id: 0 rack: null)], partitions = [])
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node 0 for sending metadata request
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 0 at localhost:9092.
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-sent
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-received
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.latency
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node 0
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 0
09:27:40.327 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[]} to node 0
09:27:40.330 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 1 : {=INVALID_TOPIC_EXCEPTION}
09:27:40.330 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 3 to Cluster(id = sFYBPokrQHOEhsifK88vjA, nodes = [localhost:9092 (id: 0 rack: null)], partitions = [])
09:27:40.430 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request {topics=[]} to node 0
09:27:40.432 [kafka-producer-network-thread | producer-1] WARN o.apache.kafka.clients.NetworkClient - Error while fetching metadata with correlation id 2 : {=INVALID_TOPIC_EXCEPTION}
09:27:40.433 [kafka-producer-network-thread | producer-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 4 to Cluster(id = sFYBPokrQHOEhsifK88vjA, nodes = [localhost:9092 (id: 0 rack: null)], partitions = [])
As you can see it tries to get the metadata from node -1 which is obviously wrong. Then from node 0 which does exist, but an empy list/set of partition which is wrong.
Marek.
Marek Svitok
Nov 22, 2016, 6:32:38 AM11/22/16
to Confluent Platform
I've found out what the problem was. A part of the topology didn't read the topic name from the configuration. The problem was that ProducerRecord doesn't throw any exception
like IllegalArgumentException when you pass empty topic name as a input parameter. They control just if it's null, but not the empty String. They accept it and then it starts behaving pretty weird.
Marek.
like IllegalArgumentException when you pass empty topic name as a input parameter. They control just if it's null, but not the empty String. They accept it and then it starts behaving pretty weird.
Marek.
Reply all
Reply to author
Forward
0 new messages