Kafka ReplicaFetcherThread org.apache.kafka.common.errors.UnknownTopicOrPartitionException

[Jing Song]

We are using kafka 0.11.0, lately seeing many ReplicaFetcherThread exception:

[2017-11-08 14:51:14,631] ERROR [ReplicaFetcherThread-1-4]: Error for partition [activity,224] to broker 4:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread).

Do you know what caused this? 

Thanks,

~Jing

[Shantanu Deshmukh]

How many brokers are there in your cluster? Is any broker down? When topic is under-replicated or not replicated, partitions for that topic may be on different brokers. And if a broker went down at some point in time and you don't have it's replica elsewhere this error may occur.

[Peter Bukowinski]

Hi, I work with Jing and this issue is still bugging us. We are still on kafka 0.11.0, zk 3.4.5  We are in the process of deploying approximately 30 five-broker clusters. We constantly encounter this issue on just-deployed clusters. More often than not, a cluster will be brought online, and the __consumer_offsets topic will be auto-created with a 20% broker skew. We then run a partition reassignment to bring it into balance.

If a cluster starts with this condition, then any new topic created with more than 1x replication -- our default is 2x for these clusters -- will also result in a 20% broker skew upon creation. If I watch the kafka server.log file, one of the brokers will throw errors like the one Jing mentioned.

Any ideas what could be causing this?

[Xin Li]

https://issues.apache.org/jira/browse/KAFKA-6221 this ?

[Nivethika Mahasivam]

Did any of you resolved this issue? I am having this on our production environment and the jira page mentioned below is not available.

Any help on how to resolve this and recover the cluster back to healthy state would be really appreciated.