Offset commit failed on partition

[danho...@gmail.com]

Hi all we have run into an odd issue where ALL of our consumer groups are getting 'stuck' in:

Offset commit failed on partition..

We have been able to replicate this on existing, and new topics we create.

We can run :

kafka-console-consumer --bootstrap-server xzy:9092 --topic testtopic --partition 0

multiple times in parallel on separate shells, for example we have 3 running, and 1 or 2 of those will begin exhibiting the following error (99% of the time) .once the error occurs that consumer will never recover.

[2019-05-02
 18:32:52,545] ERROR [Consumer clientId=consumer-1, groupId=console-consumer-94446] Offset commit failed on partition workflow.wdc-tst-1 at offset 7319765: The request timed out. (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)

In addition we have also seen (much rarer):

[2019-05-03 16:20:11,454] ERROR [Consumer clientId=consumer-1, groupId=console-consumer-77894] Offset commit failed on partition workflow3.wdc-tst-0 at offset 77: The server experienced an unexpected error when processing the request (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)

[2019-05-03 16:20:11,456] WARN [Consumer clientId=consumer-1, groupId=console-consumer-77894] Asynchronous auto-commit of offsets {workflow3.wdc-tst-0=OffsetAndMetadata{offset=77, metadata=‘’}} failed: Unexpected error in commit: The server experienced an unexpected error when processing the request (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)

Any ideas on how to diagnose/troubleshoot/fix this issue?

Also ..when do consumer groups from the console-consumer go away? if ever?..do we need to explicitly delete them?

Thanks.

Dan

[Lisheng Wang]

Hi Dan

May i know if you run both console-consumer and broker on same machine or not? 

Kafka will delete consumer group when fulfill 2 conditions as following:

1. there is no any active consumer in that consumer group

2. all offsets which that consumer group committed has been deleted

Thanks.

Wang Lisheng

<danho...@gmail.com> 于2019年5月4日周六 上午12:45写道：

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/490da730-47a2-4648-b3b9-5312f0a65d39%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.