Getting UNABLE_TO_CONNECT_TO_STREAM when resuming Kafka Supervisor

[sivapra...@gmail.com]

I have a Kafka datasource. I had to update the ingestion spec and I submitted the updated spec via a POST call. This triggered the current running index_kafka tasks for this source to go the segment hand-off phase but new index_kafka tasks were not getting created. The supervisor then moved to UNABLE_TO_CONNECT_TO_STREAM state. So I suspended the data source and tried resuming it, it stays in CREATING_TASKS for a while but eventually moves to UNABLE_TO_CONNECT_TO_STREAM. In the overlord log, I do see an WARN message saying 

WARN [KafkaSupervisor-poc_kafka_source_hourly] org.apache.druid.indexing.seekablestream.supervisor.SeekableStreamSupervisor - Exception in supervisor run loop for dataSource [poc_kafka_source_hourly]

It doesn't print the complete stack trace for the exception. Resetting the supervisor made it work eventually but we cannot do that in production. 

[Rachel Pedreschi]

Any errors in any other log, like middle manager or historical?

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/876d2f29-cf75-44bf-bd64-1ee8d2828d65n%40googlegroups.com.

--

Rachel Pedreschi

VP Developer Relations and Community

Imply.io

[Peter Marshall]

Interesting - as that state seems to imply that Druid doesn't think it's connected to that stream before:

https://druid.apache.org/docs/latest/development/extensions-core/kafka-ingestion.html#getting-supervisor-status-report

Does that make sense in your scenario?

[Peter Marshall]

And +1 to Rachel on the other log files...