kafka streams boot error and not consumed

[朱健]

when I start my kstreams application, the ksteams and kstable can consume message normally.But when doing a kstream-globalKtable join , there is no output messages. The error message is here:

2018-04-18 16:27:04,543] DEBUG [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] [Consumer clientId=client_id_transaction_crm_keeper_20180417-global-restore-consumer, groupId=] Kafka consumer has been closed (org.apache.kafka.clients.consumer.KafkaConsumer)
[2018-04-18 16:27:04,543] DEBUG [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] Closing global storage engine crm_denorm_merchant_keeper_k_store (org.apache.kafka.streams.processor.internals.GlobalStateManagerImpl)
[2018-04-18 16:27:04,544] DEBUG [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] Sending GET with input null to http://public-kafka-002:8081/schemas/ids/84 (io.confluent.kafka.schemaregistry.client.rest.RestService)
[2018-04-18 16:27:04,549] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] [Consumer clientId=client_id_transaction_crm_keeper_20180417-StreamThread-1-consumer, groupId=app_id_transaction_crm_keeper_20180417] Committed offset 15131188 for partition events.upay.trade-3 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
[2018-04-18 16:27:04,550] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_4] Committing (org.apache.kafka.streams.processor.internals.StreamTask)
[2018-04-18 16:27:04,550] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_4] Flushing producer (org.apache.kafka.streams.processor.internals.RecordCollectorImpl)
[2018-04-18 16:27:04,554] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_5] Committing (org.apache.kafka.streams.processor.internals.StreamTask)
[2018-04-18 16:27:04,554] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_5] Flushing producer (org.apache.kafka.streams.processor.internals.RecordCollectorImpl)
[2018-04-18 16:27:04,557] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_6] Committing (org.apache.kafka.streams.processor.internals.StreamTask)
[2018-04-18 16:27:04,557] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_6] Flushing producer (org.apache.kafka.streams.processor.internals.RecordCollectorImpl)
[2018-04-18 16:27:04,564] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] [Consumer clientId=client_id_transaction_crm_keeper_20180417-StreamThread-1-consumer, groupId=app_id_transaction_crm_keeper_20180417] Fetch READ_UNCOMMITTED at offset 0 for partition events.upay.trade-9 returned fetch data (error=NONE, highWaterMark=0, lastStableOffset = -1, logStartOffset = 0, abortedTransactions = null, recordsSizeInBytes=0) (org.apache.kafka.clients.consumer.internals.Fetcher)
[2018-04-18 16:27:04,565] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] [Consumer clientId=client_id_transaction_crm_keeper_20180417-StreamThread-1-consumer, groupId=app_id_transaction_crm_keeper_20180417] Fetch READ_UNCOMMITTED at offset 14967921 for partition events.upay.trade-0 returned fetch data (error=NONE, highWaterMark=14978520, lastStableOffset = -1, logStartOffset = 6920025, abortedTransactions = null, recordsSizeInBytes=1048576) (org.apache.kafka.clients.consumer.internals.Fetcher)
[2018-04-18 16:27:04,565] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] [Consumer clientId=client_id_transaction_crm_keeper_20180417-StreamThread-1-consumer, groupId=app_id_transaction_crm_keeper_20180417] Fetch READ_UNCOMMITTED at offset 15131188 for partition events.upay.trade-3 returned fetch data (error=NONE, highWaterMark=15141875, lastStableOffset = -1, logStartOffset = 7491412, abortedTransactions = null, recordsSizeInBytes=1048576) (org.apache.kafka.clients.consumer.internals.Fetcher)
[2018-04-18 16:27:04,565] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] [Consumer clientId=client_id_transaction_crm_keeper_20180417-StreamThread-1-consumer, groupId=app_id_transaction_crm_keeper_20180417] Fetch READ_UNCOMMITTED at offset 15031602 for partition events.upay.trade-6 returned fetch data (error=NONE, highWaterMark=15042291, lastStableOffset = -1, logStartOffset = 7493465, abortedTransactions = null, recordsSizeInBytes=1048576) (org.apache.kafka.clients.consumer.internals.Fetcher)
[2018-04-18 16:27:04,567] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] [Consumer clientId=client_id_transaction_crm_keeper_20180417-StreamThread-1-consumer, groupId=app_id_transaction_crm_keeper_20180417] Committed offset 15031602 for partition events.upay.trade-6 (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
[2018-04-18 16:27:04,567] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_7] Committing (org.apache.kafka.streams.processor.internals.StreamTask)
[2018-04-18 16:27:04,567] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_7] Flushing producer (org.apache.kafka.streams.processor.internals.RecordCollectorImpl)
[2018-04-18 16:27:04,569] DEBUG [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] Removed sensor with name hitRatio (org.apache.kafka.common.metrics.Metrics)
[2018-04-18 16:27:04,582] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_8] Committing (org.apache.kafka.streams.processor.internals.StreamTask)
[2018-04-18 16:27:04,582] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_8] Flushing producer (org.apache.kafka.streams.processor.internals.RecordCollectorImpl)
[2018-04-18 16:27:04,584] DEBUG [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] stream-thread [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] Released global state dir lock (org.apache.kafka.streams.processor.internals.StateDirectory)
[2018-04-18 16:27:04,585] INFO [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] global-stream-thread [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] State transition from PENDING_SHUTDOWN to DEAD (org.apache.kafka.streams.processor.internals.GlobalStreamThread)
[2018-04-18 16:27:04,585] INFO [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] stream-client [client_id_transaction_crm_keeper_20180417]State transition from RUNNING to ERROR (org.apache.kafka.streams.KafkaStreams)
[2018-04-18 16:27:04,585] WARN [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] stream-client [client_id_transaction_crm_keeper_20180417]Global thread has died. The instance will be in error state and should be closed. (org.apache.kafka.streams.KafkaStreams)
[2018-04-18 16:27:04,585] INFO [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] global-stream-thread [client_id_transaction_crm_keeper_20180417-GlobalStreamThread] Shutdown complete (org.apache.kafka.streams.processor.internals.GlobalStreamThread)
Exception in thread "client_id_transaction_crm_keeper_20180417-GlobalStreamThread" [2018-04-18 16:27:04,585] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_9] Committing (org.apache.kafka.streams.processor.internals.StreamTask)
[2018-04-18 16:27:04,585] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] task [1_9] Flushing producer (org.apache.kafka.streams.processor.internals.RecordCollectorImpl)
java.lang.NullPointerException
        at org.apache.kafka.streams.processor.internals.GlobalStateUpdateTask.update(GlobalStateUpdateTask.java:77)
        at org.apache.kafka.streams.processor.internals.GlobalStreamThread$StateConsumer.pollAndUpdate(GlobalStreamThread.java:239)
        at org.apache.kafka.streams.processor.internals.GlobalStreamThread.run(GlobalStreamThread.java:282)
[2018-04-18 16:27:04,588] DEBUG [client_id_transaction_crm_keeper_20180417-StreamThread-1] stream-thread [client_id_transaction_crm_keeper_20180417-StreamThread-1] Committed all active tasks [1_0, 1_1, 1_2, 1_3, 1_4, 1_5, 1_6, 1_7, 1_8, 1_9] and standby tasks [] in 69ms (org.apache.kafka.streams.processor.internals.StreamThread)

[Matthias J. Sax]

Not sure. I had a quick look into the code. Might be a bug.

Can you reliably reproduce the issue? Would be great to get a minimum
example that we can use to reproduce in a test.

-Matthias

> --
> You received this message because you are subscribed to the Google
> Groups "Confluent Platform" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to confluent-platf...@googlegroups.com
> <mailto:confluent-platf...@googlegroups.com>.
> To post to this group, send email to confluent...@googlegroups.com
> <mailto:confluent...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/confluent-platform/1eaf19ac-b694-45a3-b5c4-281cc4ca69be%40googlegroups.com
> <https://groups.google.com/d/msgid/confluent-platform/1eaf19ac-b694-45a3-b5c4-281cc4ca69be%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

[朱健]

I've resolved this problem by delete all local state. The "join problem" described is caused by  "null key" stream. I've not read the Stream DSL carefully.

The NPE problem can't be reproduced . I'm sorry.

在 2018年4月19日星期四 UTC+8下午8:23:32，Matthias J. Sax写道：

> an email to confluent-platform+unsub...@googlegroups.com
> <mailto:confluent-platform+unsub...@googlegroups.com>.