Druid Can't get data after get some error message.

219 views
Skip to first unread message

HJ KIM

unread,
May 18, 2016, 10:34:49 PM5/18/16
to Druid User
Hello. Can I ask some error msg?

I using kafka, tranquility, druid(imply1.1.0). And I can see data on druid side with GUI.

But after got error msg, I can't get any data. Should I change some offset configuration?

Please see below msg,

filename: /usr/local/imply-1.1.0/var/sv/tranquility-kafka.log


```
06:36:09.672 [KafkaConsumer-0-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x154c293b6050002, negotiated timeout = 40000
06:36:09.876 [KafkaConsumer-0-EventThread] INFO  o.a.c.f.state.ConnectionStateManager - State change: CONNECTED
06:36:12.230 [KafkaConsumer-0] INFO  c.metamx.emitter.core.LoggingEmitter - Start: started [true]
06:36:13.508 [main-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x154c293b6050001 after 1ms
06:36:18.179 [KafkaConsumer-0] DEBUG c.m.t.tranquilizer.Tranquilizer - Swapping out buffer with 1 messages, 1 batches now pending.
06:33:12.839 [KafkaConsumer-1-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x154c28ba6340044, negotiated timeout = 40000
06:33:12.846 [KafkaConsumer-1-EventThread] INFO  o.a.c.f.state.ConnectionStateManager - State change: CONNECTED
06:33:13.164 [KafkaConsumer-1] INFO  c.metamx.emitter.core.LoggingEmitter - Start: started [true]
06:33:14.151 [KafkaConsumer-1] DEBUG c.m.t.tranquilizer.Tranquilizer - Swapping out buffer with 1 messages, 1 batches now pending.
06:33:14.152 [KafkaConsumer-1] DEBUG c.m.t.tranquilizer.Tranquilizer - Sending buffer with 1 messages.
06:33:14.267 [KafkaConsumer-1] DEBUG c.m.t.tranquilizer.Tranquilizer - Sent 0 out of 1 messages.
06:33:14.268 [KafkaConsumer-1] DEBUG c.m.t.tranquilizer.Tranquilizer - Sent buffer, 0 batches pending.
06:33:14.269 [KafkaConsumer-1] DEBUG kafka.consumer.PartitionTopicInfo - reset consume offset of twitter:0: fetched offset = 123398: consumed offset = 116214 to 116214
06:33:14.296 [KafkaConsumer-1] ERROR c.m.tranquility.kafka.KafkaConsumer - Exception:
com.metamx.tranquility.tranquilizer.MessageDroppedException: null
        at com.twitter.finagle.NoStacktrace(Unknown Source) ~[na:na]
06:33:14.297 [KafkaConsumer-1] INFO  c.m.tranquility.kafka.KafkaConsumer - Shutting down - attempting to flush buffers and commit final offsets
06:33:14.300 [Tranquilizer-BackgroundSend-[ClusteredBeam(druid:overlord/metrics-kafka)]] DEBUG c.m.t.tranquilizer.Tranquilizer - Interrupted, exiting thread.
May 18, 2016 6:33:14 AM com.twitter.finagle.Init$$anonfun$1 apply$mcV$sp
INFO: Finagle version 6.31.0 (rev=50d3bb0eea5ad3ed332111d707184c80fed6a506) built at 20151203-164135
.......
06:36:18.180 [KafkaConsumer-0] DEBUG c.m.t.tranquilizer.Tranquilizer - Sending buffer with 1 messages.
06:36:18.806 [KafkaConsumer-0] DEBUG c.m.t.tranquilizer.Tranquilizer - Sent 0 out of 1 messages.
06:36:18.808 [KafkaConsumer-0] DEBUG c.m.t.tranquilizer.Tranquilizer - Sent buffer, 0 batches pending.
06:36:18.813 [KafkaConsumer-CommitThread] ERROR c.m.tranquility.kafka.KafkaConsumer - Commit thread failed!
com.metamx.tranquility.tranquilizer.MessageDroppedException: null
        at com.twitter.finagle.NoStacktrace(Unknown Source) ~[na:na]
06:36:18.814 [KafkaConsumer-CommitThread] INFO  c.m.tranquility.kafka.KafkaConsumer - Shutting down - attempting to flush buffers and commit final offsets
06:36:18.816 [Tranquilizer-BackgroundSend-[ClusteredBeam(druid:overlord/metrics-kafka)]] DEBUG c.m.t.tranquilizer.Tranquilizer - Interrupted, exiting thread.
May 18, 2016 6:36:18 AM com.twitter.finagle.Init$$anonfun$1 apply$mcV$sp
INFO: Finagle version 6.31.0 (rev=50d3bb0eea5ad3ed332111d707184c80fed6a506) built at 20151203-164135

```

Fangjin Yang

unread,
May 19, 2016, 9:11:05 PM5/19/16
to Druid User

Gian Merlino

unread,
May 19, 2016, 9:11:20 PM5/19/16
to druid...@googlegroups.com
Have you set reportDropsAsExceptions: true? If so then this behavior is expected. Otherwise it's not – you should see a log message about a drop but not an exception.

Also consider upgrading to imply-1.2.1, that has newer versions of Druid (0.9.0) and Tranquility (0.8.0).

Gian

On Wed, May 18, 2016 at 7:33 PM, HJ KIM <fortre...@gmail.com> wrote:
Hello. Can I ask some error msg?

I using kafka, tranquility, druid(imply 1.0.0). And I can see data on druid side with GUI.

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/3cb2d0b6-2cb6-4ec7-a639-788a4ee85f4b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages