Thanks for the suggestion, but adding a 'warming period' doesn't work for me.
Let me post my druid settings here:
overlord: druid.service=overlord
druid.port=8087
common.runtime.properties:
druid.selectors.indexing.serviceName=overlord
druid.discovery.curator.path=
/druid/discovery
zookeeper:
[zk: localhost:2181(CONNECTED) 5] get /druid/discovery/overlord/b7e1a803-b2a8-4e9d-8f81-0f702f5733d7
{"name":"overlord","id":"b7e1a803-b2a8-4e9d-8f81-0f702f5733d7","address":"
phxdbx1189.phx.ebay.com","port":8087,"sslPort":null,"payload":null,"registrationTimeUTC":1432614864207,"serviceType":"DYNAMIC","uriSpec":null}
...
Full log of tranquility app:
2015-06-09 00:56:03,367 [169] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
2015-06-09 00:56:03,368 [169] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=
druidppzk1.prodtracking.vip.lvs.ebay.com:2181,
druidppzk2.prodtracking.vip.lvs.ebay.com:2181,
druidppzk3.prodtracking.vip.lvs.ebay.com:2181,
druidppzk4.prodtracking.vip.slc.ebay.com:2181,
druidppzk5.prodtracking.vip.slc.ebay.com:2181,
druidppzk6.prodtracking.vip.phx.ebay.com:2181,
druidppzk7.prodtracking.vip.phx.ebay.com:2181,
druidppzk8.prodtracking.vip.phx.ebay.com:2181,
druidppzk9.prodtracking.vip.slc.ebay.com:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@7c637cca
2015-06-09 00:56:03,376 [169] INFO com.metamx.emitter.core.LoggingEmitter - Start: started [true]
2015-06-09 00:56:03,377 [171] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server
lvs3b02c-c310.stratus.lvs.ebay.com/10.132.112.134:2181. Will not attempt to authenticate using SASL (unknown error)
2015-06-09 00:56:03,385 [171] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to
lvs3b02c-c310.stratus.lvs.ebay.com/10.132.112.134:2181, initiating session
2015-06-09 00:56:03,418 [171] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server
lvs3b02c-c310.stratus.lvs.ebay.com/10.132.112.134:2181, sessionid = 0x34dc7c8f2110000, negotiated timeout = 60000
2015-06-09 00:56:03,418 [172] INFO org.apache.curator.framework.state.ConnectionStateManager - State change: CONNECTED
2015-06-09 00:56:03,419 [170] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[overlord] to Set()
2015-06-09 00:56:03,468 [169] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[overlord] to Set(ServiceInstance{name='overlord', id='b7e1a803-b2a8-4e9d-8f81-0f702f5733d7', address='
phxdbx1189.phx.ebay.com', port=8087, sslPort=null, payload=null, registrationTimeUTC=1432614864207, serviceType=DYNAMIC, uriSpec=null})
2015-06-09 00:56:03,470 [169] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: overlord
2015-06-09 00:56:03,471 [169] INFO com.ebay.jetstream.tracking.sojenrich.druid.TranquilityProcessor - Tranquility processor enabled.
2015-06-09 00:56:03,615 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Creating new merged beam for identifier[overlord/pulsar_event_tq] timestamp[2015-06-09T01:00:00.000-07:00] (target = 2, actual = 0)
2015-06-09 00:56:03,619 [175] INFO com.metamx.common.scala.control$ - Creating druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0 (service = overlord)
2015-06-09 00:56:03,653 [92] INFO com.metamx.common.scala.control$ - Created druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0 (service = overlord)
2015-06-09 00:56:03,716 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-08-0000-0000] to Set()
2015-06-09 00:56:03,717 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-08-0000-0000
2015-06-09 00:56:03,724 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Created beam: {"interval":"2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00","partition":0,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0","firehoseId":"pulsar_event_tq-08-0000-0000"}],"timestamp":"2015-06-09T01:00:00.000-07:00"}
2015-06-09 00:56:03,724 [175] INFO com.metamx.tranquility.druid.DruidBeam - Closing Druid beam for datasource[pulsar_event_tq] interval[2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00] (tasks = index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0)
2015-06-09 00:56:03,724 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Closing client for service: druid:firehose:pulsar_event_tq-08-0000-0000
2015-06-09 00:56:03,725 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - No longer monitoring service[druid:firehose:pulsar_event_tq-08-0000-0000]
2015-06-09 00:56:03,727 [175] INFO com.metamx.common.scala.control$ - Creating druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0 (service = overlord)
2015-06-09 00:56:03,759 [92] INFO com.metamx.common.scala.control$ - Created druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0 (service = overlord)
2015-06-09 00:56:03,822 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-08-0001-0000] to Set()
2015-06-09 00:56:03,823 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-08-0001-0000
2015-06-09 00:56:03,824 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Created beam: {"interval":"2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00","partition":1,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0","firehoseId":"pulsar_event_tq-08-0001-0000"}],"timestamp":"2015-06-09T01:00:00.000-07:00"}
2015-06-09 00:56:03,824 [175] INFO com.metamx.tranquility.druid.DruidBeam - Closing Druid beam for datasource[pulsar_event_tq] interval[2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00] (tasks = index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0)
2015-06-09 00:56:03,824 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Closing client for service: druid:firehose:pulsar_event_tq-08-0001-0000
2015-06-09 00:56:03,824 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - No longer monitoring service[druid:firehose:pulsar_event_tq-08-0001-0000]
2015-06-09 00:56:03,831 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Writing new beam data to[/tranquility/beams/overlord/pulsar_event_tq/data]: {"latestTime":"2015-06-09T01:00:00.000-07:00","latestCloseTime":"2015-06-08T19:00:00.000-07:00","beams":{"2015-06-09T01:00:00.000-07:00":[{"interval":"2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00","partition":0,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0","firehoseId":"pulsar_event_tq-08-0000-0000"}],"timestamp":"2015-06-09T01:00:00.000-07:00"},{"interval":"2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00","partition":1,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0","firehoseId":"pulsar_event_tq-08-0001-0000"}],"timestamp":"2015-06-09T01:00:00.000-07:00"}]}}
2015-06-09 00:56:03,888 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Adding beams for identifier[overlord/pulsar_event_tq] timestamp[2015-06-09T01:00:00.000-07:00]: List(Map(interval -> 2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00, partition -> 0, tasks -> ArraySeq(Map(id -> index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0, firehoseId -> pulsar_event_tq-08-0000-0000)), timestamp -> 2015-06-09T01:00:00.000-07:00), Map(interval -> 2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00, partition -> 1, tasks -> ArraySeq(Map(id -> index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0, firehoseId -> pulsar_event_tq-08-0001-0000)), timestamp -> 2015-06-09T01:00:00.000-07:00))
2015-06-09 00:56:03,923 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-08-0000-0000] to Set()
2015-06-09 00:56:03,923 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-08-0000-0000
2015-06-09 00:56:03,957 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-08-0001-0000] to Set()
2015-06-09 00:56:03,958 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-08-0001-0000
2015-06-09 00:56:04,079 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Creating new merged beam for identifier[overlord/pulsar_event_tq] timestamp[2015-06-09T00:00:00.000-07:00] (target = 2, actual = 0)
2015-06-09 00:56:04,080 [175] INFO com.metamx.common.scala.control$ - Creating druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_0_0 (service = overlord)
2015-06-09 00:56:04,113 [92] INFO com.metamx.common.scala.control$ - Created druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_0_0 (service = overlord)
2015-06-09 00:56:04,148 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-07-0000-0000] to Set()
2015-06-09 00:56:04,149 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-07-0000-0000
2015-06-09 00:56:04,149 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Created beam: {"interval":"2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00","partition":0,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_0_0","firehoseId":"pulsar_event_tq-07-0000-0000"}],"timestamp":"2015-06-09T00:00:00.000-07:00"}
2015-06-09 00:56:04,149 [175] INFO com.metamx.tranquility.druid.DruidBeam - Closing Druid beam for datasource[pulsar_event_tq] interval[2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00] (tasks = index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_0_0)
2015-06-09 00:56:04,150 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Closing client for service: druid:firehose:pulsar_event_tq-07-0000-0000
2015-06-09 00:56:04,150 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - No longer monitoring service[druid:firehose:pulsar_event_tq-07-0000-0000]
2015-06-09 00:56:04,159 [175] INFO com.metamx.common.scala.control$ - Creating druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0 (service = overlord)
2015-06-09 00:56:04,191 [92] INFO com.metamx.common.scala.control$ - Created druid indexing task with id: index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0 (service = overlord)
2015-06-09 00:56:04,226 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-07-0001-0000] to Set()
2015-06-09 00:56:04,227 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-07-0001-0000
2015-06-09 00:56:04,227 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Created beam: {"interval":"2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00","partition":1,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0","firehoseId":"pulsar_event_tq-07-0001-0000"}],"timestamp":"2015-06-09T00:00:00.000-07:00"}
2015-06-09 00:56:04,228 [175] INFO com.metamx.tranquility.druid.DruidBeam - Closing Druid beam for datasource[pulsar_event_tq] interval[2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00] (tasks = index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0)
2015-06-09 00:56:04,228 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Closing client for service: druid:firehose:pulsar_event_tq-07-0001-0000
2015-06-09 00:56:04,228 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - No longer monitoring service[druid:firehose:pulsar_event_tq-07-0001-0000]
2015-06-09 00:56:04,230 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Writing new beam data to[/tranquility/beams/overlord/pulsar_event_tq/data]: {"latestTime":"2015-06-09T01:00:00.000-07:00","latestCloseTime":"2015-06-08T19:00:00.000-07:00","beams":{"2015-06-09T01:00:00.000-07:00":[{"interval":"2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00","partition":0,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_0_0","firehoseId":"pulsar_event_tq-08-0000-0000"}],"timestamp":"2015-06-09T01:00:00.000-07:00"},{"interval":"2015-06-09T01:00:00.000-07:00/2015-06-09T02:00:00.000-07:00","partition":1,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T01:00:00.000-07:00_1_0","firehoseId":"pulsar_event_tq-08-0001-0000"}],"timestamp":"2015-06-09T01:00:00.000-07:00"}],"2015-06-09T00:00:00.000-07:00":[{"interval":"2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00","partition":0,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_0_0","firehoseId":"pulsar_event_tq-07-0000-0000"}],"timestamp":"2015-06-09T00:00:00.000-07:00"},{"interval":"2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00","partition":1,"tasks":[{"id":"index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0","firehoseId":"pulsar_event_tq-07-0001-0000"}],"timestamp":"2015-06-09T00:00:00.000-07:00"}]}}
2015-06-09 00:56:04,287 [175] INFO com.metamx.tranquility.beam.ClusteredBeam - Adding beams for identifier[overlord/pulsar_event_tq] timestamp[2015-06-09T00:00:00.000-07:00]: List(Map(interval -> 2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00, partition -> 0, tasks -> ArraySeq(Map(id -> index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_0_0, firehoseId -> pulsar_event_tq-07-0000-0000)), timestamp -> 2015-06-09T00:00:00.000-07:00), Map(interval -> 2015-06-09T00:00:00.000-07:00/2015-06-09T01:00:00.000-07:00, partition -> 1, tasks -> ArraySeq(Map(id -> index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0, firehoseId -> pulsar_event_tq-07-0001-0000)), timestamp -> 2015-06-09T00:00:00.000-07:00))
2015-06-09 00:56:04,322 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-07-0000-0000] to Set()
2015-06-09 00:56:04,323 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-07-0000-0000
2015-06-09 00:56:04,358 [175] INFO com.metamx.common.scala.net.finagle.DiscoResolver - Updating instances for service[druid:firehose:pulsar_event_tq-07-0001-0000] to Set()
2015-06-09 00:56:04,359 [175] INFO com.metamx.tranquility.finagle.FinagleRegistry - Created client for service: druid:firehose:pulsar_event_tq-07-0001-0000
2015-06-09 00:56:04,406 [92] WARN com.metamx.tranquility.finagle.FutureRetry$ - Transient error, will try again in 206 ms
java.io.IOException: Unable to push events to task: index_realtime_pulsar_event_tq_2015-06-09T00:00:00.000-07:00_1_0 (status = TaskRunning)
at com.metamx.tranquility.druid.DruidBeam$$anonfun$4$$anonfun$apply$4$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$3$$anonfun$applyOrElse$2.apply(DruidBeam.scala:160)
at com.metamx.tranquility.druid.DruidBeam$$anonfun$4$$anonfun$apply$4$$anonfun$apply$6$$anonfun$apply$7$$anonfun$apply$3$$anonfun$applyOrElse$2.apply(DruidBeam.scala:146)
at com.twitter.util.Future$$anonfun$map$1$$anonfun$apply$6.apply(Future.scala:863)
at com.twitter.util.Try$.apply(Try.scala:13)
at com.twitter.util.Future$.apply(Future.scala:90)
at com.twitter.util.Future$$anonfun$map$1.apply(Future.scala:863)
at com.twitter.util.Future$$anonfun$map$1.apply(Future.scala:863)
at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:824)
at com.twitter.util.Future$$anonfun$flatMap$1.apply(Future.scala:823)
at com.twitter.util.Promise$Transformer.liftedTree1$1(Promise.scala:100)
at com.twitter.util.Promise$Transformer.k(Promise.scala:100)
at com.twitter.util.Promise$Transformer.apply(Promise.scala:110)
at com.twitter.util.Promise$Transformer.apply(Promise.scala:91)
at com.twitter.util.Promise$$anon$2.run(Promise.scala:345)
at com.twitter.concurrent.LocalScheduler$Activation.run(Scheduler.scala:186)
at com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:157)
at com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:212)
at com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:86)
at com.twitter.util.Promise.runq(Promise.scala:331)
at com.twitter.util.Promise.updateIfEmpty(Promise.scala:642)
at com.twitter.util.Promise.update(Promise.scala:615)
at com.twitter.util.Promise.setValue(Promise.scala:591)
at com.twitter.concurrent.AsyncQueue.offer(AsyncQueue.scala:76)
at com.twitter.finagle.transport.ChannelTransport.handleUpstream(ChannelTransport.scala:45)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:108)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:194)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142)
at com.twitter.finagle.channel.ChannelStatsHandler.messageReceived(ChannelStatsHandler.scala:86)
at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142)
at com.twitter.finagle.channel.ChannelRequestStatsHandler.messageReceived(ChannelRequestStatsHandler.scala:35)
at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: com.twitter.finagle.NoBrokersAvailableException: No hosts are available for druid:firehose:pulsar_event_tq-07-0001-0000
at com.twitter.finagle.NoStacktrace(Unknown Source)
thanks.