presto-kafka Query kafka topic got error ArrayIndexOutOfBoundsException sometimes

16 views
Skip to first unread message

Chris Yuan

unread,
Sep 7, 2016, 2:26:15 AM9/7/16
to Presto

env:

  • presto v 0.149 5workers
  • kafka v0.8.1.1 5nodes, topic on 10 partitions

query sql:SELECT count(1) FROM kafka.tpch.customer


java.lang.ArrayIndexOutOfBoundsException: 5
    at kafka.api.TopicMetadata$$anonfun$readFrom$1.apply$mcVI$sp(TopicMetadata.scala:38)
    at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
    at kafka.api.TopicMetadata$.readFrom(TopicMetadata.scala:36)
    at kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31)
    at kafka.api.TopicMetadataResponse$$anonfun$3.apply(TopicMetadataResponse.scala:31)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.immutable.Range.foreach(Range.scala:141)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at kafka.api.TopicMetadataResponse$.readFrom(TopicMetadataResponse.scala:31)
    at kafka.consumer.SimpleConsumer.send(SimpleConsumer.scala:92)
    at kafka.javaapi.consumer.SimpleConsumer.send(SimpleConsumer.scala:68)
    at com.facebook.presto.kafka.KafkaSplitManager.getSplits(KafkaSplitManager.java:83)
    at com.facebook.presto.split.SplitManager.getSplits(SplitManager.java:45)
    at com.facebook.presto.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:112)
    at com.facebook.presto.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:92)
    at com.facebook.presto.sql.planner.plan.TableScanNode.accept(TableScanNode.java:135)
    at com.facebook.presto.sql.planner.DistributedExecutionPlanner$Visitor.visitLimit(DistributedExecutionPlanner.java:258)
    at com.facebook.presto.sql.planner.DistributedExecutionPlanner$Visitor.visitLimit(DistributedExecutionPlanner.java:92)
    at com.facebook.presto.sql.planner.plan.LimitNode.accept(LimitNode.java:86)
    at com.facebook.presto.sql.planner.DistributedExecutionPlanner.plan(DistributedExecutionPlanner.java:78)
    at com.facebook.presto.sql.planner.DistributedExecutionPlanner.plan(DistributedExecutionPlanner.java:83)
    at com.facebook.presto.execution.SqlQueryExecution.planDistribution(SqlQueryExecution.java:303)
    at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:226)
    at com.facebook.presto.execution.QueuedExecution.lambda$start$1(QueuedExecution.java:62)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)


this issue is not occurring frequently, How can I fix this issue?

Reply all
Reply to author
Forward
0 new messages