Re: Hadoop Indexing task failing

[saur...@inshorts.com]

Just FYI : Hadoop indexing  is working fine on druid build 0.9.1.1 with the following change merged.

On Monday, September 26, 2016 at 2:38:49 AM UTC+5:30, SAURABH JAIN wrote:

Hey ,

I am using the following build https://github.com/druid-io/druid/releases/tag/druid-0.9.2-rc1 with the following changes merged https://github.com/druid-io/druid/pull/2458.

Kafka Indexing task is working fine.

I am getting the following exception while running hadoop indexing task : 

2016-09-25 14:07:08,681 task-runner-0-priority-0 ERROR Recursive call to appender Console

2016-09-25 14:07:08,682 task-runner-0-priority-0 ERROR Recursive call to appender Console

2016-09-25 14:07:08,697 task-runner-0-priority-0 ERROR Recursive call to appender Console

2016-09-25 14:07:08,697 task-runner-0-priority-0 ERROR Recursive call to appender Console

2016-09-25T14:07:08,671 ERROR [task-runner-0-priority-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_prism-data-12_2016-09-25T14:07:01.206Z, type=index_hadoop, dataSource=prism-data-12}]

java.lang.RuntimeException: java.lang.reflect.InvocationTargetException

at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]

at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:204) ~[druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:208) ~[druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:436) [druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:408) [druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_101]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]

Caused by: java.lang.reflect.InvocationTargetException

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]

at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:201) ~[druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

... 7 more

Caused by: java.lang.NullPointerException: segmentOutputPath

at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:229) ~[guava-16.0.1.jar:?]

at io.druid.indexer.HadoopDruidIndexerConfig.verify(HadoopDruidIndexerConfig.java:585) ~[druid-indexing-hadoop-0.9.2-rc1.jar:0.9.2-rc1]

at io.druid.indexer.HadoopDruidIndexerJob.<init>(HadoopDruidIndexerJob.java:46) ~[druid-indexing-hadoop-0.9.2-rc1.jar:0.9.2-rc1]

at io.druid.indexing.common.task.HadoopIndexTask$HadoopIndexGeneratorInnerProcessing.runTask(HadoopIndexTask.java:258) ~[druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]

at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]

at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:201) ~[druid-indexing-service-0.9.2-rc1.jar:0.9.2-rc1]

... 7 more

2016-09-25T14:07:08,848 INFO [task-runner-0-priority-0] io.druid.indexing.overlord.TaskRunnerUtils - Task [index_hadoop_prism-data-12_2016-09-25T14:07:01.206Z] status changed to [FAILED].

2016-09-25T14:07:08,850 INFO [task-runner-0-priority-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {

  "id" : "index_hadoop_prism-data-12_2016-09-25T14:07:01.206Z",

  "status" : "FAILED",

  "duration" : 3927

}

What can be the issue for this? . 

Thanks,

Saurabh