I'm trying to set up Druid with Hadoop 2.6.0 CDH 5.5.2. After wading through and working around various dependency issues, I'm hitting a wall. Our default compression codec is snappy. We force the Hadoop java processes to load the native snappy libraries by setting LD_LIBRARY_PATH=/usr/lib/hadoop/lib/native. I had gotten an error about not being able to load snappy in the middleManager earlier this week, but got around it by setting LD_LIBRARY_PATH in the middleManager's environment. Now I'm to the point where an indexing task successfully completes a MapReduce job and writes out snappy files in /tmp/druid-indexing. The MapReduce job finishes fine. After it does, it looks like a Peon process attempts to read what was written. While doing so, I get a snappy loading error:
2016-05-20T20:35:27,591 INFO io.druid.indexer.DetermineHashedPartitionsJob: Job completed, loading up partitions for intervals[Optional.of([2015-09-01T00:00:00.000Z/2015-09-02T00:00:00.000Z])].
2016-05-20T20:35:27,643 ERROR io.druid.indexing.overlord.ThreadPoolTaskRunner: Exception while running task[HadoopIndexTask{id=index_hadoop_pageviews_2016-05-20T20:33:38.361Z, type=index_hadoop, dataSource=pageviews}]
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.1.jar:?]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:160) ~[druid-indexing-service-0.9.0.jar:0.9.0]
at io.druid.indexing.common.task.HadoopIndexTask.run(HadoopIndexTask.java:175) ~[druid-indexing-service-0.9.0.jar:0.9.0]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:338) [druid-indexing-service-0.9.0.jar:0.9.0]
at io.druid.indexing.overlord.ThreadPoolTaskRunner$ThreadPoolTaskRunnerCallable.call(ThreadPoolTaskRunner.java:318) [druid-indexing-service-0.9.0.jar:0.9.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_101]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_101]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_101]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_101]
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_101]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_101]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_101]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_101]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:157) ~[druid-indexing-service-0.9.0.jar:0.9.0]
... 7 more
Caused by: java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.
at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65) ~[?:?]
at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:193) ~[?:?]
at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:178) ~[?:?]
at org.apache.hadoop.io.compress.CompressionCodec$Util.createInputStreamWithCodecPool(CompressionCodec.java:157) ~[?:?]
at org.apache.hadoop.io.compress.SnappyCodec.createInputStream(SnappyCodec.java:163) ~[?:?]
at io.druid.indexer.Utils.openInputStream(Utils.java:101) ~[druid-indexing-hadoop-0.9.0.jar:0.9.0]
at io.druid.indexer.Utils.openInputStream(Utils.java:77) ~[druid-indexing-hadoop-0.9.0.jar:0.9.0]
at io.druid.indexer.DetermineHashedPartitionsJob.run(DetermineHashedPartitionsJob.java:161) ~[druid-indexing-hadoop-0.9.0.jar:0.9.0]
at io.druid.indexer.JobHelper.runJobs(JobHelper.java:323) ~[druid-indexing-hadoop-0.9.0.jar:0.9.0]
at io.druid.indexer.HadoopDruidDetermineConfigurationJob.run(HadoopDruidDetermineConfigurationJob.java:86) ~[druid-indexing-hadoop-0.9.0.jar:0.9.0]
at io.druid.indexing.common.task.HadoopIndexTask$HadoopDetermineConfigInnerProcessing.runTask(HadoopIndexTask.java:291) ~[druid-indexing-service-0.9.0.jar:0.9.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_101]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_101]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_101]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_101]
at io.druid.indexing.common.task.HadoopTask.invokeForeignLoader(HadoopTask.java:157) ~[druid-indexing-service-0.9.0.jar:0.9.0]
... 7 more
2016-05-20T20:35:27,763 INFO io.druid.indexing.worker.executor.ExecutorLifecycle: Task completed with status: {
"id" : "index_hadoop_pageviews_2016-05-20T20:33:38.361Z",
"status" : "FAILED",
"duration" : 82683
}