Hi,
I am using 0.6.171.
From the task log and Hadoop jobtracker, the hadoop map/reduce jobs ran successfully. The files on hadoop after the job failed is shown in this picture:
https://lh3.googleusercontent.com/-ebb83nGJkO8/VSTa884__EI/AAAAAAAABNg/rgE7m4wnZsw/s1600/Untitled.pngbelow is a more detailed druid task log, reducer log also attached.
2015-04-08 09:50:52,058 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Job job_1426279437401_565266 completed successfully
2015-04-08 09:50:52,236 INFO [task-runner-0] org.apache.hadoop.mapreduce.Job - Counters: 53
File System Counters
FILE: Number of bytes read=
5175293687 FILE: Number of bytes written=10880875352
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=36760981983
HDFS: Number of bytes written=2682017189
HDFS: Number of read operations=3474
HDFS: Number of large read operations=0
HDFS: Number of write operations=20
Job Counters
Killed map tasks=6
Launched map tasks=580
Launched reduce tasks=10
Other local map tasks=11
Data-local map tasks=528
Rack-local map tasks=41
Total time spent by all maps in occupied slots (ms)=97614345
Total time spent by all reduces in occupied slots (ms)=5731419
Total time spent by all map tasks (ms)=32538115
Total time spent by all reduce tasks (ms)=1910473
Total vcore-seconds taken by all map tasks=32538115
Total vcore-seconds taken by all reduce tasks=1910473
Total megabyte-seconds taken by all map tasks=133276119040
Total megabyte-seconds taken by all reduce tasks=
7825297408 Map-Reduce Framework
Map input records=62555721
Map output records=52722302
Map output bytes=36381101937
Map output materialized bytes=
5594877400 Input split bytes=90118
Combine input records=0
Combine output records=0
Reduce input groups=20
Reduce shuffle bytes=
5594877400 Reduce input records=52722302
Reduce output records=52722302
Spilled Records=104042959
Shuffled Maps =5740
Failed Shuffles=0
Merged Map outputs=5740
GC time elapsed (ms)=309442
CPU time spent (ms)=35744580
Physical memory (bytes) snapshot=1098268250112
Virtual memory (bytes) snapshot=2925780250624
Total committed heap usage (bytes)=1219227549696
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
io.druid.indexer.HadoopDruidIndexerConfig$IndexJobCounters
INVALID_ROW_COUNTER=9833415
File Input Format Counters
Bytes Read=36760891865
File Output Format Counters
Bytes Written=2682017189
ALERT:/tmp/druid-indexing
ALERT:pulsar_event_merged
ALERT:2015-04-08T093021.124Z
2015-04-08 09:50:52,293 ERROR [task-runner-0] io.druid.indexing.overlord.ThreadPoolTaskRunner - Exception while running task[HadoopIndexTask{id=index_hadoop_
pulsar_event_merged_2015-04-08T09:30:21.121Z, type=index_hadoop, dataSource=pulsar_event_merged}]
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: File /tmp/druid-indexing/pulsar_event_merged/2015-04-08T093021.124Z/segmentDescriptorIn
Caused by: java.io.FileNotFoundException: File /tmp/druid-indexing/pulsar_event_merged/2015-04-08T093021.124Z/segmentDescriptorInfo does not exist.
2015-04-08 09:50:52,306 INFO [task-runner-0] io.druid.indexing.worker.executor.ExecutorLifecycle - Task completed with status: {
"id" : "index_hadoop_pulsar_event_merged_2015-04-08T09:30:21.121Z",
"status" : "FAILED",
"duration" : 1223161
}
thanks.