FlowException

[sont...@gmail.com]

hi
i don't know how to the reason of this exception when my application
process just 1GB data , and i have three task nodes and 10 pipes:


09/07/10 17:33:40 WARN mapred.LocalJobRunner: job_local_0001
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/
output/file.out in any of the configured local directories
at org.apache.hadoop.fs.LocalDirAllocator
$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:359)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead
(LocalDirAllocator.java:138)
at org.apache.hadoop.mapred.MapOutputFile.getOutputFile
(MapOutputFile.java:50)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run
(LocalJobRunner.java:169)
09/07/10 17:33:41 WARN flow.FlowStep: [page_rate flow] completion
events count: 0
09/07/10 17:33:41 WARN flow.FlowStep: [page_rate flow] abandoning
step: (2/2) ...TextLine[['offset', 'line']->[ALL]]"]["file:///data/
hadoop/sun_test/pageclickrate__test/output/2009-05-25/"]"],
predecessor failed: (1/2) ..., 'Level1ChannelName', 'Province',
'City', 'DeviceType', 'Mobile', 'page', 'people', 'rate']]"][log_head/
379/]
09/07/10 17:33:41 WARN flow.Flow: stopping jobs
09/07/10 17:33:41 INFO flow.FlowStep: [page_rate flow] stopping:
(2/2) ...TextLine[['offset', 'line']->[ALL]]"]["file:///data/hadoop/
sun_test/pageclickrate__test/output/2009-05-25/"]"]
09/07/10 17:33:41 INFO flow.FlowStep: [page_rate flow] stopping:
(1/2) ..., 'Level1ChannelName', 'Province', 'City', 'DeviceType',
'Mobile', 'page', 'people', 'rate']]"][log_head/379/]
09/07/10 17:33:41 WARN flow.Flow: stopped jobs
09/07/10 17:33:41 WARN flow.Flow: shutting down job executor
09/07/10 17:33:41 WARN flow.Flow: shutdown complete
cascading.flow.FlowException: step failed: (1/2) ...,
'Level1ChannelName', 'Province', 'City', 'DeviceType', 'Mobile',
'page', 'people', 'rate']]"][log_head/379/]
at cascading.flow.FlowStep$FlowStepJob.call(FlowStep.java:478)
at cascading.flow.FlowStep$FlowStepJob.call(FlowStep.java:409)
at java.util.concurrent.FutureTask$Sync.innerRun
(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)


thanks !!

[Chris K Wensel]

You probably already sorted this out, but you likely need to check the
logs on the slaves. those exceptions die there and don't propagate
back to the Flow (though one day this may change).

ckw

--
Chris K Wensel
ch...@concurrentinc.com
http://www.concurrentinc.com