ConcurrentModificationException - happens when using local job runner

[ANithian]

I ran into the following when using the LocalJobRunner:
8124 [Thread-35] WARN org.apache.hadoop.mapred.LocalJobRunner -
job_local_0004
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$ValueIterator.next(HashMap.java:822)
at cascading.flow.stack.StackElement.closeTraps(StackElement.java:87)
at cascading.flow.stack.StackElement.close(StackElement.java:197)
at
cascading.flow.stack.SinkMapperStackElement.close(SinkMapperStackElement.java:
136)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.StackElement.close(StackElement.java:203)
at cascading.flow.stack.FlowMapperStack.close(FlowMapperStack.java:
220)
at cascading.flow.FlowMapper.close(FlowMapper.java:99)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.LocalJobRunner
$Job.run(LocalJobRunner.java:177)

It seems that the local job runner spawns multiple threads to do work
and perhaps it's getting caught up in a spot where the map is being
modified in the StackElement.getTrapCollector method while being
closed in the StackElement.closeTraps() method.

I haven't seen this when running on the cluster although it doesn't
happen locally very often either.. typical of a thread related bug so
I can't say if this is a bug only in local mode or not. I am running
Cascading 1.1.1

Thanks!
Amit

[Chris K Wensel]

Hey Amit

Strange. Didn't realize LocalJobRunner would spawn multiple threads. I'll see what I can do for the next maint release.

cheers,
chris

> --
> You received this message because you are subscribed to the Google Groups "cascading-user" group.
> To post to this group, send email to cascadi...@googlegroups.com.
> To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cascading-user?hl=en.
>

--
Chris K Wensel
ch...@concurrentinc.com
http://www.concurrentinc.com

[Chris K Wensel]

Should be a fix for this in the next 1.1.2 wip build in a couple hours.

http://www.concurrentinc.com/downloads/

Sorry for the delay.

cheers,
chris

On May 25, 2010, at 3:24 PM, ANithian wrote: