Step State Serialization (try again)

[jd]

Hi,

I'm running in to Step State Serialization issues related to:

1) Lots of anonymous functions

2) Adding lots of columns/fields to the output

Is there a parameter I can turn up to buy some more time while I re-write to fix this issue?

-JD

java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:348) at org.apache.hadoop.mapred.Child$4.run(Child.java:282) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1117) at org.apache.hadoop.mapred.Child.main(Child.java:271) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 9 more Caused by: cascading.flow.FlowException: step state path is empty: /tmp/mapr-hadoop/mapred/local/taskTracker/jdavis/distcache/537681335966608374_1743861847_1343348525/maprfs/tmp/hadoop-jdavis/step-state-4547177DBB3045FC9D3990471EC829AC at cascading.flow.hadoop.util.HadoopUtil.readStateFromDistCache(HadoopUtil.java:467) at cascading.flow.hadoop.FlowMapper.configure(FlowMapper.java:79) ... 14 more

[Oscar Boykin]

Just to be clear: this is with just cascading, or are you using scalding?

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/bfb8d04b-735a-4a5a-af77-8fee3056060e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Oscar Boykin :: @posco :: http://twitter.com/posco

[jd]

Plain 'ol Cascading.