Caused by: java.lang.ArithmeticException: / by zero

[jdcryans]

Hi users,

On of my devs ran into a problem:

09/11/20 16:25:47 WARN flow.Flow: stopped jobs
09/11/20 16:25:47 WARN flow.Flow: shutting down job executor
09/11/20 16:25:47 WARN flow.Flow: shutdown complete
Exception in thread "main" cascading.flow.FlowException: unhandled
exception
at cascading.flow.Flow.complete(Flow.java:607)
at com.stumbleupon.cascading.AdOrganicStumbleSummary.main
(AdOrganicStumbleSummary.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ArithmeticException: / by zero
at org.apache.hadoop.hbase.mapred.TableInputFormatBase.getSplits
(TableInputFormatBase.java:295)
at cascading.tap.hadoop.MultiInputFormat.getSplits
(MultiInputFormat.java:227)
at cascading.tap.hadoop.MultiInputFormat.getSplits
(MultiInputFormat.java:194)
at org.apache.hadoop.mapred.JobClient.writeOldSplits
(JobClient.java:810)
at org.apache.hadoop.mapred.JobClient.submitJobInternal
(JobClient.java:781)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:
730)
at cascading.flow.FlowStep$FlowStepJob.call(FlowStep.java:470)
at cascading.flow.FlowStep$FlowStepJob.call(FlowStep.java:409)
at java.util.concurrent.FutureTask$Sync.innerRun
(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run
(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

This is 1.0.9 and afaik it doesn't seems fixed in 1.0.17. Reading the
code in MultiInputFormat I see

for( int i = 0; i < inputSizes.length; i++ )
indexedSplits[ i ] = (int) Math.ceil( (double) numSplits * inputSizes
[ i ] / (double) totalSize );

So if we have 2 inputs and each has only 1 split, that gives 0? This
is a dev environment btw.

Thx,

J-D

[Jean-Daniel Cryans]

This fixes the issue for us:

- for( int i = 0; i < inputSizes.length; i++ )
- indexedSplits[ i ] = (int) Math.ceil( (double) numSplits *

inputSizes[ i ] / (double) totalSize );

+ for( int i = 0; i < inputSizes.length; i++ ) {
+ int tempSplits = (int) Math.ceil( (double) numSplits *

inputSizes[ i ] / (double) totalSize );

+ indexedSplits[ i ] = tempSplits == 0 ? 1 : tempSplits;
+ }

J-D

> --
>
> You received this message because you are subscribed to the Google Groups "cascading-user" group.
> To post to this group, send email to cascadi...@googlegroups.com.
> To unsubscribe from this group, send email to cascading-use...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cascading-user?hl=.
>
>
>

[Chris K Wensel]

Thanks!

--
Chris K Wensel
ch...@concurrentinc.com
http://www.concurrentinc.com