The magic of slices

[Markus]

We experienced an unexplainable JAQL behaviour which is really hard to
explain because of limited example possibilities. I think it has
something to do with JAQLs MapReduce Implementation.

We have a usual JSON data source - a JSON array. We read this like:

$file = read(hdfs($filename));

Then we have a part of transforms, groupings and merging. (for this
error, it is not elementary what this is exactly).
Then, for some reason, we get the following:

java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
at com.ibm.jaql.lang.expr.hadoop.MapReduceFn.eval(MapReduceFn.java:
241)
at com.ibm.jaql.lang.core.Var.setEval(Var.java:291)
at com.ibm.jaql.lang.expr.core.BindingExpr.eval(BindingExpr.java:197)
at com.ibm.jaql.lang.expr.core.DoExpr.eval(DoExpr.java:125)
at com.ibm.jaql.lang.expr.top.QueryExpr.eval(QueryExpr.java:85)
at com.ibm.jaql.lang.StreamPrinter.print(StreamPrinter.java:69)
at com.ibm.jaql.lang.Jaql.run(Jaql.java:494)
at com.ibm.jaql.lang.Jaql.run(Jaql.java:97)
at
com.ibm.jaql.util.shell.AbstractJaqlShell.run(AbstractJaqlShell.java:
53)
at
com.ibm.jaql.util.shell.AbstractJaqlShell.main(AbstractJaqlShell.java:
97)
at JaqlShell.main(JaqlShell.java:274)

As we have some experience with "Job failed" failures, we tried to
figure out where the reason was. As any statement turned out to be
valid, we did something unusual. We made a slice from the whole
source. This looks like

$file = slice(read(hdfs($filename)),0,1000000);

where the $file only contains a few thousand records. Surprise: the
code was executed without any errors. And this was the solution for
more than one script. We currently use it as a workaround for theese
JAQL errors, but this can't be a standrad solution.

So maybe our first question ist, what does a slice do internally, that
JAQL seems to do something very else?