Index out of bounds exception during BLOCKGEN By INDEX

Santhosh Swaminathan

unread,

Jul 1, 2015, 9:56:14 AM7/1/15

to cubert...@googlegroups.com

I was trying to generate blocks for a second dataset based on the blocks generating on a first dataset. The first dataset blockgen completed successfully. When I tried the second dataset by INDEX

and I got the following error when i compiled my script:

Cannot compile cubert script. Exiting.

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0

at java.util.ArrayList.rangeCheck(ArrayList.java:635)

at java.util.ArrayList.get(ArrayList.java:411)

at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.getPostCondition(DependencyAnalyzer.java:328)

at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.exitProgram(DependencyAnalyzer.java:284)

at com.linkedin.cubert.analyzer.physical.PhysicalPlanWalker.walk(PhysicalPlanWalker.java:75)

at com.linkedin.cubert.analyzer.physical.DependencyAnalyzer.rewrite(DependencyAnalyzer.java:93)

at com.linkedin.cubert.ScriptExecutor.rewrite(ScriptExecutor.java:342)

at com.linkedin.cubert.ScriptExecutor.main(ScriptExecutor.java:529)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:197)

JOB "panel image fact BLOCKGEN"

REDUCERS 20;

MAP {

input = LOAD "/path/to/other/input" USING TEXT("schema": ....");

}

BLOCKGEN data BY INDEX "/path/to/first/output" PARTITIONED ON akey;

STORE data INTO "/path/to/other/output" USING RUBIX();

END

Could someone please help me out?

-Thanks

Santhosh

Santhosh Swaminathan

unread,

Jul 1, 2015, 11:13:30 AM7/1/15

to cubert...@googlegroups.com

Few additional details: I am using MAPR version of hadoop (0.20.2) and Mapr uses maprfs instead of the traditional hdfs, would this cause the problem while getting the path?

Santhosh Swaminathan

unread,

Jul 1, 2015, 12:27:14 PM7/1/15

to cubert...@googlegroups.com

My mistake. The /path/to/other/input was incorrect. After correcting it, I am getting a new error:

[Dependency Analyzer] Program inputs: [/path/to/other/input, /path/to/first/output]

Obtaining schema of rubix file maprfs:/path/to/first/output/part-r-00000.rubix

Analyzing job [my BLOCKGEN]...

ERROR: input block data not found

At: [MAP] BLOCKGEN data BY INDEX "/path/to/first/output" PARTITIONED ON key1 SORTED ON col2;

null

Cannot compile cubert script. Exiting.

Exception in thread "main" com.linkedin.cubert.analyzer.physical.PlanRewriteException

at com.linkedin.cubert.analyzer.physical.SemanticAnalyzer.error(SemanticAnalyzer.java:730)

at com.linkedin.cubert.analyzer.physical.SemanticAnalyzer.error(SemanticAnalyzer.java:708)

at com.linkedin.cubert.analyzer.physical.SemanticAnalyzer.getPreconditions(SemanticAnalyzer.java:743)

at com.linkedin.cubert.analyzer.physical.SemanticAnalyzer.visitOperator(SemanticAnalyzer.java:305)

at com.linkedin.cubert.analyzer.physical.PhysicalPlanWalker.walk(PhysicalPlanWalker.java:46)

at com.linkedin.cubert.analyzer.physical.SemanticAnalyzer.rewrite(SemanticAnalyzer.java:140)

at com.linkedin.cubert.ScriptExecutor.rewrite(ScriptExecutor.java:342)

at com.linkedin.cubert.ScriptExecutor.main(ScriptExecutor.java:529)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:197)

How to use rubixtool to find out if the output of the first blockgen is correct?

On Wednesday, July 1, 2015 at 9:56:14 AM UTC-4, Santhosh Swaminathan wrote:

Santhosh Swaminathan

unread,

Jul 1, 2015, 2:44:54 PM7/1/15

to cubert...@googlegroups.com

Found out the issue. I was incorrect variable input vs data. Corrected it and it is working now.

On Wednesday, July 1, 2015 at 9:56:14 AM UTC-4, Santhosh Swaminathan wrote:

Reply all

Reply to author

Forward