Lipstick failing when running a pig script which performs join of hbase tables.

[Satish Chandra]

Hi,

I have a pig script which performs a join of two hbase tables. This script is working successfully when it is run individually but when it is run with lipstick-console-0.6-SNAPSHOT-withHadoop.jar, it is failing with following error

2015-06-02 02:35:08,446 [main] ERROR org.apache.pig.tools.grunt.LipstickGrunt - java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/filter/WritableByteArrayComparable

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:270)

at org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:662)

at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1320)

at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1308)

at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5158)

at org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3515)

at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1625)

at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)

at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)

at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)

at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)

at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1712)

at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1420)

at org.apache.pig.PigServer.parseAndBuild(PigServer.java:364)

at org.apache.pig.LipstickPigServer.parseAndBuild(LipstickPigServer.java:164)

at org.apache.pig.PigServer.executeBatch(PigServer.java:389)

at org.apache.pig.PigServer.executeBatch(PigServer.java:375)

at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:170)

at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:232)

at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203)

at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)

at com.netflix.lipstick.Main.run(Main.java:625)

at com.netflix.lipstick.Main.main(Main.java:171)

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.filter.WritableByteArrayComparable

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:425)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

... 24 more

I tried with explicitly setting hbase-0.92.1.jar to PIG_CLASSPATH but it didn't help. Any pointer would be much appreciated.

Thanks,

Satish