Pig-ORCStorage

126 views
Skip to first unread message

Jegan Baskar

unread,
Nov 21, 2015, 1:10:57 PM11/21/15
to Hue-Users
Hi Romain,

Is cloudera 5.1 version supports ORC File format. Am using pig 0.12 as in build installed version.

The below code am executing in HUE and its giving me an error.  Do i need to add any jar (Piggybank.jar) in my script???

A = LOAD '/home/user/days.txt' USING PigStorage(',') as (name:chararray, age:int);
STORE A INTO '/home/user/out.txt' Using OrcStorage();

<file script.pig, line 2, column 44> pig script failed to validate: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve OrcStorage using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
 at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1260)
 at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1245)
 at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5158)
 at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7756)
 at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669)
 at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102)
 at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560)
 at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
 at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:188)

And if i want to add a compression in the storage OrcStorage('-c snappy') do i need to register the snappy jar in my script else its a inbuilt compression mechanism in pig?

Regards,
Jegan

Romain Rigaux

unread,
Nov 21, 2015, 10:06:22 PM11/21/15
to Jegan Baskar, Hue-Users
Yes, You need to add the jar that contains the OTC reader !
--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

Jegan

unread,
Nov 22, 2015, 9:27:54 AM11/22/15
to Romain Rigaux, Hue-Users
Hi Romain,

Thanks for your input. i have tried with piggybank.jar and snappy.jar but not luck. Still am getting same issue. Please help me.

And let me know any other jar do i need to add and let me know the name of the jar's.

Regards,
Jegan

Romain Rigaux

unread,
Nov 22, 2015, 6:49:46 PM11/22/15
to Jegan, Hue-Users
Reply all
Reply to author
Forward
0 new messages