Sequencefile storage issue

[samee...@gmail.com]

Hi everyone ,

 I am trying to use the SequenceFileStorage Utility and I have the following problem. Thanks in advance for your help.

%declare SEQFILE_LOADER 'com.twitter.elephantbird.pig.load.SequenceFileLoader';
%declare SEQFILE_STORAGE 'com.twitter.elephantbird.pig.store.SequenceFileStorage';
%declare INT_CONVERTER 'com.twitter.elephantbird.pig.util.IntWritableConverter';
%declare TEXT_CONVERTER 'com.twitter.elephantbird.pig.util.TextConverter';
%declare LONG_CONVERTER 'com.twitter.elephantbird.pig.util.LongWritableConverter';
%declare VECTOR_CONVERTER 'com.twitter.elephantbird.pig.mahout.VectorWritableConverter';

A = LOAD 'map.dat' as (v: ( f1: chararray, f2: chararray, f3: chararray, f4: chararray));

STORE A into 'map.out' using $SEQFILE_STORAGE ('-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER');

I get the following error:

2013-12-05 23:08:14,784 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: Pig script failed to parse:
<file map-test.pig, line 23, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.store.SequenceFileStorage' with arguments '[-c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter]'


Pig Stack Trace
---------------
ERROR 1200: Pig script failed to parse:
<file map-test.pig, line 23, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.store.SequenceFileStorage' wit\
h arguments '[-c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twi\
tter.elephantbird.pig.util.TextConverter]'

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Pig script failed to parse:
<file map-test.pig, line 23, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.store.SequenceFileStorage' wit\
h arguments '[-c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twi\
tter.elephantbird.pig.util.TextConverter]'
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:991)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:604)
        at org.apache.pig.Main.main(Main.java:157)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: Failed to parse: Pig script failed to parse:
<file map-test.pig, line 23, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.store.SequenceFileStorage' wit\
h arguments '[-c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twi\
tter.elephantbird.pig.util.TextConverter]'
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
        ... 14 more
Caused by:
<file map-test.pig, line 23, column 0> pig script failed to validate: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.store.SequenceFileStorage' wit\
h arguments '[-c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twi\
tter.elephantbird.pig.util.TextConverter]'
        at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:904)
        at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7260)
        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1359)
        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
        ... 15 more
Caused by: java.lang.RuntimeException: could not instantiate 'com.twitter.elephantbird.pig.store.SequenceFileStorage' with arguments '[-c com.twitter.elephantbird.pig.util.TextConv\
erter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter, -c com.twitter.elephantbird.pig.util.TextConverter]'
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:607)
        at org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:883)
        ... 21 more
Caused by: java.lang.NoSuchMethodException: com.twitter.elephantbird.pig.store.SequenceFileStorage.<init>([Ljava.lang.String;)
        at java.lang.Class.getConstructor0(Class.java:2721)
        at java.lang.Class.getConstructor(Class.java:1674)
        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:594)
        ... 22 more

[Andy Schlaikjer]

STORE A into 'map.out' using $SEQFILE_STORAGE ('-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER', '-c $TEXT_CONVERTER');

Note that sequence files store a series of key-value pairs. The sequence file storage and loader classes accept only two ctor arguments (your statement above has four); The first is a param string controlling key (de)serialization and the second is another param string controlling value (de)serialization.

--
You received this message because you are subscribed to the Google Groups "elephantbird-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elephantbird-d...@googlegroups.com.
To post to this group, send email to elephant...@googlegroups.com.
Visit this group at http://groups.google.com/group/elephantbird-dev.
For more options, visit https://groups.google.com/groups/opt_out.