problem using BSON Store

91 views

Skip to first unread message

batman

unread,

Oct 13, 2014, 7:02:53 PM10/13/14

to mongod...@googlegroups.com

hey guys,

We are trying to follow example from link below but getting output location validation failure. https://github.com/mongodb/mongo-hadoop/blob/master/pig/README.md

STORE raw_out INTO 'file:///tmp/whatever.bson' USING com.mongodb.hadoop.pig.BSONStorage('id');

We tried following format for output folder:
hdfs:///user/content/201410131003
/user/content/201410131003
/user/content/201410131003/file.bson
file:///user/content/201410131003/file.bson

Below are the errors we got, any idea ?

Pig Stack Trace

---------------

ERROR 6000: Output Location Validation Failed for: 'hdfs:///user/content/201410131003 More info to follow:

Output directory not set.

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2042: Error in new logical plan. Try -Dpig.usenewlogicalplan=false.

at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:313)

at org.apache.pig.PigServer.compilePp(PigServer.java:1365)

at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1207)

at org.apache.pig.PigServer.execute(PigServer.java:1201)

at org.apache.pig.PigServer.access$100(PigServer.java:129)

at org.apache.pig.PigServer$Graph.execute(PigServer.java:1528)

at org.apache.pig.PigServer.executeBatchEx(PigServer.java:373)

at org.apache.pig.PigServer.executeBatch(PigServer.java:340)

at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:115)

at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:172)

at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)

at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)

at org.apache.pig.Main.run(Main.java:500)

at org.apache.pig.Main.main(Main.java:107)

Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 6000: Output Location Validation Failed for: 'hdfs:///user/content/201410131003 More info to follow:

Output directory not set.

at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:102)

at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:76)

at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)

at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)

at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)

at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)

at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:52)

at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:292)

... 13 more

Caused by: org.apache.hadoop.mapred.InvalidJobConfException: Output directory not set.

at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:123)

at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:87)

... 23 more

================================================================================

Christian Schneider

unread,

Mar 22, 2015, 9:07:40 PM3/22/15

to mongod...@googlegroups.com

Hi Batman,

have you solved the issue?

I get exactly the same stacktrace when using s3:// or s3n:// like mentioned in the docs.