Mongo-Hadoop PIG - Load from BSON and output to Mongo not working?

136 views
Skip to first unread message

LM Heah

unread,
Jul 6, 2014, 4:12:14 AM7/6/14
to mongod...@googlegroups.com
Hi All,

I am trying to load data from BSON and output to MongoStorage. 
The BSON load success but storing result to MongoStorage failed.

E.g.

raw = LOAD 'hdfs://SERVER:8020/user/hadoop/BSONFILE.bson' using com.mongodb.hadoop.pig.BSONLoader('',''); 
raw_limited = LIMIT raw 10;
STORE raw_limited INTO 'mongodb://123.1.1.123:27017/DB.TBL' USING com.mongodb.hadoop.pig.MongoInsertStorage('','');

These are the error I get, anyone can enlighten me?

Backend error message
---------------------
AttemptID:attempt_1404616432190_0125_r_000000_2 Info:Container killed by the ApplicationMaster.
Container killed on request. Exit code is 137
Container exited with a non-zero exit code 137

Backend error message
---------------------
AttemptID:attempt_1404616432190_0125_r_000000_3 Info:Error: java.io.IOException: Could not find schema in UDF context
at com.mongodb.hadoop.pig.MongoInsertStorage.prepareToWrite(MongoInsertStorage.java:130)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Error message from task (reduce) task_1404616432190_0125_r_000000
-----------------------------------------------------------------
ERROR 2997: Encountered IOException. Could not find schema in UDF context

java.io.IOException: Could not find schema in UDF context
at com.mongodb.hadoop.pig.MongoInsertStorage.prepareToWrite(MongoInsertStorage.java:130)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
================================================================================
Error message from task (reduce) task_1404616432190_0125_r_000000
-----------------------------------------------------------------
ERROR 2997: Encountered IOException. Could not find schema in UDF context

java.io.IOException: Could not find schema in UDF context
at com.mongodb.hadoop.pig.MongoInsertStorage.prepareToWrite(MongoInsertStorage.java:130)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
================================================================================
Error message from task (reduce) task_1404616432190_0125_r_000000
-----------------------------------------------------------------
ERROR 2997: Encountered IOException. Could not find schema in UDF context

java.io.IOException: Could not find schema in UDF context
at com.mongodb.hadoop.pig.MongoInsertStorage.prepareToWrite(MongoInsertStorage.java:130)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
================================================================================
Error message from task (reduce) task_1404616432190_0125_r_000000
-----------------------------------------------------------------
ERROR 2997: Encountered IOException. Could not find schema in UDF context

java.io.IOException: Could not find schema in UDF context
at com.mongodb.hadoop.pig.MongoInsertStorage.prepareToWrite(MongoInsertStorage.java:130)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.<init>(PigOutputFormat.java:125)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
================================================================================

Justin Lee

unread,
Jul 7, 2014, 9:20:54 AM7/7/14
to mongod...@googlegroups.com
That error gets thrown when the property mongoinsert.pig.output.schema can't be found. 


--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/c3f6959b-2597-4dd5-9da7-8524e2982593%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages