mongodb hadoop connector

[Zhu Wayne]

I am using CDH4 mongo hadoop connector and I got unauthorized exception:
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to calculate input splits: unauthorized
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)

The user ID is an admin user and has access to database - testz and  config (system database). I verified via mongodb command. However, I can't tell mongodb that I am using admin user using mongodb connection URL.

Here is my script:
REGISTER deploy/mongo-hadoop_cdh4b1-1.1.0-SNAPSHOT.jar;
REGISTER deploy/mongo-hadoop-core_cdh4b1-1.1.0-SNAPSHOT.jar;
REGISTER deploy/mongo-hadoop-pig_cdh4b1-1.1.0-SNAPSHOT.jar;
REGISTER deploy/mongo-java-driver-2.11.1.jar;
raw =   LOAD  'mongodb://myuid:mypasswd@mongodbhost:20000/testz.testz'
        USING com.mongodb.hadoop.pig.MongoLoader('id, wcsId, parentId, wcsParentId')
        AS    (id:chararray, wcsId:chararray, parentId:chararray, wcsParentId:chararray);
dump raw;

Here is my stack trace:

Backend error message during job submission
-------------------------------------------
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Unable to calculate input splits: unauthorized
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
        at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1064)
        at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1081)
        at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:993)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:946)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:946)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:318)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.startReadyJobs(JobControl.java:238)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:269)
        at java.lang.Thread.run(Thread.java:662)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257)
Caused by: java.lang.IllegalArgumentException: Unable to calculate input splits: unauthorized
        at com.mongodb.hadoop.util.MongoSplitter.calculateUnshardedSplits(MongoSplitter.java:116)
        at com.mongodb.hadoop.util.MongoSplitter.calculateSplits(MongoSplitter.java:85)
        at com.mongodb.hadoop.MongoInputFormat.getSplits(MongoInputFormat.java:51)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
        ... 15 more

[Bernie Hackett]

> ERROR 2118: Unable to calculate input splits: unauthorized

See this thread:

https://groups.google.com/d/msg/mongodb-user/wKTPri2rnSk/BCdaOgCAWNUJ

--
--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb
 
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Zhu Wayne]

Thanks. It works after setting "-Dmongo.input.split.create_input_splits=false" for a single split.