Mongo-hadoop connector (help)

[Fabiola Rojo Orellana]

Hi,
I try to install mongodb connector, but i get this error:
Executing hadoop job:
 /home/fabiola/hadoop-binaries/hadoop-2.4.0/bin/hadoop \
        jar \
        examples/treasury_yield/build/libs/treasury_yield-1.4.0-SNAPSHOT.jar \
        com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig \
        -Dmongo.input.split_size=8 \
        -Dmongo.job.verbose=true \
        -Dmongo.input.uri=mongodb://localhost:27017/mongo_hadoop.yield_historical.in \
        -Dmongo.output.uri=mongodb://localhost:27017/mongo_hadoop.yield_historical.out
Exception in thread "main" java.lang.NoClassDefFoundError: com/mongodb/hadoop/util/MongoTool
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:249)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:205)
Caused by: java.lang.ClassNotFoundException: com.mongodb.hadoop.util.MongoTool
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 14 more
:historicalYield FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/home/fabiola/mongo-hadoop/build.gradle' line: 517

* What went wrong:
Execution failed for task ':historicalYield'.
> Process 'command '/home/fabiola/hadoop-binaries/hadoop-2.4.0/bin/hadoop'' finished with non-zero exit value 1

[Justin Lee]

run 'gradlew jar' first then run the examples.

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/b2d5ddec-0f40-434d-a1d2-9ff768ae0cbd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Fabiola Rojo Orellana]

Justin thank! but I did it, the examples not run :(

[Justin Lee]

How are you running them exactly, then?  Everything just works on my end: both locally and on jenkins.

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/20015961-1973-4bf1-9dda-8f1d44249046%40googlegroups.com.

[Fabiola Rojo Orellana]

i am running with this page: http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-hadoop/
and i have installed mongo db, hadoop 2.4 (both locally)

[Justin Lee]

Hrm.  those are out of date and I meant to update them.  try this:  gradlew testJar historicalYield

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/40e99611-aecc-4eb0-abd7-946d835d7a89%40googlegroups.com.

[Fabiola Rojo Orellana]

ups! is the unique i found......  if you provide me some tutorial I would appreciate

[Justin Lee]

https://github.com/mongodb/mongo-hadoop/blob/master/examples/README.md

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/c7ae0213-0cd3-4730-946c-7d7dc13e6d32%40googlegroups.com.

[Fabiola Rojo Orellana]

Thanks!! ... i will try and later tell as it turned (i am noob :/)

[Fabiola Rojo Orellana]

i try and i try .... always the similar error .. when i execute ./gradlew historicalYield

El lunes, 21 de julio de 2014 09:14:04 UTC-4, Justin Lee escribió:

[Justin Lee]

You still don't seem to be running the correct command.  For clarity, run this:  gradlew jar testJar historicalYield

That should work.  If it doesn't, I don't know what else to tell you.

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/837d4335-ad18-48f8-bb0c-9d739229711b%40googlegroups.com.

[TJ Tang]

I ran into the same issue, it appears. Tried a few times with frustration. Eventually I found out by running the "hadoop classpath" command. I have installed Hadoop and set the env variable HADOOP_PREFIX to my original install. But the gradlew jar will ignore the existing Hadoop and go ahead to install its own. When try to run "gradlew historicalYield", hadoop is trying to find the Mongo libraries from  under HADOOP_PREFIX/. Hence the ClassNotFound exception.  After changing the HADOOP_PREFIX to the one installed by gradlew, it proceeded to next... exception.  (Server is Ubuntu 12.04 provisioned using Vagrant, java is oracle JDK7). I'm going to try to reinstall Java.

Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /hadoop/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.

It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

14/07/23 08:54:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Exception in thread "main" java.lang.NullPointerException

at java.lang.Class.isAssignableFrom(Native Method)

at org.apache.hadoop.conf.Configuration.setClass(Configuration.java:1968)

at org.apache.hadoop.mapreduce.Job.setOutputFormatClass(Job.java:823)

at com.mongodb.hadoop.util.MongoTool.runMapReduceJob(MongoTool.java:184)

at com.mongodb.hadoop.util.MongoTool.run(MongoTool.java:93)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)

at com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig.main(TreasuryYieldXMLConfig.java:52)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

在 2014年7月22日星期二UTC+8下午8时37分16秒，Justin Lee写道：

[TJ Tang]

Okay  I had to uncomment out this line in mongo-defaultx.xml

    <value>com.mongodb.hadoop.MongoOutputFormat</value>

Not sure why the default didn't work though. After this the hisotricalYield example succeeded.

在 2014年7月23日星期三UTC+8下午4时56分40秒，TJ Tang写道：

[Justin Lee]

HADOOP_PREFIX definitely muddles things.  You should unset that in any shell you're running these samples in.  I just pushed a change that'll remove that env var for the execution of the samples.  Give that a try and see if that helps.

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/ebec067c-3306-4cfe-be75-33eaff3d7295%40googlegroups.com.

[Fabiola Rojo Orellana]

hi it me again :/

Nothing works for me.

Please, comment  the their versions of operating system and tools used.

I have worked with various versions and operating systems.

What is better, operating systems 32 or 64 bits?

[Fabiola Rojo Orellana]

Hi, 

I'm trying to run the treasury yield example that brings the connector .... 

  now that was finally stopped running at 97% and does not advance (takes more than 12 hours so) 

Any help please !!!!! 

I really had a hard time this

This is all that show in the shell:

[Justin Lee]

The samples should only take a few minutes each to run.  Check in build/logs and see if any errors occurred when trying to launch the hadoop cluster.

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

--

You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: http://www.mongodb.org/about/support/.
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at http://groups.google.com/group/mongodb-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/505137d5-2fb6-44a9-9549-baf1c3182be4%40googlegroups.com.

[Fabiola Rojo Orellana]

I deleted all logs,
I ran the process again and checked each file. Only one had
output messages is the attachment.

help me please!!!

[Justin Lee]

Are you on master when trying this?

--------------------------------

{ name     : "Justin Lee", 

  title    : "Software Engineer",

  twitter  : "@evanchooly",

  web      : [ "10gen.com", "antwerkz.com" ],

  location : "New York, NY" }

To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/46e1a366-ba50-4eb8-a98a-b30614c8dc4a%40googlegroups.com.

[Fabiola Rojo Orellana]

Sorry but I do not know what you mean, mongo or hadoop ???