tutorial error help me

[한종빈]

mapreduce(input = small.ints, map = function(k, v) cbind(v, v^2))

14/01/08 11:10:09 WARN streaming.StreamJob: -file option is deprecated, please use generic option -files instead.

packageJobJar: [/tmp/RtmpNXyeZg/rmr-local-env3eaa18e2db97, /tmp/RtmpNXyeZg/rmr-global-env3eaa7d3e0b0, /tmp/RtmpNXyeZg/rmr-streaming-map3eaa7ec02349] [] /tmp/streamjob4421325598601398577.jar tmpDir=null

14/01/08 11:10:10 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.

14/01/08 11:10:10 INFO mapred.FileInputFormat: Total input paths to process : 1

14/01/08 11:10:10 INFO mapred.JobClient: Running job: job_201312121716_2459

14/01/08 11:10:11 INFO mapred.JobClient:  map 0% reduce 0%

14/01/08 11:10:29 INFO mapred.JobClient: Task Id : attempt_201312121716_2459_m_000000_2, Status : FAILED

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.streaming.AutoInputFormat not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1649)

at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:620)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:394)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)

at org.apache.hadoop.mapred.Child$4.run(Child.java:268)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:396)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)

at org.apache.hadoop.mapred.Child.main(Child.java:262)

Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.streaming.AutoInputFormat not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1617)

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1641)

14/01/08 11:10:29 INFO mapred.JobClient: Task Id : attempt_201312121716_2459_m_000001_2, Status : FAILED

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.streaming.AutoInputFormat not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1649)

at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:620)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:394)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)

at org.apache.hadoop.mapred.Child$4.run(Child.java:268)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:396)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)

at org.apache.hadoop.mapred.Child.main(Child.java:262)

Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.streaming.AutoInputFormat not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1617)

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1641)

14/01/08 11:10:36 INFO mapred.JobClient: Job complete: job_201312121716_2459

14/01/08 11:10:36 INFO mapred.JobClient: Counters: 7

14/01/08 11:10:36 INFO mapred.JobClient:   Job Counters 

14/01/08 11:10:36 INFO mapred.JobClient:     Failed map tasks=1

14/01/08 11:10:36 INFO mapred.JobClient:     Launched map tasks=8

14/01/08 11:10:36 INFO mapred.JobClient:     Data-local map tasks=8

14/01/08 11:10:36 INFO mapred.JobClient:     Total time spent by all maps in occupied slots (ms)=42773

14/01/08 11:10:36 INFO mapred.JobClient:     Total time spent by all reduces in occupied slots (ms)=0

14/01/08 11:10:36 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0

14/01/08 11:10:36 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0

14/01/08 11:10:36 ERROR streaming.StreamJob: Job not Successful!

Streaming Command Failed!

Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce,  : 

  hadoop streaming failed with error code 1

the error message show.. 

i don't know what can i do

please help me 

set env

/etc/profile

export HADOOP_HOME=/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop

export HADOOP_CONF_DIR=/etc/hadoop/conf

export HADOOP_CMD=/usr/bin/hadoop

export HADOOP_LIBEXEC_DIR=/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/libexec

/etc/environment

HADOOP_STREAMING=/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.4.0.jar

CLASSPATH=/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-mapreduce/hadoop-streaming.jar

i use 

R version 3.0.2 

CDH-4.4.0-1

[Antonio Piccolboni]

On Tue, Jan 7, 2014 at 6:37 PM, 한종빈 <stom...@gmail.com> wrote:

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.streaming.AutoInputFormat not found

This is the interesting part of the stack trace and your CLASSPATH setting is awfully short. Try this command

hadoop classpath

to get a complete setting.

Antonio

[bayroot22]

+1

I am experiencing the exact same issue running CDH 4.5 so I am very interested in this as well.

These are the variables I set

 export HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib

 export CLASSPATH=/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/hadoop-mapreduce/hadoop-streaming-2.0.0-cdh4.5.0.jar

 export HADOOP_HOME=/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/hadoop

 export HADOOP_CMD=/etc/alternatives/hadoop

 export HADOOP_STREAMING=/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/hadoop-mapreduce/hadoop-streaming.jar

[bayroot22]

I tried a lot of different things and ultimately I had to set my HADOOP_STREAMING variable to:

/opt/cloudera/parcels/CDH-4.5.0-1.cdh4.5.0.p0.30/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.5.0.jar

Also in order for this to work R has to be installed on all the datanodes, I missed that in the documentation. 

[Antonio Piccolboni]

I doubt that would manifest itself with a ClassNotFoundException, but it can't hurt to make sure the installation and configuration is correct.

Antonio

--
post: rha...@googlegroups.com ||
unsubscribe: rhadoop+u...@googlegroups.com ||
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.