--
post: rha...@googlegroups.com ||
unsubscribe: rhadoop+u...@googlegroups.com ||
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Error in if (file.exists(cmd)) return(cmd) : argument is of length zero
Antonio, i have resolved that error by workaround that you have mentioned in the above link by setting HDFS_CMD. Now, after running the mapreduce jobs, i am getting the below error in the job tracker in hadoop logs.java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)
But in the R terminal i am getting the error as below:
14/08/26 05:36:41 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201408221219_0019
14/08/26 05:36:41 ERROR streaming.StreamJob: Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201408221219_0019_m_000000
14/08/26 05:36:41 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce, :
hadoop streaming failed with error code 1
Calls: mapreduce -> mr
Execution halted
Warning: $HADOOP_HOME is deprecated.
Warning: $HADOOP_HOME is deprecated.
Deleted hdfs://localhost:9000/tmp/file25b230f3f310
I have copied the program also for your reference as you have not mentioned that you are not aware of the exact logic.
R Program :
===========
require(rhbase)
require(rmr2)
#require(rhdfs)
Sys.setenv(HADOOP_CMD='/mnt/servers/hadoop/bin/hadoop');
Sys.setenv(HADOOP_STREAMING="/mnt/servers/hadoop/contrib/streaming/hadoop-streaming-1.2.1.jar");
hostLoc = 'localhost' #Give your server IP
port = 9090 #Default port for thrift service
#hb.init()
hb.init(serialize="char")
#hb.list.tables()
#hb.insert("test_stream_realtime",list(list("20100101",c("df:name","df:company"), list("Mohan","cognizant"))))
#hb.insert("test_stream_realtime",list(list("20100102",c("df:name","df:company"), list("Sharan","cognizant"))))
#hb.insert("test_stream_realtime",list(list("20100103",c("df:name","df:company"), list("Mathan","cognizant"))))
hb.describe.table("test_stream_realtime")
#r = from.dfs(mapreduce(input = "input",input.format = make.input.format("hbase",family.columns = list(cf=list("x","y")),map = function(k,v) v )))
print ('Here')
#hb.scan("test_stream_realtime",start=1,end=10,colspec=c("df"))
r = mapreduce(input = "test_stream_realtime",input.format = make.input.format("hbase",family.columns = list(df=list("name","company")),key.deserialize="raw",cell.deserialize = "raw"),output.format = "text",map = function(k,v) v )
Thanks,
Mohan.
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)
could it be possible to trace the error with the help of this log now ??
Thanks,
Mohan.
http://localhost:50030/jobdetails.jsp?jobid=job_201408221219_0019
then select map tasks, failed tasks, one in particular then logs then stderr or something like this. Take a look there and unless it clarifies the issue for you please report back here. Thanks
Can you do what you need to do on a small scale outside mapreduce, in console? That would be step one. Second, I would think about the input to the mapreduce job. Third, move the rhbase code inside the map function and see what happens.Antonio
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+unsubscribe@googlegroups.com.