When running the tutorial (My first map reduce job) Problem , subprocess failed with code 1

[linetor]

HI All

I install hadoop(pseudo) , and R(by using R-studio) and RHadoop

• OS : Windows 7
  
• Hadoop : 2.6.0
  
• rmr : 3,3,1 for windows
  
• rhdfs : 1.0.8 for windwos
  

And I am having a problem when running the tutorial (My first map reduce job)  from: https://github.com/RevolutionAnalytics/rmr2/blob/master/docs/tutorial.md

The first line of code: small.ints = to.dfs(1:1000) Apparently works okay, other than the warning. And I check the file.

 15/03/26 14:42:33 WARN zlib.ZlibFactory: Failed to load/initialize native-zlib library

15/03/26 14:42:33 INFO compress.CodecPool: Got brand-new compressor [.deflate]

However when I try to execute the second line:

> mapreduce(input=small.ints, map =function(k,v) cbind(v, v^2))

I get a hadoop streaming failed and the job is therefore not executed

And I run Hadoop example word count. It is OK. So I think It's problem of R or Hadoop-streaming

Also the line of code from.dfs(small.ints) works okay.

I think where the problem might be R lib location. but In Windows. there are no different package folder(/usr/lib or /home/lib). So It's not.

And The Real Problem is there is no stderr. I saw others log. There are log about the problem. But The thing that I have is  (Full log is at the bottom)

15/03/26 14:48:45 INFO mapreduce.Job:  map 0% reduce 0%

15/03/26 14:48:53 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000000_0, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

So I can't Check the related problem. Is there any setting raleted to the log ??

Any idea or insight will be really appreciated.

Thank you

Log

> small.ints = to.dfs(1:1000)

15/03/26 14:48:34 WARN zlib.ZlibFactory: Failed to load/initialize native-zlib library

15/03/26 14:48:34 INFO compress.CodecPool: Got brand-new compressor [.deflate]

> mapreduce(input = small.ints, map = function(k, v) cbind(v, v^2))

15/03/26 14:48:36 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces

packageJobJar: [/C:/Users/linetor/AppData/Local/Temp/hadoop-unjar9072986975313623981/] [] C:\Users\linetor\AppData\Local\Temp\streamjob8412367128897596396.jar tmpDir=null

15/03/26 14:48:37 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032

15/03/26 14:48:37 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032

15/03/26 14:48:38 INFO mapred.FileInputFormat: Total input paths to process : 1

15/03/26 14:48:38 INFO mapreduce.JobSubmitter: number of splits:2

15/03/26 14:48:38 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1427347776374_0002

15/03/26 14:48:39 INFO impl.YarnClientImpl: Submitted application application_1427347776374_0002

15/03/26 14:48:39 INFO mapreduce.Job: The url to track the job: http://LGCNS-PC:8088/proxy/application_1427347776374_0002/

15/03/26 14:48:39 INFO mapreduce.Job: Running job: job_1427347776374_0002

15/03/26 14:48:45 INFO mapreduce.Job: Job job_1427347776374_0002 running in uber mode : false

15/03/26 14:48:45 INFO mapreduce.Job:  map 0% reduce 0%

15/03/26 14:48:53 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000000_0, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)

at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)

at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

15/03/26 14:48:53 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000001_0, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)

at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)

at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

15/03/26 14:49:00 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000000_1, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)

at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)

at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

15/03/26 14:49:01 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000001_1, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)

at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)

at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

15/03/26 14:49:08 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000000_2, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)

at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)

at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

15/03/26 14:49:09 INFO mapreduce.Job: Task Id : attempt_1427347776374_0002_m_000001_2, Status : FAILED

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1

at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:322)

at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:535)

at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)

at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

15/03/26 14:49:16 INFO mapreduce.Job:  map 100% reduce 0%

15/03/26 14:49:17 INFO mapreduce.Job: Job job_1427347776374_0002 failed with state FAILED due to: Task failed task_1427347776374_0002_m_000000

Job failed as tasks failed. failedMaps:1 failedReduces:0

15/03/26 14:49:17 INFO mapreduce.Job: Counters: 13

Job Counters 

Failed map tasks=7

Killed map tasks=1

Launched map tasks=8

Other local map tasks=6

Data-local map tasks=2

Total time spent by all maps in occupied slots (ms)=45607

Total time spent by all reduces in occupied slots (ms)=0

Total time spent by all map tasks (ms)=45607

Total vcore-seconds taken by all map tasks=45607

Total megabyte-seconds taken by all map tasks=46701568

Map-Reduce Framework

CPU time spent (ms)=0

Physical memory (bytes) snapshot=0

Virtual memory (bytes) snapshot=0

15/03/26 14:49:17 ERROR streaming.StreamJob: Job not successful!

Streaming Command Failed!

Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce,  : 

  hadoop streaming failed with error code 1

In addition: Warning message:

running command 'C:\HADoOP\bin\hadoop jar C:\hadoop\share\hadoop\tools\lib\hadoop-streaming-2.6.0.jar   -D     "stream.map.input=typedbytes"     -D     "stream.map.output=typedbytes"     -D     "stream.reduce.input=typedbytes"     -D     "stream.reduce.output=typedbytes"     -D     "mapred.reduce.tasks=0"     -D     "mapreduce.map.java.opts=-Xmx400M"     -D     "mapreduce.reduce.java.opts=-Xmx400M"     -files     "/Users/linetor/AppData/Local/Temp/RtmpK25Bcn/rmr-local-env20043714610b,/Users/linetor/AppData/Local/Temp/RtmpK25Bcn/rmr-global-env20047e562dda,/Users/linetor/AppData/Local/Temp/RtmpK25Bcn/rmr-streaming-map20042bbc502c"     -input     "/tmp/file200426375de7"     -output     "/tmp/file2004eb0680c"     -mapper     "Rscript --vanilla ./rmr-streaming-map20042bbc502c"       -inputformat     "org.apache.hadoop.streaming.AutoInputFormat"     -outputformat     "org.apache.hadoop.mapred.SequenceFileOutputFormat"   2>&1' had status 1 

15/03/26 14:49:22 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.

Deleted /tmp/file200466b23bcc

15/03/26 14:49:27 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.

Deleted /tmp/file20041df94b4c

15/03/26 14:49:33 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.

Deleted /tmp/file20046fc44809

[Beta]

I think the code should look like this:

mapreduce(input=small.ints, map =function(k,v) keyval(v, v^2))

I generally do like this:

mapreduce(input=small.ints, map =function(k,v){

keyval(v, v^2)

} 

)


"keyval()" is a function in mapreduce to tell which one is key and which one is value variable.

[Antonio Piccolboni]

The console output doesn't help in this and most cases. If you can't find the stderr log of a failing task as you said, please refer to your Hadoop documentation to fix that first, because it's a problem in and of itself and because we can't make any progress without that information. There may be a setting to disable the collection of stderr, but I doubt anyone would be crazy enough to change that. If stderr is empty, there are other logs related to failing tasks, not the console output. Let's take a look at them.

--
post: rha...@googlegroups.com ||
unsubscribe: rhadoop+u...@googlegroups.com ||
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[linetor]

Thanks for your concern.

And I found What Problem is.

The problem is tmp file and folder. And I run that script R make temp File in my Windows temp Folder

ex) rmr-global-env2e2c40617c7a , rmr-local-env2e2c7d60395b ... my case is C:\Users\linetor\AppData\Local\Temp\RtmpuIUkgW

and when I see that file It use "./rmr-global-env2e2c40617c7a" and My R current directory is C:\Users\linetor\Documents

So I change MY TMP C:\Users\linetor\Documents and tempfile

using this

Sys.setenv(TMP="c:\\users\\linetor\\Documents") 

tempfile(tmpdir=Sys.getenv("TMP")) 

And I found that jar file is streamjob4108145292356635898.jar in tmp file but rmr-global-env2e2c40617c7a , rmr-local-env2e2c7d60395b still in  C:\Users\linetor\AppData\Local\Temp\RtmpuIUkgW 

Can I change that global and local value in rmr??

2015년 3월 27일 금요일 오전 1시 55분 43초 UTC+9, Antonio Piccolboni 님의 말:

[linetor]

Warning: running command 'C:\HADoOP\bin\hadoop jar C:\hadoop\share\hadoop\tools\lib\hadoop-streaming-2.6.0.jar   -D     "stream.map.input=typedbytes"     -D     "stream.map.output=typedbytes"     -D     "stream.reduce.input=typedbytes"     -D     "stream.reduce.output=typedbytes"     -D     "mapred.reduce.tasks=0"     -D     "mapreduce.map.java.opts=-Xmx400M"     -D     "mapreduce.reduce.java.opts=-Xmx400M"     -files     "/users/linetor/Documents/RtmpQLNuZt/rmr-local-env72c3dba588a,/users/linetor/Documents/RtmpQLNuZt/rmr-global-env72c37e53b3e,/users/linetor/Documents/RtmpQLNuZt/rmr-streaming-map72c6c4b2d07"     -input     "/tmp/file72c5278318"     -output     "/tmp/file72c3f257c8c"     -mapper     "Rscript --vanilla ./rmr-streaming-map72c6c4b2d07"       -inputformat     "org.apache.hadoop.streaming.AutoInputFormat"     -outputformat     "org.apache.hadoop.mapred.SequenceFileOutputFormat"   2>&1' had status 1

Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce,  : 

  hadoop streaming failed with error code 1

2: file(filename, "r", encoding = encoding)

1: source("./rmr-streaming-map72c6e091d34")

Error during wrapup: 

Warning: closing unused connection 4 (C:\Windows\system32\cmd.exe /c "C:/Program Files/R/R-3.1.3/library/rmr2/bin/x64/catwin.exe")

Warning: closing unused connection 3 (C:\Windows\system32\cmd.exe /c "C:/Program Files/R/R-3.1.3/library/rmr2/bin/x64/catwin.exe")

Is This log about you said??

And using web UI I saw the stderr log application log

there are just this

log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

empty

2015년 3월 27일 금요일 오전 1시 55분 43초 UTC+9, Antonio Piccolboni 님의 말:

The console output doesn't help in this and most cases. If you can't find the stderr log of a failing task as you said, please refer to your Hadoop documentation to fix that first, because it's a problem in and of itself and because we can't make any progress without that information. There may be a setting to disable the collection of stderr, but I doubt anyone would be crazy enough to change that. If stderr is empty, there are other logs related to failing tasks, not the console output. Let's take a look at them.

[Antonio Piccolboni]

Yes, see help(rmr.options) argument hdfs.tempdir. For local, help(tempdir).

[Antonio Piccolboni]

As far as the log, that looks like a log4j log, which means it's a java program's log, not an R program. If the problem is really the tempdir (you share your conclusions, not the evidence or reasoning that led there), it is possible that R doesn't even get started, so no R logs will be available, but then the console error should be more telling.

[linetor]

Thanks for your reply.

In weekend, I solved that log problem. It;s config problem. Thanks. So I saw the log. It's about rJava, rmr2 not available.

But In Rstudio and in cmd  R, I can use rjava and rmr2. But in rhadoop , they can't use rJava, and rmr2.

And I wonder why Rhadoop use 'C:/Program Files/R/R-3.1.3/library/rJava/libs/i386/rJava.dll' i't's not x64 I use 64bit java and I use R 64bit. Why RHadoop use i386??

 .Random.seed

  small.ints

???: namespace 'rmr2' is not available and has been replaced

by .GlobalEnv when processing object 'small.ints'

Loading objects:

  backend.parameters

  combine

  combine.file

  combine.line

  debug

  default.input.format

???: namespace 'rmr2' is not available and has been replaced

by .GlobalEnv when processing object 'default.input.format'

  default.output.format

  in.folder

  in.memory.combine

  input.format

  libs

  map

  map.file

  map.line

  out.folder

  output.format

  pkg.opts

  postamble

  preamble

  profile.nodes

  reduce

  reduce.file

  reduce.line

  rmr.global.env

  rmr.local.env

  save.env

  tempfile

  vectorized.reduce

  verbose

  work.dir

Loading required package: methods

Loading required package: rJava

Error : .onLoad failed in loadNamespace() for 'rJava', details:

  call: inDL(x, as.logical(local), as.logical(now), ...)

  error: ?????? ??? 'C:/Program Files/R/R-3.1.3/library/rJava/libs/i386/rJava.dll'?? ?ε? ?? ?? ??????:

  LoadLibrary failure:  ?????? ????? ??? ?? ??????.

2015년 3월 27일 금요일 오후 12시 20분 47초 UTC+9, Antonio Piccolboni 님의 말:

[linetor]

And If I install R 64bit. there is No R log(stderr). But I install 32bit R there is log there.

So I think why rmr(Rhadoop) use 32bit R and there is issue about 64bit R in windows7

Is it right??

2015년 3월 30일 월요일 오후 2시 46분 16초 UTC+9, linetor 님의 말:

[Antonio Piccolboni]

For rjava, first rmr2 doesn't need it. So for the sake of simplifying things, just don't load it for now. As far as win 32/64 bit I will have to defer to people who know windows better. It seems like it should make no difference.

[Antonio Piccolboni]

On Thursday, March 26, 2015 at 7:55:04 PM UTC-7, linetor wrote:

Warning: running command 'C:\HADoOP\bin\hadoop jar C:\hadoop\share\hadoop\tools\lib\hadoop-streaming-2.6.0.jar   -D     "stream.map.input=typedbytes"     -D     "stream.map.output=typedbytes"     -D     "stream.reduce.input=typedbytes"     -D     "stream.reduce.output=typedbytes"     -D     "mapred.reduce.tasks=0"     -D     "mapreduce.map.java.opts=-Xmx400M"     -D     "mapreduce.reduce.java.opts=-Xmx400M"     -files     "/users/linetor/Documents/RtmpQLNuZt/rmr-local-env72c3dba588a,/users/linetor/Documents/RtmpQLNuZt/rmr-global-env72c37e53b3e,/users/linetor/Documents/RtmpQLNuZt/rmr-streaming-map72c6c4b2d07"     -input     "/tmp/file72c5278318"     -output     "/tmp/file72c3f257c8c"     -mapper     "Rscript --vanilla ./rmr-streaming-map72c6c4b2d07"       -inputformat     "org.apache.hadoop.streaming.AutoInputFormat"     -outputformat     "org.apache.hadoop.mapred.SequenceFileOutputFormat"   2>&1' had status 1

Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce,  : 

  hadoop streaming failed with error code 1

2: file(filename, "r", encoding = encoding)

1: source("./rmr-streaming-map72c6e091d34")

Error during wrapup: 

Warning: closing unused connection 4 (C:\Windows\system32\cmd.exe /c "C:/Program Files/R/R-3.1.3/library/rmr2/bin/x64/catwin.exe")

Warning: closing unused connection 3 (C:\Windows\system32\cmd.exe /c "C:/Program Files/R/R-3.1.3/library/rmr2/bin/x64/catwin.exe")

Is This log about you said??

It looks like a mix of several things. I am not sure.