I am beginning to use RHadoop and have successfully run some simple mapreduce programs going line-by-line in R. I would now like to execute the same program as an R Markdown script that outputs to HTML or PDF.
However, I am getting the OutofMemoryError: Java heap space when I try to knit (run) any Rmd file. It does not matter how large the object I am passing to the DFS.
Config-wise, I have a VM (8gb) with Ubuntu 14.04. I have already set my yarn-site.xml config to 4g RAM and set the RHadoop rmr.options("backend.parameters") to 4g for each of map and reduce parameters.
Again, I can run everything as individual commands but cannot knit the Rmd file. This Java heap space error also occurs regardless if I use RStudio or the command line.
Is there some different config I need to setup for knitr and/or rmarkdown in conjunction with RHadoop/rmr? Thanks in advance.