Rhipe 0.73.1(stable) example

[Luke Hallett]

Hi,

I am trying to validate my Rhipe setup, and am having a hard time finding constant documentation that gives a good example that I can use to verify my Rhipe/Hadoop configuration.

I have installed the following under CentOS 6.2:

   Hadoop v1.2.1

   R v3.1.0

   Protocol Buffer v2.4.1  (found that v2.5.0 was incompatible)

   Rhipe v0.73.1 (stable)

I originally tried to follow through a sample posted here (http://ml.stat.purdue.edu/rhafen/rhipe/), but found that the rhmr() function had been replaced by rhwatch() somewhere around v0.72.  I tried to convert the rhmr command unsuccessfully, which led me to trying to simplify the example.  I found the following example on this discussion board:

map <- function(a,b)

  for(i in 1:1000)

   rhcollect(runif(1),1)

m <- rhwatch(map=map, reduce=0, input=c(1000,3),read=FALSE)

But I continue to get the following error every time I try the rhwatch function:

> map <- function(a,b)

+   for(i in 1:1000)

+    rhcollect(runif(1),1)

> m <- rhwatch(map=map, reduce=0, input=c(1000,3),read=FALSE)

Loading required package: codetools

Saving 2 paramaters to /tmp/rhipe-temp-params-342a041a6eec973f79df11f851cc54d1 (use rhclean to delete all temp files)

java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setUserClassesTakesPrecedence(Z)V

at org.godhuli.rhipe.RHMR.setJob(RHMR.java:222)

at org.godhuli.rhipe.RHMR.init(RHMR.java:135)

at org.godhuli.rhipe.RHMR.run(RHMR.java:149)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)

at org.godhuli.rhipe.RHMR.fmain(RHMR.java:101)

at org.godhuli.rhipe.PersonalServer.rhex(PersonalServer.java:166)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at RJavaTools.invokeMethod(RJavaTools.java:386)

Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl,  :

  java.lang.NoSuchMethodError: org.apache.hadoop.mapreduce.Job.setUserClassesTakesPrecedence(Z)V

Can someone either give me a simple example or suggest change(s) I should make to my environment to get this to work?

Thanks!

Luke.

[Saptarshi Guha]

Hello,
Apologies for the bad documentation.

1. I think you're using Apache hadoop.

2. RHIPE , the one you have, works with CDH 4 (MR v1)

The function setUserClassesTakesPrecedence

is not found in Apache HAdoop.

If you're adventurous, check out the rearch branch and follow the build instructions there to build a version that (ought to )work with Apache Hadoop.


Cheers
Saptarshi

--

---
You received this message because you are subscribed to the Google Groups "rhipe" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhipe+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.