rhlapply not found

[sowmya sree]

hello,

i installed Rhipe ..when i try to run the code given in
http://www.stat.purdue.edu/~sguha/rhipe/ (the map reduce code), im
getting :

java.lang.RuntimeException: PIPEMapRed.waitOutputThreads(): subprocess
failed with code 127

and alternatively try

rhwordcount(infolder,outfolder)

it says : object "rhwordcount" not found .. Same is the case with
"rhlapply" and some others too ..

pls tell me what went wrong with my installation .. the version of
protobuffers is 2.3.0 .. Rhipe 0.65.3 and R is 2.11.1

thanks

[Shekhar]

Hi Sowmya,
I have also faced the same issue with some of the functions, you might
be using newer version of Rhipe where some of the functions got
deprecated like rhs2m(function for sequential to map)not tried
rhlapply...well whats the problem you are trying to solve?
U should also check whether you are able to load the Rhipe library or
not???

for checking whether the Rhipe is working properly or not, try a very
basic problem consisting of simple map expression and let there be
identity reducerThen use rhmr function for encapsulating the map and
reduce expressions and Hadoop related parameters and use rhex for
submitting the job.


Well for installing Rhipe, follow the procedure given below:
The version of R i am using is 2.12.0
protobuf.pc:2.3.0
Rhipe: 0.63

Installing protobuf.pc

I.Download protobuf.pc (protobuf-2.3.0.tar.gz) from
http://code.google.com/p/protobuf/downloads/list .

II. Execute the following commands serially after unzipping the
contents
A. sh configure
B. make
C. make install

Create an environment variable PKG_CONFIG_PATH=/usr/local/lib/
pkgconfig .

III. Check for the proper installation of protobuf.pc
Execute the command : pkg-config –-modversion protobuf
The output should be 2.3.0

IV. Check for the proper installation ofprotobuf libraries.
Execute the command : pkg-config –libs protobuf

Installing RHIPE

I. Download RHIPE from http://www.stat.purdue.edu/~sguha/rhipe/ and
place the contents in a directory Rhipe. For example : /home/user/
Rhipe

II. Create the following environment variables in .bashrc

HADOOP = location to hadoop installation
HADOOP_LIB = $HADOOP/lib
HADOOP_CONF_DIR=$HADOOP/conf

III. Execute the following command from the directory /home/user/
Rhipe.

R CMD INSTALL Rhipe

As a check open the R console and load the Rhipe library using the
command library(Rhipe),it shouldn’t throw error.


The installation will fail if the libraries are not properly linked.
The error is as shown below:

Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/lib64/R/library/Rhipe/libs/
Rhipe.so':
libprotobuf.so.6: cannot open shared object file: No such file or
directory
ERROR: loading failed
* removing â/usr/lib64/R/library/Rhipeâ

In order to link the libraries to LD_LIBRARY_PATH, create the file
Protobuf-x86.conf under /etc/ld.so.conf.d/ and add the line /usr/local/
lib to the file.
Sometimes even after creating the file Protobuf-x86.conf, library
libprotobuf.so is not found. This is because of the presence of stale
cache ld.so.cache. For removing the stale cache execute the command
ldconfig.

Let us know if you are still facing some problem

Regards,
Som Shekhar

On Apr 13, 11:20 pm, sowmya sree <sowmyasree.bhagavat...@gmail.com>
wrote:
> hello,
>
> i installed Rhipe ..when i try to run the code given inhttp://www.stat.purdue.edu/~sguha/rhipe/  (the map reduce code), im

[Sowmya Sree Bhagavatula]

map n reduce r working now :)
 i changed the mapred.job.tracker to local instead of localhost: some port in mapred-site.xml in hadoop/conf directory ..
It surprisingly worked ..

And yes May b ur right some of the functions r probably deprecated ..
Ill try using some other version of rhipe ( im still learning RHipe .. So i need those functions for the tutorials online )
If u can point me out to some good tutorial other than documentation and http://ml.stat.purdue.edu/rhafen/rhipe/ .. It'll be very helpful..

Thank u alot for ur reply

and

[Saptarshi Guha]

Hi,

Sorry about the out -of -sync landing page.
rhlapply and rhwordcount have been removed.

rhlapply can be implemented like this

rhlapply(N, F) #apply F to 1..N

map = expression({
lapply(map.values, function(r) {
rhcollect(r, F(r))
})
})
rhmr(map, inout=c("lapply","seq"), N=N)

Should work

[Shekhar]

Hi,
Are you running Hadoop on Pseudo mode? if that is the reason then
changing that parameter, mapred.job.tracker=local might be working for
you....i am not sure..need to check it out

Try Rhipe on cluster atleast on two nodes, then you might encounter
some more interesting problems :-)....Well as such there is no
tutorial for Rhipe.There are some pdfs avaialable for each and every
version, so you can refer that....But i suggest that you choose your
problem first like simple wordcount or grep problem, then if you wish
you can try out the weather data set which Saptarishi has solved using
Rhipe...

Regards,
Som Shekhar

On Apr 14, 10:40 am, Sowmya Sree Bhagavatula

<sowmyasree.bhagavat...@gmail.com> wrote:
>  map n reduce r working now :)
>  i changed the mapred.job.tracker to local instead of localhost: some port
> in mapred-site.xml in hadoop/conf directory ..
> It surprisingly worked ..
>
> And yes May b ur right some of the functions r probably deprecated ..
> Ill try using some other version of rhipe ( im still learning RHipe .. So i
> need those functions for the tutorials online )

> If u can point me out to some good tutorial other than documentation andhttp://ml.stat.purdue.edu/rhafen/rhipe/.. It'll be very helpful..

>
> Thank u alot for ur reply
>
> and
>

> > I.      Download RHIPE fromhttp://www.stat.purdue.edu/~sguha/rhipe/ and

> >www.stat.purdue.edu/~sguha/rhipe/<http://www.stat.purdue.edu/%7Esguha/rhipe/>