General Question to users

[Saptarshi Guha]

Hello,
For those who have successfully installed RHIPE and used it, do you

have any feedback ?

All manner of comments are allowed.

Cheers
Saptarshi

[Marek Bejda]

I started putting together a small installation script for RHIPE, because it took me a good while to get everything working. Following the instructions on the website I kept getting incorrect protobuf version errors. But eventually by some miracle I found the 0.75 version of RHIPE 

Rhipe_0.75.0_cdh5mr2.tar.gz that worked with the protobuf version 2.5.0 and it actually installed really smoothly with R v.3.1.2

https://github.com/marek5050/Sparkie/blob/master/RHIPE/install.sh
Unfortuntely,  the script isn't fully functioning yet.

wget http://ml.stat.purdue.edu/rhipebin/Rhipe_0.75.0_cdh5mr2.tar.gz
R CMD INSTALL Rhipe_0.75.0_cdh5mr2.tar.gz

[dotcz12@login TESTING]$ lsb_release -a
LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.5 (Final)
Release: 6.5
Codename: Final

[Marek Bejda]

[dotcz12@login TACCPROJECT]$ hadoop version
> Hadoop 2.3.0-cdh5.1.0
> Subversion git://github.sf.cloudera.com/CDH/cdh.git -r 8e266e052e423af592871e2dfe09d54c03f6a0e8
> Compiled by jenkins on 2014-07-12T13:49Z
> Compiled with protoc 2.5.0
> From source with checksum 7ec68264497939dee7ab5b91250cbd9
> This command was run using /usr/lib/hadoop/hadoop-common-2.3.0-cdh5.1.0.jar

[Saptarshi Guha]

Hello,

Thanks much for working on this.

Ryan, now that github's main is 0.75,, let's try making the 0.75 binary download less miraculous and more obvious. Or is the recommended way to clone github and build from source?I think that is the preferred method and works for CDH and Apache Hadoop

Cheers

S

[Ryan Hafen]

Whenever I build an update of RHIPE, I put it out on ml.stat.purdue.edu/rhipebin.  I’ve been thinking for a while about how to automate this or make it easier.  If I get our RHIPE CI server running, we can have it push artifacts out somewhere.  Tagged releases on github might be a nice way to go.  But wget from ml.stat.purdue.edu works well for now.  

I have not tested this, but I have been told several times by an engineer at my previous employer that if you build it against Apache Hadoop, it should work with all distributions, because all distributions are supposed to be compatible with Apache.

You can also find documented many up-to-date examples where we install RHIPE on various systems.

For example, our Vagrant VM (Ubuntu - CDH5mr2):

https://github.com/tesseradata/install-vagrant/blob/master/cdh5mr2-rhipe0.75/tessera.sh

Or our EMR scripts (Amazon Linux (CentOS-ish) - Hadoop 2):

https://github.com/tesseradata/install-emr/blob/master/emr-3.2.1/scripts/install-tessera.sh

If anyone has ideas for how the documentation of these issues could be improved, please submit a PR.

Thanks!

Ryan