DataStax Enterprise + Virgil

[Jeremy Brenner]

Just wanted to add a couple notes on using DataStax enterprise + virgil-hadoop in case anybody else wanted to give it a shot.

I set virgil up on its own machine and pointed it to the first node in my cluster, there are a couple roadblocks but nothing major.

First you need to copy /usr/share/dse/dse.jar from a dse deploy to mapreduce/jars/dse.jar in the virgil install.

Second, after you have copied your xml files into mapreduce/conf notice that in mapreduce/conf/mapred-site.xml dse uses a variable to set the job tracker that will not be available to virgil:

<!-- Auto detect the dse job tracker -->

<property>

  <name>mapred.job.tracker</name>

  <value>${dse.job.tracker}</value>

  <description>

    The address of the job tracker

  </description>

</property>

I used the host and port of the job tracker on my first node:

  <value>cassandra01:8012</value>

That is all it took to get up and running,  hope it helps someone.

[Brian O'Neill]

Thanks Jeremy.  I'll add this to the wiki as well.

-brian

---- 

Brian O'Neill
Lead Architect, Software Development
Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406
p: 215.588.6024

blog: http://weblogs.java.net/blog/boneill42/

blog: http://brianoneill.blogspot.com/