QFS to/from HDFS 2.7.2

[cand...@domaintools.com]

Noticed the outstanding JIRA bug for testing the 2.7.x branch of Hadoop with QFS so thought I'd post this here. I've been testing getting the latest HDFS and QFS to work together and it took days to figure out why the containers couldn't find qfs_access and was hoping this could help someone else or possibly find someone with a cleaner solution. I had all of the libs in the right place ($HADOOP_HOME/lib/native/) on all nodes, plugins (${HADOOP_HOME}/share/common/hadoop/lib/) and the Hadoop binary working for cp, but wasn't able to do anything involving map/reduce (ex. distcp) due to the containers never being able to find qfs_access due to what I assume is this bug involving container CLASSPATHs being overwritten. In the end these options to the command worked as it copied the libraries and JARs to each host under the CLASSPATH that was overwritten (/tmp/hadoop/nm-local-dir/very/long/string/and/more/directories/to/traverse):

${HADOOP_HOME}/bin/hadoop distcp -libjars=${QFS_HOME}/lib/qfs-access-master.jar,${QFS_HOME}/lib/hadoop-2.5.1-qfs-master.jar -files=${QFS_HOME}/lib/libqfs_access.so,${QFS_HOME}/lib/libqfs_client.so,${QFS_HOME}/lib/libqfs_io.so,${QFS_HOME}/lib/libqfs_common.so,${QFS_HOME}/lib/libqfs_qcdio.so,${QFS_HOME}/lib/libqfskrb.so,${QFS_HOME}/lib/libqfs_qcrs.so,${QFS_HOME}/lib/libJerasure.so.2,${QFS_HOME}/lib/libgf_complete.so.1 <src> <dest>

Hopefully this saves someone some troubleshooting time trying to get the MR containers working -- and maybe if I'm lucky one of you know of a cleaner way so I don't have to push a bunch of aliases around for my developers. I have not yet had a chance to upgrade our development cluster to 2.7.3 to see if that bug being resolved fixed the issue. Will report back if it does. As always, thanks for the amazing product and all the work you guys do.

[Kevin Stinson]

Thanks for the post.

We are working on collecting together information on QFS and Hadoop compatibility that should appear on our wiki page soon.

I will include this information and will check to see if 2.7.3 is any different.