GlusterFS 3.4.6 and Tachyon 0.7.0-SNAPSHOT : Cannot format master with Glusterfs as underFileSystem

[Fanny Martin]

Hello, 

I changed last week for the new version of Tachyon but I can't format anymore when using glusterfs as underfilesystem.

I have run :

mvn clean install -Dtest.profile=glusterfs -Dhadoop.version=2.3.0 -Dtachyon.underfs.glusterfs.mounts=/mnt/gluster -Dtachyon.underfs.glusterfs.volumes=testvol -DskipTests

But I get the following exception for formatting the master when launching ./bin/tachyon format :

Connecting to 212.128.53.12 as root...

Connecting to 212.128.53.13 as root...

Pseudo-terminal will not be allocated because stdin is not a terminal.

Pseudo-terminal will not be allocated because stdin is not a terminal.

Formatting Tachyon Worker @ gfsnode-1.localdomain

Formatting Tachyon Worker @ gfsnode-2.localdomain

Formatting Tachyon Master @ 212.128.53.12

Exception in thread "main" java.lang.IllegalArgumentException: All eligible Under File Systems were unable to create an instance for the given path: glusterfs:///212.128.53.12:9000/tmp/tachyon/data

java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.glusterfs.GlusterFileSystem not found

java.lang.IllegalArgumentException: Can not create a Path from a null string

        at tachyon.underfs.UnderFileSystemRegistry.create(UnderFileSystemRegistry.java:132)

        at tachyon.underfs.UnderFileSystem.get(UnderFileSystem.java:99)

        at tachyon.underfs.UnderFileSystem.get(UnderFileSystem.java:83)

        at tachyon.Format.formatFolder(Format.java:38)

        at tachyon.Format.main(Format.java:77)

 

I use the same tachyon-env.sh file as the one as used for the previous version of tachyon 0.7.0-SNAPSHOT I used :

export TACHYON_UNDERFS_ADDRESS=glusterfs:///212.128.53.12

export TACHYON_UNDERFS_GLUSTER_VOLUMES=testvol

export TACHYON_UNDERFS_GLUSTER_MOUNTS=/mnt/gluster

export TACHYON_UNDERFS_GLUSTER_MR_DIR=glusterfs:///mapred/system

Can you help me please to resolve this issue ? Am I missing some new configuration steps ? Why is the org.apache.hadoop.fs.glusterfs.GlusterFileSystem class not found ?

Thanks in advance for your answer, 

Fanny

[Calvin Jia]

Hi Fanny,

Could you try adding this dependency to underfs/glusterfs/pom.xml

<dependency>

  <groupId>org.gluster</groupId>

  <artifactId>glusterfs-hadoop</artifactId>

  <version>2.3.13</version>

</dependency>

Thanks,

Calvin

[Fanny Martin]

Hi Calvin,

Thanks for your answer !

After adding this dependency to underfs/glusterfs/pom.xml

and adding the following properties to TACHYON_HOME/conf/core-site.xml, it works well.

<property>

    <name>fs.glusterfs.impl</name>

    <value>org.apache.hadoop.fs.glusterfs.GlusterFileSystem</value>

  </property>

  <property>

    <name>fs.AbstractFileSystem.glusterfs.impl</name>

    <value>org.apache.hadoop.fs.local.GlusterFs</value>

  </property>

  <property>

    <name>fs.default.name</name>

    <value>glusterfs:///</value>

  </property>

  <property>

    <name>fs.glusterfs.volumes</name>

    <value>testvol</value>

  </property>

  <property>

    <name>fs.glusterfs.mount</name>

    <value>/mnt/gluster</value>

  </property>

  <property>

    <name>fs.glusterfs.volume.fuse.testvol</name>

    <value>/mnt/gluster</value>

  </property>

[Haoyuan Li]

Great! Thanks for the update!

Best,

Haoyuan

--
You received this message because you are subscribed to the Google Groups "Tachyon Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tachyon-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.