Provider tachyon.hadoop.TFS could not be instantiated

Madhumita Bharde

unread,

Nov 17, 2016, 11:38:59 PM11/17/16

to Alluxio Users

Hi all,

My project involves changes to tachyon-0.8.2 and spark 1.6.1 and I am trying to mimic an already existing distributed setup which has local directory as tachyon underfs (I could already get single node setup working).

I am completely new to all this and I am guessing it might have got to do with tachyon-env.sh but my tachyon-env.sh looks exactly similar (with obvious changes) to the working setup.

So I have no clue why I get this error on worker node(s) during spark-submit --

 java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider tachyon.hadoop.TFS could not be instantiated

Could somebody help? Let me know if I need to provide more info..

Madhumita Bharde

unread,

Nov 18, 2016, 1:51:49 AM11/18/16

to Alluxio Users

Also, when I use tachyon tfs copyFromLocal to ingest files, and check using tachyon tfs fileinfo, all blocks are laid out on node when I run the command from. what could be the issue? Thanks!

Haoyuan Li

unread,

Nov 18, 2016, 9:06:21 AM11/18/16

to Madhumita Bharde, Alluxio Users

Hi Madhumita,

Any chance to upgrade to latest Alluxio version (1.3.0)? Here is 1.3.0 doc with Spark: http://www.alluxio.org/docs/1.3/en/Running-Spark-on-Alluxio.html

Cheers,

Haoyuan

Alluxio, Inc.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gene Pang

unread,

Nov 18, 2016, 9:12:08 AM11/18/16

to Alluxio Users

Hi,

Would you be able to use a newer version of Alluxio (tachyon)? Alluxio 1.3.0 was recently released.

Also, for copyFromLocal it will copy it to the local worker since there is a local worker. In newer releases of Alluxio, there is the option to use different location policies when writing data into Alluxio.

Thanks,

Gene

Madhumita Bharde

unread,

Nov 22, 2016, 4:59:27 AM11/22/16

to Alluxio Users

Thank you so much Li and Gene, for your prompt replies. I found/fixed some issues with my tachyon-env.sh and /etc/hosts files and am thankfully past this!

Porting our changes to latest alluxio codebase soon. Thanks!

Gene Pang

unread,

Nov 22, 2016, 8:58:38 AM11/22/16

to Alluxio Users

Glad we could help!

Thanks,

Gene

Reply all

Reply to author

Forward