How to load a file from local file system to Tachyon?

[Jia Zou]

Dears,

I am using Tachyon 0.8.2. I've tried following but failed. Your help will be very appreciated! Thanks.

ubuntu@ip-10-149-8-254:~/tachyon-0.8.2/bin$ ./tachyon loadufs tachyon://127.0.0.1:19998/HiBench/Kmeans /home/ubuntu/Kmeans

java.io.IOException: tachyon.exception.TachyonException: /tmp/HiBench/Kmeans/Input/cluster/part-00000

at tachyon.client.TachyonFS.createFile(TachyonFS.java:322)

at tachyon.client.AbstractTachyonFS.createFile(AbstractTachyonFS.java:83)

at tachyon.client.TachyonFS.createFile(TachyonFS.java:66)

at tachyon.client.UfsUtils.loadUfs(UfsUtils.java:165)

at tachyon.client.UfsUtils.loadUfs(UfsUtils.java:79)

at tachyon.client.UfsUtils.main(UfsUtils.java:222)

Caused by: tachyon.exception.TachyonException: /tmp/HiBench/Kmeans/Input/cluster/part-00000

at tachyon.client.FileSystemMasterClient.loadMetadata(FileSystemMasterClient.java:444)

at tachyon.client.TachyonFS.createFile(TachyonFS.java:319)

... 5 more

Usage: java -cp target/tachyon-0.8.2-jar-with-dependencies.jar tachyon.client.UfsUtils <TachyonPath> <UfsPath> [<Optional ExcludePathPrefix, separated by ;>]

Example: java -cp target/tachyon-0.8.2-jar-with-dependencies.jar tachyon.client.UfsUtils tachyon://127.0.0.1:19998/a hdfs://localhost:9000/b c

Example: java -cp target/tachyon-0.8.2-jar-with-dependencies.jar tachyon.client.UfsUtils tachyon://127.0.0.1:19998/a file:///b c

Example: java -cp target/tachyon-0.8.2-jar-with-dependencies.jar tachyon.client.UfsUtils tachyon://127.0.0.1:19998/a /b c

In the TFS, all files under local FS /b will be registered under /a, except for those with prefix c

Best Regards,

Jia

[Jiří Šimša]

Hi Jia,

try this: $ ./tachyon/bin tfs copyFromLocal /home/ubuntu/Kmeans tachyon://127.0.0.1:19998/HiBench/Kmeans 

Best,

--
You received this message because you are subscribed to the Google Groups "Tachyon Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tachyon-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Jiří Šimša

Tachyon Nexus, Inc.

[Jia Zou]

It works, thanks, Jiří!

Best Regards,

Jia

[Bin Fan]

glad that works for you.

meanwhile, loadufs is a different command from copyFromLocal.

loadufs assumes there is an existing dir structure in under file system and brings the directory structure in Tachyon (without really loading the data),

while, copyFromLocal does the copy from your local file system to Tachyon space.

you can find more explanation here: http://tachyon-project.org/documentation/Command-Line-Interface.html

- Bin

[Jia Zou]

Bin, thanks for the detailed explanation!

Best Regards,

Jia