Tachyon client (version 0.7.1) is failing to connect with master @ localhost/127.0.0.1:19998

[Ricardo Bravo]

Hi,

I'm testing Tachyon. version 0.71 in cloudera sandbox (linux centos 6.4), after finishing all the installations steps, I want to init tachyon (I did it before rebooting the system) and this is the error trace:

[root@quickstart tachyon]# bin/tachyon format

Connecting to localhost as cloudera...

root@localhost's password: 

Formatting Tachyon Worker @ quickstart.cloudera

Connection to localhost closed.

Formatting Tachyon Master @ localhost

[root@quickstart tachyon]# bin/tachyon-start.sh local

Killed 0 processes on quickstart.cloudera

Killed 0 processes on quickstart.cloudera

Connecting to localhost as cloudera...

root@localhost's password: 

Killed 0 processes on quickstart.cloudera

Connection to localhost closed.

Formatting RamFS: /mnt/ramdisk (1gb)

Starting master @ localhost

Starting worker @ quickstart.cloudera

[root@quickstart tachyon]# bin/tachyon runTest Basic CACHE_THROUGH

Failed to connect with master @ localhost/127.0.0.1:19998 after 29 attempts

2015-10-05 10:12:58,626 INFO   (MasterClient.java:connect) - Tachyon client (version 0.7.1) is trying to connect with master @ localhost/127.0.0.1:19998

2015-10-05 10:12:58,648 ERROR  (MasterClient.java:connect) - Failed to connect (0) with master @ localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused

2015-10-05 10:12:58,699 INFO   (MasterClient.java:connect) - Tachyon client (version 0.7.1) is trying to connect with master @ localhost/127.0.0.1:19998

2015-10-05 10:12:58,700 ERROR  (MasterClient.java:connect) - Failed to connect (1) with master @ localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused

2015-10-05 10:12:58,800 INFO   (MasterClient.java:connect) - Tachyon client (version 0.7.1) is trying to connect with master @ localhost/127.0.0.1:19998

2015-10-05 10:12:58,802 ERROR  (MasterClient.java:connect) - Failed to connect (2) with master @ localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused

2015-10-05 10:12:59,103 INFO   (MasterClient.java:connect) - Tachyon client (version 0.7.1) is trying to connect with master @ localhost/127.0.0.1:19998

2015-10-05 10:12:59,104 ERROR  (MasterClient.java:connect) - Failed to connect (3) with master @ localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused

2015-10-05 10:12:59,404 INFO   (MasterClient.java:connect) - Tachyon client (version 0.7.1) is trying to connect with master @ localhost/127.0.0.1:19998

2015-10-05 10:12:59,406 ERROR  (MasterClient.java:connect) - Failed to connect (4) with master @ localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused

Anyone can help me?

Regards 

[Bin Fan]

Hi Ricardo,

Could you check if http://your_host:19999 is on or not?

this it the webUI to show Tachyon master status. 

if the UI is not showing, it is possible that your Tachyon Master is dead (likely killed for some reason).

Also, you may want to go to $TACHYON_HOME/logs and check out the master and workers logs stored there.

- Bin

--
You received this message because you are subscribed to the Google Groups "Tachyon Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tachyon-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gene Pang]

Hi,

Are there any useful information in the master log file?

Thanks,

Gene

[Bin Fan]

(putting tachyo...@googlegroups.com back to cc'ed list in case the discussion is useful for other ppl)

Checking out ${TACHYON_HOME}/logs/master.log and ${TACHYON_HOME}/logs/master.out as one is for stdout and one is for stderr.

To start master standalone using:

 ${TACHYON_HOME}/bin/tachyon-start.sh master 

Then checkout http://localhost:19999 again

Best,

-Bin

On Mon, Oct 5, 2015 at 11:32 AM, Ricardo Bravo <rbra...@gmail.com> wrote:

Hi,

http://localhost:19999 is  off

There's no information useful in master.log file

Any idea for restarting tachyon master?

Regards

[Ricardo Bravo]

Hi,

thanks for your help. When I start as master:

[root@quickstart tachyon]# bin/tachyon format

Connecting to localhost as cloudera...

root@localhost's password: 

Formatting Tachyon Worker @ quickstart.cloudera

Connection to localhost closed.

Formatting Tachyon Master @ localhost

[root@quickstart tachyon]# bin/tachyon-start.sh master

Starting master @ localhost

[root@quickstart tachyon]#

the master.log has the next information

2015-10-05 11:49:38,045 ERROR MASTER_LOGGER (TachyonMaster.java:<init>) - Tachyon was not formatted! The journal folder is /var/tachyon/journal/

java.lang.IllegalStateException: Tachyon was not formatted! The journal folder is /var/tachyon/journal/

at com.google.common.base.Preconditions.checkState(Preconditions.java:149)

at tachyon.master.TachyonMaster.<init>(TachyonMaster.java:151)

at tachyon.master.TachyonMaster.main(TachyonMaster.java:63)

2015-10-05 11:49:38,048 ERROR MASTER_LOGGER (TachyonMaster.java:main) - Uncaught exception terminating Master

java.lang.IllegalStateException: Tachyon was not formatted! The journal folder is /var/tachyon/journal/

at com.google.common.base.Preconditions.checkState(Preconditions.java:149)

at tachyon.master.TachyonMaster.<init>(TachyonMaster.java:151)

at tachyon.master.TachyonMaster.main(TachyonMaster.java:63)

[Bin Fan]

it seems that for some reason, the journal folder for Tachyon master is not really formatted though you have already run "bin/tachyon format" as instructed.

Could you check you have writing permission in  /var/tachyon/journal/ ?

- Bin

[Ricardo Bravo]

I have writing permission in   /var/tachyon/journal/ :

drwxrwxrwx 2 root root 4096 Oct  5 09:49 journal

 

I've modified with chmod, but the same error message appears when I try to start tachyon.

Regards

[Bin Fan]

could you try removing /var/tachyon/journal before running the format script?

the script is doing very simple things: just create a journal subdir (by default under $TACHYON_HOME) and

a bunch of sub-subdirs under journal.

this is what looks like under my journal dir.

$ la -r journal

total 0

-rwxrwxrwx@ 1 binfan  staff     0B Sep 15 13:49 _format_1442350194122*

drwxrwxrwt@ 3 binfan  staff   102B Sep 15 13:50 RawTableMaster/

drwxrwxrwt@ 3 binfan  staff   102B Sep 15 13:50 FileSystemMaster/

drwxrwxrwt@ 4 binfan  staff   136B Sep 15 13:50 BlockMaster/

[Ricardo Bravo]

Hi,

I've solve the problem. 

I had to delete hdfs temporary files created with superuser status (I launched tachyon scripts as superuser) 

2015-10-05 14:11:03,712 INFO  USER_LOGGER (Format.java:formatFolder) - Formatting UNDERFS_DATA_FOLDER:hdfs://localhost:8020/tmp/tachyon/data

2015-10-05 14:11:03,801 INFO  USER_LOGGER (Format.java:formatFolder) - Failed to remove UNDERFS_DATA_FOLDER:chyon/data/44

...

2015-10-05 14:51:21,808 INFO  USER_LOGGER (Format.java:formatFolder) - Formatting UNDERFS_DATA_FOLDER:hdfs://localhost:8020/tmp/tachyon/data

2015-10-05 14:51:22,008 INFO  USER_LOGGER (Format.java:formatFolder) - Formatting UNDERFS_WORKERS_FOLDER:hdfs://localhost:8020/tmp/tachyon/workers

Thank you very much for your help.

Regards

[Bin Fan]

Great! Thanks for posting the solution.

- Bin

[薛晨浩]

Did you recompile the Tachyon? I recompile tachyon0.7.1 with hadoop2.6 and java 1.8. However, when I format tachyon, I encounter a problem "Could not find or load main class tachyon.Format". Could you give me some advice? Thanks a lot!

在 2015年10月6日星期二 UTC+8上午1:14:34，Ricardo Bravo写道：

[Takechiyo]

It seems that it didn't really recompile successfully. And I reloaded the source codes and recompiled it. It works!

在 2015年10月17日星期六 UTC+8下午12:35:57，Takechiyo写道：

[Haoyuan Li]

Great! Thanks for the updates!

PS, we have released Tachyon 0.8.2, which is significant improvement and more features than Tachyon 0.7.1. It will be great to give it a try!

Cheers,

Haoyuan 

--

You received this message because you are subscribed to the Google Groups "Tachyon Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tachyon-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Haoyuan Li

CEO, Tachyon Nexus