worker info doesn't show on web UI

[SuDo]

Hi, 

I have installed spark standalone to a set of clusters. And I tried to lunch clusters through cluster lunch script. I have added clusters address into conf/slaves file. The master connects to all slaves through password less ssh. 

After running ./bin/start-slaves.sh script, It gives message  

"starting org.apache.spark.deploy.worker.Worker, logging to /root/spark-0.8.0-incubating/bin/../logs/spark-root-org.apache.spark.deploy.worker.Worker-1-jbosstest2.out" 

But web UI (localhost:8080) is not showing about worker. The worker is blank. But when I add localhost onto my conf/slaves file the worker info of localhost is shown. 

There are no error messages, the message says the worker is started on terminal, but the Web UI is not showing any workers.  

[Robin Chen]

Hi,

I think if you attach the worker to the master successfully, you may see the log something like "worker has successfully register with master". If you didn't see this, I think maybe you didn't register worker correctly and successfully, furthermore you cannot see it on the ui page.

Thanks.

Chen Jingci

[SuDo]

I've done everything as mentioned in doc. I tried starting cluster manually, which again gives the following error: 

[root@jbosstest1 spark-0.8.0-incubating]# ./spark-class org.apache.spark.deploy.worker.Worker spark://192.168.50.42:7077

13/11/08 14:04:32 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started

13/11/08 14:04:51 INFO worker.Worker: Starting Spark worker jbosstest1:40353 with 2 cores, 512.0 MB RAM

13/11/08 14:04:51 INFO worker.Worker: Spark home: /root/spark-0.8.0-incubating

13/11/08 14:04:56 INFO server.Server: jetty-7.x.y-SNAPSHOT

13/11/08 14:04:56 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/json,null}

13/11/08 14:04:56 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/static,null}

13/11/08 14:04:56 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/log,null}

13/11/08 14:04:56 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/logPage,null}

13/11/08 14:04:56 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/json,null}

13/11/08 14:04:56 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{*,null}

13/11/08 14:04:57 INFO server.AbstractConnector: Started SelectChann...@0.0.0.0:8081

13/11/08 14:04:57 INFO ui.WorkerWebUI: Started Worker web UI at http://jbosstest1:8081

13/11/08 14:04:57 INFO worker.Worker: Connecting to master spark://192.168.50.42:7077

13/11/08 14:05:04 ERROR worker.Worker: Connection to master failed! Shutting down.

I have set SPARK_MASTER_IP=192.168.50.41 and SPARK_LOCAL_IP on each machine. Yet gives the same error. 

Log from worker node is : 

[root@jbosstest2 logs]# cat spark-root-org.apache.spark.deploy.worker.Worker-1-jbosstest2.out

Spark Command: java -cp :/root/spark-0.8.0-incubating/conf:/root/spark-0.8.0-incubating/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.0.4.jar -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.worker.Worker spark://192.168.50.41:7077

========================================

13/11/08 11:28:26 WARN util.Utils: Your hostname, jbosstest2 resolves to a loopback address: 127.0.0.1; using 192.168.50.42 instead (on interface eth0)

13/11/08 11:28:26 WARN util.Utils: Set SPARK_LOCAL_IP if you need to bind to another address

13/11/08 11:28:28 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started

13/11/08 11:28:28 INFO worker.Worker: Starting Spark worker 192.168.50.42:45953 with 2 cores, 512.0 MB RAM

13/11/08 11:28:28 INFO worker.Worker: Spark home: /root/spark-0.8.0-incubating

13/11/08 11:28:28 INFO server.Server: jetty-7.x.y-SNAPSHOT

13/11/08 11:28:28 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/json,null}

13/11/08 11:28:28 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/static,null}

13/11/08 11:28:28 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/log,null}

13/11/08 11:28:28 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/logPage,null}

13/11/08 11:28:28 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/json,null}

13/11/08 11:28:28 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{*,null}

13/11/08 11:28:28 INFO server.AbstractConnector: Started SelectChann...@0.0.0.0:8081

13/11/08 11:28:28 INFO ui.WorkerWebUI: Started Worker web UI at http://192.168.50.42:8081

13/11/08 11:28:28 INFO worker.Worker: Connecting to master spark://192.168.50.41:7077