Cluster scheduler exiting abruptly

[Mayuresh Kunjir]

Hello Spark developers,

I am running a standalone Spark(0.7.2) cluster. Occasionally, I get disconnected from the cluster soon after initializing Spark context. 

13/08/05 13:26:00 ERROR client.Client$ClientActor: Connection to master failed; stopping client

13/08/05 13:26:00 ERROR cluster.SparkDeploySchedulerBackend: Disconnected from Spark cluster!

13/08/05 13:26:00 ERROR cluster.ClusterScheduler: Exiting due to error from cluster scheduler: Disconnected from Spark cluster

The message is coming from RemoteClientShutdown case of ClientActor. This behavior is pretty random. Sometimes it just needs a restart of master to fix it. 

Does anyone know what may cause this?

Apologies if this has been asked before. 

Regards,

~Mayuresh

p.s. I didn't get any solution for this question on users list, hence posting it here. 

[Patrick Wendell]

Hey There,

This is really odd... it's possible that this is just caused by a connectivity problem in your cluster, for some reason the connection gets dropped, firewalled, or otherwise messed up.

It's also possible it's something on the Spark side. One thing to look at is if the master logs (in /logs) say anything indicating it has explicitly dropped the connection.

- Patrick 

--
You received this message because you are subscribed to the Google Groups "Spark Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spark-develope...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.