How to resolve this error? Run K Means in Cluster

[Paulo Henrique Rates]

Usage: SparkLocalKMeans <master> <file> <k> <convergeDist>

master@akuma:/opt/spark-0.7.0$ ./run spark.examples.SparkKMeans spark://10.1.1.2:7077 kmeans_data.txt 2 0.01

13/04/18 16:19:58 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started

13/04/18 16:19:58 INFO storage.BlockManagerMaster: Registered BlockManagerMaster Actor

13/04/18 16:19:58 INFO storage.MemoryStore: MemoryStore started with capacity 323.9 MB.

13/04/18 16:19:58 INFO storage.DiskStore: Created local directory at /tmp/spark-local-20130418161958-88b0

13/04/18 16:19:58 INFO network.ConnectionManager: Bound socket to port 38648 with id = ConnectionManagerId(akuma,38648)

13/04/18 16:19:58 INFO storage.BlockManagerMaster: Trying to register BlockManager

13/04/18 16:19:58 INFO storage.BlockManagerMaster: Registered BlockManager

13/04/18 16:19:58 INFO server.Server: jetty-7.5.3.v20111011

13/04/18 16:19:58 INFO server.AbstractConnector: Started SocketC...@0.0.0.0:34458 STARTING

13/04/18 16:19:58 INFO broadcast.HttpBroadcast: Broadcast server started at http://10.1.1.2:34458

13/04/18 16:19:58 INFO spark.MapOutputTracker: Registered MapOutputTrackerActor actor

13/04/18 16:19:58 INFO spark.HttpFileServer: HTTP File server directory is /tmp/spark-9e18f077-f84d-497a-baf5-e448d34f4c68

13/04/18 16:19:58 INFO server.Server: jetty-7.5.3.v20111011

13/04/18 16:19:58 INFO server.AbstractConnector: Started SocketC...@0.0.0.0:47275 STARTING

13/04/18 16:19:58 INFO io.IoWorker: IoWorker thread 'spray-io-worker-0' started

13/04/18 16:19:58 INFO server.HttpServer: akka://spark/user/BlockManagerHTTPServer started on /0.0.0.0:50323

13/04/18 16:19:58 INFO storage.BlockManagerUI: Started BlockManager web UI at http://akuma:50323

13/04/18 16:19:58 INFO spark.SparkContext: Added JAR /opt/spark-0.7.0/examples/target/scala-2.9.2/spark-examples_2.9.2-0.7.0.jar at http://10.1.1.2:47275/jars/spark-examples_2.9.2-0.7.0.jar with timestamp 1366312798678

13/04/18 16:19:58 INFO client.Client$ClientActor: Connecting to master spark://10.1.1.2:7077

13/04/18 16:19:59 WARN spark.SizeEstimator: Failed to check whether UseCompressedOops is set; assuming yes

13/04/18 16:19:59 INFO storage.MemoryStore: ensureFreeSpace(32239) called with curMem=0, maxMem=339585269

13/04/18 16:19:59 INFO storage.MemoryStore: Block broadcast_0 stored as values to memory (estimated size 31.5 KB, free 323.8 MB)

13/04/18 16:19:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

13/04/18 16:19:59 WARN snappy.LoadSnappy: Snappy native library not loaded

13/04/18 16:19:59 INFO mapred.FileInputFormat: Total input paths to process : 1

13/04/18 16:19:59 INFO spark.SparkContext: Starting job: takeSample at SparkKMeans.scala:49

13/04/18 16:19:59 INFO scheduler.DAGScheduler: Got job 0 (takeSample at SparkKMeans.scala:49) with 2 output partitions (allowLocal=false)

13/04/18 16:19:59 INFO scheduler.DAGScheduler: Final stage: Stage 0 (map at SparkKMeans.scala:45)

13/04/18 16:19:59 INFO scheduler.DAGScheduler: Parents of final stage: List()

13/04/18 16:19:59 INFO scheduler.DAGScheduler: Missing parents: List()

13/04/18 16:19:59 INFO scheduler.DAGScheduler: Submitting Stage 0 (MappedRDD[2] at map at SparkKMeans.scala:45), which has no missing parents

13/04/18 16:19:59 INFO scheduler.DAGScheduler: Submitting 2 missing tasks from Stage 0 (MappedRDD[2] at map at SparkKMeans.scala:45)

13/04/18 16:19:59 INFO cluster.ClusterScheduler: Adding task set 0.0 with 2 tasks

13/04/18 16:20:14 WARN cluster.ClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered

[Eduardo Alfaia]

Hi Paulo Henrique,

Could you available your file kmeans_data.txt or say me how to create one?

Best Regards

[Paulo Henrique Rates]

Hi Eduardo,

the file in this folder kmeans_data.txt spark 7.0, however this database is very small, I'm looking for a larger database, do you have any?

[Matei Zaharia]

Paulo,

The log message at the bottom means you don't have any workers in your cluster (or they have less RAM than the job requested). Go to the cluster's web UI, http://10.1.1.2:8080, in a web browser, and see if there are any workers. If not, you need to launch some (look at the "standalone cluster" part of the documentation).

Matei

--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spark-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.