Spark Users

As part of Spark's move to Apache, we have now made this Google Group read-only. To keep

Hey folks, FYI, the talk submission deadline for this is October 25th. We've gotten a lot of

Does it stop with an error? Also after how long? Can you give an idea of the different values of

hi, I use maven to build the spark0.7.2, and I successfully build spark-core,streaming,bagel. But

I got it all, your posts are really helpful. Thanks very much. On Friday, January 3, 2014 4:34:21 PM

I saw Code of sortByKey: def sortByKey(ascending: Boolean = true, numPartitions: Int = self.

I've tried it bulding spark with mesos 0.14.2 and I have the exact same error Stack: [

Hi, I followed instructions in the document of 0.8.1 to run a program on EC2 cluster. 1. I can setup

Hi all, Wonder if someone can point me in the right direction ... I'm using the 0.81 binaries

Need not mention Workers could be seen on the UI. On Thu, Jan 2, 2014 at 5:01 PM, Archit Thakur <

Yes, I am using My Custom Data Structures (for Key and Value) and have registered different

I am still getting it. I googled and found a similar open problem on stackoverflow: http://

Command: SPARK_JAR=./assembly/target/scala-2.9.3/spark-assembly-0.8.1-incubating-hadoop2.2.0.jar \ ./

Hi, Sorry that was my mistake. I corrected it. On Friday, December 27, 2013 6:52:20 PM UTC+1, Patrick

What is the difference between including my app (dependencies) jars in SPARK_CLASSPATH and putting

Apache-spark rencently update the version to 0.8.1, in which yarn-client mode is available. My

Hi Kaden. Are you running your application on a Spark cluster? If so, `f' is executed by the

Hi, I'm deploy spark in YARN. Spark is 0.8.1 Hadoop is 2.2.0 But when I run driver program. It

To configure the web UI port on the worker, you need to set SPARK_WORKER_WEBUI_PORT appropriately in

Never mind , I found out I followed the wrong version of documentation . yarn-client mode is

Hey Andy - these Nabble groups look great! Thanks for setting them up. On Tue, Dec 24, 2013 at 10:49

I have used the elasticsearch-hadoop successfully, mapreduce run over elasticsearch instead of hdfs

Hey thanks again, that's what I feared anyway, but I had hope they would assure

Ya this can be confusing (the logging output gets interpolated with the shell). Just hitting "

I would not recommend putting your text files in via ADD_JARS. The better thing to do is to put those

Hi, I'm testing my spark program recently, in which I have 4 RDD of different data, after a

Hi, I'm testing my spark program recently, in which I have 4 RDD of different data, after a

private var nodes: RDD[(Int, StaticNode)] = ....some code for init.... ....... val tmp = nodes nodes

It might work using the `yarn-client` mode (https://spark.incubator.apache.org/docs/latest/running-on

Definitely you can stream it. You just need to do val rdd = sc.textFile("hdfs://....") rdd.

Thanks for the reply. The second issue also has been resolved by rebuild use the same version as our

Either of these approaches will work well. Both HDFS and HBase will handle the input splits so spark