Error: Could not find or load main class org.test.spark.SparkTest

[Navin Manaswi]

 In Scala IDE, i am trying to run this code but i get the error as mentioned.

The code is :

package org.test.spark

import org.apache.spark.mllib.tree.DecisionTree

import org.apache.spark.mllib.tree.model.DecisionTreeModel

import org.apache.spark.mllib.util.MLUtils

object SparkTest extends App {

  // Load and parse the data file.

  val data = MLUtils.loadLibSVMFile(sc, "TrainingNagios1.csv")

val splits = data.randomSplit(Array(0.7, 0.3))

val (trainingData, testData) = (splits(0), splits(1))

// Train a DecisionTree model.

//  Empty categoricalFeaturesInfo indicates all features are continuous.

val numClasses = 2

val categoricalFeaturesInfo = Map[Int, Int, Int,Int]((0,31), (1,17),(2,10),(3,167))

val impurity = "gini"

val maxDepth = 5

val maxBins = 32

val model = DecisionTree.trainClassifier(trainingData, numClasses,     categoricalFeaturesInfo,  impurity, maxDepth, maxBins)

  // etc.

}

I ran as Scala Application in Scala IDE. Can someone help me how to overcome it ?

[Oliver Ruebenacker]

     Hello,

  How are you running this code?

  I don't know what is happening in your case, but generally, Spark applications need to be run by submitting them to Spark.

     Best, Oliver

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Oliver Ruebenacker

Senior Software Engineer, Diabetes Portal, Broad Institute

[AG]

Do not worry about that error, I had the same problem with Scala IDE.

It seems that, for some reason, Scala IDE does not auto-detect the main class. Just go the up menu -->"Run"-->"Run Configurations"-->"New Scala configuration" and manually write your main class on the Text Field. Then save the configuration and run it.

I don't have the the IDE installed right now, so tell me if you could follow my steps :)

[Navin Manaswi]

Hi,

I ran as Scala Application on Scala IDE ..

 Can u help what to do ?

 Regards,

Navin

[AG]

Ok, I'll try to make it step by step. This IS a problem of Scala IDE, who doesn't recognise your main class.
Try this:

1. Open Scala IDE
2. Go to the up menu --> "Run" --> "Run configurations". This should pop up a new window.
   
3. Click on "Scala application" and on the icon for "New launch configuration"
4. Now, you will see two text fields. In "Project" just browse for your project.
5. THE PROBLEM COMES WITH THE MainClass TEXT FIELD. Scala does not detect your main class, so you should write it manually. For example, in your case you should write "org.test.spark.SparkTest".
6. Click "Apply" and "Run" buttons.

Now, there is A SECOND PROBLEM IN YOUR CODE. If you want to run Spark, you'll need a Spark Context. You should add some lines at the beginning of your class. Something like this:

    val sparkConf =
      new SparkConf().setMaster("local[2]").setAppName("THIS_IS_AN_EXAMPLE")
    val sc = new SparkContext(sparkConf)

[NavinManaswi]

Thank you very much, AG



--
View this message in context: http://scala-language.1934581.n4.nabble.com/Error-Could-not-find-or-load-main-class-org-test-spark-SparkTest-tp4644206p4644269.html
Sent from the Scala - User mailing list archive at Nabble.com.