How to run Scala code for Spark using an editor like scalaIDE

[Deb Midya]

Hi,

Thanks in advance.

I am new to Scala as well as to Spark.

I am using ScalaIDE 4.3.0 for Eclipse 4.4.2 (Luna) on Windows 10. I've also installed Spark 1.5.2. Spark is working under pyspark & command prompt (e.g., spark-shell).

I am looking for an Editor suitable for writing and running Scala code for Spark.

May I request you to assist me for the following please.

1. Is it possible to write & run Scala code for Spark using ScalaIDE?

   If yes, how to do it?

2. Any alternative to write & run Scala code for Spark

Once again, thank you very much for the time you have given.

Regards,

Deb

[iulian dragos]

On Wed, Dec 23, 2015 at 2:27 AM, Deb Midya <midy...@gmail.com> wrote:

Hi,

Thanks in advance.

I am new to Scala as well as to Spark.

I am using ScalaIDE 4.3.0 for Eclipse 4.4.2 (Luna) on Windows 10. I've also installed Spark 1.5.2. Spark is working under pyspark & command prompt (e.g., spark-shell).

I am looking for an Editor suitable for writing and running Scala code for Spark.

May I request you to assist me for the following please.

1. Is it possible to write & run Scala code for Spark using ScalaIDE?

   If yes, how to do it?

Yes. Spark code is just a Scala application, so there isn't really anything out of the ordinary that you need to do. Create a new Scala project, add Spark dependencies (usually spark-assembly.jar is enough), create a Main object where you initialize a Spark context and then you program and run this project as any other Scala (or Java) project. If you are new to Eclipse as well, it might be a good idea to check one of the existing tutorials for Java (or scala-ide.org).

You could also run the Scala interpreter inside Eclipse (there's a Run configuration to do that), but you'd still need to start a Spark context yourself. Generally you won't get anything more than you get on the command line so I don't recommend this option.

iulian

2. Any alternative to write & run Scala code for Spark

Once again, thank you very much for the time you have given.

Regards,

Deb

--
You received this message because you are subscribed to the Google Groups "Scala IDE User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-ide-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scala-ide-user/a253e76b-3486-4e4a-b4b0-b4f79708848d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

[Deb Midya]

Hi all,

Thanks in advance.

May I request you to assist me for the following please?

1. How can I add Spark dependencies for my scala project?

2. Then how can I run it?

Once again, thank you very much for the time you have given.

Regards,

Deb

[Simon Schäfer]

On 01/10/2016 02:44 AM, Deb Midya wrote:

Hi all,

Thanks in advance.

May I request you to assist me for the following please?

1. How can I add Spark dependencies for my scala project?

I suggest to use a build tool like sbt. It can manage all of your dependencies. With the sbt plugin sbt-eclipse you can generate Eclipse project files by running the 'eclipse' task.

2. Then how can I run it?

Once sbt is setup correctly you can just create a main method (or extend the App trait) and then run your code as a Scala App.

Once again, thank you very much for the time you have given.

Regards,

Deb

On Wednesday, December 23, 2015 at 6:45:34 PM UTC+11, Deb Midya wrote:

Hi,

Thanks in advance.

I am new to Scala as well as to Spark.

I am using ScalaIDE 4.3.0 for Eclipse 4.4.2 (Luna) on Windows 10. I've also installed Spark 1.5.2. Spark is working under pyspark & command prompt (e.g., spark-shell).

I am looking for an Editor suitable for writing and running Scala code for Spark.

May I request you to assist me for the following please.

1. Is it possible to write & run Scala code for Spark using ScalaIDE?

   If yes, how to do it?

2. Any alternative to write & run Scala code for Spark

Once again, thank you very much for the time you have given.

Regards,

Deb

--

You received this message because you are subscribed to the Google Groups "Scala IDE User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-ide-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-ide-user/8e2be5b4-d5f5-4345-9953-e05df4e50b35%40googlegroups.com.

[Deb Midya]

HI,

Thanks in advance.

I've checked that one of the contents for ScalaIDE 4.3.0 for Eclipse 4.4.2 (Luna) on Windows 10 (which I've installed) is Sbt 0.13.8.

1. Is this sbt OK? Otherwise please advise me (which sbt for eclipse and how to install). I like to move to step 2 when this sbt problem is solved.

2. I've created created a samle scala program for my spark project in \FirstSpark\src by name SampleApp.SCALA

Then how can I compile  SampleApp.SCALA using sbt.

Once again, thank you very much for the time you have given.

Regards,

Deb

[Simon Schäfer]

On 01/11/2016 03:38 AM, Deb Midya wrote:

HI,

Thanks in advance.

I've checked that one of the contents for ScalaIDE 4.3.0 for Eclipse 4.4.2 (Luna) on Windows 10 (which I've installed) is Sbt 0.13.8.

1. Is this sbt OK? Otherwise please advise me (which sbt for eclipse and how to install). I like to move to step 2 when this sbt problem is solved.

2. I've created created a samle scala program for my spark project in \FirstSpark\src by name SampleApp.SCALA

Then how can I compile  SampleApp.SCALA using sbt.

It looks like you should first read a tutorial about what sbt is and how to use it. You can start with Activator, which is an application built on top of sbt and which provides templates that let you start in an easy way. See: https://www.typesafe.com/community/core-tools/activator-and-sbt#overview

Activator also provides an editor, which means that you don't have to use Scala IDE at the beginning. I suggest to not start with an IDE until you understand the very basics of working with sbt.

To view this discussion on the web visit https://groups.google.com/d/msgid/scala-ide-user/ce711c0e-5f69-41cb-8009-999302d8fb78%40googlegroups.com.

[Deb Midya]

Thanks Simon for your quick response. I'll go through Activator.

Regards,

Deb