Apache Spark project launcher roadmap

[Eric Clairambault]

Hi,

Congratulations for version 4.5 !

What about Apache Spark project launcher ? It seems to be planned in the roadmap, is it included within  version 4.5 or next version ?

It could be interesting to set up an integration with Apache Toree (I'm just wondering ...).

Many thanks for your infos.

[wpopie...@virtuslab.com]

Hi,

so the plan is following:

I hope we manage to release version 4.5.1 before the end of year just with support of scala 2.12.0. We wanted to switch directly to 2.12.1 but this one is not source compatible with 2.12.0.

So next version, 4.6.0, we plan to have first Spark support (what exactly it will be depends on potential difficulties to overcome). We weren't able to do this in 4.5.0 because of some trammels with sbt server integration (there is separate experimental branch of code available on scala-ide GitHub repo).

Kind regards

W.

[Rafał Krzewski]

W dniu poniedziałek, 19 grudnia 2016 08:37:54 UTC+1 użytkownik wpopie...@virtuslab.com napisał:

I hope we manage to release version 4.5.1 before the end of year just with support of scala 2.12.0.

 

That's excellent news! I'm really looking forward to that :)

 

We wanted to switch directly to 2.12.1 but this one is not source compatible with 2.12.0.

 

Oh, that's interesting. I don't see any mention of that in release notes for 2.12.1 (https://www.scala-lang.org/news/2.12.1) can you elaborate?

 

So next version, 4.6.0, we plan to have first Spark support (what exactly it will be depends on potential difficulties to overcome). We weren't able to do this in 4.5.0 because of some trammels with sbt server integration (there is separate experimental branch of code available on scala-ide GitHub repo).

Sbt project import support, at least at the level provided by sbteclipse plugin would be nice, but there's a usable alternative. Editing .sbt files in Eclipse with completions and so on would be more of a game changer, but I imagine it's quite hard to implement, because Eclipse is really fussy about overlapping project locations. 

Cheers,

Rafał

[wpopie...@virtuslab.com]

W dniu poniedziałek, 19 grudnia 2016 13:39:04 UTC+1 użytkownik Rafał Krzewski napisał:

W dniu poniedziałek, 19 grudnia 2016 08:37:54 UTC+1 użytkownik wpopie...@virtuslab.com napisał:

I hope we manage to release version 4.5.1 before the end of year just with support of scala 2.12.0.

 

That's excellent news! I'm really looking forward to that :)

 

We wanted to switch directly to 2.12.1 but this one is not source compatible with 2.12.0.

 

Oh, that's interesting. I don't see any mention of that in release notes for 2.12.1 (https://www.scala-lang.org/news/2.12.1) can you elaborate?

 

scala.tools.nsc.ast.parser.Scanner.skipComment()

 is final in 2.12.1 and wasn't in 2.12.0

and scala-refactoring uses its to find out comments in source.

So next version, 4.6.0, we plan to have first Spark support (what exactly it will be depends on potential difficulties to overcome). We weren't able to do this in 4.5.0 because of some trammels with sbt server integration (there is separate experimental branch of code available on scala-ide GitHub repo).

Sbt project import support, at least at the level provided by sbteclipse plugin would be nice, but there's a usable alternative. Editing .sbt files in Eclipse with completions and so on would be more of a game changer, but I imagine it's quite hard to implement, because Eclipse is really fussy about overlapping project locations. 

There is quite advanced work done on integration but Simon decided that we are not able to fully support sbt build, so we decided to shunt it aside for some time. Not sure which version of sbt-eclipse we used then but current 5.0.1 looks promising so maybe we resuscitate it.

 

Cheers,

Rafał

[Rafał Krzewski]

W dniu poniedziałek, 19 grudnia 2016 14:47:36 UTC+1 użytkownik wpopie...@virtuslab.com napisał:

scala.tools.nsc.ast.parser.Scanner.skipComment()

 is final in 2.12.1 and wasn't in 2.12.0

and scala-refactoring uses its to find out comments in source.

I see... I thought you meant incompatibility in the accepted compiler input :)

 

On that note, I'm wondering if you have been looking into scala.meta already? It would eliminate surprises of this kind.

Integrating scalafmt as a alternative or replacement for scalariform could be a relatively easy first step. Have you considered it?

 

There is quite advanced work done on integration but Simon decided that we are not able to fully support sbt build, so we decided to shunt it aside for some time. Not sure which version of sbt-eclipse we used then but current 5.0.1 looks promising so maybe we resuscitate it.

 

I think importing SBT projects is pretty high up on the list of expectations of new users of Scala IDE, so it might indeed be a good idea to look into it once 2.12 support is done.

Cheers,
Rafał

 

[wpopie...@virtuslab.com]

W dniu poniedziałek, 19 grudnia 2016 16:09:36 UTC+1 użytkownik Rafał Krzewski napisał:

W dniu poniedziałek, 19 grudnia 2016 14:47:36 UTC+1 użytkownik wpopie...@virtuslab.com napisał:

scala.tools.nsc.ast.parser.Scanner.skipComment()

 is final in 2.12.1 and wasn't in 2.12.0

and scala-refactoring uses its to find out comments in source.

I see... I thought you meant incompatibility in the accepted compiler input :)

 

On that note, I'm wondering if you have been looking into scala.meta already? It would eliminate surprises of this kind.

Integrating scalafmt as a alternative or replacement for scalariform could be a relatively easy first step. Have you considered it?

 

Rationale for using scala scanner directly is not to duplicate ways of code parsing. Personally I think that the finest solution would be adding new token for comment or expose comment scanner simile to DocScanner. And yes, it seems that we end up with scala.meta or another tool to this job.