An idea: compiling Java source code via scalac

[杨博]

I always think about how to make Scala more popular.

I noticed that the most successful frameworks and tools in Scala community usually support Java API, Kafka, Play, Akka, Spark for example. At the same time, there are a much larger population of Java developers than Scala. 

So what prevent Java developers move to Scala even when their most brilliant library authors choose Scala?

I think syntax is the largest block. People worry about that a new language would bring risks into their working production.

On the contrary, there are some successful languages like C++, JSX and Generic Java. They all keep source-level compatibility with their original languages (C, JavaScript, and Java).

Scala's author, Martin Odersky, himself is not a syntax fan, in The Goals of Scala's Design, he said:

It's a totally minor point, and people can get used to either approach. So we decided to not fight convention in these minor things, when there were other places where we did want to make a difference.

What the battles in Scala's design did Martin pick? In my honest opinion, these battles should be the typer and the backend.

So, I wonder if we can enable the Java source code compilation in scalac, and still keep using scala's AST, typer and backend. As a result, we could bring the power of Scala's macro system and Scala.js into Java world.

In order to achieve the powers from Scala world, a Java developer only need to switch the java compiler in his Gradle or Maven configuration, without learning any new language.

I guess all the Scala frameworks would benefit from this idea, especially Scala.js, which may be called Java.js then.

Do you think this idea attractive, guys?

[杨博]

IIRC, we already have a Java parser under scala.tools.nsc.javac for the purpose of type resolving. We will need to update the Java parser to support complete Java syntax and convert Java source code to Scala AST.

在 2016年4月24日星期日 UTC+8上午2:27:12，杨博写道：

[Simon Schäfer]

On 04/23/2016 08:27 PM, 杨博 wrote:

I always think about how to make Scala more popular.

I noticed that the most successful frameworks and tools in Scala community usually support Java API, Kafka, Play, Akka, Spark for example. At the same time, there are a much larger population of Java developers than Scala. 

So what prevent Java developers move to Scala even when their most brilliant library authors choose Scala?

I think syntax is the largest block. People worry about that a new language would bring risks into their working production.

On the contrary, there are some successful languages like C++, JSX and Generic Java. They all keep source-level compatibility with their original languages (C, JavaScript, and Java).

Scala's author, Martin Odersky, himself is not a syntax fan, in The Goals of Scala's Design, he said:

It's a totally minor point, and people can get used to either approach. So we decided to not fight convention in these minor things, when there were other places where we did want to make a difference.

What the battles in Scala's design did Martin pick? In my honest opinion, these battles should be the typer and the backend.

So, I wonder if we can enable the Java source code compilation in scalac, and still keep using scala's AST, typer and backend. As a result, we could bring the power of Scala's macro system and Scala.js into Java world.

At least the Java compilation through Scala.js is a planned project for GSoC: https://summerofcode.withgoogle.com/organizations/4954864794730496/#5512830375690240

In order to achieve the powers from Scala world, a Java developer only need to switch the java compiler in his Gradle or Maven configuration, without learning any new language.

I don't see what powers Java developers would get. Macros are not exactly a feature that is interesting to Java developers and Scala language features can't be used in Java anyway, therefore why do you want to go through the Scala backend?

I guess all the Scala frameworks would benefit from this idea, especially Scala.js, which may be called Java.js then.

Do you think this idea attractive, guys?

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[杨博]

I didn't understand why there is a project that compiles Java into Scala.js IR. Why not reuse the existing Java to AST parser?

Simon Schäfer <ma...@antoras.de>于2016年4月24日周日 上午3:31写道：

You received this message because you are subscribed to a topic in the Google Groups "scala-internals" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scala-internals/GCKWlh4z1Zo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scala-interna...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

杨博 (Yang Bo)

[Simon Schäfer]

On 04/23/2016 09:46 PM, 杨博 wrote:

I didn't understand why there is a project that compiles Java into Scala.js IR. Why not reuse the existing Java to AST parser?

I guess because there doesn't existing a Java to scalac AST parser yet.

Simon Schäfer <ma...@antoras.de>于2016年4 月24日周日 上午3:31写道：

[Adriaan Moors]

Our Java parser only does the "outlines" so that we can generate signatures. It does not go into method bodies etc. A full Java front end would be a  significant effort. A (rough) Java to Scala converter (such as intellij's) seems like an easier way to help people migrate their code. 

[杨博]

So will the GSoC project reuse the nsc typer for Java compilation? Would the JVM target also benefit from the GSoC project's Java parser?

Adriaan Moors <adr...@lightbend.com>于2016年4月24日周日 上午8:53写道：

--

杨博 (Yang Bo)

[Sébastien Doeraene]

Hello,

No, that GSoC project will not use any of the nsc infrastructure. It will reuse an existing Java parser and typechecker, and compile directly from Java AST to Scala.js IR.

Cheers,
Sébastien

[杨博]

So why not reuse the nsc's Scala AST and typer? I guess the Scala AST is able to represent most of Java syntax.

Sébastien Doeraene <sjrdo...@gmail.com>于2016年4月25日周一 下午1:31写道：

--

杨博 (Yang Bo)

[Naftoli Gugenheim]

I'm very confused. The only point of scalac is to convert source files written in a certain language called Scala into JVM bytecode. What would be the point of scalac compiling java source code? You can already do that with javac!

On Sat, Apr 23, 2016 at 2:27 PM 杨博 <pop....@gmail.com> wrote:

I always think about how to make Scala more popular.

I noticed that the most successful frameworks and tools in Scala community usually support Java API, Kafka, Play, Akka, Spark for example. At the same time, there are a much larger population of Java developers than Scala. 

So what prevent Java developers move to Scala

You mean from writing code in the Scala language?

 

even when their most brilliant library authors choose Scala?

I think syntax is the largest block. People worry about that a new language would bring risks into their working production.

Well syntax is pretty much inherent to a language...

 

On the contrary, there are some successful languages like C++, JSX and Generic Java. They all keep source-level compatibility with their original languages (C, JavaScript, and Java).

How does that prevent issues of risk or learning curve?

Anyway, here it sounds like you are okay with syntactic differences, but only ones that are backwards-compatible.

 

Scala's author, Martin Odersky, himself is not a syntax fan, in The Goals of Scala's Design, he said:

It's a totally minor point, and people can get used to either approach. So we decided to not fight convention in these minor things, when there were other places where we did want to make a difference.

What the battles in Scala's design did Martin pick?

The article answers that:

Martin Odersky: The first thing we cared about was to have as clean an integration of functional and object-oriented programming as possible. We wanted to have first-class functions in there, function literals, closures. We also wanted to have the other attributes of functional programming, such as types, generics, pattern matching. And we wanted to integrate the functional and object-oriented parts in a cleaner way than what we were able to achieve before with the Pizza language. That was something we deeply cared about from the start. 

Almost all of that involves syntax. And a lot of it is not backwards-compatible.

 

In my honest opinion, these battles should be the typer and the backend.

So, Java syntax, but a different typer? It would type the same code differently? Sounds confusing! And what would be the point of a different backend for javac?

So, I wonder if we can enable the Java source code compilation in scalac, and still keep using scala's AST, typer and backend. As a result, we could bring the power of Scala's macro system and Scala.js into Java world.

How could java code fit into scala's AST? How would you do statics, for instance? There are plenty of things in java syntax that don't exist in scalac's AST.

Macro system? What does that have to do with anything? It's nice that scala recently added experimental macros support but to suggest that that's all scala brings to the table? It's an extremely minor thing in the scheme of things.

And what does scala.js have to do with anything? Scala.js just compiles scala to javascript. If you want to use Java then what's the point of scala.js?

In order to achieve the powers from Scala world, a Java developer only need to switch the java compiler in his Gradle or Maven configuration, without learning any new language.

Sorry to disappoint you, but Scala doesn't have magic powers, that you gain access to by taking some old code and conjuring scalac. The thing that's powerful about scala IS the language!

Can you figure out a way to achieve the powers of Chinese (https://www.quora.com/What-are-some-advantages-of-complex-languages-such-as-Chinese) in English?

 

I guess all the Scala frameworks would benefit from this idea, especially Scala.js, which may be called Java.js then.

Do you think this idea attractive, guys?-- 

I'm still not clear if you want to drop a bunch of the design of the scala language (type/identifier order?), or if you want scala to be "backwards compatible" to Java somehow.

You realize that mixing Java and Scala is quite seamless (on file boundaries: a given file is either java or scala), and supported very well by the tooling (sbt, eclipse scala ide, intellij)? If the point is to move from Java to Scala gradually, that's always been quite doable and a major "feature" of scala.

[Lukas Rytz]

Without knowing all the answers or details, I'd expect quite a lot of subtle difficulties trying to represent Java as Scala ASTs. Areas that come to mind are:

 - constructors

 - field access

 - non-existing language constructs (break/continue, for loops)

 - annotations

 - static members (SI-9111 for example)

 - possibly type checking (wildcards)

 - some compiler transformations would need to be avoided (generating getters/setters)

 - disable implicit search when compiling Java

It would probably be easy to make the easy part working, but very difficult to get the project to a point where you have a faithful Java compiler.

Attached is a PDF report of Cédric Lavanchy, an EPFL Student that investigated some of these difficulties in a small project in 2009.

[Kevin Wright]

Sure, the AST could represent most of Java’s syntax, but that’s not the point.  Right now the Scala parser has very limited support indeed.

AFAIK, scalac against java source just parses enough to determine externally visible symbols and types… meaning just interfaces, classes, and their member declarations (fields and methods); all of which are explicitly typed, so no inference is needed.

Aside: This means a fair bit of work would be needed to support Java 9 if they adopt the proposal to implement type inference via the var keyword.

As for the actual content of Java methods… Scalac just ignores it right now.  So there are large section of the Java language spec, fairly large and significant sections, that are completely absent from the parser. (e.g. The entirety of chapter 5)

The tl;dr is that the amount of logic required for the parsing of the rest of the Java language is far, far more than the amount of logic we have right now for the small fraction we can currently parse.

Java Lang Spec here: https://docs.oracle.com/javase/specs/jls/se8/html/index.html

Kevin Wright
mail / hangouts / msn : kev.lee...@gmail.com

quora: http://www.quora.com/Kevin-Wright

google+: http://gplus.to/thecoda

twitter: @thecoda

vibe / skype: kev.lee.wright

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra