Java vs Scala in Livy

[Alex Bozarth]

I've been working on ClientConf.java in client-common and trying to understand why it and many other modules in Livy are written in java and not scala. Currently I'm trying to implement the ability to deprecate confs similar to how Spark does it, but I've been running into issues creating a java implementation inspired by a scala implementation.

So why java and not scala? Was it chosen for a reason? Would updating to scala be a future interest? Based on the git history it looks like the decision was originally made by Marcelo.

[Alex Bozarth]

Pinging Marcelo directly for input, is the reasoning behind java to make it easier to write either java or scala client applications? I think it's odd though that a server class (LivyConf) extends a client class (ClientConf), was this out of necessity or simplicity? In relation to my current work, I'm currently considering the following options: only implement conf deprecation in LivyConf, recreate it using java so it resides in ClientConf, or attempt to update/switch ClientConf to scala without breaking its java dependancies. I'm just looking for some context to the java/scala reasoning here to help in my decision making moving forward.

[Marcelo Vanzin]

The RSC code and API is written in Java mainly for three reasons:

- it comes from Hive's "spark-client" module which is written in Java
- I prefer to write public APIs in Java (no dealing with Scala's ever
changing ABI)
- Because of the above, you can use the same RSC jar against any Spark version

As far as what people prefer going forward, I don't really mind either
way since I'm not really active in the project currently.

--
Marcelo

[Alex Bozarth]

Thanks Marcelo, that helps

[Saisai Shao]

Alex, is there any issue we have to use Scala instead of Java for ClientConf? 

--
You received this message because you are subscribed to the Google Groups "Livy Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to livy-dev+unsubscribe@cloudera.org.

[Alex Bozarth]

Not a big issue, I was using the Spark implementation of conf deprecation as a starting point, but it leverages case classes and scala objects. I plan to take a stab at recreating it with java tomorrow to see how messy it would be. The more I look at the problem though the smaller the scala/java issue seems, though I am glad to understand the initial reasoning.

Alex Bozarth Software Engineer Spark Technology Center

E-mail: ajbo...@us.ibm.com GitHub: github.com/ajbozarth	505 Howard Street San Francisco, CA 94105 United States

Saisai Shao ---03/02/2017 05:53:46 PM---Alex, is there any issue we have to use Scala instead of Java for ClientConf?

To unsubscribe from this group and stop receiving emails from it, send an email to livy-dev+u...@cloudera.org.