A Critique of Scala in The Practical World

[kraythe]

I have been using Scala now for two years and have integrated it into projects I am working on in the professional business world. Just so there is no misunderstanding, I consider myself to have a very decent skill level with the language and I have been programming in general for 31 years, 23 of them professionally. I have written two books on Java and consider myself an expert on that language. Furthermore, I am actually a FAN of the scala language itself (minus the issues below) so this is not an attempt to flame war scala into hell but rather a desire to offer constructive criticism. Minus the issues below I would much rather write scala than Java despite my java expertise. This post is not targeted, therefore, to tear down the language but to recommend that it take some different directions. 

Having worked for 2 years on one particular very complicated project, I have formed some conclusions about Scala from the trenches. The project uses the latest version of Play Framework and SBT and integrates Akka as well. It is written partially in Java and partially in Scala with Scala being many of the controllers and only a few of the entities. The design of the project is a standard play application with Non-Anemic domain model, since I rather strongly agree with Martin Fowler's condemnation of the POJO pattern. The application handles tens of thousands of concurrent users, must be transactional and has real time streaming components. 

In using Scala on this project I have a number of observations to convey. 

1) In my opinion the absolute worst aspect of Scala is the rampant use of Implicit conversions. They make the code nearly impossible to follow when reading it in the IDE in many situations. While I understand their motivation, it tends to go seriously sideways and results in the creation of Opaque DSLs that just serve to confuse. When it comes to cross training Java developers into Scala I might as well be stirring a pot of witch brew and doing some Voodoo for all the junior developers understand. My company has neither the resources nor the time to send the staff to a couple months or even weeks of training. Since implicit have become so rampant in Scala, the language is being polluted and I find it hard to defend against the notion that it is an Obfuscated C Code contest on Steroids. Furthermore since most conversions between Java and scala are done with implcits (for collections) it makes writing interoperable code a complete pain. 

2) A close second to Implicit Conversions is the introduction of Macros. Again i understand the base reasoning but the resulting mess it creates makes code nearly impossible to debug. Generated code has always been an impediment to effective debugging but this takes it to a whole new level. Furthermore, since macros exist they have prevented better solutions from being invented to the problems that macros address. I am reminded of STL in C++ and some of the nightmares with #IFDEF and #DEFINE that are well known to render C++ programs unreadable. 

3) SBT should be renamed "Screwed Up Build Tool". A language lives and dies on its tools and SBT needs some serious work. I suppose that is why it hasn't even reached a 1.0 release. Its slow, cumbersome, often flighty. Our play application, for example, will spontaneously recompile the entire codebase when I change a comment in a file or introduce a new private variable. It will sometimes do so twice, iterating the whole codebase. The fact that SBT is sensitive to double spacing is fundamentally FUBAR. 

4) Integration with major development environments like IntelliJ and Eclipse is minimal and works but often just barely. Again, a language lives and dies on its tools. In my opinion Trpesafe (the org pushing scala) should be spending SIGNIFICANT resources making the perfect plugins for these two major toolsets. I am an intelliJ fan and scala plugin for that platform works ok but not nearly to the extent that the java integration works. For example, intellij fails to understand and deal with any of the scala templates for views in Playframework. This is with the Play plugin and integration working.

5) While Implicit and Macros have been added to make the language more complicated significantly more simple additions to the language for more practical features have been flamed by the community and not implemented. For example a keyword readonly (or similar keyword) on a scala var to allow the variable to be written by the class but not by external members of the class. The only solution is to declare a private member then declare getters and not setter. This is a silly miss on the part of the language since it would be so useful. The standard response from the community is that everything should be val but this is not realistic when you have to integrate with existing code and have complexity of language mixing to deal with. 

6) And this is meant in the most honest constructive way possible. The community has some issues when it comes to dealing with others. New users are often ridiculed for lack of knowledge and derided for not knowing some of the most obscure facets of the language. I suppose this makes the "experts" in the community feel superior but does nothing to advance the cause of scala. Quite the opposite. In addition much of the advice is given from an academic perspective rather than a practical one. People in the business world have no ROI to re-implement the whole system to Scala purity and simply want do so. Advising them to do so is tantamount to advising them "scala is not for you" and that is hardly beneficial to the development of the language. Most developers that make a language go big will be business developers and not academics. Business developers have to balance things like ROI and resources, not simply advocate developmental purity and this is not a scala specific mantra either. Its just reality of business life. 

Anyway those are my opinions and I am sure I will be flamed below but these are my honest impressions and in fact much more mild than most of my colleagues that i have tried to push to learn scala over the last 2 years. 

[martin odersky]

Just wanted to respond the the first issue (which is not to say the
others are not important).
Since Scala 2.10, implicit conversions are discouraged and they
require an explicit language import:

import language.implicitConversions

Are you saying that their use is ubiquitous nevertheless? Are there
some specific libraries to blame for this? Note that for Java/Scala
collections interop the recommended way is to use JavaConverters, not
JavaConversions.

Maybe we should make it even less convenient than now to use implicit
conversions? Or warn users of libraries that leak implicit conversions
about the dangers?

- Martin

> --
> You received this message because you are subscribed to the Google Groups
> "scala-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scala-user+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Martin Odersky
EPFL

[Dennis Haupt]

i only have 11 professional years of around 16, but here is my opinion anyway:

java, at least until 7 (didn't use 8) is terrible to write compared to scala. i had to rely heavily on code generation via my ide.

scala, on the other hand, was easy to write even without real ide support (i used the intellij plugin in its early stages)

 

as for code reading, i don't have problems with either language. thanks to <insert ide here> you can easily follow call hierachies, variables etc.

neither implicit conversions nor parameters give me trouble, because .... i don't try to understand them :D

i started to try to guess the general idea, and what usually happens is that i get a few ideas on how i would use implicits here and there, and it turns out the guy who wrote the code had similar thoughts. it is necessary to know what the code is trying to accomplish. without that, implicits indeed make things harder. imho, the benefits greatly outweights the disadvantages.

for example, our codebase has lots of implicit conversions to do things like foo.toDate("dd.MM.yyyy"), math.pi.decimalFormat(2), collection.headAssertSizeOne, map.mapKeys, map.mapValuesStrict, collection.toMapBy(...) and i don't want to sacrifice this.

 

what i don't need are *actual* conversions with the intent of converting type a into type b, like

 

def x(i:Int) = ...

x("12345")

 

they are always used for some "higher purpose"

 

 

 

Gesendet: Samstag, 20. Februar 2016 um 16:17 Uhr
Von: "martin odersky" <martin....@epfl.ch>
An: kraythe <kra...@gmail.com>
Cc: scala-user <scala...@googlegroups.com>
Betreff: Re: [scala-user] A Critique of Scala in The Practical World

[Matthew Pocock]

Hi Kraythe,

I've been using scala for a while now and have a similar amount of experience to you. I agree with many of your observations.

1. Developing scala for anything non-trivial without an IDE is not easy. Developing things that rely upon implicits without an IDE that understands implicits is very difficult indeed. Even then, I spent 20 minutes yesterday working out what I hadn't imported within some slick code to get .result added as a decorator. I got there in the end, but the IDE didn't seem to know what to import. Implicits are really awesome, but they come with great responsibility.

2. The IDE support needs improving. It is already much better than it was when I started out, but it is still not there. Mainly around support for implicits and macros.

3. I can't agree with you on macros to be honest. They are an amazingly better than the mind-bending mess you have in Java and other ecosystems of code generation. We need better IDE support, to show the expansions in-place within the code. Developing macros is possible but not for the faint hearted.

4. SBT is growing on me. Yes, it sucks. It is weird. It doesn't seem to have a lot of consistency. But compared to the alternatives? I mean would you honestly like to port complex builds from sbt back into maven? I make heavy use of the ~ modifier and the sbt-revolver plugin, which gives me a real-time development experience I can't get outside of interpreted platforms like node.js.

sbt-revolver

Anyway, I like Scala, and it does seem to be constantly improving. Increasingly I deploy things either as Jars, or restful services or dockerised apps, so the implementation language really doesn't matter much.

Matthew

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Dr Matthew Pocock

Turing ate my hamster LTD

mailto: turingate...@gmail.com

Integrative Bioinformatics Group, School of Computing Science, Newcastle University

mailto: matthew...@ncl.ac.uk

gchat: turingate...@gmail.com

msn: matthew...@yahoo.co.uk

irc.freenode.net: drdozer

skype: matthew.pocock

tel: (0191) 2566550

mob: +447535664143

[Mark_Grand]

I have a background similar to yours, but I have some different conclusions. Implicits and macros are powerful tools. Their use involves a trade-off between complexity and readability. Yes their use makes code harder to debug. When used wisely, the simplicity that they add more than offsets the added complexity. The good news is that the complexity can be mitigated by the IDE. However, I see no way around the problem that inexperienced programmers do not have the wisdom to use implicts well.

The IDEs could be a lot better than they are. I would like to be able to push a button and see an expansion of an implicit reference or a macro invocation.  I would also like to be able to trace through the logic needed to produce the expansion. 

[Erik Bruchez]

> The fact that SBT is sensitive to double spacing is fundamentally FUBAR. 

Do you mean having to insert an empty line between declarations? That's been fixed in sbt 0.13.7. See:

    http://stackoverflow.com/questions/21780787/why-does-sbt-version-%E2%89%A4-0-13-6-require-blank-lines-between-settings-in-sbt-fil

-Erik

[Rex Kerr]

On Sat, Feb 20, 2016 at 6:38 AM, kraythe <kra...@gmail.com> wrote:

In using Scala on this project I have a number of observations to convey. 

1) In my opinion the absolute worst aspect of Scala is the rampant use of Implicit conversions.

Any chance you can solve this problem locally by adopting as best-practice not using them in your own code base?  There's already a compiler warning, as Martin pointed out.

 

They make the code nearly impossible to follow when reading it in the IDE in many situations. While I understand their motivation, it tends to go seriously sideways and results in the creation of Opaque DSLs that just serve to confuse.

When used properly, implicit conversions make code enormously clearer and less error-prone and boilerplatey (even discounting their use in type classes):

  - They can replace dependency injection, where it is even harder to figure out where things are coming from

  - They allow configuration while maintaining sane defaults that you don't need to think about

  - Trusting me to type the same execution context into every Future operation is folly

  - So is thinking I can see what the Future logic is doing any more after inserting an extra parameter on every single thing

  - They let you not worry so much about external libraries not implementing the trait you need them to implement

So, given that the advantages are large, it seems like it's worth some really serious thought about how to prevent it "going seriously sideways" without simply abandoning the feature entirely.

 

When it comes to cross training Java developers into Scala I might as well be stirring a pot of witch brew and doing some Voodoo for all the junior developers understand. My company has neither the resources nor the time to send the staff to a couple months or even weeks of training.

This might be a case where your expectations for Scala just aren't the best thing for the programming community as a whole.  Not every improvement in language design is necessarily accessible to a typical junior developer with not "even weeks of training".  Other languages have that as a specific design goal (e.g. Groovy).  Scala is more ambitious / independent.

Maybe you don't have much input into company policy in this regard, but if you're keeping your junior developers for more than a few months each, do you really have time to *NOT* give them weeks of training (so they're reasonably productive for the rest of their time)?

 

Since implicit have become so rampant in Scala, the language is being polluted and I find it hard to defend against the notion that it is an Obfuscated C Code contest on Steroids. Furthermore since most conversions between Java and scala are done with implcits (for collections) it makes writing interoperable code a complete pain.

The recommendation for a long time now has been to use JavaConverters, which provide explicit asScala and asJava methods to go back and forth.  Again, this one I think you solve with best practices locally: use JavaConverters.

 

 

2) A close second to Implicit Conversions is the introduction of Macros. Again i understand the base reasoning but the resulting mess it creates makes code nearly impossible to debug. Generated code has always been an impediment to effective debugging but this takes it to a whole new level.

Well, it's both better and worse.  With generated code, figuring out where a runtime bug is is often nigh-impossible (using typical approaches).  With macros, figuring out where a compile-time bug is is often pretty hard, but at least at runtime you can typically track it back to approximately the right place.

 

Furthermore, since macros exist they have prevented better solutions from being invented to the problems that macros address.

I'm not sure that's true.  Before macros, there was a long time where those solutions also weren't being invented.  I agree that something better would be great, but I am not sure that in the absence of macros there'd have been any improvement at all.

 

5) While Implicit and Macros have been added to make the language more complicated significantly more simple additions to the language for more practical features have been flamed by the community and not implemented. For example a keyword readonly (or similar keyword) on a scala var to allow the variable to be written by the class but not by external members of the class. The only solution is to declare a private member then declare getters and not setter. This is a silly miss on the part of the language since it would be so useful.

Agreed.  If you can't do this sort of thing generically in a sensible way (e.g. macros cannot do it for you), it would make life easier to have it baked in.  It's not like there aren't a bunch of other conveniences baked in:

  - for comprehensions

  - insertion of unit

  - elision of superfluous parens in method calls

  - { case x => } shorthand for { _ match { case x => } }

and so on.

I do spend a lot of boilerplate budget on exactly the public-readable private-writable thing.

 

6) And this is meant in the most honest constructive way possible. The community has some issues when it comes to dealing with others. New users are often ridiculed for lack of knowledge and derided for not knowing some of the most obscure facets of the language.

I hope that's been getting better lately with the introduction terms of conduct and various other factors.  For instance, the Gitter channel is usually pretty friendly.  You might get advice to use an "obscure facet of the language", but it's almost always friendly advice.  (To newbies.  Established users argue with each other about the best way to do things.)

I agree that this has been a problem in the past, but do you still find it to be true in a substantial number of public forums?

  --Rex

 

[martin odersky]

Robert,

I am still not sure whether it's *implicit conversions* you are
arguing against, or all kinds of implicits (i.e. also implicit
parameters and implicit classes. That makes a huge difference in my
mind. Scala would not be Scala if it did not have implicit parameters
and classes.

Cheers

- Martin

> --
> You received this message because you are subscribed to the Google Groups
> "scala-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scala-user+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

Martin Odersky
EPFL

[Pedro Larroy]

My experience is just the opposite. I just see Scala usage growing and slowly taking over the enterprise. I had a great experience with Scala which allowed me to become more familiar with functional programming concepts coming from an imperative background.

Implicit conversions are extremely useful for ad hoc polymophism, and intellij with the Scala (and vim) plugin is an excellent IDE. I never enjoyed an IDE before, but for a while already I'm extremely happy with Intellij and Scala.

Intellij seems to help you with implicit conversions:

https://confluence.jetbrains.com/display/IntelliJIDEA/Working+with+Scala+Implicit+Conversions

I think the best features of the language are implicits (both implicit parameters and conversions) and higher kinded types.

For me the less shiny part of developing in Scala are pitfalls like autoboxing and creating many closures that can make code run slower than it could. For some of these, is part of the price to pay for  running on the JVM.

About SBT, I don't know if it could be better, probably the documentation could, but I have dealt in my career with many build systems for a plethora of languages and platforms and there are much, much worse and inconvenient. Not to mention having to maintain one different build system for every platform for a multiplatform C++ project, or dinosaurs like make or Cmake. 

You can create you SBT build using plan scala if you dislike the DSL, for example: https://github.com/SandroGrzicic/ScalaBuff/blob/master/project/ScalaBuffBuild.scala

Regards.

Pedro.

[Lanny Ripple]

Programming for 30 years, professionally for 23.  Scala professionally for the last 5 and I hate Java with a passion.  Computers (and languages) should work for people and not the other way around.  That said I'm glad to read this post and get a view from a different perspective since I know my biases are blatantly for Scala.

(1) I hope you are not using JavaConversions.  You have to be explicit with JavaConverters but that's sort of the point and bleeds off much of the confusion around, "How did my type just get mugged?"  (Martin, don't make implicit conversions more difficult.  Just deprecate JavaConversions.)

There are good times to use implicit conversions but they generally involve translating from a specific type in a library into another specific type in the same library.  If the conversions are leaking over into grabbing general types then that would definitely cause confusion.  (Which is what JavaConversions does.)

(3) Used to agree 100%.  It got much better in 0.13.x, enough that we were able to drop Maven.  What build system would you recommend for a Java project?  I can only guess Maven or Ant and those are no picnic either for anything larger than a small project.

(5) How would `readonly` work with access modifiers?

(6) Can you offer specific examples?  I know this board has made a great effort to prevent ridicule and be accepting.  I also know it's easy to say things like, "Just do voodoo() because you have an Insert-Magic-Here.", which can come across as very opaque to someone that isn't aware of what an Insert-Magic-Here is. That said if I suggest you use a Visitor or Flyweight I'm essentially doing the same thing.  Once you understand an abstraction it's difficult not to use the short-hand.  I have yet to see an expert tell someone, "You should just be able to figure it out."  (I have seen, "Here's some links/keywords to get started with.", but I don't consider that the same thing.  YMMV.)

  -ljr

[martin odersky]

On Fri, Mar 25, 2016 at 7:00 PM, Lanny Ripple <lanny....@gmail.com> wrote:
> Programming for 30 years, professionally for 23. Scala professionally for
> the last 5 and I hate Java with a passion. Computers (and languages) should
> work for people and not the other way around. That said I'm glad to read
> this post and get a view from a different perspective since I know my biases
> are blatantly for Scala.
>
> (1) I hope you are not using JavaConversions. You have to be explicit with
> JavaConverters but that's sort of the point and bleeds off much of the
> confusion around, "How did my type just get mugged?" (Martin, don't make
> implicit conversions more difficult. Just deprecate JavaConversions.)
>

Good idea. Are they not deprecated yet?

> There are good times to use implicit conversions but they generally involve
> translating from a specific type in a library into another specific type in
> the same library. If the conversions are leaking over into grabbing general
> types then that would definitely cause confusion. (Which is what
> JavaConversions does.)

Exactly. That could be a good case for a linter: Flag implicit
conversions that leak from a library. In that case not just the
designer but also the user of such a library should get a warning.

Cheers

- Martin

[Seth Tisue]

On Friday, March 25, 2016 at 3:05:59 PM UTC-4, martin odersky wrote:

On Fri, Mar 25, 2016 at 7:00 PM, Lanny Ripple <lanny....@gmail.com> wrote:
>  (Martin, don't make
> implicit conversions more difficult.  Just deprecate JavaConversions.)
>
Good idea. Are they not deprecated yet? 

Not yet, but there’s a ticket on it https://issues.scala-lang.org/browse/SI-9684 that’s targeted for 2.12. We just haven’t gotten around to it yet.

Hey open source! Would someone out there like to put together a pull request for this…? It would be welcome. https://github.com/scala/scala/pulls

Seth