(empirical) Study on Scala vs. Java: Scala takes slightly more time, results in slightly less code.

[Reinier Zwitserloot]

http://www.neverworkintheory.org/?p=375

Findings:

* It takes longer to write the same thing in scala than in java, but the difference is not that much.

* The end result is less lines in scala than in java, but the difference is not that much.

* Programming skill was not a statistically relevant influence on how well people managed to handle the scala side of the experiment. This goes against widespread belief (including what I thought, and will continue to presume might be true until more studies come out, but it has tempered this belief for me) that scala is less suited to Joe 'I just work here' programmer.

* Scala code is just as performant as java code. (Is anyone who knows how VMs work and what scala does under the hood surprised by this?)

It's just one study, and it is notoriously difficult to study effectiveness of programming languages. Nevertheless, some highlights which your random 'wow, I switched to (newlang) and my code is like 10% in size and I can write it 5 times faster' anecdote failed to do:

* Half the group, at random, programmed java first and scala second, and the other half went the other way around. The considerable benefit of having experience with the project domain in the second go around is thus balanced.

* The subject base is 13 people, not 1.

* They measured things. No part of what this summary publishes involves asking the test subjects any subjective questions. 

Of course, there are also some parts I noticed that aren't as nice:

* It's a very specific domain: Implement this 17-page VSLI layout algorithm specificatino (essentially: Here are a truckload of little regions that all need to be fit and interconnected on a chip. Please write an algorithm that'll pack it all in on the smallest possible surface. I'll actually tell you how to do it in this 17 page document, but you have to translate it to code). There are a million-and-one problem domains in programming, and I wouldn't at all be surprised if the results for i.e. code size in domain A is 50% in favour of scala (scala code half size), and in domain B it's actually the other way around. So, we can extrapolate the results of this to programming in general, or is this primarily useful only if you are considering a project that is similar to writing VLSI layout algorithms?

* The 13 test subjects had 4 years of java experience coming into this project, and no scala experience. They were, fortunately, given a warmup assignment in scala to try and eliminate that part of the learning process, but was it enough? Then again, if you're a java programmer trying to figure out if you should switch, this part of the test means it should be more accurate for you, so there's that.

[Kevin Wright]

The line that most caught my attention was:

"One hypothesis (based on interviews) is that Scala’s automatic type inference actually made debugging more difficult."

Anecdotally (I didn't do a study), this is absolutely true!  One practice that I'm seeing repeated over multiple teams is to explicitly specify a type on a method whenever it isn't immediately obvious to a casual reader.  Amongst other things, this helps the compiler catch any mistakes you make, it also speeds up compilation.

This typically doesn't happen until a team is properly "bedded down" and has begun to establish some best practices.  In my experience, 4 weeks is not enough to establish such a level of group intelligence.

[Matthew Farwell]

Salut,

This is one of the reasons I introduced a check into scalastyle for public methods that have an inferred return type. The ostensible reason is that public methods form part of an interface, so if you change the return type by 'accident', it can have unintended consequences, but it does help with understanding the code as well.

Matthew Farwell.

2012/8/28 Kevin Wright <kev.lee...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Java Posse" group.

To post to this group, send email to java...@googlegroups.com.
To unsubscribe from this group, send email to javaposse+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/javaposse?hl=en.

[Kevin Wright]

Speaking of which...

Programming languages nowadays are rarely used in isolation.  The size of code and time taken to write it is determined in no small part by the libraries/frameworks you'll be using.

For me, much of the joy in Scala comes from the growing ecosystem as much as from the language itself, and I'm finding libraries written in Scala to be far more powerful and usable than libraries written in Scala.

A lot of this comes from having a chance to learn from past experience, and can be seen even without changing languages (e.g. log4j -> slf4j, calendar -> JodaTime).  But there's also a lot which comes from the richer type system, from symbolic method names, and from the potential for DSLs.

I'm still not sure if it would be possible to craft a study that can compare two languages, *with their ecosystems*, and not be in some way biased.

[Kevin Wright]

correction: "I'm finding libraries written in Scala to be far more powerful and usable than libraries written in Java."

[Artie Peshimam]

The other question I have is just how idiomatic is the code that was produced for this effort. I would guess that it's probably quite imperative with hints of functional programming by using things such has mapping, folds, Options, etc. I've been working with scala on a daily basis and while it's probably taken me longer than the normal person, Im starting to really grok idiomatic scala at this point. 4 weeks training vs. 4 years of java experience. Im not sure how this is even comparable. 

[Ricky Clarkson]

It's not comparable, but it is realistic. If a manager is trying to
work out how difficult it would be to move over to Scala this is a
useful data point, though it needs information about what happens
after months or years to give a full picture.

This tells us that moving from Java to Scala does not cause an instant
halving in productivity such as you might find when moving from C++ to
Java or Python to Ruby. Not that in either of those cases the
productivity would stay at half, but it's feasible that you'd see a
big drop during transition.

Even with experienced Scala programmers, if the spec is by its nature
quite imperative ("to do this, first do that then do the other") then
you might not gain much from Scala. E.g., I bet there'd be little to
be gained by implementing MPEG-4 in Scala instead of Java or even C.
You'd get more if the spec states *what* must be calculated but not
*how*, and even more so if it can be implemented in terms of
collections (or other monads) in some way.

> --
> You received this message because you are subscribed to the Google Groups
> "Java Posse" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/javaposse/-/elr5HjIIogcJ.