On the cryptic error messages, if you compare messages produced by
2.7.x, 2.8.x, 2.9.x and 2.10.x, you'll see there has been steady
improvement. String representations of types have been altered to
better show how different types differ (for example, tuples vs
multiple parameters), common issues now come with suggestion for the
usual cause/solution, failed implicit resolution gained an annotation
so that the implicit author can write the error message, and so on.
IMHO, error messages on 2.9.x cannot be called "cryptic" anymore, with
a few exceptions. Nowadays, questions about error messages on Stack
Overflow are usually answered with "Yes, the problem is exactly what
the error message is saying, and the solution too." If you (or anyone
else!) feel otherwise, perhaps it is time to put forward examples of
error messages that are still deemed cryptic.
Mind you, one can get error messages that are difficult to comprehend
when one uses advanced features such as the ones Martin wants to put
behind a fence with SIP-18, but the problem is not so much the error
message, but the code that has been written.
With regards to compiler speed, this has been a constant focus of Paul
Phillips for as long as I have been involved with Scala (and I assume
before that as well), and there have been and there are huge efforts
to improve it. The whole interruptible presentation compiler is one
part of that effort, as is the better incremental compilation of the
next version of Scala IDE.
But, ultimately, there are hard limits on how much it can be improved.
JVM cannot represent many concepts in Scala, which makes it necessary
for the compiler to do more work than javac has to. Traits, for
instance, means tracing class file dependency is much harder for Scala
than for Java. Methods provided through implicit conversion are much
slower to compile than methods provided by the static type. Implicit
parameter search is inherently slow, and type inference is hard.
One can improve compilation speed of one own's code by specifying
types, avoiding methods provided implicitly, reducing number of
implicits visible as much as possible, as well as implicit usage.
As for stack traces, there's nothing the compiler can do about that,
and the JVM, again, doesn't help: it doesn't have a field for column
range of code, for example. External tools will have to be developed
to deal with it, and the present effort for that is on Scala IDE.
Given the number of improvements that Scala IDE must gain, I fear the
priority for this is just not there.
--
Daniel C. Sobral
I travel to the future all the time.
One thing that I have noticed about the attendees of these
gatherings it that they generally have a much stronger computer
science background than most Java programmers. I wonder if
appreciating the advantages of Scala requires a deeper knowledge of
computer science than most Java programmers have.
I wish it were so easy. :/
I'm skeptical that SBT is a silver bullet - my experience with it hasn't
been entirely positive** - but even if it is, switching to SBT isn't an
option for many projects.
Where I work, we've been introducing Scala bit by bit into a
decent-sized multi-module Maven project. We can't - and don't want to!
- switch to SBT. This is a common scenario.
Scala compilation speed is a very big issue because it has wide-ranging
effects. For example, using the latest Scala IDE for Eclipse, I
frequently get modal popups because I'm editing several files (or just
the same file) too quickly, because builds triggered by one change
aren't finished when the next build is triggered. The Eclipse plugin
uses SBT for dependency tracking, as I understand it, and (in theory)
should be recompiling only as many files as SBT would from the command
line. :(
** This is not a knock on SBT. Dependency management and
build-wrangling for large multi-module projects is inherently complex,
and probably won't ever be pretty.
With regards to speed, though, I think it has already been said that
it won't ever match the speed of javac. Java is a very simple language
compared to Scala and the fact that source file names map to class
names makes it trivial to find sources to compile when doing
dependency analysis; Scala on the other hand allows you to name your
files as you like and cram several classes in a file, so tools like
sbt must be aware of that when recomputing dependencies.
That being said, if speed is the utmost concern, there are other
languages that run on the JVM that share some similarities with Scala.
Kotlin by JetBrains, Xtend by the Eclipse Foundation and Ceylon by
RedHat probably all compile much faster than Scala. In fact, one of
the design goals of Kotlin is to compile at roughly the same speed as
javac while being "simpler than Scala."
There might also be an issue with the command line parameters for sbt;
I know that on my machine, sbt often fills the entire permgen when
compiling a Play 2 app and then dies there with the VM needing to be
kill -9'ed. By default, it will only run with -Xmx512M, but bumping
the permgen a bit and running with -client (on Mac, since -server is
horribly broken there) makes a big difference. YMMV. Try checking what
happens inside of sbt using VisualVM and see if there's anything
needing to be fixed.
I wouldn't complain though if the Scala compiler was faster. It should
also come with free cupcakes, courtesy of Typesafe. :)
For me, the biggest productivity win for Scala is its support for
multiple programming paradigms.
For compilation time, it more or less is. SBT has all kinds of other problems, but the maven scala compiler is double-plus ungood.
Of course you can switch to SBT. You are using the same compiler. You mean you don't want to make the investment, and that is a sentiment I can understand. We stuck around with maven for more than a year before we switched to SBT.
Geir
On 03/30/2012 10:16 AM, Aleksey Nikiforov wrote:1. Using SBT resolves 95% of slow compilation issues. Giving more ram to compiler also helps.I wish it were so easy. :/ I'm skeptical that SBT is a silver bullet - my experience with it hasn't been entirely positive** - but even if it is, switching to SBT isn't an option for many projects. Where I work, we've been introducing Scala bit by bit into a decent-sized multi-module Maven project. We can't - and don't want to! - switch to SBT. This is a common scenario.
Scala compilation speed is a very big issue because it has wide-ranging effects. [...]
-- Francois ARMAND http://fanf42.blogspot.com http://www.normation.com
But 1000 lines a second is not bad at all if you are smart about it. A
Play template is maybe 100 lines, so that should be 1/10sec, not the
reported 13 sec. The main challenge here is (1) keep the compiler
warm, and keep as much info as is safe loaded, and (2) make that warm
compiler available in as many environments as possible. (1) is
essentially solved by fsc. Simply call fsc instead of scalac from the
command line, and you are set. Or use fsc for IntelliJ. sbt is a bit
more conservative and therefore a bit slower, but still much faster
than compiling from scratch with scalac. The main problem is that if
you use other build tools, like maven, then you are stuck, and have to
suffer really bad build times.
We are currently working on making incremental compilation available
to maven and other build tools. Hopefully that will be a constructive
solution to the issue.
Cheers
- Martin
As a reference, have you any idea of the speed of javac (so that I could
have a feeling of the gap) ? 10 000 ? 100 000 ? Even more ?
> [...] We are currently working on making incremental compilation available
> to maven and other build tools. Hopefully that will be a constructive
> solution to the issue.
That's awesome ! Hope it would
I often use sbt with mountains of scala code -- a project depending on
internal libraries depending on scalaz, etc. I rarely wait longer than 3
or so seconds for compiler feedback. The incremental compilation helps
immensely, so long as you have enough memory.
--
Tony Morris
http://tmorris.net/
That's great news! Any place we can read about/test improved build-times
for a multi-module Maven scala-project?
--
Andreas Joseph Krogh<and...@officenet.no> - mob: +47 909 56 963
Senior Software Developer / CEO - OfficeNet AS - http://www.officenet.no
Public key: http://home.officenet.no/~andreak/public_key.asc
Given that Odersky has written the basis to the current javac
(http://www.scala-lang.org/node/241), I think it should be taken for
granted that there _are_ inherent reasons for the huge performance
gap.
As someone who reads the e-mails and commits by Paul Phillips in his
quest for the holy speed, I can also tell you that serious work has
been put in improving that speed. All the same, anyone can grab
profilers and other performance tools, see what's going on with the
compiler, and take a shot at getting a few more seconds out of it. I
can't think of any commit I'd rather have in Scala than one that
begins with "Improve compile speed". :-)
>
> /Jesper Nordenberg
--
thanks and keep up the good work,
wade
We are currently working on making incremental compilation available
to maven and other build tools. Hopefully that will be a constructive
solution to the issue.
(2) make that warm
compiler available in as many environments as possible.
Compiler has been running in parallel for many years now. Trying compiling a reasonable size project and watch your task manager.
Nor was scalac 1.x. :-)
Your memory is fine. scalac doesn't parallelize compilation of files.
SBT, however, can parallelize compilation of independent sub-projects.
-jason
Similar complexity? OCaml's type system isn't even turing complete!
-jason
For me, the compilation speed of a single file is not that bad, it's
the recompilation of the "solution" that is problematic. The
turnaround time for making a simple change in a single file often
causes a compilation of most files in the project.
I think one of the main issues is that even with SBT, the incremental
compilation compiles too much (some details here
https://groups.google.com/forum/#!msg/scala-ide-user/h00tWkpw1Lk/ZFZW4XUc1QoJ)
Add to that use of e.g. the cake pattern, suddenly everything depends
on the cake and even minor changes causes full recompilation.
I'm seriously thinking of adding e.g. Guice just to break this compile
time dependency.
/Jeppe
No. OCaml lacks dependent types and a proper kind system. Also,
subtyping is not ubiquitous in common OCaml programs. It may have row
polymorphism, but we have something similar in Scala, too.
Well, with first-class modules, you do have some ways to encode GADTs,
so I'd be quite cautious with these kind of assertions.
> and a proper kind system. Also, subtyping is not ubiquitous in common
> OCaml programs. It may have row polymorphism, but we have something
> similar in Scala, too.
Generally I believe Scala's type system to be richer, however, the real
differences in type systems, aside from usability issues, are in my
opinion:
-1- lack of any implicits in OCaml (which comprises Scala's version of
type classes)
-2- less pervasive inference in Scala (although type inference
frameworks such as ML^F may in the long run bridge the two approaches,
as far as I understand it).
Anyway, to me, it's comparing apples and oranges.
--
Guillaume Yziquel
Crossing-Tech
Parc Scientifique EPFL
It isn't? I means parsing a single line can take arbitrarily long,
which is certainly not the case for any rational non-turing complete
type system.
For example, parsing line 42 of https://gist.github.com/66925#L42
involves producing the answer to the Hanoi Towers problem with four
discs. And while that is a gross example of abuse of the type system,
the mechanism used in it is very common, and can easily produce long
searches to solve.
>
> In fact, I think it requires in general more features from the type system
> to
> be expressive enough but not Turing complete at the same time.
--
So maybe I could reorganize my code to compile faster, but I don't
want to have to know enough about the compiler to know what kinds of
designs and source files it can compile at a reasonable pace.
The issue, is, from the first e-mail, "1. Compile time is super slow". So:
> 1) The run-time complexity of type checking
> 2) The (informal) complexity of language features
You said that "OCaml is a language of similar complexity to scala and
it compiles much quicker (>10K lines/sec)." I'm showing an example of
how compiling a single line can take arbitrarily long, a feature which
is directly derived from the fact that Scala's type system is turing
complete.
>
> (BTW it is not the "parsing" of line 42 that takes long, but its type
> checking.)
I misused "parsing" to mean everything from lexing to code generation.
I thought of using "typer", but the culprit here is implicit
resolution, which might be done at the typer phase, but is not what
people usually think of when they think "type checking".
> In both languages you can have exponentially slow few-liners that are
> exponentially slow to type-check in the input size.
So look at the program again. It's speed is not related to input size:
I can make it slower or faster without changes to the input size.
That's what I mean when I say "arbitrary".
hey, i think the 1980's are calling and want their c++ metaprogramming back!
You couldn't do metaprogramming with standard C++ back in the 80s.
You'd have to go to the 90s for that.
Besides, metaprogramming is boring. Metaprogramming is what Forth
programmers call "writing code". Metacompiling, now, that's fun.
yeah, and Forth doesn't get adopted by enterprise, either.
but for some strange reason, C++ is a shoe-in.
what is up with that?
If you're implying that writing FORTH is boring then I would have to
disagree. I can think of other adjectives, but never boring.
-- Erik
> If [...] implying that writing FORTH is boring then I would have to
> disagree. I can think of other adjectives, but never boring.
Boring not FORTH is. FORTH was actually used in the enterprise, sort of—the grand enterprise to boldly go where no one has gone before.
http://solarsystem.nasa.gov/galileo/
"Galileo plunged into Jupiter's crushing atmosphere on Sept. 21, 2003. The spacecraft was deliberately destroyed to protect one of its own discoveries - a possible ocean beneath the icy crust of the moon Europa."
http://www.flownet.com/gat/jpl-lisp.html
"... in 1993 I used MCL to help generate a code patch for the Gallileo magnetometer. The magnetometer had an RCA1802 processor, 2k each of RAM and ROM, and was programmed in Forth using a development system that ran on a long-since-decommissioned Apple II. The instrument had developed a bad memory byte right in the middle of the code. The code needed to be patched to not use this bad byte. The magnetometer team had originally estimated that resurrecting the development environment and generating the code patch would take so long that they were not even going to attempt it. Using Lisp I wrote from scratch a Forth development environment for the instrument (including a simulator for the hardware) and used it to generate the patch." [Ron Garret]
On 30/03/2012 10:48, amulya rattan wrote:
> Cool developers,
>
> I must preface this by saying I absolutely love Scala and frameworks
> that are built on it. I wanna marry them all, move to south of France
> and start a band named "Scalalala", dont ask why. But there are some
> serious _serious_ __serious__ problems with it that are definitely
> avoiding a fast adoption of Scala everywhere.
>
> On the face of it, comparison of Scala and Java leaves no doubt that
> one should make a good investment in the future and that future is
> Scala. One reads a Scala book, scribbles few lines of code on REPL and
> goes, "This is so cool! I'll go tomorrow and introduce my team to some
> of this awesomeness". Learning/Understanding its intricate Type system
> feels like an acid trip, and a good one at that. One comes out a
> better man. Implicits are yummy. Traits are simple and powerful.
> Introduction of Typeclasses from Haskell is more than welcome. Merging
> of OO and FP to seamlessly transition from one style to other is neat.
> Frameworks like Scalaz can be mind-bending, requiring a lot of effort
> and diligence, but you know if you apply yourself, you can understand
> it. And dont even get me started about Akka, its a work of art.
>
> Wow, so many positives!! So why isn't everyone investing their
> resources into it?? Here are few silly/rebuttable reasons/arguments:
>
> 1. Some people are just adverse to change: And you can't do anything
> about it. They want to continue living in null land and so be it.
> 2. Some people are just BIG java fans and don't want to use anything
> else: To them Java, like many other languages, is like a religion and
> they don't wanna convert. Java is turing-complete and you can do
> everything in it, just more verbosely and less runtime-safe-ly.
> 3. Some people make effort but fallback on "Scala is too complex" cuz
> they read it is somewhere: Well this has come up many times before and
> doesn't need any more argumenting. For my 2 cents, Complex is just
> intertwining of a lot of simple things. Plus since most programmers
> are engineers and hence smart people, one would expect them not to shy
> away from complex but embrace it, break it down and see what sort of
> scary organism is living down there. However, thats not how the world
> works. And there is a flip-side to it, since some people would love to
> understand the "hairy" concepts but are busy otherwise.
>
> However, here are few arguments that are legitimate and definitely
> something that Scala team and contributors, perhaps need to
> contemplate on:
>
> 1. Compile time is super slow - And yes, running a seperate server
> especially for compilation doesn't really help a lot. It doesn't. And
> its not a single incident. Bigger your code base is in scala, worse
> are the times. I am afraid to pitch Scala for our products, since I am
> sure he'll show me this http://xkcd.com/303/ I thought we were out of
> C++ compilation era. I am sure scala compiler has to perform a lot of
> magic to convert to optimized bytecode and go through all its 25
> elaborate stages. However, instead of working on all other cool
> features added to Scala for now, couldn't one release be dedicated to
> making compiler faster. Or am I just asking for too much? Also:
>
> Recently Play framework moved to 2.0 with a very aggressive
> introduction of Scala in it and the response has been pretty much
> violent. If you have been following play's mailing list, you'd know
> what I am talking about. First comes the "scala is a nice language but
> too complex" argument which looses Play a lot of territory with early
> adopters moving to Django. Next people who are trying to use Scala are
> faced with horrible compile times in DEV mode. Here is a guy with
> similar issue of how using Play with Scala took all the speed away:
> https://groups.google.com/forum/#!msg/play-framework/AcZs8GXNWUc/IanbqC-c-MkJ
> <https://groups.google.com/forum/#%21msg/play-framework/AcZs8GXNWUc/IanbqC-c-MkJ>
>
> Now this guy at least tried using scala, but failed not because scala
> is complex but because Scala has very slow compilation. And he
> presents a _very_ legitimate problem that other organizations are
> facing or will face if they started using scala unless Scala team does
> something about it.
>
> 2. Cryptic errors - Everyone has seen them so I don't need to elaborate.
>
> 3. Those run time stack traces just leave me hanging
>
> These three arguments are as legitimate as anything can be and as far
> as I know, every Scala developer has seen them. And they are all
> compiler-magic centric. And while Scala is all about about
> compiler-magic, does it leave no scope for optimization or errors that
> are more _day-to-day_ developer friendly than _rockstar_ developer
> friendly. Not everyone who is a programmer is super-smart and can go
> around those issues using their black-magic skills. A day-to-day
> developer in an organization wants to get through a task as fast as
> possible because that's what business end demands off of him/her.
> Software engineering doesn't exist on its own, there is a big layer of
> business on top of it and biggest business need is smallest
> Time_To_Market.
>
> If Scala needs to take off as it aspires to or as it should, it needs
> to cater to day-to-day developer. The developer who just wants a fast
> compile time, a error explaining what he did wrong or what could go
> wrong and a stack trace explaining what kind of muck he created. Not
> every organization would want to have a Typesafe subscription to
> ensure they avoid scala related issues. They expect their everyday
> developer to take care of language-related issues.
>
> Now I spent my time writing this to extract what the other developers
> in scala feel about this and to know if Scala people do consider this
> a major issue and have a plan to solve it very soon or dont know of
> any existing solution to it or simply dont give a muck about it.
>
> Expecting some responses.
>
> ~Amulya
What telescope did you work at ;-)
Cheers, Eric