What is highest priority for Scala to succeed in corporate world (Should be in scala-debate?) ?

1205 views
Skip to first unread message

Ken McDonald

unread,
Nov 5, 2011, 5:51:52 PM11/5/11
to scala...@googlegroups.com
Even though this perhaps should be in scala-debate, I'm posting it here because:
    1) It's really very important.
    2) I've never seen a similar question posted.

This comes after reading a few of the "White elefant" posts.

The question is simple: List, in descending order of priority, what you feel needs to be addressed/fixed/whatever for Scala to succeed in the corporate (and hence IMHO ultimately the "real") world.

My list:

1. IDE Support. Scala must have rock-hard support in at least one IDE, and must have "pretty good" support in Eclipse, since it's the de facto standard. I don't think we're close yet. This alone could lose Scala the language wars.

2. Documentation. Scaladoc documentation needs to be expanded on greatly; at the same time Scaladoc itself could stand some enhancements, though exactly where is less obvious. I am complicit in this; I keep on intending to devote some serious time to enhancing scaladoc, and keep failing to do so, dangit!

3. A "Scala Cookbook". I'm amazed one isn't out or on the web already. I know, there's lots on stackoverflow and other sites but it's not the same. Scala is such a neat language, that a newcomer could easily get caught up and read for hours on a good cookbook, saying things like, "Wow, I can do it in just one line...".


And your suggestions? Recommended three or less because probably no one will read past the third one :-).



Ken

Rex Kerr

unread,
Nov 5, 2011, 5:58:37 PM11/5/11
to scala...@googlegroups.com
I think your list is exactly right; if anything I'd just emphasize the first two points.

On Sat, Nov 5, 2011 at 5:51 PM, Ken McDonald <ykke...@gmail.com> wrote:
Even though this perhaps should be in scala-debate, I'm posting it here because:
    1) It's really very important.
    2) I've never seen a similar question posted.

This comes after reading a few of the "White elefant" posts.

The question is simple: List, in descending order of priority, what you feel needs to be addressed/fixed/whatever for Scala to succeed in the corporate (and hence IMHO ultimately the "real") world.

My list:

1. IDE Support. Scala must have rock-hard support in at least one IDE, and must have "pretty good" support in Eclipse, since it's the de facto standard. I don't think we're close yet. This alone could lose Scala the language wars.

I think Scala must have rock-hard support in _most major IDEs_.  This includes both Eclipse and NetBeans; IDEA is also a good candidate since it seems pretty advanced already.  If I had to switch my IDE as _well_ as my language, that would be an additional barrier, I'd think.
 
2. Documentation. Scaladoc documentation needs to be expanded on greatly; at the same time Scaladoc itself could stand some enhancements, though exactly where is less obvious. I am complicit in this; I keep on intending to devote some serious time to enhancing scaladoc, and keep failing to do so, dangit!

Indeed.  Scala's core API should be as well documented as Java's.  I rarely find that the Java documentation says far more than I need, but I often find that it just barely says enough for me to do something useful.  In Scala, it's often several hours of poking around, looking at source code, blog posts, and StackOverflow, before I figure out something nontrivial that I haven't used before.

  --Rex

HamsterofDeath

unread,
Nov 5, 2011, 6:19:25 PM11/5/11
to scala...@googlegroups.com
all my votes go to "ide support". and a little tiny enhancement to the jvm: i want to be able to write List(1,2,3).filter(_ == 2) in my debugger

never had a problem with documentation. maybe i have magic eyes or i am a google prime customer, but i never have to search long until i find an answer.

Matthew Pocock

unread,
Nov 5, 2011, 7:34:07 PM11/5/11
to scala...@googlegroups.com
I agree with your list. I'd probably add making the core libs (particularly thinking of collections) faster and lower-memory/burn. This probably includes making specialization work (better?) and aggressively inlining away typeclasses. When performance really does matter there is no substitute for fast code. If the scaladocs are beefed up, I'd probably place performance at 3 in place of the cookbook.
--
Dr Matthew Pocock
Integrative Bioinformatics Group, School of Computing Science, Newcastle University
skype: matthew.pocock
tel: (0191) 2566550

Andreas Flierl

unread,
Nov 5, 2011, 8:00:53 PM11/5/11
to scala...@googlegroups.com
Hi,

Ken McDonald wrote:
> 1. IDE Support. Scala must have rock-hard support in at least one IDE, and must have "pretty good" support in Eclipse, since it's the de facto standard. I don't think we're close yet. This alone could lose Scala the language wars.

What other JVM language (Java aside) has better IDE support than Scala?

(Also, I find "language wars" a strange term.)

Kind regards
Andreas

Tony Morris

unread,
Nov 5, 2011, 8:03:57 PM11/5/11
to scala...@googlegroups.com
On 06/11/11 10:00, Andreas Flierl wrote:
> Also, I find "language wars" a strange term.

You must accept legitimacy of the idea of "language wars" in order to
succeed in a corporate environment. If you are having difficulty with
the self-deception required to adopt this illusion, try calling it "real
world" -- HTH.


--
Tony Morris
http://tmorris.net/


Erik Engbrecht

unread,
Nov 5, 2011, 9:08:03 PM11/5/11
to scala...@googlegroups.com
I think you need to define what "rock-hard support" in an IDE means.  For example, the Eclipse plugin keeps on moving forward in terms of functionality, but where exactly is the version of it for Scala 2.9.1?  It also consumes a ton of memory and tends to "just sit there" a lot.  It's basically unusable in my work environment, while the version quite a while back, while buggier and with far fewer features, was usable.  I think the NetBeans plugin also took a step sideways when it integrated with the presentation compiler.  In my opinion the only really usable IDE for Scala right now is Emacs+ENSIME.  Maybe the ENSIME integrations for Vim and JEdit work well, too.  I'm fine with Emacs, but I wouldn't try to force in on anyone.



Brian Schlining

unread,
Nov 5, 2011, 9:52:29 PM11/5/11
to scala...@googlegroups.com

The question is simple: List, in descending order of priority, what you feel needs to be addressed/fixed/whatever for Scala to succeed in the corporate (and hence IMHO ultimately the "real") world.


1) The speed of the compiler.
 
--
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Brian Schlining
bschl...@gmail.com

Thorpe, Michael

unread,
Nov 5, 2011, 10:17:51 PM11/5/11
to Brian Schlining, scala...@googlegroups.com
That basically. 

William la Forge

unread,
Nov 5, 2011, 10:54:29 PM11/5/11
to scala-user
When you talk about real world, it is important that some of those very nice features of the language which are noticably slower than what a novice would expect should be de-emphasized a bit and come with a bit of a warning. Nothing wrong with using them, but some awareness of the costs of these features will save a lot of work tracking down why a piece of code is so slow and will help a lot in preventing the impression that Scala is a purely academic language.

These "nice" features then shouldn't be the first thing a Scala novice reads about, even though they demonstrate the elegance of the language.

Bill

Michael Schmitz

unread,
Nov 6, 2011, 12:34:36 AM11/6/11
to William la Forge, scala-user
+1 rock-hard IDE support. When I developed in C# on .NET in a
corporate environment, I wanted to use F# (while it was beta) for
experimentation but the overhead of learning a new language alongside
IDE problems was too much. This also shows how hard it is to create
solid IDE integration since Microsoft controls Visual Studio and
releases both C# and F#.

I almost didn't use Scala because of how buggy ScalaIDE used to be.
It's a lot better now, but if it were as solid as Eclipse Java support
it'd be easy for someone to switch over and start adding Scala to
their projects.

Of course, rock-solid IDE support relates to the speed of the compiler!

Peace. Michael

Eric Kolotyluk

unread,
Nov 6, 2011, 12:58:46 AM11/6/11
to scala...@googlegroups.com
My humble take on things...
  1. The very real fear of Scala needs to be addressed. I cannot remember when I have ever seen such a controversial programming language and there just seems to be so many "Scala is too complex" concerns raised by too many people.

  2. IDE Support. I still struggle with installing new versions of Scala with Eclipse and run into obscure errors that I would never tolerate in Java. To coin a phrase from Steve Jobs, when he was creating NeXT, "it just works." Scala in the IDE has to be close to as dependable as Java in the IDE - it just has to work - no exceptions, no excuses. Troubleshooting Scala plug-in problems is a very serious turn-off and makes the technology seem like it is still an academic curiosity.

  3. Every time I delve into the Scaladocs I cringe at the complexity - it makes me feel stupid, as in, I am not intelligent enough for this.
Cheers, Eric

Simon Ochsenreither

unread,
Nov 6, 2011, 8:16:03 AM11/6/11
to scala...@googlegroups.com
I think we really need SIQ or whatever we call it shipping as soon as possible.

Being able to say

here are the hundreds of lines of code you need to work with a database:
  • you need some classes for ORM,
  • dozens of annotations to describe the connection between them
  • dozens of lines for simple type safe queries
  • you need to know the quirks of Hibernate/JPA/EclipseLink and
  • you probably still need to write SQL for more complex tasks

and here is how you do that in Scala, with a single line of code:

<example>

I think if we would have this, we have more or less won the whole enterprise stuff.

Erik Engbrecht

unread,
Nov 6, 2011, 10:13:43 AM11/6/11
to scala...@googlegroups.com

On Saturday, November 5, 2011 8:00:53 PM UTC-4, Andreas Flierl wrote:

(Also, I find "language wars" a strange term.)

A corporate environment with enough legacy (basically meaning "it's been around for a while") will almost certainly have a contingent of "defenders of the status quo" for any given technology that's had some success.  This is especially true if some influential individual(s) have linked their personal success to the success of this technology.  In a large enough environment there will likely be several of these groups, and they will fight with each other as much as with anything that's new.  On one hand this is all well and good, because change costs serious time and money, and often multiple overlapping technologies exist for a reason, so to the extent that fights stay on technical grounds they can be productive.  On the other hand, the fights often leave the rational world and exist almost entirely in the realm of politics.

A corporate environment with enough R&D investment in enabling technologies will almost certainly have one or more contingent of "evangelists of the new doodad."  Their aggressiveness will be roughly equivalent to the square of the amount of money invested in their doodad divided by the amount of success it has obtained.  On one hand this is can be productive, so long as the evangelists stick to technical grounds.  On the other hand, once a given technology has passed some degree of investment, the evangelism starts leaving the technical realm and becomes mostly political.

A "technology" could be a programming language, web framework, operating system, process, tool, semiconductor substrate, etc...

Now, the poor sap who has to actually create a product (meaning most engineers) has to contend with all these people who have opinions about what technologies he's supposed to use.  Furthermore, if he decides to toss his own competing technology into the mix because he thinks it fits his problem better than the ones associated with existing factions, he's going to face challenges from all sides.

If you're at a startup you probably don't have to worry about this much.  If you're at a Fortune 100 company (or one is your customer), you probably have more legacy hanging around than is humanly comprehendible and a fair bit of R&D, too.

If this poor sap is trying to use Scala, then he is "fighting a battle" for Scala in the "language wars."  Calling a bunch of nerds and managers arguing a "battle" in a "war" seems to involve a high amount of hyperbole, but it's a popular analogy none-the-less.

Anyway, if you're trying to use Scala, you could follow Tony's advice:

Except that I think the barriers identified in this thread represent rational reasons for not using Scala at this time, so rational objections will inevitably be mixed in with all the political nonsense.  You should also remember the Golden Rule: He who has the gold makes the rules.  Sticking to principles works pretty well so long as the funding isn't cut off, so you can dismiss the irrational objections of others so long as the person with the gold is doing the same.

Of course, if you dismiss all these objections and run with Scala, and then the project fails, you run a high risk that Scala will be blamed regardless of actual fault, and you will be blamed as the person who ignored the wisdom of others and charged ahead with an immature technology.

HamsterofDeath

unread,
Nov 6, 2011, 11:22:35 AM11/6/11
to scala...@googlegroups.com
that already exists, it's called db4o :)

Simon Ochsenreither

unread,
Nov 6, 2011, 1:00:21 PM11/6/11
to scala...@googlegroups.com
Which no one uses ...

I certainly don't want to choose a database based on the API.

HamsterofDeath

unread,
Nov 6, 2011, 1:14:21 PM11/6/11
to scala...@googlegroups.com
Am 06.11.2011 19:00, schrieb Simon Ochsenreither:
> Which no one uses ...
>
> I certainly don't want to choose a database based on the API.
you can always fall back to just saving hashmaps using an
objectoutputstream.

there is no solution which is both simple and efficient. you can have
one or the other.

martin odersky

unread,
Nov 6, 2011, 3:30:00 PM11/6/11
to scala...@googlegroups.com
On Sat, Nov 5, 2011 at 10:51 PM, Ken McDonald <ykke...@gmail.com> wrote:
Even though this perhaps should be in scala-debate, I'm posting it here because:
    1) It's really very important.
    2) I've never seen a similar question posted.

This comes after reading a few of the "White elefant" posts.

The question is simple: List, in descending order of priority, what you feel needs to be addressed/fixed/whatever for Scala to succeed in the corporate (and hence IMHO ultimately the "real") world.

My list:

Your list is pretty much spot on as far as I can tell. Here's what we (Typesafe & EPFL) are doing about it.
 

1. IDE Support. Scala must have rock-hard support in at least one IDE, and must have "pretty good" support in Eclipse, since it's the de facto standard. I don't think we're close yet. This alone could lose Scala the language wars.

Typesafe now has 3 FTEs working on the Scala Eclipse IDE, plus 2 part-timers (one of them myself). None of this is paid for by anyone. We are doing this because we think its important, and universities can't do it. It's grunt work. Have you tried a recent beta of Scala IDE 2.0? If you have serious problems we'd like to see a ticket!

Also, I realize that even with this amount of committed resources we are still moving too slowly. So if you have time & expertise to contribute, this would be hugely appreciated!
 
2. Documentation. Scaladoc documentation needs to be expanded on greatly; at the same time Scaladoc itself could stand some enhancements, though exactly where is less obvious. I am complicit in this; I keep on intending to devote some serious time to enhancing scaladoc, and keep failing to do so, dangit!

Heather Miller and Josh Suereth have recently released http://docs.scala-lang.org/. Now there's no more excuse not to contribute!

3. A "Scala Cookbook". I'm amazed one isn't out or on the web already. I know, there's lots on stackoverflow and other sites but it's not the same. Scala is such a neat language, that a newcomer could easily get caught up and read for hours on a good cookbook, saying things like, "Wow, I can do it in just one line...".

I think Cay Horstmann's excellent "Scala for the Impatient" comes close. A preprint of the first half is available for free from the typesafe.com site.

Cheers

 -- Martin

Simon Ochsenreither

unread,
Nov 6, 2011, 3:49:33 PM11/6/11
to scala...@googlegroups.com
Did you have a look at the things happening with SIQ/...?

1. One definition of the database url and the credentials
2. .fromDb
3. Use the API known from the collection stuff

That's how it should be done.

Luc Evers

unread,
Nov 7, 2011, 4:56:09 AM11/7/11
to scala...@googlegroups.com
  Hi, Ken

  Scala Cookbook , indeed a missing book.
  I should extend your list with :

  - Scala Swing book
  - Scala RIA?
       Not bad is Scala Vaadin but no real good information.
  - Scala Android developer book.

   Luc.

Peter Walkley

unread,
Nov 7, 2011, 5:54:09 AM11/7/11
to scala...@googlegroups.com
Rock solid IDE and maven support.
 
My experience so far is that the maven plugins all work absolutely fine - BUT so much emphasis (here and the lift forum) is around sbt that it is very easy to pick up the misunderstanding that you need to switch build tool.  That would be an automatic fail in many enviroments as maven is so heavily entrenched. Configuring builds is boring crap no-one wants to do twice.
 
For this reason, I'm making a deliberate point of not using sbt as I climb the scala learning curve. Should I find a bug, I'll be raising it very promptly !
 
There's some points in a similar question on linkedin that may be of relevance: Is scala the future of java

Bastian, Mark

unread,
Nov 7, 2011, 10:50:16 AM11/7/11
to scala...@googlegroups.com
IDE support is far and away #1 for me. This comes from two standpoints:
  1. I like a good IDE, even though Scala has made me less IDE-dependent than I used to be since I use the repl and other iterative tools so much more than I used to.
  2. Even more important, when I am trying to get other developers to adopt Scala the first thing they want is a good IDE plugin. Getting others to try out Scala requires a gentle introduction. The first thing they want to do is write Java-as-Scala in their IDE of choice, not dive into FP. From what I have seen, the Eclipse plugin has come a long way towards making this happen (good job, team).
As an aside, it seems like "better IDE support" is frequently translated as "better Eclipse support." However, it seems that many experienced Scala developers use some sort of "Glorified Code Editor"+build tool to avoid many of the Eclipse headaches and to take advantage of the great features of mvn/sbt/fsc/etc. (they all take some learning, but are more powerful the deeper you dive). Most of the time I use NetBeans as a glorified code editor with some combination of sbt and/or mvn running in a console. To this end, I would really like to see better support for the NB plugin.

Just my $0.02.

-Mark

Peter C. Chapin

unread,
Nov 7, 2011, 12:35:46 PM11/7/11
to scala...@googlegroups.com
On Mon, 2011-11-07 at 10:50 -0500, Bastian, Mark wrote:

> I use NetBeans as a glorified code editor with some combination of sbt
> and/or mvn running in a console. To this end, I would really like to
> see better support for the NB plugin.

I'm also a NetBeans user and would love to see the NetBeans plugin
continue to evolve. Yes, I know... why don't I contribute? Well, as for
many of us I just don't have the personal resources to do that right
now.

I can't wait until retirement. Then I'll be able to get some real
programming done. :)

Peter

John Cheng

unread,
Nov 7, 2011, 1:28:53 PM11/7/11
to scala...@googlegroups.com



And your suggestions? Recommended three or less because probably no one will read past the third one :-).



Ken

I strongly agree with IDE as #1. However, I am skeptical about the value of better Scala Docs. Perhaps I am a pessimist, but while I agree that better Scala Doc would be incredibly useful to me as a developer, I think it means very little to CTOs and people who are on the fence about learning Scala. 

I think organizing meet ups and sharing success stories in the development community will be key. At a recent Scala meet up in Los Angeles, I met a small group of developers who are interested in getting started with Scala, but noone with in depth knowledge and success stories. I think that can be disheartening to new developers. In this respect, I think I can only rely on companies like TypeSafe getting out and help organize more meetups in major development hubs. 


--
---
John L Cheng

Luc Duponcheel

unread,
Nov 7, 2011, 1:49:11 PM11/7/11
to Luc Evers, scala...@googlegroups.com
about Scala RIA

maybe ScalaFX is an interesting technology to consider
http://code.google.com/p/scalafx

btw: here is ScalaFX demo that I have developed
http://code.google.com/p/scalafx/source/browse/demo/scalafx/JumpingFrogsPuzzle.scala?spec=svn280146c3b01da9dac672e9b3bfedaddb00fe7a78&r=280146c3b01da9dac672e9b3bfedaddb00fe7a78

Luc
--
   __~O
  -\ <,
(*)/ (*)

reality goes far beyond imagination

Arif Mustafa

unread,
Nov 8, 2011, 10:09:13 AM11/8/11
to scala...@googlegroups.com
think Scala's weakness (with regards to its adoption in the "Enterprise") is that it never projected/marketed  itself as such... the focus has always been on esoteric topics like bloom filters, actors etc. Now if we were to divide Enterprises into two broader categories (1) Traditional Enterprises (which are against any disruption/innovation...like most major financial institutions) (b) New/Innovative Enterprises (like Facebook, Google etc) ... Scala's appeal is to the second types of Enterprises. For first type of Enterprises you need many examples like "Duke's shopping cart" ;-) ... thus showing that Scala (Like Java/JEE) can answer the needs for presentation and business tier and Web Services ... so yes ..such a book is required.

my two cents worth

cheers

Daniel Sobral

unread,
Nov 8, 2011, 2:52:55 PM11/8/11
to Arif Mustafa, scala...@googlegroups.com
If only Scala were *not* being adopted in major financial
institutions, like it is, that argument would be stronger.

Alas, someone once divided financial institutions between loaning
banks and investment banks, where the first are very conservative and
the second will try anything that can give it an edge.

--
Daniel C. Sobral

I travel to the future all the time.

Channing Walton

unread,
Nov 8, 2011, 4:09:15 PM11/8/11
to scala...@googlegroups.com, Arif Mustafa


On Tuesday, 8 November 2011 19:52:55 UTC, Daniel Sobral wrote:
If only Scala were *not* being adopted in major financial
institutions, like it is, that argument would be stronger.

Alas, someone once divided financial institutions between loaning
banks and investment banks, where the first are very conservative and
the second will try anything that can give it an edge.


Spot on. From what I can see, Scala is rapidly gaining traction in most major financial orgs. The devs love it and don't mind that the IDE's aren't great right now, the benefits of the language outweigh the short-term problem of good IDEs.

What needs to happen is for everyone to stop worrying about a problem that doesn't really exist.

Kevin Wright

unread,
Nov 8, 2011, 4:28:59 PM11/8/11
to scala...@googlegroups.com, Arif Mustafa

For my part, I'm seeing a lot of adoption in media.  Including broadcast, papers, online gaming, social, etc, etc.

Basically, lots of people who're after an edge in being more responsive to the market than their competition, so not just investment banking :)

Banks are just more visible because they tend to pay more for support contacts, and use recruiters who are far spammier on LinkedIn and the like.

Arif Mustafa

unread,
Nov 8, 2011, 6:10:04 PM11/8/11
to Kevin Wright, scala...@googlegroups.com
@Kevin spot on... "broadcast, papers, online gaming, social, etc, etc." ...and thats where I have observed the adoption rate of Scala to be high ... and ..."Banks"  (Tradional model, who would rather pay IBM'ers and/or Tata Consultancy etc to run their IT) have low adoption rate of Scala is low.

Jordi Salvat i Alabart

unread,
Nov 8, 2011, 9:48:16 PM11/8/11
to scala-user
> The question is simple: List, in descending order of priority, what you
> feel needs to be addressed/fixed/whatever for Scala to succeed in the
> corporate (and hence IMHO ultimately the "real") world.

I'll only list 1: reliability.

As a software development manager, I recently shied away from Scala in
a newly starting project mainly because I judged the compiler to be
unreliable.

As for IDE support, while it is very important (even essential to
attract the Java lot), it is only an absolute must because the correct/
compile/run cycle is so slow. PHP, for example, is quite successful in
spite of having extremely poor IDE/tooling support.

--
Salut,

Jordi.

Naftoli Gugenheim

unread,
Nov 8, 2011, 9:57:16 PM11/8/11
to Jordi Salvat i Alabart, scala-user

On Tue, Nov 8, 2011 at 9:48 PM, Jordi Salvat i Alabart <jordi.salva...@gmail.com> wrote:
mainly because I judged the compiler to be
unreliable.

Why?

Tomás Lázaro

unread,
Nov 8, 2011, 10:31:06 PM11/8/11
to scala-user
IDE Support would be the most important issue for me. I use Netbeans to write the code and SBT on a terminal to compile and run. I really do not want to use Eclipse, I don't like it as a whole (I don't have a problem with the Scala IDE itself). How hard would it be to have the Eclipse plugin work in Netbeans? It supports OSGI so maybe it could re-use a lot of stuff. Off course integration with the editor itself is definitely going to be different but Caoyuan has gone a long way with the Scala plugin for NB and there should be enough stuff already figured out there.

I started using Scala in the "real world" more than 3 years ago, I think 4. We had a custom server over TCP, hundreds of clients, several databases and many other legacy servers using different custom communication protocols as well as some desktop applications for internal use and for clients... all of that was done in Java. One day we discovered Scala, thought it was cool, bought the first book the day it came out and started using it right away when we noticed we could mix Scala code with our current code base without much trouble. We ended up converting most of the code to Scala and only produced new code with it. We never looked back. I don't work there anymore but they still use Scala. There is no compelling reason not to use it in the "real world".

I never had a problem with documentation or the available books regarding the language itself. I never minded googling for stuff or browsing stack overflow. I never had a problem asking questions on the mailing list either.

jilen

unread,
Nov 8, 2011, 10:57:29 PM11/8/11
to Tomás Lázaro, scala-user
The problem with eclipse scala-ide I found
1. Performance, too much heap memory usage , too slow.
2. Syntax color, I always see only two color, that is somewhat boring.
3.Coding express is not good enough. Code completion sometimes not available.

The problem with Netbeans
1.Swing seems not work well with linux.

martin odersky

unread,
Nov 9, 2011, 3:43:17 AM11/9/11
to Tomás Lázaro, scala-user


2011/11/9 Tomás Lázaro <tlaz...@gmail.com>

IDE Support would be the most important issue for me. I use Netbeans to write the code and SBT on a terminal to compile and run. I really do not want to use Eclipse, I don't like it as a whole (I don't have a problem with the Scala IDE itself). How hard would it be to have the Eclipse plugin work in Netbeans? It supports OSGI so maybe it could re-use a lot of stuff. Off course integration with the editor itself is definitely going to be different but Caoyuan has gone a long way with the Scala plugin for NB and there should be enough stuff already figured out there.

I think Caoyuan is already using the presentation compiler, which is at the core of the Eclipse IDE. And so is ENSIME. Hopefully it will only take minor upgrade works to the latest version to get many improvements. The other large trend is that we are going to
put more functionality out of the Eclipse SDT and into the presentation compiler. That should also help the other plugins. 

Cheers

 -- Martin

bryan hunt

unread,
Nov 9, 2011, 5:51:58 AM11/9/11
to scala-user
Thomas, Wow.

Have you been using Netbeans to compile real world projects? My
experience of Scala Netbeans integration is:
The guy who was maintaining the plug-in left 2 years ago and that it
currently chokes on anything more serious than Hello World.

Scala-IDE, the Eclipse plugin while not quite there compared to
Eclipse's Java support is still WAAAEEEEY ahead of the competition.

And yes, I use IDEA for ActionScript but it's Scala support is at
best, enthusiast grade.

I say this having written an approx 30,000 LOC Scala project last year
all of which parses and compiles (slowly) using Scala-IDE and/or
Maven.

Regards,

Bryan Hunt

Geir Hedemark

unread,
Nov 9, 2011, 7:04:52 AM11/9/11
to bryan hunt, scala-user
On 2011, Nov 9, at 11:51 AM, bryan hunt wrote:
> Scala-IDE, the Eclipse plugin while not quite there compared to
> Eclipse's Java support is still WAAAEEEEY ahead of the competition.
>
> And yes, I use IDEA for ActionScript but it's Scala support is at
> best, enthusiast grade.

Oh, I dont know. My team all use intellij since last year or so. Eclipse just did not work out for us - it was too buggy.

We all seem to think intellij is good enough. So do the consultants I know around here who actively use Scala.

YMMV.

Geir


Aleksey Nikiforov

unread,
Nov 9, 2011, 7:49:40 AM11/9/11
to bryan hunt, scala-user
You may be having problem with inference. Reducing inference by annotating types can help reduce ram consumption and compilation time significantly. This is mostly true for multi-level inference that has to go through several layers of parentheses.

For example: val x = A(B(C(a, b, c)))
If the type of x is something like A[X[Y[Z]]], then you are better off declaring it: val x = A[X[Y[Z]]](B(C(a, b, c)))

It may take a bit of extra work, at least with large existing projects, but it will make everything compile in reasonable amount of time.

This problem has been fixed for command line compilations, but it still makes Eclipse IDE eat up all ram and slow down to a crawl. My guess it will be fixed in Eclipse sometime soon as well.

iulian dragos

unread,
Nov 9, 2011, 8:13:41 AM11/9/11
to scala...@googlegroups.com
On Sun, Nov 6, 2011 at 2:08 AM, Erik Engbrecht <erik.en...@gmail.com> wrote:
> I think you need to define what "rock-hard support" in an IDE means.  For
> example, the Eclipse plugin keeps on moving forward in terms of
> functionality, but where exactly is the version of it for Scala 2.9.1?

If you refer to the download page, the version number is indeed 2.9.2,
because of changes to the presentation compiler. Since the
presentation compiler is part of the compiler, we need an updated
version to take advantage of the new features. However, since
yesterday we ship with the standard 2.9.1 library, so you should not
feel any difference.

The Eclipse IDE is really low on features. We're basically focusing on
completion, hyperlinking and interactive error reporting. IntelliJ has
been a lot more active in adding new features, though, so maybe you
mixed them.

>  It
> also consumes a ton of memory and tends to "just sit there" a lot.  It's
> basically unusable in my work environment, while the version quite a while
> back, while buggier and with far fewer features, was usable.

What versions are we talking about, or when did you try Eclipse last?
This sounds really surprising, since my own experience, and most
people that cared to try Eclipse one year ago, agreed that it's much
more usable now.

>I think the
> NetBeans plugin also took a step sideways when it integrated with the
> presentation compiler.  In my opinion the only really usable IDE for Scala
> right now is Emacs+ENSIME.  Maybe the ENSIME integrations for Vim and JEdit

ENSIME uses the presentation compiler, so it can't be much better
(assuming the presentation compiler is the problem).

iulian

> work well, too.  I'm fine with Emacs, but I wouldn't try to force in on
> anyone.
>
>
>

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

Peter C. Chapin

unread,
Nov 9, 2011, 12:59:39 PM11/9/11
to scala-user
On Wed, 2011-11-09 at 05:51 -0500, bryan hunt wrote:

> Thomas, Wow.
>
> Have you been using Netbeans to compile real world projects? My
> experience of Scala Netbeans integration is:
> The guy who was maintaining the plug-in left 2 years ago and that it
> currently chokes on anything more serious than Hello World.

I'm using NetBeans on a Scala project. It's not a huge project, but
definitely larger than "Hello, World." The Scala plugin is working fine.
It's a bit quirky but it gets the job done. I should say that I'm using
the latest version from the git repository compiled from source. The
person maintaining the plugin is still active with it, although activity
is light. The last commit was maybe a couple of months ago (I haven't
tried a pull recently though).

Peter


Brian Schlining

unread,
Nov 9, 2011, 1:30:54 PM11/9/11
to scala-user
> I'm using NetBeans on a Scala project. It's not a huge project, but
> definitely larger than "Hello, World." The Scala plugin is working fine.
> It's a bit quirky but it gets the job done. I should say that I'm using
> the latest version from the git repository compiled from source.


For the curious, the git repo is https://github.com/dcaoyuan/nbscala

--
Brian Schlining

Donald McLean

unread,
Nov 9, 2011, 1:54:09 PM11/9/11
to scala-user
I find this very hard to believe. The JetBrains folks have been
working steadily on their plugin and they have made tremendous
progress in features and usability. In the beginning, it's syntax
parsing was often a bit sketchy but those problems have pretty much
disappeared and all of the IDEA niceties have started showing up.

I have found that IDEA vs Eclipse is a chocolate/vanilla or
purple/green question. I've had to use Eclipse a couple of times and I
just didn't like it.

Donald

Andreas Joseph Krogh

unread,
Nov 9, 2011, 3:21:24 PM11/9/11
to scala...@googlegroups.com
Agreed.
It's not just how the IDE understands Scala itself, it's also about how the IDE integrates other features of the IDE in Scala-files/projects. IDEA is just fantastic here. My JPA-model written in Scala shows up in the "Show ER-diagram", my JPQL is highlighted correctly to give me code-completion in my JPQL working with JPA-entities written in Scala. SQL code-completion in scala-classes. Spring-integration for Scala is *almost* just as good as in JAVA. The list is long...

I use IDEA-11 EAP with the latest scala-plugin and I really don't consider IDE-support at all to be an issue anymore.

I only have one thing which I think is important to succeed in the corporate world:
1. Binary compatibility among version-upgrades

I have another nit-picking-list though:-)
- JAVA interoperability (yes, there are cases where they don't play nicely [1][2])
- Consistent handling of "val" (finals) [3]

[1] Should be able to use JAVA-enums and other "public static final" in annotation-parameters.
[2] https://issues.scala-lang.org/browse/SI-4549
[3] http://stackoverflow.com/questions/7762838/scala-forward-references-why-does-this-code-compile
-- 
Andreas Joseph Krogh <and...@officenet.no> - mob: +47 909 56 963
Senior Software Developer / CTO - OfficeNet AS - http://www.officenet.no
Public key: http://home.officenet.no/~andreak/public_key.asc

Francois Armand

unread,
Nov 9, 2011, 6:30:01 PM11/9/11
to scala...@googlegroups.com
Le 05/11/2011 22:51, Ken McDonald a écrit :
[...]

> The question is simple: List, in descending order of priority, what you
> feel needs to be addressed/fixed/whatever for Scala to succeed in the

> corporate (and hence IMHO ultimately the "real") world.
>
> My list:
>
> 1. IDE Support. [...]
>
> 2. Documentation. [...]
>
> 3. A "Scala Cookbook". [...]
>


My shop list would be:

1. A compiler as fast as the java one. And a modular scala-lib. Yeah, I
now...
2. An as powerful and great library to deals with immutable data
structure modification as the collection one is for collection (lens,
zipper, etc)
3. Being able to use Scala for scripting. Than implies 1 and a better
starting time + something to include automagically dependencies,
essentially. And it will be so good to show how cool and powerful Scala
is in some little lines. I really would *love* to convert the sysadmin I
work with to use it as possible replacement of perl or python (or bash),
but for now, the lauching time are killing that possibility.

IDE support is reaching the point where Eclipse does not go in my way
anymore and is of real help for boring things (imports, completion of
long method names, formatting, things like that).

Documentation is OK, especially with so many books and blogs post online.

A "Scala Cookbook" could be intersting, but I would much more prefer a
repository of Scala scripts, if they were usable for real.

Cheers,

--
Francois Armand
http://fanf42.blogspot.com

Daniel Sobral

unread,
Nov 9, 2011, 6:39:09 PM11/9/11
to Francois Armand, scala...@googlegroups.com
On Wed, Nov 9, 2011 at 21:30, Francois Armand <fan...@gmail.com> wrote:
> Le 05/11/2011 22:51, Ken McDonald a écrit :
> [...]
>>
>> The question is simple: List, in descending order of priority, what you
>> feel needs to be addressed/fixed/whatever for Scala to succeed in the
>> corporate (and hence IMHO ultimately the "real") world.
>>
>> My list:
>>
>> 1. IDE Support. [...]
>>
>> 2. Documentation. [...]
>>
>> 3. A "Scala Cookbook". [...]
>>
>
>
> My shop list would be:
>
> 1. A compiler as fast as the java one. And a modular scala-lib. Yeah, I
> now...
> 2. An as powerful and great library to deals with immutable data structure
> modification as the collection one is for collection (lens, zipper, etc)
> 3. Being able to use Scala for scripting. Than implies 1 and a better
> starting time + something to include automagically dependencies,
> essentially. And it will be so good to show how cool and powerful Scala is
> in some little lines. I really would *love* to convert the sysadmin I work
> with to use it as possible replacement of perl or python (or bash), but for
> now, the lauching time are killing that possibility.

You mean like this?

#!/usr/bin/env scalas
!#

/***
scalaVersion := "2.9.0-1"

libraryDependencies ++= Seq(
"net.databinder" %% "dispatch-twitter" % "0.8.3",
"net.databinder" %% "dispatch-http" % "0.8.3"
)
*/

import dispatch.{ json, Http, Request }
import dispatch.twitter.Search
import json.{ Js, JsObject }

def process(param: JsObject) = {
val Search.text(txt) = param
val Search.from_user(usr) = param
val Search.created_at(time) = param

"(" + time + ")" + usr + ": " + txt
}

Http.x((Search("#scala") lang "en") ~> (_ map process foreach println))

See https://github.com/harrah/xsbt/wiki/Scripts. Works like a charm.

>
> IDE support is reaching the point where Eclipse does not go in my way
> anymore and is of real help for boring things (imports, completion of long
> method names, formatting, things like that).
>
> Documentation is OK, especially with so many books and blogs post online.
>
> A "Scala Cookbook" could be intersting, but I would much more prefer a
> repository of Scala scripts, if they were usable for real.
>
> Cheers,
>
> --
> Francois Armand
> http://fanf42.blogspot.com
>

--

Francois Armand

unread,
Nov 9, 2011, 6:48:49 PM11/9/11
to Daniel Sobral, scala...@googlegroups.com
Le 10/11/2011 00:39, Daniel Sobral a �crit :

> On Wed, Nov 9, 2011 at 21:30, Francois Armand<fan...@gmail.com> wrote:
[...]

>> 3. Being able to use Scala for scripting. Than implies 1 and a better
>> starting time + something to include automagically dependencies,
>> essentially. And it will be so good to show how cool and powerful Scala is
>> in some little lines. I really would *love* to convert the sysadmin I work
>> with to use it as possible replacement of perl or python (or bash), but for
>> now, the lauching time are killing that possibility.
>
> You mean like this?
>
[...]

Exactly like this, thanks for pointing it !

Now, it only remains compiler speed and starting time :)

(wow, that's really cool, what a pity I missed that... Does it exists
for a long time ?)

Jordi Salvat i Alabart

unread,
Nov 10, 2011, 3:16:37 AM11/10/11
to Naftoli Gugenheim, scala-user
I found one bug in the compiler and 1 in the Language Spec in a few hours of tinkering around (well, actually I found more, but I was playing with experimental features so I didn't count those).

As a result of that research I fell in love with the language and have been playing with it in personal projects since. A couple of months later, I've filed a total of 21 bug reports (my username is "jsalvata" in Jira, if you want to see the listing).

Of course some of those may be false positives, and many are at the edges (playing with Xexperimental, problems internal to the compiler, ...), but I still stand by my initial assessment: Scala is not ready for the industry at large, except possibly for specially complex problems where it provides significant advantage (such as Scala's flagship use in industry, Twitter's engine)

--
Salut,

Jordi.

2011/11/9 Naftoli Gugenheim <nafto...@gmail.com>

Channing Walton

unread,
Nov 9, 2011, 4:13:58 PM11/9/11
to scala...@googlegroups.com

On 9 Nov 2011, at 20:21, Andreas Joseph Krogh wrote:

> I use IDEA-11 EAP with the latest scala-plugin and I really don't consider IDE-support at all to be an issue anymore.

This is also my experience too although I use IDEA-11 and Eclipse. The only thing that irritates me in IDEA is that one still needs to do a full compile occasionally to see if the red lines in editors are real problems or not. But a couple of bugs I submitted recently were fixed in 1 day which made the experience much better.

The IDE is simply not an issue any more, certainly not one that should prevent anyone from benefitting from Scala.

Channing Walton

unread,
Nov 8, 2011, 4:36:43 PM11/8/11
to scala...@googlegroups.com
Its all good!

Chris Marshall

unread,
Nov 10, 2011, 5:25:31 AM11/10/11
to scala...@googlegroups.com
> On Sun, Nov 6, 2011 at 2:08 AM, Erik Engbrecht <erik.en...@gmail.com> wrote:
> > I think you need to define what "rock-hard support" in an IDE means. 

Here are some of the things that IDEA can do for me in Java files that have not-much to do with refactoring/navigation. These things will (I suspect) come as a *big* surprise to the people who are used to VIM and emacs but they are the sorts of things that save a lot of time.

1. Intelligent string parsing. IDEA can tell me when my format string passed to String.format is malformed wrt to the types of the arguments I pass to it (it is highlighted as a bug). It can render *strings* which represent SQL or regex such that they have coloring appropriate to SQL (different color keywords etc).

2. Auto-complete *within Strings*. In a String value in my program, IDEA detects whether the string looks like a path and can auto-complete within the string. For example, if I am creating an ImageIcon with the path to an image resource, IDEA can offer me the choices of images within the directory. It even pops up an *image viewer* as I scroll down the options, even though all I have written is something like: "/path/to/images/ CTRL-SPACE"

3. Color choosing. IDEA spots when a java.awt.Color is created in my program and offers a little square of that color in the sidebar. Clicking on the square gives me a color picker which I can then use to change the color's parameters (requires enterprise edition)

4. Warnings: IDEA warns me when a piece of code is redundant, or when a return value or parameter is not used. It tells me when a method call is being placed against something that might reasonably be null (under certain circumstances), and when a code block (like an if-statement) can be simplified, or its sense reversed. etc etc.

5. When I perform a refactoring, IDEA spots whether other snippets of code in the same file are duplicates (and hence can also be replaced by the refactored item). Refactoring options are many; allowing me to extract fields, constants, parameters, perform renamings which are detected across other file types (such as properties files or XML config files).

6. I can navigate to my types and auto-complete methods and class names from within config or properties files. I can, from within a config file, get a popup of the constructor-params which a type takes. etc etc.

Admittedly these individually are minor things (and a few are already possible with the scala IDEA plugin) but they do add up in such a way that stepping into scala-land can feel less enterprisey (not always a bad thing). Obviously I feel that, on balance, scala is still way ahead in the productivity stakes - but you can at least imagine the reaction of developers when *features that they are used to and use every day* are no longer available.

Chris

Dennis Haupt

unread,
Nov 10, 2011, 5:41:01 AM11/10/11
to Chris Marshall, scala...@googlegroups.com

-------- Original-Nachricht --------
> Datum: Thu, 10 Nov 2011 10:25:31 +0000
> Von: Chris Marshall <oxbow...@hotmail.com>
> An: scala...@googlegroups.com
> Betreff: RE: [scala-user] Re: What is highest priority for Scala to succeed in corporate world (Should be in scala-debate?) ?

>
> > On Sun, Nov 6, 2011 at 2:08 AM, Erik Engbrecht
> <erik.en...@gmail.com> wrote:
> > > I think you need to define what "rock-hard support" in an IDE means.
>
> Here are some of the things that IDEA can do for me in Java files that
> have not-much to do with refactoring/navigation. These things will (I suspect)
> come as a *big* surprise to the people who are used to VIM and emacs but
> they are the sorts of things that save a lot of time.
> 1. Intelligent string parsing. IDEA can tell me when my format string
> passed to String.format is malformed wrt to the types of the arguments I pass
> to it (it is highlighted as a bug). It can render *strings* which represent
> SQL or regex such that they have coloring appropriate to SQL (different
> color keywords etc).

it works for nested languages in general (sql, hibernate query stuff). for example, i have native javascript inside my gwt java classes. idea offers full support for these snippets. i pity my eclipse co-workers every time :)


> 2. Auto-complete *within Strings*. In a String value in my program, IDEA
> detects whether the string looks like a path and can auto-complete within
> the string. For example, if I am creating an ImageIcon with the path to an
> image resource, IDEA can offer me the choices of images within the directory.

really helpful inside xml files

> It even pops up an *image viewer* as I scroll down the options, even
> though all I have written is something like: "/path/to/images/ CTRL-SPACE"
> 3. Color choosing. IDEA spots when a java.awt.Color is created in my
> program and offers a little square of that color in the sidebar. Clicking on the
> square gives me a color picker which I can then use to change the color's
> parameters (requires enterprise edition)

it also shows the color in the debugger.

bryan hunt

unread,
Nov 10, 2011, 7:08:56 AM11/10/11
to scala-user
Ok, an example, one of many, Scala-Query (the project) is not parsed
correctly by Intellij.

Having spent a number of years on theserverside.com, I'm well versed
in the world of AstroTurfing(TM).

I am for the record, a fully paid up, subscribed Intellij Enterprise
customer, but I keep a copy of Eclipse for when I need to do Scala.

Intellij cannot accurately parse more than about 20 (Scala) classes
without crapping itself, Java and Flex support are outstanding, even
if the UI is a bit sluggish.

Regards,

Bryan Hunt

bryan hunt

unread,
Nov 10, 2011, 7:22:40 AM11/10/11
to scala-user
> 3. Being able to use Scala for scripting. Than implies 1 and a better
> starting time + something to include automagically dependencies,
> essentially. And it will be so good to show how cool and powerful Scala
> is in some little lines. I really would *love* to convert the sysadmin I
> work with to use it as possible replacement of perl or python (or bash),
> but for now, the lauching time are killing that possibility.

My 2c.

System administrators, in general, have rather particular
requirements.

Here's a couple that I've observed:

1) Simplicity.
2) Stability.
3) Speed, but simplicity is much more important.
4) An API, (where applicable) which wraps OS level functionality in
simple fashion.
5) Can be installed, without playing version bingo, from the OS
package management system.
6) Fast code/test/deploy cycle, for a 200 line program, compile/
startup time of more than 2 seconds is too long.
7) Is Bourne, Python, Ruby or for those hangers on from the dot-com
boom, Perl.

System administrators, in general, don't like the following things.

1) Dynamic behaviour redefinition, think implicits.
2) Factory classes, design patterns, types, static typing.
3) Monads, Monoids, things that make more than 2 minutes to make sense
of.
4) Stuff that needs to constantly auto-update itself from some server
in Switzerland, every time you do a build.

I could be wrong....

Geir Hedemark

unread,
Nov 10, 2011, 7:31:41 AM11/10/11
to bryan hunt, scala-user
On 2011, Nov 10, at 1:22 PM, bryan hunt wrote:
>> is in some little lines. I really would *love* to convert the sysadmin I
>> work with to use it as possible replacement of perl or python (or bash),
>> but for now, the lauching time are killing that possibility.
> System administrators, in general, have rather particular
> requirements.

System administrators, in general, are not developers. They need a different toolset than us devvie guys do. I don't think it is possible to build a tool that does both well.

I would love to be proven wrong on this one, tough.

Geir


Daniel Sobral

unread,
Nov 10, 2011, 8:06:44 AM11/10/11
to bryan hunt, scala-user
On Thu, Nov 10, 2011 at 10:22, bryan hunt <sentimen...@gmail.com> wrote:
>> 3. Being able to use Scala for scripting. Than implies 1 and a better
>> starting time + something to include automagically dependencies,
>> essentially. And it will be so good to show how cool and powerful Scala
>> is in some little lines. I really would *love* to convert the sysadmin I
>> work with to use it as possible replacement of perl or python (or bash),
>> but for now, the lauching time are killing that possibility.
>
> My 2c.
>
> System administrators, in general, have rather particular
> requirements.
>
> Here's a couple that I've observed:
>
> 1) Simplicity.
> 2) Stability.

System administrators love stability from others. For their own
script, I have never seen one give a fig. And I think the wild ride
that are the scripting languages is proof enough of that attitude.

> 3) Speed, but simplicity is much more important.

If speed wasn't important, Perl would never have been born. Log files
can be HUGE, and handling them efficiently is important.

> 4) An API, (where applicable) which wraps OS level functionality in
> simple fashion.

I'd go even beyond that. They want "a library for that". CPAN is not
simply a result of Perl's popularity: it is an integral part of it. I
can *browse* the available libraries from CLI by typing "perl -MCPAN
-e shell". I can search for a Ruby library on shell with "gem search".
I'm not familiar with Python, but I bet it has the same thing.

So, while sbaz, as it is, doesn't work, having a working sbaz, and a
web site equivalent, would be fundamental in actually getting Scala
accepted as a system administrator tool.

> 5) Can be installed, without playing version bingo, from the OS
> package management system.

Well, they do like that, but I can tell you this is not a situation
one encounters that often. Again, this is true much more for stuff
sent by devel than stuff done by sysadmin themselves.

> 6) Fast code/test/deploy cycle, for a 200 line program, compile/
> startup time of more than 2 seconds is too long.

Well, there are third party solutions to that, but it can't be helped
except by having a JVM daemon running.

> 7) Is Bourne, Python, Ruby or for those hangers on from the dot-com
> boom, Perl.
>
> System administrators, in general, don't like the following things.
>
> 1) Dynamic behaviour redefinition, think implicits.

Actually, I think they'd love implicits.

> 2) Factory classes, design patterns, types, static typing.

Mostly (well, see stuff like Python-written Ganeti or Perl-written
OTRS for counter-examples). As for static typing, the problem is not
static typing, the problem is boilerplate. In fact, you could replace
that whole line with "boilerplate" and be done with it and be more
accurate, to boot.

> 3) Monads, Monoids, things that make more than 2 minutes to make sense
> of.

Word.

> 4) Stuff that needs to constantly auto-update itself from some server
> in Switzerland, every time you do a build.

As if there isn't a package manager that isn't download-happy. They
don't like being FORCED to update at a time they don't want to. Other
than that... business as usual.

Daniel Sobral

unread,
Nov 10, 2011, 8:08:48 AM11/10/11
to Geir Hedemark, bryan hunt, scala-user

Ruby and Python were born as system administration scripting
languages, as far as I know. I do know that my first contact with Ruby
was in such a role. And I think it is undeniable that Perl is the
ultimate system administrator tool. :-)

Which is more and more besides the point. Devops is here to stay.

Chris Marshall

unread,
Nov 10, 2011, 8:40:30 AM11/10/11
to scala...@googlegroups.com
Whilst I agree - I have been bitten countless times by malformed format strings. Many times these lead to stuff like actors shutting down (with no accompanying stack trace) and can be a complete nightmare to detect. Particularly when the string format was itself supposed to indicate some exceptional situation (so the system fails silently, possibly months after the code was written):

 catch { case e => log.info("Fallen over with: %d".format(e)) }

I have become hypersensitive to checking my own format strings carefully, whereas with IDEA I needed only to be aware of the fact that they were not highlighted as malformed

Chris


> Subject: Re: [scala-user] Re: What is highest priority for Scala to succeed in corporate world (Should be in scala-debate?) ?
> From: channin...@googlemail.com
> Date: Wed, 9 Nov 2011 21:13:58 +0000
> To: scala...@googlegroups.com

jherber

unread,
Nov 10, 2011, 9:24:14 AM11/10/11
to scala-user
I don't post often, but when I do...

This thread is becoming an IDE flame war, but in the meantime, take a
look at this article just published:

http://www.informationweek.com/news/software/info_management/231902645

The Big Data Stack is probably the next big thing to work its way into
large enterprises. What offering does Scala have here? Scala
ecosystem recently bet big on parallel collections, lang
virtualization, and absorbing AKKA (all truly amazing stuff). But,
these are just different directions than big enterprises are taking.
With the Big Data Stack, numerous industries are realizing the value
of a generic, supported framework for the storage and analysis of
unlimited data in structured and unstructured forms. This solves a
huge industry problem as general trend is more real-time results, more
introspection of your customer's data so you can sell or offer more
value than competitor, and of course it solves the data explosion
problem in an affordable, predictable way without vendor lock in.

I am consulting with a big financial center currently, so let me give
some background and context of the environment. There might be a
dozen or two of us who could write code in Scala right now. But the
other 10,000 developers will not have a clue how to read it, how to
get trained, how to get started, how to get Management to pay for all
their training and lost productivity while they train, rewrite, and
learn on the job. We'd no doubt see the range of Scala as it is very
flexible, making it very difficult to maintain and read. This
contradicts the process and control lock down big companies must have
to make sure strategy flows from top to bottom as it is translated
into execution. From a ground up perspective, the first time a
Manager who codes "Enterprise Style Java", risks his reputation (and
budget and bonus) on some smart consultant's Scala recommendation,
only to experience first hand a random long pauses using an IDE or has
an unexpected class explosion from specialization or a code review
with someone pushing the type system, he will shut it down and
vocalize his perception to other managers and that consultant is
booted.

In a big company, it is not uncommon to find millions of lines of code
written in Java, with huge chunks of it pre generics. I don't see
that code going anywhere. It is simple to read, buggy as heck to
modify beyond its intended extension points, but in Management's mind
it has powered the company for almost a decade so why change it? They
see so many other areas that would have a higher return on investment.

I think Scala could make inroads in another way. Big companies buy
smaller companies all the time. If eventually enough small companies
are ingested that are using Scala, the base make up of the big company
changes. Big companies also go with proven advantages from vendors or
open source, but they typically do not make big risky bets, and they
only change tooling and paradigms when competition forces change.
There's also ingestion in the small - great libraries.

But this would take a long time, and I suspect great progress is not
happing in this direction. Why? I'm not sure, but I don't see a
great migration in open source to replace Java for enterprise tooling
with Scala (I am aware of the contributions flowing from startups). I
do not see dozens of Apache projects banging out new libraries and
Java Enterprise API implementations using Scala, even where they are
small, lightweight subsets like JAXB and JAX-RS. So something is
amiss with the value proposition of Scala, such that it is not getting
ingested by the Java ecosystem in the small. There is some progress,
but not as much as I thought there would be at the "Apache" level.

How about in the large? The Big Data Stack is the next big thing, and
it has been incubating on the JVM almost as long as Scala has, yet I
see very little to no influence by Scala. If Scala is such a great
productive language, why are not all the new tools around the Big Data
Stack and pieces of the Big Data Stack written in Scala? Again, I'm
just not sure why this didn't happen, just that it appears not to be,
and thus ingestion as a mechanism to break out appears not to be
happening.

Then there is the operational requirements of big enterprise. How do
we constrain Scala for Enterprise? Enterprises follow a model that is
the opposite of Academia in terms of the composition and use of labor
(many cogs following directed, repeatable process to generate globally
usable artifacts vs Acadmia's accretion of new knowledge using search
and investigation to generate narrow use artifacts.) How would you
get 10,000 or maybe 1000 developers or 100 or 10 to follow an approved
subset of the language? Trust? Ugh. Scala has turing complete type
system (side stepping tractability issues) - amazing for research and
exploration of the possible - not so good for 9 to 5 coder who might
be proficient in 2004-2008 era Java writing code that any of those
other 10,000 devs must be able to read and modify.

I used to think Scala was going to crack the enterprise by 2009-2010
(been an avid follower since 2007). Then the great collection rewrite
showed us that type abstraction in Scala would allow us to do things
more correctly and keep type safety. However that type safety had a
cost and the collection API is now non-navigable or extendable by new
users. How do we make the same kinds of jumps in usability
abstraction?

In many ways Scala's direction appears to be the antithesis of big
enterprise tooling. But, every consumer having a ubiquitous computing
device that is always connected in their hand is forcing big
enterprises to increase capabilities or lose their relationship with
those customers. This is an opportunity for Scala, but maybe not the
Scala we all know and love. Maybe it is time to evaluate a Scala
"light"?

Scala's hypothesis of a scalable language fusing FP and OO was and
continues to be needed (thanks Martin), but what has been learned here
can certainly feed into a subset language that leverages Scala, but
answers another hypothesis: How do we bridge a scalable language to a
constrained environment to deliver value to the masses that don't grok
type theory, category theory/abstract algebra, and that don't have
much time to learn anything that does not fit their current analogies
or at least have obvious, predictable value model? I used to think
DSLs were the vehicle by which Scala would crack into the enterprise.
While they are targeted, and offer high levels of productivity - in
the end they require you must understand all of Scala to extend, grow,
and therefore require passing the enterprise test before they can be
used to bring the language in.

Is working with the constrained big enterprise something the Scala
community even wants to solve? It cuts into resources and that cuts
into all the great edge problems Scala allows us to interact with.
You could think of Scala as an incubator - somewhere between edge
research and somewhere between getting real stuff done at tomorrows
big companies (startups). These environments have a great deal in
common, and it isn't surprising Scala can thrive there. But
enterprise is a re-think if you want to target them. I don't think
providing more documentation or a flawless GUI based environment
solves the problem outright. Imagine one small team using modern day
fighter jet fighter for missions. The training, the documentation,
the hours to effectively use it are unreal so you have to be a top
student - but the result is very capable, fast, and far reaching
instrument a handful can use. Well, enterprises are more like big
battleships. slow to turn. lots of hands on deck taking orders in a
hierarchy to accomplish things. Sure battleships might have some cool
bolt-on technology, and some really smart people on board (and even
more 9-5er types). But, I don't think providing more doc, training,
or a flawless heads up display will turn battleship hands into jet
fighter pilots, let alone battleships into aircraft carriers. Maybe
it is time to reframe the problem.

Just some food for thought on this issue.

Kind Regards,

Jim Herber

Razvan Cojocaru

unread,
Nov 10, 2011, 1:55:32 PM11/10/11
to jherber, scala-user
WOW. I hear you and used to think so, but then I also find it amazing that
these "big companies" don't use Cobol that much anymore... that builds kind
of a contradiction there...

Our customers - many orders of magnitude larger than us, have vey funky
environments. I have seen all sorts of scripting environments known to man
(and a few in-house) in production all over the place.

Also - the moment I stopped pushing for it, some guys here started picking
it up... now we have at least one "official" project using an internal scala
DSL :)

Maybe the most important thing we all need to do is just chill... of course
a great UI won't hurt the chilling process!

------

About Big data - yeah. Kind'a easy to explain to someone: map-reduce this
sh!t. somehow that sounds better to the masses than "flat-map this sh!t"?

That however will be just a stepping stone kicking in new paradigms... Yo,
Greg - how's your book coming along?

------

To answer the original "what scala needs to succeed in the enterprise" ?

Karma!

Cheers,
RAzie

-----Original Message-----
From: scala...@googlegroups.com [mailto:scala...@googlegroups.com] On
Behalf Of jherber
Sent: November-10-11 9:24 AM
To: scala-user
Subject: [scala-user] Re: What is highest priority for Scala to succeed in
corporate world (Should be in scala-debate?) ?