Why I and my company have moved away from Scala. An honest critique.

[kraythe]

I work in a company that uses the Playframework and Akka to implement a business system. For a long time i was trying to push the implementation towards Scala and away from java. However, I have ultimately given up on that effort and have almost completed refactoring the work to Java. Here is why. 

First of all the learning curve on Scala is very steep but unnecessarily so. Scala syntax is very brief and but that isn't what holds up the learning curve, nor is it the functional programming concepts, functions as first class entities. These are in fact the good facets of Scala. Pattern matching is absolutely awesome, immutable collections are something that Java is really lacking. In short there is a ton of good things in Scala that attract a developer from Java. But the problem is Scala shoots itself in the foot. 

Without too much fanfare I will declare that Implicit conversions and Macros make the language very difficult to learn and, outside the academic world, almost inaccessible to business. Implicit conversions and indeed the entire 'implicit" keyword is just lethal to the learning curve as well as code maintenance. In a business world where effort is judged in ROI, anything that just magically transparently happens is a disaster waiting to happen. In fact such disasters do happen. We found that implicits float around in our code interacting in strange ways that were non-deterministic and opaque to reading the code. The beginner looking at a method wonders, "wait how can you call that method on that object?" Then they think and say "oh there is an implicit somewhere in some of my imports somewhere down the tree that convert the type. Its obscure, opaque, confusing and the antithesis of everything that an experienced business developer wants. I used to think that it was just a matter of tooling but IntelliJ has actually decent scala tooling and still implicits are a total disaster. Second problem is Macros. Like implicits they make things happen that are difficult to discern by looking at the code. The introduction of both of these and their rampant use in the language is, in my opinion, going to push Scala to be yet another niche language like lisp, scheme, Erlang and so on. 

Secondly is the compilation tooling and interaction with java. Scala interacts transparently with java. Well, at least that is what the promotional materials say. In practice to achieve that interaction is a major PITA and I am not talking about flat bread here. Conversions of types require gymnastics. For example, try passing a list of Scala longs to a java method that takes a list of java.util.Long objects. Gymnastics. The function lambdas are non translatable so you cant just pass a Scala lambda to a java function that takes a java.util.Function. You can doctor this with ... wait for it ... an implicit conversion that wraps one into the other. Sigh. In our codebase we had tons of code in both languages and it was just a nightmare going between the two. The tooling doesn't help either because SBT gets in some wacked out logic loops and simply rebuilds the whole project multiple times. After years of Scala development SBT is still slow, and in such a bad state that it isn't even at 1.0. 

I think Scala has become too academic. Too controlled by those with PhDs. in computer science figuring out what they can do, rather than wondering if they actually should do these things and as a result the language has abandoned some of its roots. It started out as an alternative to Ruby and Java. It was a statically bound deterministic language with functional and Object oriented concepts but it has now strayed into things that happen "magically" and code that is generated with Macros. I personally think one of two things will happen. Either the maintainers of Scala will reverse this direction and get back to basics, dump the implicits and macros and find more deterministic ways to do things. Or, quite frankly, it will die on the vine as another Erlang, Lisp, Node.js or you name it. A language used in a small limited sector for a few projects but without wide acceptance. 

I, for one would think that is a shame, a horrible waste. Scala could be the next super language if academics stop messing with it like mad scientists with a new toy. Until then, I cant have my whole staff down six months learning scala. 

-- Robert, 

-- Principal Software Architect. 

[Dennis Haupt]

have you heard about kotlin? sounds like it's what you want

 

Gesendet: Sonntag, 12. Februar 2017 um 18:16 Uhr
Von: kraythe <kra...@gmail.com>
An: scala-user <scala...@googlegroups.com>
Betreff: [scala-user] Why I and my company have moved away from Scala. An honest critique.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Adriaan Moors]

I never thought of myself as a mad scientist, but now that you mention it, I finally know what to put on my next batch of Lightbend business cards!

Thanks for the feedback (although it came with a slight sense of déjà vu -- where have I seen your post before?). I am proud of the progress that we've made with making Scala a language for everyone to be productive with, and my team is hard at work making it even better -- essentially along the lines that you describe.

In the mean time, I think the sharp edges such as macros and implicits are well documented. Avoid them when they don't warrant the investment. When you do need them, you do need them.

Cheers
Adriaan

Scala tech lead @ Lightbend.
Mad Scientist. PhD. Safe typer.

[Patrik Andersson]

Mr krythe started out his journy through Scala-land with these kinds of posts.

[Rex Kerr]

It's unfortunate that Scala hasn't been working out as well as hoped for your team.

There are three issues that you address: (1) is Scala working well for your team, and if not why not?; (2) is Scala suitable for business and/or other widespread (non-niche) use?; and (3) to the extent that there is a problem, is it because the development is driven by too many Ph.D.'s?

For your own productivity and the benefit of people with teams like yours, it's best not to mix the three up too much. 

Taking the first two, "It didn't work for us" is different than "it doesn't work for anyone in business", and both are different than "it is a niche language".

For instance, Python isn't exactly a niche language, and its "monkey-patching" is way, way harder to track down than implicits are.  C and C++--not exactly niche languages!--have #define which can make anything look like anything else, pretty much.  Even in Java, reflection can be used to make pretty much everything impossible to follow.

None of these things have prevented these languages from enjoying widespread use, because a combination of lore (i.e. "best practices") and tooling can be employed to tame the worst of the problems while still benefiting from the advantages of the feature or at least of the language as a whole.

And, I would argue, exactly the same is true of Scala.

So I don't think you make a very compelling case that there is a problem with (2) in general, though there may certainly be difficulties for various application areas.

On the other hand, (1) is undoubtedly true if you have felt it necessary to rewrite in Java.  I have had trouble fully understanding why from the reasons you gave, which makes it difficult to either think of how to improve Scala (without turning it into a different language) or understand how to warn others not to fall into the same difficulties you did.

For instance, you complain about implicits and mention that the tooling helps but still isn't enough.  But then, when implicits stand to rescue you in an easy-to-inspect way from "gymnastics", you blame them?  I don't understand how having a compact library that transforms huge boilerplatey gymnastics into something that happens automatically or with what looks like a simple method call is a problem.

So I can't really tell if you mean, "The way Play and Akka use implicits is too extensive and we can't find any practices that adequately keep them under control", or you mean, "We used implicits heavily and now we can't keep track of anything any more".  If the former, well, that's no consolation to you as a user of the library, but you needn't consign the entire language to "niche" because someone can (and presumably will) write a more business-friendly set of libraries if there's demand for it.  If the latter, well, sadly there aren't great resources for learning Scala best practices the way there are for Java.  It's kind of an organic process.  Maybe you ended up with a chapter or two's worth of antipatterns.

Anyway, you wrote a post on this about 11 months ago, and didn't engage with the community to try to find ways to improve the practices of your team, as far as I can tell (e.g. I gave you a bunch of reasons why implicits are better than other solutions in a number of areas, and suggested trying to limit uses where they cause trouble; other people shared other perspectives; but you didn't reply to the thread at all).

So it didn't work out; that's too bad.  But I don't think your post has gone into enough detail to enable others to avoid the same mistakes (except maybe to commiserate if they already had the same problems--"yeah, I hate implicits and macros too").  And it doesn't seem, though it's hard to tell for sure, like you've tried very hard to understand how and why Scala is working well for those who claim it is.

Maybe in a few more years after a few "Scala: Patterns and Antipatterns for Robust Business Systems"-style books, you'll be able to hire people who don't need infeasible amounts of training.  And hopefully tooling will be better yet so the essential information leaps out at you all the more and faster.

But implicits will still be there, if it's Scala, because implicits, despite having the capacity to confuse and mislead, are the best way to tell the compiler to safely take some of the repetitive burden off your hands, so you can remember less, need to pay less attention to minutia, and spend your concentration on big/difficult logic where the real challenge ought to be.

  --Rex

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[kraythe]

Shrug .. Whatever. Was trying to provide an honest critique. I have used and will continue to use Akka, but my staff will use the Java API. I guess this is the reception in this community that you get (not just your post but the other two on the thread) when you post anything critical in the slightest. Not, in my opinion, the best way to engage but heck, it's the internet. Par for the course I guess. Actor Systems are a great idea and Lightbend is a great product, despite you blasting me personally. Furthermore, I even credited many ideas and aspects of the Scala language but if you dare critique it the community drops on you like a ton of bricks? Whatever. What do you think my motivation was in posting this? Boredom? Fame? No, neither. It was a hope that maybe some criticism will make things better but if Scala is so far down the road that criticism is not welcome, then they might as well drop the 'its an evolving language' PR because there is no functional difference between the inertia in the JDK and that of Scala. And yes, I have posted similar concerns before but I gave the language time and benefit of the doubt and now am even more certain my original feeling was correct. 

At any rate I am apparently wasting my time. So don't worry, I will see my own way out. I think the door is over here. I apologize for trying. 

-- Robert

[Vlad Patryshev]

Oh, and Java of course is just perfect, compared to Scala.

Enjoy nulls and NPE, and mutability, and all the dumb frameworks designed by ignorants. You probably won't need generics; Java people just hate generics. They (Java people) are cheaper too.

The world is moving on; you decided to take a rest; it's okay, but don't expect others to join.

I wonder also what would be your opinion of Rust. Also worse than Java? I'm sure it is. No garbage collection. No Hibernate.

Thanks,
-Vlad

On Sun, Feb 12, 2017 at 9:16 AM, kraythe <kra...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[kraythe]

I have some replies inline. 

On Sunday, February 12, 2017 at 3:29:43 PM UTC-6, Rex Kerr wrote:

It's unfortunate that Scala hasn't been working out as well as hoped for your team.

There are three issues that you address: (1) is Scala working well for your team, and if not why not?; (2) is Scala suitable for business and/or other widespread (non-niche) use?; and (3) to the extent that there is a problem, is it because the development is driven by too many Ph.D.'s?

For your own productivity and the benefit of people with teams like yours, it's best not to mix the three up too much. 

Taking the first two, "It didn't work for us" is different than "it doesn't work for anyone in business", and both are different than "it is a niche language".

Given the hiring potential its hard to argue this point. I live and work in one of the biggest tech areas around. Finding Scala people who even know the language, not even looking for experts, is near impossible. Furthermore, I work with a group of people I would consider to be at the top of their game development wise and they have difficulty grasping the language issues. Sure they can absorb the basics quickly but become immensely frustrated at things happening magically and non-deterministic code in the system. These aren't lightweight developers but guys that code million transaction a day systems for breakfast. If it stalls these people how is it going to react to the commodity engineers out there. After 2 years I have the answer to that. it doesn't work at all. This is my empirical observation from business. People can drop scorn on that all they want but its still an empirical observation. 

 

For instance, Python isn't exactly a niche language, and its "monkey-patching" is way, way harder to track down than implicits are.  C and C++--not exactly niche languages!--have #define which can make anything look like anything else, pretty much.  Even in Java, reflection can be used to make pretty much everything impossible to follow.

Which is the reason its much easier for staff a company with Java engineers than either of those languages. Java didnt take over the market because its cross platform. All of that PR is rubbish. It took over because of the difficulties in the C and C++ world made time to market a nightmare. Very few companies other than tool vendors care one whit about cross platform. They care about the fact that they can staff and implement a system to get it to the market in the smallest time. So Scala's answer is to derive a java compatible language, implement some frankly awesome constructs and then completely shoot it in the foot by introducing some of the difficult and "magical" issues of C and C++ that cause the mass dev market to flee from those languages? Perplexing! At least to this developer. 

 

None of these things have prevented these languages from enjoying widespread use, because a combination of lore (i.e. "best practices") and tooling can be employed to tame the worst of the problems while still benefiting from the advantages of the feature or at least of the language as a whole.

A comment on tooling, after years of the language being out, why hasn't the community teamed up with IntelliJ and Eclipse to get the tooling right? Not a priority? Ahh but this is how languages die. 

 

And, I would argue, exactly the same is true of Scala.

So I don't think you make a very compelling case that there is a problem with (2) in general, though there may certainly be difficulties for various application areas.

On the other hand, (1) is undoubtedly true if you have felt it necessary to rewrite in Java.  I have had trouble fully understanding why from the reasons you gave, which makes it difficult to either think of how to improve Scala (without turning it into a different language) or understand how to warn others not to fall into the same difficulties you did.

I just finished converting the vast majority of our Scala code to Java, which was actually a bummer for me because I see a lot of the brilliant things in Scala. However, as a matter of practical business reality it made zero sense to continue the duality. I know people here would say "well you should reacthitect this way or that, or dont use RDBMS or use Akka persistence." Sure, in an academic would I could do that, but if I walk up to my CEO and say, "Hey, lets drop 5 million in capital investment in the system, rebuild it and in a year I will have the same requirements again but it will be way cooler." If I do this, I get fired, and rightly so. 

 

For instance, you complain about implicits and mention that the tooling helps but still isn't enough.  But then, when implicits stand to rescue you in an easy-to-inspect way from "gymnastics", you blame them?  I don't understand how having a compact library that transforms huge boilerplatey gymnastics into something that happens automatically or with what looks like a simple method call is a problem.

So I can't really tell if you mean, "The way Play and Akka use implicits is too extensive and we can't find any practices that adequately keep them under control", or you mean, "We used implicits heavily and now we can't keep track of anything any more".  If the former, well, that's no consolation to you as a user of the library, but you needn't consign the entire language to "niche" because someone can (and presumably will) write a more business-friendly set of libraries if there's demand for it.  If the latter, well, sadly there aren't great resources for learning Scala best practices the way there are for Java.  It's kind of an organic process.  Maybe you ended up with a chapter or two's worth of antipatterns.

What I mean is implicits THEMSELVES are an antipattern. Scala went to a lot of effort to make a statically typed functional programming language and then crushed it with Rubyesque "magic" that is not statically typed. Unfortunate. 

 

Anyway, you wrote a post on this about 11 months ago, and didn't engage with the community to try to find ways to improve the practices of your team, as far as I can tell (e.g. I gave you a bunch of reasons why implicits are better than other solutions in a number of areas, and suggested trying to limit uses where they cause trouble; other people shared other perspectives; but you didn't reply to the thread at all).

I responded to many people but I do work a lot and thread responses sometimes slip through the cracks. Also, being flamed for heresy gets a bit old when all I am trying to do is help. I dont make any money saying this and it certainly doesn't earn me any friends here. Im not by nature masochistic so my motivation must be something else.

 

So it didn't work out; that's too bad.  But I don't think your post has gone into enough detail to enable others to avoid the same mistakes (except maybe to commiserate if they already had the same problems--"yeah, I hate implicits and macros too").  And it doesn't seem, though it's hard to tell for sure, like you've tried very hard to understand how and why Scala is working well for those who claim it is.

You cant avoid implicits in Scala. They are pervasive. 

 

Maybe in a few more years after a few "Scala: Patterns and Antipatterns for Robust Business Systems"-style books, you'll be able to hire people who don't need infeasible amounts of training.  And hopefully tooling will be better yet so the essential information leaps out at you all the more and faster.

Maybe. But then maybe Scala will re-evaluate some features and decide that they were a good thought but didn't work out in reality. But again, I cant afford to have the staff of our small little development company down 6 months and not seriously productive for another 6 months when I could have the product out by then. Learning a cool language is great and all but something has to pay the bills. 

 

But implicits will still be there, if it's Scala, because implicits, despite having the capacity to confuse and mislead, are the best way to tell the compiler to safely take some of the repetitive burden off your hands, so you can remember less, need to pay less attention to minutia, and spend your concentration on big/difficult logic where the real challenge ought to be.

I can think of half a dozen techs that said they would take away repetition and went wrong. Repetition is a part of life for developers. Incurring a framework to keep me from casting an object or wrapping it in a decorator by hand is a little overkill don't you think? 

 

  --Rex

Anyway as I said in my other reply, clearly my opinions arent welcome. Good luck to the community. I tried to help. 

 

On Sun, Feb 12, 2017 at 9:16 AM, kraythe <kra...@gmail.com> wrote:

I work in a company that uses the Playframework and Akka to implement a business system. For a long time i was trying to push the implementation towards Scala and away from java. However, I have ultimately given up on that effort and have almost completed refactoring the work to Java. Here is why. 

First of all the learning curve on Scala is very steep but unnecessarily so. Scala syntax is very brief and but that isn't what holds up the learning curve, nor is it the functional programming concepts, functions as first class entities. These are in fact the good facets of Scala. Pattern matching is absolutely awesome, immutable collections are something that Java is really lacking. In short there is a ton of good things in Scala that attract a developer from Java. But the problem is Scala shoots itself in the foot. 

Without too much fanfare I will declare that Implicit conversions and Macros make the language very difficult to learn and, outside the academic world, almost inaccessible to business. Implicit conversions and indeed the entire 'implicit" keyword is just lethal to the learning curve as well as code maintenance. In a business world where effort is judged in ROI, anything that just magically transparently happens is a disaster waiting to happen. In fact such disasters do happen. We found that implicits float around in our code interacting in strange ways that were non-deterministic and opaque to reading the code. The beginner looking at a method wonders, "wait how can you call that method on that object?" Then they think and say "oh there is an implicit somewhere in some of my imports somewhere down the tree that convert the type. Its obscure, opaque, confusing and the antithesis of everything that an experienced business developer wants. I used to think that it was just a matter of tooling but IntelliJ has actually decent scala tooling and still implicits are a total disaster. Second problem is Macros. Like implicits they make things happen that are difficult to discern by looking at the code. The introduction of both of these and their rampant use in the language is, in my opinion, going to push Scala to be yet another niche language like lisp, scheme, Erlang and so on. 

Secondly is the compilation tooling and interaction with java. Scala interacts transparently with java. Well, at least that is what the promotional materials say. In practice to achieve that interaction is a major PITA and I am not talking about flat bread here. Conversions of types require gymnastics. For example, try passing a list of Scala longs to a java method that takes a list of java.util.Long objects. Gymnastics. The function lambdas are non translatable so you cant just pass a Scala lambda to a java function that takes a java.util.Function. You can doctor this with ... wait for it ... an implicit conversion that wraps one into the other. Sigh. In our codebase we had tons of code in both languages and it was just a nightmare going between the two. The tooling doesn't help either because SBT gets in some wacked out logic loops and simply rebuilds the whole project multiple times. After years of Scala development SBT is still slow, and in such a bad state that it isn't even at 1.0. 

I think Scala has become too academic. Too controlled by those with PhDs. in computer science figuring out what they can do, rather than wondering if they actually should do these things and as a result the language has abandoned some of its roots. It started out as an alternative to Ruby and Java. It was a statically bound deterministic language with functional and Object oriented concepts but it has now strayed into things that happen "magically" and code that is generated with Macros. I personally think one of two things will happen. Either the maintainers of Scala will reverse this direction and get back to basics, dump the implicits and macros and find more deterministic ways to do things. Or, quite frankly, it will die on the vine as another Erlang, Lisp, Node.js or you name it. A language used in a small limited sector for a few projects but without wide acceptance. 

I, for one would think that is a shame, a horrible waste. Scala could be the next super language if academics stop messing with it like mad scientists with a new toy. Until then, I cant have my whole staff down six months learning scala. 

-- Robert, 

-- Principal Software Architect. 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

[kraythe]

No its not. Scala has gained ZERO significant traction in the business world. It is not the next java. 

Thanks,
-Vlad

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

[Rex Kerr]

On Sun, Feb 12, 2017 at 1:52 PM, kraythe <kra...@gmail.com> wrote:

I have some replies inline. 

On Sunday, February 12, 2017 at 3:29:43 PM UTC-6, Rex Kerr wrote:

It's unfortunate that Scala hasn't been working out as well as hoped for your team.

There are three issues that you address: (1) is Scala working well for your team, and if not why not?; (2) is Scala suitable for business and/or other widespread (non-niche) use?; and (3) to the extent that there is a problem, is it because the development is driven by too many Ph.D.'s?

For your own productivity and the benefit of people with teams like yours, it's best not to mix the three up too much. 

Taking the first two, "It didn't work for us" is different than "it doesn't work for anyone in business", and both are different than "it is a niche language".

Given the hiring potential its hard to argue this point. I live and work in one of the biggest tech areas around. Finding Scala people who even know the language, not even looking for experts, is near impossible.

I grant it isn't easy.  This doesn't mean that Scala isn't destined to be a niche language, though.  Its rise is not as meteoric as Java's was, admittedly.

 

Furthermore, I work with a group of people I would consider to be at the top of their game development wise and they have difficulty grasping the language issues.

Maybe you should tell us more about what you think a reasonable training/learning scenario is?  I mean, it's totally fair to say, "You know, I have these top-notch Java devs with 15 years of experience apiece and dropping them into Scala lowers their productivity and makes them make a bunch of newbie mistakes."  I'd expect that, actually.  So if your take home message is, "Scala isn't a good language for rapid productivity gains when you're staffed with top-notch Java developers," it sounds eminently plausible.

 

Sure they can absorb the basics quickly but become immensely frustrated at things happening magically and non-deterministic code in the system.

Implicit resolution is deterministic.  Any time it isn't, that's a bug.  Macros should be deterministic (but it's technically possible to make them not).

 

These aren't lightweight developers but guys that code million transaction a day systems for breakfast.

I don't see how handling a million transactions a day in one language is going to make you an automatic expert in another language.  You're no lightweight, obviously, if you can do that in *any* language.  But it doesn't mean from knowing Java you'll just be able to pick up, I don't know, Node.js, and immediately do as well with it as the Node.js experts.

 

If it stalls these people how is it going to react to the commodity engineers out there. After 2 years I have the answer to that. it doesn't work at all. This is my empirical observation from business. People can drop scorn on that all they want but its still an empirical observation.

Sure, but you asserted in previous years that you *weren't* willing to invest in more than minimal training.  With *that* proviso, sure.  If your language isn't just "Java with better syntax and an immutable collections library", you may well run into problems.

Kotlin and Ceylon are both trying harder to be Java with better syntax.  (Groovy 3 too, I guess, now that some types are in evidence.)  So far, the market response seems to be, "Meh, you know, that's kind of nice but not really enough to be worth the effort of switching/adding a language."

 

 

For instance, Python isn't exactly a niche language, and its "monkey-patching" is way, way harder to track down than implicits are.  C and C++--not exactly niche languages!--have #define which can make anything look like anything else, pretty much.  Even in Java, reflection can be used to make pretty much everything impossible to follow.

Which is the reason its much easier for staff a company with Java engineers than either of those languages. Java didnt take over the market because its cross platform.

So you basically are dividing the world into "Java" and "niche"?  Fair enough.  It could well be that Scala remains a "niche" language.

It's really hard to tell if that is the only standard we're measuring by.

Otherwise, I reiterate: other languages that by most standards are not considered niche have features that are a lot worse, so the presence of the features you don't like isn't necessarily enough to keep a language as a niche language.

 

What I mean is implicits THEMSELVES are an antipattern. Scala went to a lot of effort to make a statically typed functional programming language and then crushed it with Rubyesque "magic" that is not statically typed. Unfortunate.

I am baffled by how you can say this and claim to be more than marginally familiar with Scala.

Implicits are completely statically typed.  In fact, they are the main user-invokable mechanism for performing type computations.  They underlie a huge amount of the machinery in libraries that do advanced type wizardry like scalaz, cats, and shapeless.

Maybe you mean something other than what you're saying.

Do you mean "explicitly typed" rather than "statically typed"?

 

You cant avoid implicits in Scala. They are pervasive. 

This is true, but now this just sounds like I Hate Implicits For Reasons I Can't Express, because you've only made some handwavy comments about running into trouble with them, and I and (some) others don't run into (much) trouble with them.

I assume you/your team is doing something else with them than I am if you are *actually* having problems rather than just recoiling in horror when you realize one is there.

Again, consider Java reflection for comparison.  You can get amazingly incomprehensible and awful stuff to happen with reflection.  So Java devs don't do that.  (Except occasionally when they must, and in almost every case I've seen there's a much clearer and less error-prone solution with implicits in Scala.)  They don't say, "Oh, both these things have a `close` method but they're not related, so we'll make a method that takes an `Object` and reflect", because BOOM! type safety gone, runtime errors galore.  Instead, they either write a box that encapsulates the two different things with close methods, or they overload the method to take the two different things at the position of that argument.

So--is the *problematic* part of implicits pervasive?

Or is it just once burned twice shy, so you want to get rid of all aspects of it, and can't?  (There, I agree--they are too pervasive to get rid of the non-problematic aspects.)

 

But implicits will still be there, if it's Scala, because implicits, despite having the capacity to confuse and mislead, are the best way to tell the compiler to safely take some of the repetitive burden off your hands, so you can remember less, need to pay less attention to minutia, and spend your concentration on big/difficult logic where the real challenge ought to be.

I can think of half a dozen techs that said they would take away repetition and went wrong. Repetition is a part of life for developers. Incurring a framework to keep me from casting an object or wrapping it in a decorator by hand is a little overkill don't you think? 

No, it's not overkill if the repetition and boilerplate is bad enough that it is one of your two main points.

Java has 143 different SAM function types.  Being able to just bam! .asJava a Scala function you've already got to get the right one in Java saves an amazing amount of work.  Trying do do it manually is awful.  All your logic is completely lost inside boilerplate.

Repetition is part of life but it doesn't mean it's desirable.

  --Rex

[Kevin Wright]

“Scala has gained ZERO significant traction in the business world.”

Assuming that you don’t want your statements to be immediately labeled as FUD (and they certainly look to be tending that way), you’ll need to specify here how you’re defining both “significant traction” and “business world” in order to justify such a claim.

My personal experience would lead me to conclude the exact opposite; specifically as regards the rate at which I’m approached by headhunters/recruiters, and how “business worldy” their clients are… presumably investment banks, hedge funds, petro-chemicals, media conglomerates, and other assorted multinationals can all be reasonably classed as belonging to the business world.

[Russ P.]

On Sunday, February 12, 2017 at 2:51:04 PM UTC-8, Rex Kerr wrote:

On Sun, Feb 12, 2017 at 1:52 PM, kraythe <kra...@gmail.com> wrote:

I have some replies inline. 

What I mean is implicits THEMSELVES are an antipattern. Scala went to a lot of effort to make a statically typed functional programming language and then crushed it with Rubyesque "magic" that is not statically typed. Unfortunate.

I am baffled by how you can say this and claim to be more than marginally familiar with Scala.

Implicits are completely statically typed.  In fact, they are the main user-invokable mechanism for performing type computations.  They underlie a huge amount of the machinery in libraries that do advanced type wizardry like scalaz, cats, and shapeless.

Yeah, that one puzzled me too. Implicits not statically typed? Say what? No way.

As an aerospace engineer who works fairly independently, I am not qualified to weigh in on the issue of Scala adoption in industry. However, I will say that I am very glad that I have the option of using Scala instead of Java. As far as I am concerned, there is no comparison in terms of elegance and simplicity once you learn to use Scala properly (e.g., lots of immutable case classes). If someone forced me to switch from Scala to Java, I would quit programming and find something else to do (programming is not a required part of my job).

As for implicits, I use them fairly rarely, but they do come in handy at times. For example, I have a implicit class that I call Vectorx, which extends the Scala Vector to do standard vector addition, subtraction, and multiplication or division by a scalar. It works beautifully, and it saved me the pain of finding another name for what really should be called a Vector (because it is implicit, the name Vectorx never needs to be explicitly invoked by the user).

[Vlad Patryshev]

Thank you Russ! That's an exemplary attitude toward judgments. Our knowledge, our scope is limited. Nobody can speak for the whole world of business, the whole scala community, the whole FP or math. We have our partial opinions, and share them. Generalization is no good even if we talk about Church thesis.

Thanks,
-Vlad

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[kraythe]

What I meant is that implicits are. It directly linked to code you can see. If you look at a method and see the code it is not apparent that an implicit is involved. Because they happen "magically.

But I got the message. It was "F off, your opinions aren't wanted no matter how honestly you offer them." Message received. No more reason for me to post on this thread or indeed group anymore. Twitter abandoned Scalia. Linked in abandoned Scala. So by all means keep beating people upside the head when they offer constructive advice. Could have been wonderful but toss it on the heap of could have been languages.

Peace and good luck.

[Dennis Haupt]

my ide (intellij) highlights implicit conversions. it's the same for "call by name" parameters. this problem only exists for text editors, and in those, everything is magic because i can't jump to declaractions or check the type of an expression

 

Gesendet: Dienstag, 14. Februar 2017 um 07:45 Uhr

Von: kraythe <kra...@gmail.com>
An: scala-user <scala...@googlegroups.com>

Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

What I meant is that implicits are. It directly linked to code you can see. If you look at a method and see the code it is not apparent that an implicit is involved. Because they happen "magically.

But I got the message. It was "F off, your opinions aren't wanted no matter how honestly you offer them." Message received. No more reason for me to post on this thread or indeed group anymore. Twitter abandoned Scalia. Linked in abandoned Scala. So by all means keep beating people upside the head when they offer constructive advice. Could have been wonderful but toss it on the heap of could have been languages.

Peace and good luck.

[Filippo De Luca]

To be fair, this is everything but not an honest and constructive critique. Your statement:

> I think Scala has become too academic. Too controlled by those with PhDs. in computer science figuring out what they can do, rather than wondering if they actually should do these things and as a result the language has abandoned some of its roots. It started out as an alternative to Ruby and Java.

Is not constructive and quite aggressive and populist. `those with PhDs` are the same people that we have to thank for the progress. Also saying that Scala was an alternative to Ruby is a nonsense. As Ruby and scala have completely different use cases, and so Ruby and Java.

You say Twitter and Linkedin have abandoned Scala, fair enough, but at the same time other, much bigger companies, like Expedia, are embracing it because more and more developers want to write Scala code.

My 2cents.

--

Filippo De Luca about.me/FilippoDeLuca

[jez prime]

With respect, I agree that implicits and the "magic" therein can mean that what's going on isn't immediately obvious to an external caller of functions involving them. 

That said, lots of Java programmers use Spring and its magic without worrying about the fact that they don't know all about what's going on under the hood during the wiring/bean discovery processes (and the vast majority simply don't know, they just accept the Spring magic as long as it works). I don't really see why implicit magic at compile time is any worse than Spring magic at runtime. There are very strong arguments in favour of using implicits around/in (where necessary) to make code calling APIs, and its intent, much easier to read. I'm sure they can be overused, but it's also true that over the last few weeks, working on an API I'm writing in Java, I've more than once wished for implicits and implicit conversions to make a few very specific scenarios much easier to read and work with, both for me and the people who will be consuming the API. I'd prefer to write it in Scala but that's not an option given it will belong primarily to a Java-based team over which I do not have any control). 

So yes, implicits can be a two-edged sword. They can also be a fabulous tool for helping convey intent more obviously by eliminating boilerplate adapters. On balance, while I wouldn't go out of my way to use them, there are definitely situations where I would vastly prefer to have them in my toolkit. 

I won't comment on macros as, frankly, I've never used them at all so I haven't run into the pain points in any way. 

[martin odersky]

On Tue, Feb 14, 2017 at 12:21 PM, Filippo De Luca <m...@filippodeluca.com> wrote:

To be fair, this is everything but not an honest and constructive critique. Your statement:

> I think Scala has become too academic. Too controlled by those with PhDs. in computer science figuring out what they can do, rather than wondering if they actually should do these things and as a result the language has abandoned some of its roots. It started out as an alternative to Ruby and Java.

Is not constructive and quite aggressive and populist. `those with PhDs` are the same people that we have to thank for the progress. Also saying that Scala was an alternative to Ruby is a nonsense. As Ruby and scala have completely different use cases, and so Ruby and Java.

You say Twitter and Linkedin have abandoned Scala, fair enough, but at the same time other, much bigger companies, like Expedia, are embracing it because more and more developers want to write Scala code.

For what it's worth Twitter just joined the Scala Center advisory board as a full member. Don't believe the FUD.

 

 

My 2cents.

On Tue, Feb 14, 2017 at 6:55 AM Dennis Haupt <h-s...@gmx.de> wrote:

my ide (intellij) highlights implicit conversions. it's the same for "call by name" parameters. this problem only exists for text editors, and in those, everything is magic because i can't jump to declaractions or check the type of an expression

 

Gesendet: Dienstag, 14. Februar 2017 um 07:45 Uhr

Von: kraythe <kra...@gmail.com>
An: scala-user <scala...@googlegroups.com>

Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

What I meant is that implicits are. It directly linked to code you can see. If you look at a method and see the code it is not apparent that an implicit is involved. Because they happen "magically.

But I got the message. It was "F off, your opinions aren't wanted no matter how honestly you offer them." Message received. No more reason for me to post on this thread or indeed group anymore. Twitter abandoned Scalia. Linked in abandoned Scala. So by all means keep beating people upside the head when they offer constructive advice. Could have been wonderful but toss it on the heap of could have been languages.

Peace and good luck.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Filippo De Luca about.me/FilippoDeLuca

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Martin Odersky
EPFL and Lightbend

[Seyed H. HAERI (Hossein)]

Dear Robert,

Being a C++ person, I too have had a lot of headache getting the Scala
community to help when one spares an honest criticism about the
language. Alas, I can't witness it getting any better. Yet, I think
Rex does have some valid point in trying to separate concerns. Maybe
his (1), (2), and (3) were not exactly the best. But, that's at least
an effort to see what lessons the language designers can take out of
your report.

Social aspects aside, there is something technical in your words which
absolutely tickles me to respond:

> What I mean is implicits THEMSELVES are an antipattern. Scala went to a lot of effort to make a statically typed functional programming language and then crushed it with Rubyesque "magic" that is not statically typed. Unfortunate.

I cannot help falling in love with your above words. :)

Being a C++ person, I am used to all sorts of gotwas when it comes to
implicit conversions. Yet, I am amused by the C++ way of doing it and
have been severely beaten up by the Scala implicits many times. It
took me a lot of time to figure out what the reasons can be. It turns
out that the kind of applications I deal with all require what is
called "lightweight family polymorphism"; whereas, Scala is pro
"family polymorphism". (Sorry for the PL jargon; I'm a PL maverick.)
The specially relevant consequence is that the Scala implicits work
(and do creepy things) at runtime; whereas, the C++ conversions are
secured at compile-time. A Scala implicit, being an object, (by
design) enjoys all sorts of late-binding. In C++, however, the exact
conversions are completely known statically. Clearly, the former comes
with extra flexibility. But, for me too, the price is way beyond
affordable for it makes code unpredictable and out of the programmer's
control.

And, BTW, C++ has constantly been loosing market share for its
excessive difficulty to learn. Java, OTOH, ramped up because it lucky
coincidence to facilitate massive rubbish development with the time
when people could no longer focus on efficiency due to physical
barriers.

Cheers,
--Hossein

[Dennis Haupt]

i do not share this opinion. i never had a problem with understanding implicits once i understood the concept (unless it was a code puzzle intended to confuse)

 

Gesendet: Dienstag, 14. Februar 2017 um 16:35 Uhr
Von: "Seyed H. HAERI (Hossein)" <hossei...@gmail.com>
An: Kein Empfänger
Cc: scala-user <scala...@googlegroups.com>

Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

[Dennis Haupt]

implicits are statically typed - this was already mentioned. 

 

Gesendet: Dienstag, 14. Februar 2017 um 16:35 Uhr
Von: "Seyed H. HAERI (Hossein)" <hossei...@gmail.com>
An: Kein Empfänger
Cc: scala-user <scala...@googlegroups.com>
Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

[Seyed H. HAERI (Hossein)]

Maybe you're right Dennis and I simply haven't still understood
implicits. :p But, did you ever had to simulate lightweight family
polymorphism in Scala? I suspect not. Because, me personally aside,
doing that is known in the PL community to be next-to-impossible for a
language supporting only family polymorphism. See the nice work of
Madsen and Ernst [1], for example.

Cheers,
--Hossein

[1] A. B. Madsen and E. Ernst, Revisiting Parametric Types and Virtual
Classes, Obj., Models, Components, Patterns, 48th Int. Conf. Proc. (J.
Vitek, ed.), Lect. Notes in Comp. Sci., vol. 6141, Springer, June
2010, pp. 233--252.

--
--------------------------------------------------------------------------------------------------------------

Seyed H. HAERI (Hossein), Dr.

Post-Doctoral Research Fellow
Department of Computing Science and Engineering
Catholic University of Louvain
Louvain-la-Neuve, Belgium

ACCU - Professionalism in programming - http://www.accu.org/
--------------------------------------------------------------------------------------------------------------

[Seyed H. HAERI (Hossein)]

Late-binding and static typing can be orthogonal. My hunch is that
Robert meant the former and he simply didn't have the terminology. (No
offence meant Robert.)

[Jasper-M]

Hossein,

I'm sorry but you are very much mistaken. Implicits are 100% compile time statically checked concepts.

The point being made by Robert, for quite a few years now, is that "Scala will never penetrate the business world" like Java, because of the magical things that happen with implicits. The irony here is that in basically all popular Java frameworks a lot of the magical stuff happens completely at runtime with reflection. In Scala every magical thing happens at compile time and is statically checked by the compiler, unless you are using runtime reflection but almost nobody does that in Scala.

I can agree that if implicit conversions are used excessively things can get out of hand. That's why there already has been an effort to make it more difficult to define implicit conversions without knowing the consequences. Perhaps it should be made even more difficult.

Stating that Scala would be a great language, if only it did not have this, this, and this concept, but would have this, this and this new keyword, is like saying "ice cream is the best dessert in the whole world, but as long as it's sweet and cold it will be a niche dessert for all eternity".

Robert's communication has always been pretty aggressive on this mailing list. That might not have been his intention, but it is the case either way. And in spite of that most community members have always gone to great lengths to help him out and go into discussions and have tried their best to be polite. Yet every counter-argument seems be received as a personal attack or flame against him. It's hard not to flame a person and at the same time help him or have a healthy discussion if that means not disagreeing at all.

Kind regards,

Jasper

Op dinsdag 14 februari 2017 16:35:55 UTC+1 schreef Hossein:

[Kris Nuttycombe]

Among long-time users of Scala, you'll find something interesting with respect to their use of implicits - that is, that long-time users barely remember that implicit conversions exist, except insofar as implicit classes are used as a mechanism to provide extension methods, frequently in combination with the "implicit typeclass" pattern. We forget that in the broader Scala world, people still use direct implicit type conversions to "magically" transform a value of an unrelated source type to a target type that is used directly. We've forgotten about the fact that some people use *stateful* implicit values to control how their program functions. We've forgotten these antipatterns because we learned that they're catastrophically bad ideas nearly a decade ago, and have abandoned their use ever since. 

The problem is that for many, these inappropriate uses of implicits are their first introduction to the concept, and consequently they properly react in horror at the whole feature, without realizing the power of their principled use. If one was able to go back in time, I think one might choose to make implicit classes a language feature, and promote the use of (stateless) implicit parameters, but would otherwise eliminate implicit conversions from the language.

All that being said, many of Robert's complaints seem somewhat disingenuous to me. He's clearly upset about Scala, but at the same time I have to wonder whether he's ever taken the opportunity to look at any Scala codebase other than his own. The (purely) functional Scala way of doing things can certainly appear alien, but I have yet to find anyone who, after having the power of real compositionality demonstrated to them, wouldn't concede that this approach is really the correct one for at least some people.

Kris

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[Adriaan Moors]

Thanks, Kris, and everyone else for sharing your thoughtful feedback.

The point about antipatterns is an important one. All languages have those, and all language communities acquire cultural knowledge about how to avoid them. It's up to us (the creators and the community together) to document them and share the wisdom with new members of the community. As the creators, we are also always looking for ways to have the language itself guide you away from the antipatterns, where possible. 

I believe Scala does a good job at shielding you from many existing antipatterns, but implicits (and their interaction with type inference, which may look like magic, even though it is deterministic) are a harder nut to crack in terms of finding a good balance in the (new) user experience and expressive power.

I also would like to reiterate that we value everyone's opinion (that includes criticism, obviously), and we take all feedback to heart. I prefer the actionable kind of feedback, though (I'm not about to give up being a mad scientist ;-)). We care deeply about the productivity of all Scala programmers out there, and are dedicated to improving your experience with Scala in every release.

cheers

adriaan

Scala team lead at Lightbend

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

[Eduardo M. Cavalcanti]

I can agree that if implicit conversions are used excessively things can get out of hand. That's why there already has been an effort to make it more difficult to define implicit conversions without knowing the consequences. Perhaps it should be made even more difficult.

 

Hi, I´ve been getting along with implicits very well. No big deal. Please do not restrict them. Scala evolution has been in very good track.

 

Cheers!

 

[Dennis Haupt]

make it difficult: no, that would defeat the purpose

make the developer sign a contract where he states that he will to be blamed for misuse: yes

 

Gesendet: Dienstag, 14. Februar 2017 um 19:12 Uhr
Von: "Eduardo M. Cavalcanti" <eduardo.m....@uol.com.br>
An: scala-user <scala...@googlegroups.com>

Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

I can agree that if implicit conversions are used excessively things can get out of hand. That's why there already has been an effort to make it more difficult to define implicit conversions without knowing the consequences. Perhaps it should be made even more difficult.

 

Hi, I´ve been getting along with implicits very well. No big deal. Please do not restrict them. Scala evolution has been in very good track.

 

Cheers!

 

 

--

[Eduardo M. Cavalcanti]

Agreed. That is how it already is. Isn´t it?

 

import language.implicitConversions

 

 

---

De: h-s...@gmx.de
Enviada: Terça-feira, 14 de Fevereiro de 2017 19:34
Para: eduardo.m....@uol.com.br
Assunto: Aw: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

make it difficult: no, that would defeat the purpose

make the developer sign a contract where he states that he will to be blamed for misuse: yes

Gesendet: Dienstag, 14. Februar 2017 um 19:12 Uhr
Von: "Eduardo M. Cavalcanti" <eduardo.m....@uol.com.br>
An: scala-user <scala...@googlegroups.com>
Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

I can agree that if implicit conversions are used excessively things can get out of hand. That's why there already has been an effort to make it more difficult to define implicit conversions without knowing the consequences. Perhaps it should be made even more difficult.

 

Hi, I´ve been getting along with implicits very well. No big deal. Please do not restrict them. Scala evolution has been in very good track.

 

Cheers!

 

 

--

[Kris Nuttycombe]

I disagree, to a point. What we choose to make difficult in a language is as important as what we choose to make easy - what we choose to make difficult tends to shape the entire ecosystem of a language. Rust chooses to make concurrent nonlocal modification of memory difficult, and this has an effect upon their whole ecosystem; Haskell chooses to make uncontrolled IO difficult, and this has a profound effect. In both cases, the ecosystem of libraries is significantly better for these choices.

The feature-flag-based disabling of implicit conversions was one of the few parts of SIP-18 that I supported. I guess though that this thread is evidence that the SIP-18 changes to feature visibility didn't have that great of an effect. In retrospect that's predictable; when desirable features like higher-kinded types get lumped in with unsafe features like implicit conversions, it's hard for users to use this modularization as a guideline for which features to use and which not. I know that I face the same confusion when exploring the myriad of Haskell extensions; I always have to go read blog posts to find out whether an extension I'm considering using is widely considered desirable or not.

Kris

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[Seyed H. HAERI (Hossein)]

Hi Jasper,

> I'm sorry but you are very much mistaken. Implicits are 100% compile time
> statically checked concepts.

Is an implicit not just like an ordinary object in that it enjoys late
binding? It is. Then, I'm afraid, they are not 100% compile-time. They
might have been if Scala had chosen to take a classes-as-families [1]
approach to family polymorphism. Right now, it's objects-as-families
-- like the original family polymorphism crafted by Ernst and all
those beautiful following works...

Cheers,
--Hossein

[1] P. Jolly, S. Drossopoulou, C. Anderson, and K. Ostermann, Simple
Dependent Types: Concord, Proc. 6th W. Formal Tech. Java-like Prog.,
June 2004, Tech. Rep. nr. NIII-R0426, Uni. Nijmegen.

[Seyed H. HAERI (Hossein)]

Hi Kris,

I liked your response for collecting some of the anti-patterns related
to implicits. Has anyone ever done a comprehensive list of such
anti-patterns?

Cheers,
--Hossein

>> email to scala-user+...@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "scala-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to scala-user+...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--

[Adriaan Moors]

I think what Jasper meant was that implicit resolution happens at compile time, and that the static type of the required implicit value directly determines the expression that will be evaluated to produce a value of that type (of course during run-time evaluation there's late binding, but that's orthogonal to implicit search). 

Implicit resolution provides the same kind of ad-hoc polymorphism as method overloading, where the static types of the arguments determine which method overload is chosen, or Haskell's type classes, where the instance is directly defined by the type class. Here's a paper we wrote on the equivalence between type classes and implicits: https://ropas.snu.ac.kr/~bruno/papers/TypeClasses.pdf

cheers

adriaan

[Dennis Haupt]

i don't know what late binding is, but i know all the magic happens at compile time. 

also: one of the criticisms was that scala is "too phd", but now i see the anti scala side argue using fancy expressions that i do not understand. just wanted to point that out because... contradiction :D

 

Gesendet: Dienstag, 14. Februar 2017 um 23:58 Uhr
Von: "Seyed H. HAERI (Hossein)" <hossei...@gmail.com>
An: Kein Empfänger
Cc: scala-user <scala...@googlegroups.com>

Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

[Adriaan Moors]

Late binding is also called virtual dispatch. When you call a method, you don't know which subclass implementation of the method you'll get. All you know is that it's there (thanks to static typing ;-)).

I think of the type system as an approximation of what your program could do when it runs. Late binding introduces the needed slack between the type checker's rigor and the run-time environment's "open-endedness".

[Dennis Haupt]

i don't think it really makes sense to collect a full list of possible antipatterns regarding implicits. it would be like asking a master chef to write down things like "do not use too much salt" and then try to cook an awesome cake

it's the wrong level

 

the opposite should be done: "how should i use implicits? what are they good for?"

my answer to that is "they should do what i otherwise would do manually every time"

 

Gesendet: Mittwoch, 15. Februar 2017 um 00:05 Uhr

Von: "Seyed H. HAERI (Hossein)" <hossei...@gmail.com>
An: Kein Empfänger
Cc: scala-user <scala...@googlegroups.com>
Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

[Seyed H. HAERI (Hossein)]

Evening Adriaan,

> I think what Jasper meant was that implicit resolution happens at compile
> time, and that the static type of the required implicit value directly
> determines the expression that will be evaluated to produce a value of that
> type (of course during run-time evaluation there's late binding, but that's
> orthogonal to implicit search).

Good point. I'm afraid though path-dependant types can kick in. Do I
not get it correctly that path-dependant types are also considered
static in Scala? If they are, then, the late-binding can indeed play a
significant role.

> Implicit resolution provides the same kind of ad-hoc polymorphism as method
> overloading, where the static types of the arguments determine which method
> overload is chosen, or Haskell's type classes, where the instance is
> directly defined by the type class. Here's a paper we wrote on the
> equivalence between type classes and implicits:
> https://ropas.snu.ac.kr/~bruno/papers/TypeClasses.pdf

Yes, I read that work in the exact same year of its publication.
Splendid work! :)

Cheers,
--Hossein

[Adriaan Moors]

You can absolutely statically remove all implicit "magic" in the IDE before compiling without changing the behavior of your program. This is what we mean when we say implicit resolution is static. IntelliJ offers a refactoring that does this (and much more!). The same idea applies to method overloading. You could avoid overloading by name mangling yourself, instead of having the compiler do it based on the static types of the arguments used in an overloaded method call. It would just be tedious.

I think this is getting a bit too technical for this forum, but path equivalence is decided statically, so, there's no uncertainty there either. Of course, every time you select a member on a path, you don't statically know the run-time value of the target. This is where late binding kicks in for value members. For type members, this is not the case, however: there is no notion of late-bound types in Scala, as opposed to Beta, Ernst's language (which leads to some interesting language design to maintain soundness).

Glad to hear you liked the paper!

cheers

adriaan

[Seyed H. HAERI (Hossein)]

Sorry Dennis for throwing too much jargon at the list. This is, after
all, a language design issue and it takes its own terminology I'm
afraid.

OTOH, for me as a C++ person, Scala is indeed only a research
language. I publish my research papers in it! :D

--

[Seyed H. HAERI (Hossein)]

Hi Adriaan,

> I think this is getting a bit too technical for this forum,

Agree. But, I think it needs to be taken up once at least -- just for
the record.

> but path
> equivalence is decided statically, so, there's no uncertainty there either.
> Of course, every time you select a member on a path, you don't statically
> know the run-time value of the target. This is where late binding kicks in
> for value members.

And, that's one of the few places where things get out of my hands
wanting to simulate lightweight family polymorphism in Scala.

Cheers,
--Hossein

[Adriaan Moors]

Always happy to discuss, but prefer https://contributors.scala-lang.org/ for language design discussions.

[Gabriel Claramunt]

Since we're talking about removing or obscuring a feature with lots of magic, that can't be determined at compile time and causes unneeded complexity, can we get rid of subtyping? :)

---

From: Adriaan Moors
Sent: ‎2/‎14/‎2017 20:47
To: Seyed H. HAERI (Hossein)
Cc: scala-user
Subject: Re: [scala-user] Why I and my company have moved away from Scala. Anhonest critique.

[Vlad Patryshev]

This may be a beautiful idea, switching from Curry types to Church types. Haskellers will totally love us. And no OOP, guaranteed! (very bad, total disaster)

Thanks,
-Vlad

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[Vincent Marquez]

+1 so much. 

On Tue, Feb 14, 2017 at 4:07 PM, Gabriel Claramunt <gabriel....@gmail.com> wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[scala solist]

I'm very amused no one mentioned that at the present time scala libraries uses macroses (macro paradise) to generate implicits. Now you have zero chances to find origins of given code sample. Just somewhere clever macros provided implicits for you to use. And, of course, it may be used to trigger implicit macro expansion. All things quickly go out of control. I really like puzzles and implicit type classes gives you ton of interesting challenges like sorting list in compile time. But honestly I don't like to see none of such canny tricks outside pet projects.

[Seyed H. HAERI (Hossein)]

Hi Gabriel,

To answer your question below, I need to deviate a bit from the original topic.

> Since we're talking about removing or obscuring a feature with lots of
> magic, that can't be determined at compile time and causes unneeded
> complexity, can we get rid of subtyping? :)

Well, I never said things that are determined at the runtime are
categorically bad. Indeed, certain flexibilities are only at hand when
things get done dynamically. It, however, is the question of what to
leave to the runtime and how. Personally, I think deciding to pair up
with Java was such a poor decision. A fundamentally disturbing
consequence that relates to your question is that the programmer has
no control on when the dynamic behaviour kicks in. In particular,
virtual function calls are automatic. Even worse, there is no way to
override that by a non-virtual call. That in addition to implicits
being objects (as opposed to classes) mean that late binding of
implicit values cannot be evaded -- causing things to get out of the
programmer's hands. (Let alone the widely used macros that generate
implicits on the fly.)

To avoid all that, one needs not to eradicate subtyping. It only needs
to be engineered differently.

Cheers,
--Hossein

[Oliver Ruebenacker]

     Hello,

  Regarding whether Scala is good for business: before doing Scala at a major research center, I did Scala at two software startups. You can search for job ads containing "Scala" and convince yourself that there are plenty.

  Drawback: it is not as popular as Java. Popularity is an obvious advantage: larger pool of staff candidates, more tools, etc. If popularity with its perks is the all-deciding factor, then you go with Java, of course.

  Regarding implicit things, I may share part of Robert's frustration. There is no problem once I have working code in my project: IntelliJ know exactly what it does.

  The problem is when trying to learn something new. There is a library, and the documentation lists an example code fraction, not self-contained, with wild-card imports, omitted types and either implicit conversions or implicit parameters. It is hard to understand how it is supposed to work, and error messages are often not helpful.

  It can be difficult to find things in the Scaladoc. For example, try to find Tree.

     Best, Oliver

  

On Tue, Feb 14, 2017 at 1:45 AM, kraythe <kra...@gmail.com> wrote:

What I meant is that implicits are. It directly linked to code you can see. If you look at a method and see the code it is not apparent that an implicit is involved. Because they happen "magically.

But I got the message. It was "F off, your opinions aren't wanted no matter how honestly you offer them." Message received. No more reason for me to post on this thread or indeed group anymore. Twitter abandoned Scalia. Linked in abandoned Scala. So by all means keep beating people upside the head when they offer constructive advice. Could have been wonderful but toss it on the heap of could have been languages.

Peace and good luck.

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Oliver Ruebenacker

Senior Software Engineer, Diabetes Portal, Broad Institute

[Russ P.]

On Monday, February 13, 2017 at 10:45:04 PM UTC-8, kraythe wrote:

What I meant is that implicits are. It directly linked to code you can see. If you look at a method and see the code it is not apparent that an implicit is involved. Because they happen "magically.

But I got the message. It was "F off, your opinions aren't wanted no matter how honestly you offer them." Message received. No more reason for me to post on this thread or indeed group anymore. Twitter abandoned Scalia. Linked in abandoned Scala. So by all means keep beating people upside the head when they offer constructive advice. Could have been wonderful but toss it on the heap of could have been languages.

I don't know where you got the impression that anyone was telling you to "F off, your opinions aren't wanted no matter how honestly you offer them." The replies to your post seemed civil and reasonable to me.

When you offer a criticism of something that others value, you should expect some pushback. Did you expect Scala users to just say, "Yeah, you're right. Scala is dying and will soon be dead"?

And yes, you were corrected for claiming erroneously that implicits are not statically typed. That's fairly basic stuff. Your criticism of implicits took a major hit when you wrote that, at least in my mind.

As others have said a million times, implicits can be useful, elegant, and convenient when used properly, and they can also be overused or used improperly. The same thing can be said for many other language features, including vars.

[kraythe]

On Tuesday, February 14, 2017 at 11:22:46 AM UTC-6, Adriaan Moors wrote:

Thanks, Kris, and everyone else for sharing your thoughtful feedback.

The point about antipatterns is an important one. All languages have those, and all language communities acquire cultural knowledge about how to avoid them. It's up to us (the creators and the community together) to document them and share the wisdom with new members of the community. As the creators, we are also always looking for ways to have the language itself guide you away from the antipatterns, where possible. 

The point I have been trying to make is that this is great in a resource unlimited setting where you can just say "hey, lets all spend six months learning to not be newbies at scala" and the real business world where deadlines loom and managers want delivered product. You can train people in the basics of scala in a short amount of time if they are competent and they will get it, then they hit the minefield of implicit conversions and macros and they go straight backwards back down the slope. Enough times and they tire of climbing the slope. They haven't even gotten to hard things like currying which at least are visible from the code you are looking at. With implicits especially things are happening to your code that are not right in front of you, not to be found with a cmd-click to navigate to method. They are magically imported from god knows where and changing things and you are just supposed to "know" they are happening.

Case in point, I know a very experienced developer, 10 years experience with high quality systems, who jumped into Scala. All was going well but then he ended up encountering implicit conversions and the unpredictable effects that occurred from importing them into third party libraries. They acted like non-deterministic code in the platform. You counter that he needed more training? I counter he had deadlines to meet.. Now the developer was frustrated, irritated and completely non productive. After encountering this multiple times they had enough. So now I have a 10 year experienced engineer, he is not productive. I am forcing him to learn a new language that has brief (in his mind "cryptic") in syntax and oh, by the way, stuff magically happens that has no relationship to the code you are looking at. The developer is not preparing code, not implementing features, not being productive. Hence, developer wants NOTHING to do with Scala anymore. He goes back to java and cranks out more productive code in a week than he had Scala code in 2 months. Unfortunately he will also now have to deal with dual languages, mixed mode compiling, converting between java and scala which don't understand each other's lambda syntax and he will assuredly never ever do Scala again if he can avoid it. This is not a hypothetical, it is real life and i have 4 staff members in the same boat. 

Solution? Drop Scala. Problem solved. Indeed that is what we have done. We have only 4 more classes in Scala and by the end of next month I will have converted them to java. The change in code volume is not appreciable. The Java code is quite understandable. Other than pattern matching and immutable types I am quite content. 

So I come to post about the issues and my experiences and, yes, I am blunt. I don't have time to be non-blunt. I guess people don't like blunt. But the reality is when the language cross wires that many EXPERIENCED, intelligent, professional developers, it will never penetrate beyond niche in business. Hence many major companies have dropped it. I have nothing to gain through these posts, I gather neither fame nor fortune and spend a boatload of time reading replies and writing when I could be doing other things.

I love Lightbend's products. But then I use them with the java API. Why? Because I can staff my team and produce code. 

 

I believe Scala does a good job at shielding you from many existing antipatterns, but implicits (and their interaction with type inference, which may look like magic, even though it is deterministic) are a harder nut to crack in terms of finding a good balance in the (new) user experience and expressive power.

I also would like to reiterate that we value everyone's opinion (that includes criticism, obviously), and we take all feedback to heart. I prefer the actionable kind of feedback, though (I'm not about to give up being a mad scientist ;-)). We care deeply about the productivity of all Scala programmers out there, and are dedicated to improving your experience with Scala in every release.

cheers

adriaan

Scala team lead at Lightbend

Oh by all means be a mad scientist but realize this:

1. I have nothing to gain from this in either fame or fortune, so what is my motivation?  

2. If you have a language that an experienced senior java programmer can't grasp in a couple weeks and be good at in 2 months or requires expensive training, then it will not get past those startups and little projects. It will never full scale penetrate like Java. If that is fine, then you consent to being an niche language. 

3. Major companies have dropped Scala because of point 2. 

4. Current companies that adopt Scala will hit point 2 and eventually have to make a decision of do they delay products and revenue or switch back. 

5. After years of being out there Scala has meagre penetration, and still bad tooling. 

6. Again, I gain nothing by making this post. It CERTAINLY hasn't made me any friends here and it is not an avenue for fame or fortune so maybe, just maybe, my motivation is that I would love this to change. 

Peace ... out!

-- Robert

[kraythe]

Whatever FUD is, I don't know. 

I get harassed by headhunters, a good portion of them want to place me in inappropriate jobs. I get 60 mails a day sometimes. Its stupid. Ive thought of cancelling my linked-in account at times. That doesnt translate to penetration. Scala developers cannot be hired. Companies that are stuck in scala with millions in capital investment would MUCH RATHER be able to staff their shops, so they go back to Java. Because the scala language is too difficult to learn, the shortage of developers means new projects aren't started in scala and those with the ability either black box their old software and build on it with java or just start over in java. 

That is just business reality. Good or bad hardly matters. You can't take an average dev and train them in a couple weeks and have them reading the Scala SDK with comprehension and have them good at the language in a month. Businesses don't have time for their staff to spend six months getting good or the money to send them to expensive 3 week seminars. Hence no penetration. 

I realize my posts aren't popular and I am not making friends. So my motivation must be something else. I am not nearly bored enough to want to sit here answering for no reason and I have a quite excellent live with a wonderful family and no need to go troll anyone. Therefore my intentions must be honorable. They aren't given credit for being so but I am not surprised really. 

I would love Scala to replace java. On this trajectory it doesn't have a chance. 

-- Robert

On Sunday, February 12, 2017 at 5:58:27 PM UTC-6, Kevin Wright wrote:

“Scala has gained ZERO significant traction in the business world.”

Assuming that you don’t want your statements to be immediately labeled as FUD (and they certainly look to be tending that way), you’ll need to specify here how you’re defining both “significant traction” and “business world” in order to justify such a claim.

My personal experience would lead me to conclude the exact opposite; specifically as regards the rate at which I’m approached by headhunters/recruiters, and how “business worldy” their clients are… presumably investment banks, hedge funds, petro-chemicals, media conglomerates, and other assorted multinationals can all be reasonably classed as belonging to the business world.

On 12 February 2017 at 21:54, kraythe <kra...@gmail.com> wrote:

No its not. Scala has gained ZERO significant traction in the business world. It is not the next java. 

On Sunday, February 12, 2017 at 3:38:13 PM UTC-6, Vlad Patryshev wrote:

Oh, and Java of course is just perfect, compared to Scala.

Enjoy nulls and NPE, and mutability, and all the dumb frameworks designed by ignorants. You probably won't need generics; Java people just hate generics. They (Java people) are cheaper too.

The world is moving on; you decided to take a rest; it's okay, but don't expect others to join.

I wonder also what would be your opinion of Rust. Also worse than Java? I'm sure it is. No garbage collection. No Hibernate.

Thanks,
-Vlad

On Sun, Feb 12, 2017 at 9:16 AM, kraythe <kra...@gmail.com> wrote:

I work in a company that uses the Playframework and Akka to implement a business system. For a long time i was trying to push the implementation towards Scala and away from java. However, I have ultimately given up on that effort and have almost completed refactoring the work to Java. Here is why. 

First of all the learning curve on Scala is very steep but unnecessarily so. Scala syntax is very brief and but that isn't what holds up the learning curve, nor is it the functional programming concepts, functions as first class entities. These are in fact the good facets of Scala. Pattern matching is absolutely awesome, immutable collections are something that Java is really lacking. In short there is a ton of good things in Scala that attract a developer from Java. But the problem is Scala shoots itself in the foot. 

Without too much fanfare I will declare that Implicit conversions and Macros make the language very difficult to learn and, outside the academic world, almost inaccessible to business. Implicit conversions and indeed the entire 'implicit" keyword is just lethal to the learning curve as well as code maintenance. In a business world where effort is judged in ROI, anything that just magically transparently happens is a disaster waiting to happen. In fact such disasters do happen. We found that implicits float around in our code interacting in strange ways that were non-deterministic and opaque to reading the code. The beginner looking at a method wonders, "wait how can you call that method on that object?" Then they think and say "oh there is an implicit somewhere in some of my imports somewhere down the tree that convert the type. Its obscure, opaque, confusing and the antithesis of everything that an experienced business developer wants. I used to think that it was just a matter of tooling but IntelliJ has actually decent scala tooling and still implicits are a total disaster. Second problem is Macros. Like implicits they make things happen that are difficult to discern by looking at the code. The introduction of both of these and their rampant use in the language is, in my opinion, going to push Scala to be yet another niche language like lisp, scheme, Erlang and so on. 

Secondly is the compilation tooling and interaction with java. Scala interacts transparently with java. Well, at least that is what the promotional materials say. In practice to achieve that interaction is a major PITA and I am not talking about flat bread here. Conversions of types require gymnastics. For example, try passing a list of Scala longs to a java method that takes a list of java.util.Long objects. Gymnastics. The function lambdas are non translatable so you cant just pass a Scala lambda to a java function that takes a java.util.Function. You can doctor this with ... wait for it ... an implicit conversion that wraps one into the other. Sigh. In our codebase we had tons of code in both languages and it was just a nightmare going between the two. The tooling doesn't help either because SBT gets in some wacked out logic loops and simply rebuilds the whole project multiple times. After years of Scala development SBT is still slow, and in such a bad state that it isn't even at 1.0. 

I think Scala has become too academic. Too controlled by those with PhDs. in computer science figuring out what they can do, rather than wondering if they actually should do these things and as a result the language has abandoned some of its roots. It started out as an alternative to Ruby and Java. It was a statically bound deterministic language with functional and Object oriented concepts but it has now strayed into things that happen "magically" and code that is generated with Macros. I personally think one of two things will happen. Either the maintainers of Scala will reverse this direction and get back to basics, dump the implicits and macros and find more deterministic ways to do things. Or, quite frankly, it will die on the vine as another Erlang, Lisp, Node.js or you name it. A language used in a small limited sector for a few projects but without wide acceptance. 

I, for one would think that is a shame, a horrible waste. Scala could be the next super language if academics stop messing with it like mad scientists with a new toy. Until then, I cant have my whole staff down six months learning scala. 

-- Robert, 

-- Principal Software Architect. 

[Dennis Haupt]

this makes perfect sense. if you study "business software development for the real world" anywhere, you most likely are taught mainstream languages, not scala, f#, haskell or clojure. those are languages for people that want to learn new things in their free time.

all the scala developers i know either learned it by accident or because they were curious people (of which a large part doesn't/didn't like java because of a lack of features)

fud is "fear, uncertainty + doubt" 

 

Gesendet: Donnerstag, 16. Februar 2017 um 07:41 Uhr
Von: kraythe <kra...@gmail.com>
An: scala-user <scala...@googlegroups.com>
Cc: kra...@gmail.com
Betreff: Re: [scala-user] Why I and my company have moved away from Scala. An honest critique.

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

[Martijn Hoekstra]

> Whatever FUD is, I don't know.

Fear, Uncertainty and Doubt. It was the strategy unofficially deployed by IBM - nobody ever got fired for choosing IBM.

Twitter is joining the Scala Center Advisory Board as a full member. The claim that Twitter has moved away from Scala may instill Fear, Uncertainty and Doubt, but it's not true. 

>  Scala developers cannot be hired. Companies that are stuck in scala with millions in capital investment would MUCH RATHER be able to staff their shops, so they go back to Java. 

So that means demand for Scala devs is so high that supply can't keep up with it. That can be a good reason for companies to move back to Java, but that doesn't show the lack of traction Scala has. 

>  You can't take an average dev and train them in a couple weeks and have them reading the Scala SDK with comprehension and have them good at the language in a month. 

Of course not. You can't take any average dev and train them in a couple of weeks in a different language by reading the SDK of that language with comprehension and have them be good at the language in a month for any language that's appreciably different from the languages they know. 

 

--
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[Alan Burlison]

On 14/02/2017 11:21, Filippo De Luca wrote:

> You say Twitter and Linkedin have abandoned Scala, fair enough, but at the
> same time other, much bigger companies, like Expedia, are embracing it
> because more and more developers want to write Scala code.

I think one of the things that often clouds these sort of discussion is
they often don't distinguish between issues with Functional Programming
in general and issues with Scala - they all get lumped together. If you
are coming from an traditional imperative background then switching to
FP is a big leap, but it's pretty much the same size of leap no matter
which FP language you choose. Yes, Scala has a reputation for being
"difficult" but I think that's mostly because of the concepts it relies
on rather than the language itself.

Yes, Scala has its dark corners, as do all other languages. Part of the
art of being truly competent in any language is to know where the dark
corners are, avoid them if you can and if you do have use them, make
sure it's tightly-bounded and documented. Several very experienced Scala
people have already said that in this thread.

There is a much older example of similar "mindset" issues - SQL. Being
declarative it requires a completely different approach from procedural
programming to write well. A significant proportion of the SQL I've seen
written by imperative programmers is awful - they end up using the
database like a flat file - join and predicate free SELECTs which they
then loop over in the application code. The "You need to switch your
paradigm" problem is nothing new.

--
Alan Burlison
--

[Vlad Patryshev]

So, the problem is that experienced developers can't grok it.

Experience means nothing in our world. One has to learn all the time.

Thanks,
-Vlad

[Vlad Patryshev]

Get an average Java developer and train them to become a good FP Scala programmer? Done it; developers are happy. There are known, and well-explained steps to get Java OOP programmers into Scala FP world.

Thanks,
-Vlad

[jez prime]

Devil's advocate time. There are vast numbers of Java developers who have no interest in learning anything new for themselves - i've worked with plenty in the past. Like it or not, that's the way it is. Similarly there are many organisations who are quite simply not prepared to spend money to train their existing staff to do new things. For a self-employed mercenary such as myself, your point holds true for the very simple reason that it's in my direct financial interest to find the time to learn new things. That isn't as clear for a developer in a permanent post in a fairly comfortable position. So yes, a steep learning conceptual learning curve does limit adoption. I recently worked in a sizeable UK government organisation that absolutely rejected Scala from the word go, despite a recommendation from one of their own architects to adopt it. Their concern was purely and simply (and correctly) about the size of the pool they'd have available to recruit from. 

I'm speaking as a Scala fan by the way, but I've witnessed this for myself as outlined above. They disregarded the argument that they wouldn't need as big a pool because the developers they would recruit would be more productive and ultimately the codebase would end up cleaner.

Thanks,
-Vlad

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

[Russ P.]

You keep coming back to implicits and how they frustrate even experienced programmers. That may well happen at times, but here's the problem with your claims. It could simply be that the code base you are working with has misused or overused implicits. If so, that is not a knock on Scala per se but rather on it's misuse in that particular code base. You cannot hold any language responsible for its misuse unless the misused feature has no value when used properly (certainly not the case for implicits).

Here's the other thing that I am wondering about. Many IDEs will take you right to the definition of the implicit. Perhaps your experienced developers should use an IDE that does that. Short of that, can't they simply grep the source code for the implicit? How about something like

grep implicit *.scala | grep SomeClassName

Is that so hard? You could even make an alias for it to reduce the typing.

[Oliver Ruebenacker]

     Hello,

  If your team is a mix of Scala engineers and Java engineers, the Scala engineers can teach the Java engineers and help them find appropriate tasks for their level. Such team will be quickly productive and suitable even in a startup, where survival depends on your ability to produce results quickly.

  We can't predict the future of Scala, but many features of Scala are great features and will be part of the most popular language within the next five to ten years, regardless of whether that most popular language will be Java, Scala or something else. Then you want to have people who are familiar with these features.

     Best, Oliver

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[joe.p...@gmail.com]

On Wednesday, February 15, 2017 at 10:32:41 PM UTC-8, kraythe wrote:

1. I have nothing to gain from this in either fame or fortune, so what is my motivation?  

That's an excellent question.  What is your motivation?  What did you hope to accomplish when you posted the message that started this thread?

To be honest, it seems like you wanted to vent your frustration, not that you wanted to somehow make Scala better.  You offered criticism, but not constructive criticism.  Please show us how we can make Scala better, but don't be surprised if "remove feature X" is met with, "Okay, tell us how we can achieve the benefits we currently derive from feature X without it."  And if your reaction is, "I don't benefit from feature X" then the problem may be that your needs can and should be met by a different language—which is what you seem to have decided.

It sounds like you really wanted Scala to be "a better Java".  Scala is not Java with FP glued on, Scala is a new language, one that has proven useful to many but, like any language, will never be "one size fits all".

Best wishes for your future endeavors,

joe

[Brian Maso]

The learning curve is indeed steep. It took me at least a couple years working evening and weekends with Scala -- with no real FP background -- to start feeling like I was guaranteed more productive in Scala than Java. At this point I see FP on the JVM as my extreme advantage over Java developers.

I know there are shops who don't want to change to other languages -- similarly there weren't shops who wanted to move from classic VB to .Net, from craptaculous HTML and server-side code to CSS/JavaScript, from COBOL to almost anything else, etc. There are definitely niches and industrial enclaves where those technologies make sense from a business POV even to this day.

If you are in a position to ride the Java wave all the way to retirement, and you are satisfied and enjoy the solutions you are able to create, that's great. Both personally and professionally I am more than satisfied with Scala, at least in part because I can look forward to the staff-heavy Java shops that I can compete with very handily.

Brian Maso

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Best regards,
Brian Maso
(949) 395-8551
Follow me: @bmaso
br...@blumenfeld-maso.com

[jez prime]

Again, I'd agree with you, but firstly this wasn't a startup but a fundamentally conservative organisation and secondly to be fair 2 or 3 Scala-experienced devs across a team of 50+ in total (that's really across multiple teams but you get the point), where those most experienced with Scala also have design and architecture responsibilities is not going to teach anyone Scala quickly enough to avoid a very sizeable productivity dip. I actually think adoption in a startup with a small team would be easier than in the scenario with which we were faced.

Like I said, I'm a fan, and I would vastly prefer to be working in Scala, but I can also see the counter-argument. We had trouble trying to teach these guys the Java 8 stream approach, but on the upside, at least now they get lambdas in that scenario, one part of a Scala move might be easier for them :) 

Many organisations, when faced with short term cost for long term benefits, will simply reject that option. Half their management won't be there 2 years from now, so what do they care about the long term benefit?

[jez prime]

I genuinely think I've got more hope of getting Kotlin adopted in some of my clients than Scala. I even had the "let's just use the nicer object construction syntax and not much else" argument rejected (maybe they realised I was really planning a Trojan horse approach - get the devs familiar with the basics, then build out from there at a pace the team could work with).

[jez prime]

Another small note, I'm not as pessimistic as the OP, despite my comments on what I've observed recently, because I also observed something else.

The people I've seen having most problems learning Scala (or the functional stuff in Java 8) are mainly the most experienced Java devs on the team. In other words it's their experience working in Java that holds them back most. The ones who took it on faster were more junior devs who hadn't already shaped a mental model based purely on OO and the Java language. So there is room for evolution in thinking. 

On Thursday, February 16, 2017 at 7:34:02 PM UTC, Brian Maso wrote:

[Rex Kerr]

On Wed, Feb 15, 2017 at 10:32 PM, kraythe <kra...@gmail.com> wrote:

Oh by all means be a mad scientist but realize this:

1. I have nothing to gain from this in either fame or fortune, so what is my motivation?  

Frustration is another possible motivator.

 

2. If you have a language that an experienced senior java programmer can't grasp in a couple weeks and be good at in 2 months or requires expensive training, then it will not get past those startups and little projects. It will never full scale penetrate like Java. If that is fine, then you consent to being an niche language. 

It will certainly slow adoption.  You can't predict the endpoint from this analysis because you're assuming "experienced senior java programmer" as the starting point.  But plenty of people aren't, and as time goes on the fraction will if anything decrease (if only because more people are coming in from JS and Python and such).

 

3. Major companies have dropped Scala because of point 2.

Indeed.  "Scala is not Java with better syntax" has caught some folks by surprise.  "Here, you can do things the old way, but we have this awesome other stuff that you have to think about in a whole new way" is, as you point out, not really compatible with short deadlines and exchangeable programmers.

But some companies seem okay with this.  I'm not in a position to know why, but you list some of the reasons yourself--maybe it's a longer-term project and they can afford to invest up front, or maybe they can get enough programmers who are experienced with Scala.  Or maybe they can scrupulously keep themselves to a "Java with better syntax" subset of the language.

 

4. Current companies that adopt Scala will hit point 2 and eventually have to make a decision of do they delay products and revenue or switch back. 

They may already have anticipated this, though.  If I had a project where the natural choice of technologies was some standard Java stack and I needed to have something done in three months, I'd probably try to hire Java developers and do it in Java unless I could hire already-trained talent.  (Actually, I'd probably quit and say "I don't like working on projects on that timescale."  But I digress.)  But if I had a year, I'd expect to do a lot of training up front (lots of pair coding and code reviews, plus plenty of time on tutorials and the Scala online courses and so on) and catch up in the remaining, I don't know, 8 months or so.  I would feel confident doing this *except* if I hired expert Java devs who weren't inclined to learn new ways of thinking.

But I haven't done it, so my impressions should be taken with appropriate levels of salt.

5. After years of being out there Scala has meagre penetration, and still bad tooling. 

Meteoric rises are fun to watch, but Scala is nonetheless gaining traction rather quickly compared to other languages that have been around for a while.  I'm not aware of any language that was adopted as widely and as quickly as Java, but that doesn't mean that Java is always going to be vastly dominant.  And Java tooling took a long time to get as good as it is now.  Really *nothing* is as good as Java tooling.  It's a competitive advantage that Java will keep for a good while (and I really mean this--it *is* a serious advantage that helps compensate for a variety of other flaws), but probably not forever.

  --Rex

[Seth Tisue]

On Thu, Feb 16, 2017 at 11:52 AM, jez prime <mr.n...@gmail.com> wrote:

The people I've seen having most problems learning Scala (or the functional stuff in Java 8) are mainly the most experienced Java devs on the team. In other words it's their experience working in Java that holds them back most. The ones who took it on faster were more junior devs who hadn't already shaped a mental model based purely on OO and the Java language. So there is room for evolution in thinking. 

This! +1

[Raoul Duke]

Or, you know, those experienced devs who hatecwaiting for a bloody compiler to ever finish...

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

[Dennis Haupt]

i usually watch a short youtube video every time i do a clean compile. still getting paid the same. wouldn't call that a disadvantage :D

[Oliver Ruebenacker]

https://xkcd.com/303/

[Raoul Duke]

i appreciate the humo(u)r but still prefer fluid trains of thought to making youtube earn ad revenue :-)

not that any other functional-supporting language ecosystem (haskell, ocaml, f#, flow, typescript, you name it) is overall better really.

also, i am old and so is my laptop cpu (tho does have ssd) so i am particularly hard to please.

and of course have no kegs to stand on sincrme I've never made any proframming language, outside of college, let alone one used by zillions of demanding industry folks!!!!

c'est la vie.

[Siddhartha Gadgil]

In spite of (or because?) my experience is atypical, here it is regarding implicits. 

* I overused implicits all over the place at first but learnt - from the community and official documentation - that they come in different flavours, and what to use and what not.

* Implicit classes for extension methods are a nice syntactic convenience. Logically one can always just use functions, but these are safe, elegant and worth using.

* The deepest use of implicits is as functions from Types to Objects, using implicit resolutions. This is wonderful stuff, making scala having some elements of a dependently typed language, and I use this a lot both directly and indirectly. The only catch is that these are effectively modelling relations, not functions, and  the compiler has to figure that the particular relation is a function (at least for a specific argument).

* Implicit objects can also be used for configuration.

* What one can justify ranting against is _implicit conversions_, but as I mentioned I constantly get the message that one should avoid them - the latest being when switching to 2.12.1, the compiler suggested using extension methods for Java Conversions.

* The one case where I have seen very well justified use of implicit conversions (rather than classes) is when using _literals_, such as in Spire.

 

regards,

Siddhartha

[Seyed H. HAERI (Hossein)]

Hi Siddhartha,

This is going to be a change of topic I guess. But, I'm too lazy to
spawn a new thread. So, I'm going to ask it here:

> * The deepest use of implicits is as functions from Types to Objects, using implicit resolutions. This is wonderful stuff, making scala having some elements of a dependently typed language,

Would you mind expanding on that a bit? My understanding has always
been that its in the other direction, i.e., dependently typed
programming gives objects individual type identities. For example, 1
has a different type (representation) than 2 -- at the same time that
they might still be regarded as integers. This way, it's from objects
to types.

> The only catch is that these are effectively modelling relations, not functions, and the compiler has to figure that the particular relation is a function (at least for a specific argument).

I would also be interested in reading more about this one. (I do know
the difference between relations and functions.) Relations between
what and what?

Cheers,
--Hossein

[Bardur Arantsson]

On 2017-02-18 10:55, Seyed H. HAERI (Hossein) wrote:
> Hi Siddhartha,
>
> This is going to be a change of topic I guess. But, I'm too lazy to
> spawn a new thread. So, I'm going to ask it here:
>
>> * The deepest use of implicits is as functions from Types to Objects, using implicit resolutions.
>> This is wonderful stuff, making scala having some elements of a
dependently typed language,
>
> Would you mind expanding on that a bit? My understanding has always
> been that its in the other direction, i.e., dependently typed
> programming gives objects individual type identities. For example, 1
> has a different type (representation) than 2 -- at the same time that
> they might still be regarded as integers. This way, it's from objects
> to types.
>

Unless something changes drastically soon, the whole Dependent Types in
Scala is a red herring for all practical purposes. Encoding DTs in a
language not meant for it is an absolutely horrible experience -- Scala
and Haskell alike. Just a) look at the implementation of Shapeless and
b) try to use Shapeless for... anything[1]. Usability is, well, let's
just say "less than stellar". This is in large part due to the encodings
which rely on implicit conversions to fire all over the place... and
when they don't, you're left scratching your head as to with your
"simple" code doesn't compile like the examples do.

Don't get me wrong, Shapeless (and the like) are *engineering* marvels
that show that incredible things are possible, but it *really* shows
(ergonomically) that most of that power is accidental. Much like C++
templates.

(Hopefully Scala macros can be rescued to give us that much needed
metaprogramming power without the obtuseness of type-level programming
in Scala.)

[1] People have managed, but it's a *lot* of effort compared to what it
could be.

Regards,

[Roland Kuhn]

18 feb. 2017 kl. 11:15 skrev Bardur Arantsson <sp...@scientician.net>:

On 2017-02-18 10:55, Seyed H. HAERI (Hossein) wrote:

Hi Siddhartha,

This is going to be a change of topic I guess. But, I'm too lazy to
spawn a new thread. So, I'm going to ask it here:

* The deepest use of implicits is as functions from Types to Objects, using implicit resolutions.
This is wonderful stuff, making scala having some elements of a

dependently typed language,

Would you mind expanding on that a bit? My understanding has always
been that its in the other direction, i.e., dependently typed
programming gives objects individual type identities. For example, 1
has a different type (representation) than 2 -- at the same time that
they might still be regarded as integers. This way, it's from objects
to types.

Unless something changes drastically soon, the whole Dependent Types in
Scala is a red herring for all practical purposes. Encoding DTs in a
language not meant for it is an absolutely horrible experience -- Scala
and Haskell alike. Just a) look at the implementation of Shapeless and
b) try to use Shapeless for... anything[1]. Usability is, well, let's
just say "less than stellar". This is in large part due to the encodings
which rely on implicit conversions

Please, everyone, be precise: this has nothing to do with implicit conversions, which are universally acknowledged as an anti-pattern. Shapeless’ implementation is almost exclusively based on implicit arguments, and it is a beautiful illustration of the Curry-Howard isomorphism—which thankfully the users of the library do not need to know or care about in order to use it.

to fire all over the place... and
when they don't, you're left scratching your head as to with your
"simple" code doesn't compile like the examples do.

This is indeed the main problem, I also recently ran into this while implementing session types for Akka Typed, with detailed effect tracking. My preliminary conclusion is that it will be necessary to write a macro whose sole purpose is to customise the error reporting in a domain-specific fashion (i.e. printing a nice message that you forgot to send that message of type A at step 3 in the protocol).

Don't get me wrong, Shapeless (and the like) are *engineering* marvels
that show that incredible things are possible, but it *really* shows
(ergonomically) that most of that power is accidental. Much like C++
templates.

(Hopefully Scala macros can be rescued to give us that much needed
metaprogramming power without the obtuseness of type-level programming
in Scala.)

Hmm, that may sound promising at first, but since types are propositions and only programs are proofs, I fear that such an approach would be wide open for the introduction of unsoundness in the usage of the type system. As you say, the proper solution is to add dependent types.

Regards,

Roland

[1] People have managed, but it's a *lot* of effort compared to what it
could be.

Regards,

-- 
You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

[Bardur Arantsson]

On 2017-02-18 11:45, Roland Kuhn wrote:
>
> Hmm, that may sound promising at first, but since types are propositions
> and only programs are proofs, I fear that such an approach would be wide
> open for the introduction of unsoundness in the usage of the type
> system.

Oh, certainly, but luckily the JVM's (mostly) got our backs with
ClassCastException and similar. Granted, it'd be better to have full
compile-time verification of the soundness of macro implementations, but
as they say... "it's a highly non-trivial problem".

At least macros could hopefully be reasonably understandable
implementation-wise and debugged over time to become "good enough".

> As you say, the proper solution is to add dependent types.
>

I'm not sure "just add dependent types" is an answer that applies to
Scala. I think it mostly has to be *designed* into the language from the
start (e.g. Idris, Agda, etc.) or you might end up with a
Frankenlanguage. It's just not something you can necessarily add as an
afterthought.

(Though, actually, Haskell is undergoing the addition of full-on
Dependent Typing at the moment. AFAIUI it's a ~decade-long effort that
just gotten underway in the last couple of years or so. I guess we'll
see how that turns out. At least there's a lot less "friction" when
adding DTs to a Haskell-like language.)

Regards,

[Roland Kuhn]

> 18 feb. 2017 kl. 11:52 skrev Bardur Arantsson <sp...@scientician.net>:
>
> On 2017-02-18 11:45, Roland Kuhn wrote:
>>
>> Hmm, that may sound promising at first, but since types are propositions
>> and only programs are proofs, I fear that such an approach would be wide
>> open for the introduction of unsoundness in the usage of the type
>> system.
>
> Oh, certainly, but luckily the JVM's (mostly) got our backs with
> ClassCastException and similar. Granted, it'd be better to have full
> compile-time verification of the soundness of macro implementations, but
> as they say... "it's a highly non-trivial problem".
>
> At least macros could hopefully be reasonably understandable
> implementation-wise and debugged over time to become "good enough".
>
>> As you say, the proper solution is to add dependent types.
>>
>
> I'm not sure "just add dependent types" is an answer that applies to
> Scala. I think it mostly has to be *designed* into the language from the
> start (e.g. Idris, Agda, etc.) or you might end up with a
> Frankenlanguage. It's just not something you can necessarily add as an
> afterthought.

Ah, yes, sorry for my sloppy language, we mean the same thing: if you want dependent typing, then the language needs to have first-class dependent types.

Regards,

Roland

> (Though, actually, Haskell is undergoing the addition of full-on
> Dependent Typing at the moment. AFAIUI it's a ~decade-long effort that
> just gotten underway in the last couple of years or so. I guess we'll
> see how that turns out. At least there's a lot less "friction" when
> adding DTs to a Haskell-like language.)
>
> Regards,
>

[Seyed H. HAERI (Hossein)]

+1

Just because (admittedly accidental) dependent types of Scala are
powerful enough to make great tools like Shapeless, it doesn't mean
they are the right tool for the purpose. In fact, I too think that's a
misuse of power. :D Although, I'm not sure what the proper alternative
would be with what currently is in Scala. Macros are the first to come
to mind. Not sure how they'll end up though. They are taking the
Template Haskell flavour which gets runtime stuff into the game and I
just don't like that. I'd say metaprogramming is compile-time only for
its for instructing compilation after all. Furthermore, Generic
Haskell is known to cause incredible runtime overhead with the
compiler ingeniously leaving some of the metaprogramming resolution to
the runtime. Sorry I have nothing constructive to offer here...

--
--------------------------------------------------------------------------------------------------------------

Seyed H. HAERI (Hossein), Dr.

Post-Doctoral Research Fellow
Department of Computing Science and Engineering
Catholic University of Louvain
Louvain-la-Neuve, Belgium

ACCU - Professionalism in programming - http://www.accu.org/
--------------------------------------------------------------------------------------------------------------

[Roland Kuhn]

Hi Seyed,

could you outline what makes you say that macros “get runtime stuff into the game”? Macros are fully evaluated at compile-time—if they choose to emit a computation to be evaluated at runtime is up to them.

Regards,

Roland

[Seyed H. HAERI (Hossein)]

Hi Roland,

Sorry for being late on the response. Didn't mean to be impolite; I'm
a but unwell these days.

> could you outline what makes you say that macros “get runtime stuff into the game”? Macros are fully evaluated at compile-time—if they choose to emit a computation to be evaluated at runtime is up to them.

Of course, if they choose to do something at runtime, that's their
choice. But, that wasn't my point. I had two points:

Firstly, I don't have runtime figures for the consequences of using
Scala macros. Neither am I aware of any empirical study on that. Nor
have I taken a look into the macro engine of the Scala compilers. My
impression though is that the Scala macros are highly influenced by
the quasi-quoting culture of Template Haskell. (Please correct me if
I'm wrong.) There, the compiler is known not to be able to evade
certain sorts of splicing at runtime -- causing unpredictably high
runtime overhead. The Haskell 12 talk of Michael Adams [1] should be
your good starting point.

Secondly, with Scala choosing every object to be a reference, there
essentially is no clear cut between compile-time and runtime. This is
because every function call is then virtual and the programmer cannot
ever rely on the runtime cost being zero. I know the folklore in this
community is the other way around. But, I'm afraid that's pure
misconception. So, it bothers me that I need to come back to this from
time to time. Maybe exemplifying C++ here can help you get the point.
In C++, when the programmer needs late-binding, they use pointers or
references. Otherwise, no runtime action will take place and even
wrong call-by-value will cause slicing.

As a matter of fact, pure functional programming also has similar
drawbacks on runtime overheads. See the WGP12's closing discussion for
real-world Haskell users talking on the matter [2]. So, more into this
on the Template Haskell quasi-splicing culture penetrating into
Scala...

Having said all that, I am not advising Scala to take the
metaprogramming approach of C++. That's completely accidental and too
much of a pain. On the other hand, the Template Haskell semantics is
so well-defined. However, from what I have witnessed, the latter is
too much of an unleashed power that I am not sure about the real
usefulness of. I'm afraid I don't have an opinion on what the best bet
here might be.

HTH,
--Hossein

[1] http://www.youtube.com/watch?v=8bIBTxHgptQ

[2] https://www.youtube.com/watch?v=fqkf_T4PrmA

[Roland Kuhn]

Thanks for your answer, Seyed, and don’t worry about the delay: it will take me a few days to look into Template Haskell before I can formulate an informed response.

Regards,

Roland

[oss.m...@gmail.com]

Why implicit conversions are not listed in ScalaDoc just like subtyping? They should be in a section just below subtyping. At least those defined directly in type companion object. But if some generic one applies it should be there as well. I have to look to Scala source code to find out that Option type can be used where Iterable is required.

Is it possible in any IDE to list all available implicit instances for a scope I have cursor in (or highlighted scope)?

One reason why nobody is confused by subtyping, outside of more a decade of familiarity and having only one option to consider, is easy discoverability. In a sense implicit conversions are similar to subtyping because they enable type substitution through a different mechanism.

Just compare:

class MyType extends Iterable { ??? }

object MyType {
  implicit def toIterable(x: MyType): Iterable = ???
}

As a side note, for those with PhDs, some call it coercive subtyping:

"The basic idea of coercive subtyping is that subtyping is modelled as an abbreviation mechanism: A is a subtype of B,

if there is a unique coercion c from A to B, written as A <c B. Then, if a hole in a context requires an object of type B, it

is legal to supply an object a of type A – it is equivalent to supplying the object c(a). This simple idea, when used with

the rich type structures in type theories with inductive types, provides a powerful and general subtyping mechanism. It

subsumes injective subtyping, projective subtyping and structural subtyping."

http://www.cs.rhul.ac.uk/~zhaohui/Subtyping12J.pdf
http://www.cs.rhul.ac.uk/~zhaohui/subtyping.html

And mad scientists may like this one:)
http://www.cs.cmu.edu/~neelk/cattheodesign.pdf

Petr

[Jasper-M]

Hi Petr,

ScalaDoc does allow you to filter all members that are added by certain implicit conversions. For instance in the ScalaDoc for Option, you can select to show all members added by option2Iterable. And if you select such a member the doc gives more information about the implicit conversion and all types that are involved.

In the ScalaDoc of older Scala versions (for instance 2.11.7) you can open up a nice diagram "Type Hierarchy" which also shows types to which implicit conversions are available. I don't know why that was removed in the newest versions. Perhaps because it's too hard to find the right balance between extensiveness and clarity.

Kind regards,

Jasper

Op vrijdag 24 februari 2017 12:39:18 UTC+1 schreef oss.m...@gmail.com:

[Seyed H. HAERI (Hossein)]

Dear Petr,

I had never heard of coercive subtyping. It indeed was very fascinating as a piece of theory. Thanks! :)

Beware that most people on this list will perhaps find that too technical. :p But, there is more to it than just that: Such a theoretical indirection whilst thinking about implicit conversions is too essentially counter-intuitive for the practitioner. The reason why OOP was so warmly embraced by the industry was its beautiful match with the human intuition.

Cheers,

--Hossein

--

You received this message because you are subscribed to the Google Groups "scala-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[oss.m...@gmail.com]

Hi Hossein,
I might be one of few but my experience is different. Maybe others find it counter-intuitive. When I started to use Scala, most of implicits patterns were a black box, confusing. I started to grasp it only once I realize that subtyping, implicit conversions and typeclasses are just different techniques how to achieve type substitution. Only then I started to see more similarities that differences, before it I tried to treat them as a completely distinct concepts and I couldn't reuse any of my existing knowledge. It stopped to look that alien when I realized that for subtyping implicit cast happens to allow me to substitute one type for another on a call site, so why implicit conversion couldn't happen to allow the same (perf aside)? And when this conversion is defined in the same file and in a companion object of the same name as defined type, it is for me as easy to understand as subtyping. Because both define "IS A" relation. Actually not. There is another perspective. Subtyping extends keyword is only "MAYBE IS A", to prove it is "IS A" it has to satisfy LSP and this can't be covered by type-system/compiler, we have to do it informally. The reason is that we implicitly cast to supertype but possibly with changed behaviours (polymorphism) and there is typically mutation involved on top. Now let's compare it to implicit conversion - it is actually "IS A", we convert to another type completely, no dynamic dispatch, it is more restricted, more safe. Isn't it interesting? I can safely convert Square to Rect, I can't make Square subtype of Rect. But math doesn't say that Rect can't have equal sides. So why there is so much problem with implicit conversions? How they are less intuitive?

Another aha moment was with typeclasses. I was constantly asking myself why just not simply pass the instance for its implementation explicitly? But the only purpose of this value is to carry (prove) "IS A" type relation. I don't have to pass this evidence on call site for subtyping, it makes sense I shouldn't have to for typeclasses either. I just need to look into type companion object to see which typeclasses it implements (is assigned to) - just like I would look for extends. Understanding implicit values as an evidence about type relation was a breakthrough for me.

This is the basic model which worked great for me. Once I got familiar with it I could move to the actual reason why to have these beasts. They allow to add behaviours to types we don't have under control and can't add that magic "extends" keyword (implements, with, & etc.). They help to fight Expression problem. Now there is discoverability problem - these ad-hoc implicits don't come from companion objects (which are straightforward just like subtyping). But they come from packages/modules/mixins which typically have some explicit suffix of *Implicits, *Integration,*Dsl. I import these locally to separate them from other imports on top. I can't remember I had an issue last time. But here I can see source of confusion if not understood and treated well and apply ad-hoc carelessly.

I just can't relate to such a OP frustration. Across my team. Maybe developers do the same mistake as I did? Trying to treat it as a completely different concepts? Probably, it comes from the fact how majority of enterprise developers learn - top down. I did. It matches great with OOP and that human intuition. But there is not much intuition beyond basics, one has to get familiar with it and understand how it actually works underneath. And not every program can be modeled like a real world. Today I would pay money not to learn that way and rather start with type theory, simplified, no dense math, just understand well what type is, how it represents values, type composition, type substitution, various polymorphisms and trying to understand it as much as possible in a language agnostic way. Then build OOP and FP on top.

Trying to see concepts differently doesn't work in Scala. Scala unifies concepts a lot on top of types. Maybe I unify further, maybe it is confusing to others but it works for me.

Petr

To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[oss.m...@gmail.com]

Thank you. I still had trouble to find it till I accidentally hovered over Search box and it showed that it can be expanded. Scala shouldn't be that good at hiding implicits:)

Supertypes | Subclasses are below long text and off-screen, hence hidden for one who is new. Namely if I'm used to JavaDoc. Implicitly is one of the most important information in Scala, it should not be away only as a filtering option in a search box. It belongs as well at the same level as Supertypes | Subclasses | Implicitly. 

It seems that filtering By Inheritance doesn't not work. In this view white color is used on light shadow background in "Inherited from ...".

Petr

[oss.m...@gmail.com]

Scala implicits are generally misunderstood. Most of hate posts I see miss the point of them and use them worng. Implicits are primarily a construct for type level programming. They are construct how we can extend type system so that it can help us. It is the reason why they are passed around implicitly, it is enough to state the requirement in the type declaration.

Here is the confusing part. While they are values they typically don't carry a value information for our value level program. This is important distinction to realize. They carry information for the type-system/compiler.

If it is a value for our value level program, don't hide this information with implicits. We should never get into situation saying that implicit hides my data-flow or something like this.

There is an use for them for value level programming. Mainly to inject external dependencies, e.g. via constructor. In this case they are equivalent to Spring @Autowired. Spring @Autowired is implicit value injected by its runtime, while in Scala compiler injects them. The same considerations if to use them or not applies in both cases. Would I autowire important value for for my data flow with Spring? No, I would create the same mess.

How any implicit values compiles to Java class? As its constructor arguments. Magic is gone, it is just plain old constructor DI, use it like that and you will be fine. It is as well a reason why you can't use implicit values if you need to pass this class to framework which is using reflection and no-arg constructor to instantiate it. Because there is no no-arg constructor in this case.

There are exceptions when an important value is passed implicitly. Example is Akka HTTP which inject/passes request value around. In such a case it seems to me reasonable because it is a central design decision to that framework. It is that central that it is not surprising after getting a bit familiar with the framework. And after all this request value is completely controlled by the framework, not me, there is inversion of control involved (maybe it can be formulated as a general characteristic for this exception?).

What is completely wrong with Scala implicits is its documentation. Implicits are central to Scala, they make the language interesting. Remove them and I see little point to use Scala, just like Kotlin (it even doesn't have pattern matching). Java8 makes it bearable and it will very slowly converge closer to Kotlin over-time.

I have just look at the documentation after long time. No way I could understand Scala implicits with what is there. I even scanned Scala Specs Chapter 7 about implicits. I just can't find explanation for a mechanic how implicits values with types with multiple type parameters resolves. Without this knowledge nobody can understand how CanBuildFrom works which is used in standard collection library! I prefer CanBuildFrom any day against Java8 collect. It is such a genius pattern, and I see at least one side why developers have problem to understand it. If I relate it to other complexities in my development it is very simple to understand how it works, but it has to be explained first. I lived for long with the wrong assumption that I have to declare all type parameters. I had figure it out myself that my assumption is wrong just looking at call which just didn't do that.

And explain limitations of Scala type inference. I think it is missing in the docs as well. Every starting developer will assume that it just works always. And get lost as soon as it doesn't not even knowing what to look for. And without understanding type inference limitations there is little chance to understand couple of design patterns which mitigate it. Which means not understanding library code which uses them and majority do.

The other part of the problem is that I dare to say that majority of enterprise developers just avoid generics and write only invariant programs. But it is another story.

Petr

[Michel Daviot]

I completely agree with Petr's post about lack of documentation ! 

I'd love to get a pointer to better explainations about implicits, CanBuildFrom, type inference and its limitations.

I could use these to teach to colleagues newcomers to Scala who are still puzzled by these 3.

--

[Alexandru Nedelcu]

On Tue, Feb 14, 2017, at 17:35, Seyed H. HAERI (Hossein) wrote:
> And, BTW, C++ has constantly been loosing market share for its
> excessive difficulty to learn.

The problem with C++ isn't only difficulty in learning, but also
difficulty in using, which doesn't get better in time.

Because of memory management issues, along with the memory unsafe
constructs of the language coming from C which still lurks within, code
written in C++ is always a minefield. For those that witnessed what
happened recently with Cloudflare dumping sensitive data all over the
net, that's yet another memory leak caused by C/C++ code. Heartbleed
before it? Another memory leak caused by C/C++.

If C++ is indeed losing market share, that's because developing
applications in C++ is insane. And a comparison with Scala, or any other
memory safe language, is comparing fruits with bacteria.

[Naftoli Gugenheim]

On Wed, Mar 1, 2017 at 2:59 AM Michel Daviot <mic...@daviot.info> wrote:

I completely agree with Petr's post about lack of documentation ! 

I'd love to get a pointer to better explainations about implicits, CanBuildFrom, type inference and its limitations.

I could use these to teach to colleagues newcomers to Scala who are still puzzled by these 3.

The documentation could certainly afford to be improved. Fortunately some improvements have been happening very recently:

https://github.com/scala/scala.github.com/pull/698

https://github.com/scala/scala.github.com/pull/699

https://github.com/scala/scala.github.com/pull/704

https://github.com/scala/scala.github.com/pull/705

https://github.com/scala/scala.github.com/pull/707

https://github.com/scala/scala.github.com/pull/708

https://github.com/scala/scala.github.com/pull/713

https://github.com/scala/scala.github.com/pull/714

https://github.com/scala/scala.github.com/pull/715

https://github.com/scala/scala.github.com/pull/716

https://github.com/scala/scala.github.com/pull/717

https://github.com/scala/scala-lang/pull/608

Hopefully they'll be merged soon.

[oss.m...@gmail.com]

Mine main message was that if implicits are seen as analogous with subtyping/inheritance it is possible to realize that a lot of best practicies are analogous as well and that there are similar risks.

Consider. But don't take it literally, there are differences in implementation details:

1) Implicit resolution resembles inheritance. Based on my experience from a team it can take long till user realizes if left to learn alone. If the first search stage is not successful (mostly explicit implicits and imports) all companion objects of all supertypes of a given type are searched for applicable implicits. Hence it can be seen as that type B inherits them.

2) What if multiple applicable are found? Static resolution rules are used to find the one which is the most specific for the given type. It resembles "override".

3) Now even without reading docs there is inevitable question. Is there diamond like problem? Yes. If compiler can't decide which one is the most specific it fails. There is no linearization like with inheritance and order how traits are mixed doesn't play a role (AFIK). How it can be resolved? In the same manual way how we did in C++, "override" with implicit implementation in subtype. Or in the case of implicit parameter by explicitly passing selected one.

Try to ask within your team who knows all of them.

The dangerous power of implicit conversions is that they accept subtypes, suptype polymorphism applies inside them and they can call any side-effects including updating external state. The sybtype polymorphism means that if implicit conversion is using some method of a type it is defined for than subtype can override it and put some side-effects and trigger them just by passing a value.

Because of it implicit conversions should be as simple as possible. I even think they should be enforced to be pure by compiler. Or there should be annotation for it to enforce that compiler checks it. They should only call side-effect free getters (those generated by a compiler) to convert a type. Only to allow type substitution. If there is another use, it should be very well established and known pattern.

It is interesting to take all this and contrast it with Golang. It can be seen as that Golang has hardcoded implicit conversion from type B to type A which converts B to A if A is contained in B and allows seamless type substitution. You can find articles which explain Golang concepts in terms of OOP subtyping/inheritance. It is static dispatch (no polymorphism), for dynamic there is interface type.

Scala implicit conversions are as flexible that it is possible to do the same thing as Golang does. Just define type A as contained in B (composition) and define implicit conversion from B to A which only returns contained A. It will be static dispatch (no polymorphism). Golang is considered as very simple language and they can program quite a bit using technique, typically called delegation. For me it seems like an example of a valid pattern for invisible implicit conversions, more verbose though.

Otherwise implicit conversions could be reserved only to seamlessly map type to 3rd party type to seamlessly use its API. Just like scala.Long to java.Long. But only if this conversion is always very fast. This is a reason why Java collections to Scala collections implicit conversions are not applicable.

This way we should end-up with handful of well described use cases for implicit conversions. The rest should be covered by implicit parameters which are visible in type system and namely typeclasses.

This applies:

"Poetry is the art of giving different names to the same thing."

"Mathematics is the art of giving the same name to different things." Henri Poincaré.

For illustration there is Golang like code:

object GolangImplicits {

  class A(val a: Int) {

    def message = s"From A: $a"

  }

  class B(val a: A, b: Int) {

    def message = s"From B: $b"

  }

  object B {

    implicit def toA(b: B): A = b.a

  }

  def show(a: A): Unit = println(s"Result: ${a.message}")

  def run(): Unit = {

    val b = new B(new A(1), 2)

    show(b) // prints "Result: From A: 1"

  }

}

Petr

[Naftoli Gugenheim]

Incidentally, C# has had implicit conversions apparently since the beginning: https://msdn.microsoft.com/en-us/library/z5z9kes2(v=vs.71).aspx

(It has to be in a user-defined type though.)

[oss.m...@gmail.com]

C# implicit conversions are very limited in comparison to Scala implicits. C# is a sling here, Scala a machine gun. They are only type declaration site hence they have no ad-hoc power. They as well ignore subtype relations during implicit resolution. I don't remember from my short time with C# how they work (if at all) for parametrized types.

In C#, if type Bar has implicit conversion to Int and Baz is subtype of Bar without its own implicit conversion to Int than Baz will be never converted to Int implicitly.

In Scala it would convert Baz implicitly to Int. And this implicit conversion can be defined ad-hoc for Baz. Both these aspects is what makes Scala powerful. In case of plain conversions it seems to be too powerful. The question is what we would loose if it would allow to call only compiler generated getters on input type and disallow other side-effects within implicit conversions.

Golang "implicit selection" is type declaration site only as well (no ad-hoc). Methods can be added to type only within the same package where type is defined (AFIK).

I think documentation is missing code snippets which would show all implicit resolution aspects one by one.

Here is an example for subtyping and polymorphism aspect (use :paste to run it in REPL):

object ImplicitSubtypeResolution {

  class Bar(val x: String, val y: String) {

    def omg(): Unit = println("OMG from Bar!!!")

  }

  object Bar {

    // NOTE: Avoid to convert implicitly to such a common type

    implicit def toArrayString(bar: Bar): Array[String] = {

      // polymorphic side-effect!!!

      bar.omg()

      println("Implicit conversion Bar -> Array[String]")

      // bar.x and bar.y are polymorphic but it is fine as far as simple.

      // After conversion, whatever is called inside its consuming method

      // on this output is called on Array[String]

      Array(bar.x, bar.y)

    }

  }

  // can be in a galaxy far far away

  class Baz(override val x: String, override val y: String) extends Bar(x, y) {

    override def omg(): Unit = println("OMG from Baz!!!")

  }

  // can be in a galaxy even more far far away

  def show(a: Array[String]): Unit = {

    println(s"Shown: ${a.mkString(",")}")

  }

  def run(): Unit = {

    val bar = new Bar("Bar", "Hello World")

    println("Show with Bar called:")

    show(bar)

    // Because B is subtype of A it looks into companion object of A

    val baz = new Baz("Baz", "Goodbye")

    println("\nShow with Baz called:")

    show(baz)

  }

}

Output:

Show with Bar called:

OMG from Bar!!!

Implicit conversion Bar -> Array[String]

Shown: Bar,Hello World

Show with Baz called:

OMG from Baz!!!

Implicit conversion Bar -> Array[String]

Shown: Baz,Goodbye

Petr

[oss.m...@gmail.com]

As well it seems that in C# there is very strong best practice - implicit conversion should never loose information. Which I understand as if A1 is converted to B there must exist such a conversion from B to A2 so that A1 equals A2. Otherwise conversion should be explicit. It is stronger than subtyping. I think it is because they apply during equality checks. If A would have conversion to B, and B to A than this is considered confusing in C# to evaluate to false: A == B == A (A is converted to B and compared, B is converted to A and compared). Scala implicit conversions are (should be) analogous with (variant of) subtyping (known as well as coercive subtyping) where this mathematical equality doesn't hold.

C# technically allows side-effects in implicit conversions, I don't know but I would guess it is hardly considered a good practice (outside of some console printing). 

Petr

[Naftoli Gugenheim]

On Mon, Mar 6, 2017, 12:12 PM <oss.m...@gmail.com> wrote:

C# implicit conversions are very limited in comparison to Scala implicits. C# is a sling here, Scala a machine gun. They are only type declaration site hence they have no ad-hoc power. They as well ignore subtype relations during implicit resolution. I don't remember from my short time with C# how they work (if at all) for parametrized types.

In C#, if type Bar has implicit conversion to Int and Baz is subtype of Bar without its own implicit conversion to Int than Baz will be never converted to Int implicitly.

I find that particularly interesting in light of the fact that the c# docs seemingly call using a Baz where a Bar is expected an implicit conversion.

--

[Seyed H. HAERI (Hossein)]

Hi Petr,

> Scala implicits are generally misunderstood. Most of hate posts I see miss
> the point of them and use them worng.

... which is another way for saying they are comparably much less
intuitive than subtyping.

> Here is the confusing part. While they are values they typically don't carry
> a value information for our value level program. This is important
> distinction to realize. They carry information for the type-system/compiler.

Unfortunately, however, implicits get the same plain object semantics.
In particular, as I also formerly clarified it on this thread, they
get the same unchangeable automatic late-binding of every other Scala
object. That is, their semantics is what you call value-level. Scala
doesn't give dependant type theory a first class support. Its
dependant method types -- although sometimes nifty -- is just
accidental. In such a situation, it is so unnatural to choose the
objects-as-families approach. (See my former posts on this thread.)
And, that's a great source of confusion for implicits.

> I have just look at the documentation after long time. No way I could
> understand Scala implicits with what is there. I even scanned Scala Specs
> Chapter 7 about implicits. I just can't find explanation for a mechanic how
> implicits values with types with multiple type parameters resolves. Without
> this knowledge nobody can understand how CanBuildFrom works which is used in
> standard collection library! I prefer CanBuildFrom any day against Java8
> collect. It is such a genius pattern, and I see at least one side why
> developers have problem to understand it.

That's a very fair point. And, I think that might very well be one of
the reasons why Robert initiated this thread.

> And explain limitations of Scala type inference. I think it is missing in
> the docs as well. Every starting developer will assume that it just works
> always. And get lost as soon as it doesn't not even knowing what to look
> for.

Indeed.

> And without understanding type inference limitations there is little
> chance to understand couple of design patterns which mitigate it. Which
> means not understanding library code which uses them and majority do.

Well, even with proper type theory background, one aught to get lost
in many dark corners. For example, I make money out of type theory for
God's sake. :D

Cheers,
--Hossein

[Seyed H. HAERI (Hossein)]

Hi Alexandru,

This isn't a thread about C++ vs Scala. So, I'm not going to response
to comments along those lines. My words below were on lessons Scala
can take from the downfall of C++.

>> And, BTW, C++ has constantly been loosing market share for its
>> excessive difficulty to learn.
>
> The problem with C++ isn't only difficulty in learning, but also
> difficulty in using, which doesn't get better in time.

Not really. Using C++ is not that difficult for the one who has
properly learned it. It's the learning that is increasingly
unaffordable to most people. I'm afraid the same pattern is observable
for Scala.

Cheers,
--Hossein

[Rex Kerr]

I beg to differ.  It is not that difficult to make mostly-working C++, but it is very, very difficult to be adequately rigorous such that one has actually zero possible buffer overruns, zero dangling pointers and other resource disposal issues, and so on.  Features like templates interact in tricky ways with these basic difficulties: they give you abstraction mechanisms to help enable safety by design, but the same mechanisms can give you problems "by design" if you forget to think of some corner case, and comprehending a set of templates that someone else wrote (or I wrote long ago) can be an arduous process.

So my experience with C++ is that the difficulty doesn't get better with time; you hit a plateau after a while, and then keep having the same struggles with shooting yourself in the foot in various ways.  I'm sure some people end up in a less error-prone plateau than I do, but I think I'm close to maxed out.  I don't hit my foot very often, but it still really hurts when I do.  It can take days to track down a subtle memory corruption issue, for instance.

I'd be perfectly happy to learn something that had the same use cases as C++ and was *harder* to learn, as long as the plateau was higher.  (FWIW, I'm very likely going to switch to Rust once I have a decent way to write vector processing instructions inline.  Rust is not harder to learn than C++, though, IMO.)

Anyway--yes, it's hard to afford time to learn things, but C++ has two strikes against it in this regard, in my book at least.

  --Rex

[Seyed H. HAERI (Hossein)]

Hi Petr,

> The dangerous power of implicit conversions is that they accept subtypes,
> suptype polymorphism applies inside them and they can call any side-effects
> including updating external state.

Indeed, this is more into my point on the unnaturalness of giving type
level duties to object level creatures -- i.e., implicits actually
being ordinary Scala objects in that they live at runtime.

> Because of it implicit conversions should be as simple as possible. I even
> think they should be enforced to be pure by compiler. Or there should be
> annotation for it to enforce that compiler checks it. They should only call
> side-effect free getters (those generated by a compiler) to convert a type.
> Only to allow type substitution. If there is another use, it should be very
> well established and known pattern.

Sorry Petr. But, I have difficulty reading your sentences. I'm afraid
they look too cryptic to me. I often fail to parse them sensibly.

Anyway, what I humbly seem to understand -- in your absence of proper
language terminology on top -- is that you're trying to recommend the
classes-as-families approach [1] of lightweight family polymorphism
[2]. Far away from the Scala mindset... :( (My apologise for sounding
sniffy here. I'm afraid the programming language terminology is
crucial for such a language design discussion.)

> It is interesting to take all this and contrast it with Golang. It can be
> seen as that Golang has hardcoded implicit conversion from type B to type A
> which converts B to A if A is contained in B and allows seamless type
> substitution. You can find articles which explain Golang concepts in terms
> of OOP subtyping/inheritance.

Would you mind pointing us to the exact article please?

> It is static dispatch (no polymorphism), for
> dynamic there is interface type.

<snip>

> For illustration there is Golang like code:
>
> object GolangImplicits {
> class A(val a: Int) {
> def message = s"From A: $a"
> }
>
> class B(val a: A, b: Int) {
> def message = s"From B: $b"
> }
>
> object B {
> implicit def toA(b: B): A = b.a
> }
>
> def show(a: A): Unit = println(s"Result: ${a.message}")
>
> def run(): Unit = {
> val b = new B(new A(1), 2)
> show(b) // prints "Result: From A: 1"
> }
> }

Hmm... Where does the above code manage to avoid dynamic dispatch?
From what I can see, it has solely not used one.

Cheers,
--Hossein

[1] P. Jolly, S. Drossopoulou, C. Anderson, and K. Ostermann, Simple
Dependent Types: Concord, Proc. 6th W. Formal Tech. Java-like Prog.,
June 2004, Tech. Rep. nr. NIII-R0426, Uni. Nijmegen.

[2] C. Saito, A. Igarashi, and M. Viroli, Lightweight Family
Polymorphism, J. Func. Prog. 18 (2008), no. 3, 285--331.

[Seyed H. HAERI (Hossein)]

Evening Rex,

>> Not really. Using C++ is not that difficult for the one who has
>> properly learned it. It's the learning that is increasingly
>> unaffordable to most people. I'm afraid the same pattern is observable
>> for Scala.
>
> I beg to differ.

C++ and Scala are of course different in many ways. Yet, I fail to see
the relevant difference to this thread. Robert started this thread,
crying out for Scala's difficulty to learn, for example. And, more
evidence was later provided by say Petr and Naftoli (?).

> It is not that difficult to make mostly-working C++, but
> it is very, very difficult to be adequately rigorous such that one has
> actually zero possible buffer overruns, zero dangling pointers and other
> resource disposal issues, and so on.

(And, your words above read like early 2000s C++ to me. Have you ever
tried C++11 and after? You might want to come back offline on this one
to me for its off-topic here.)

Cheers.
--Hossein

[Jasper-M]

I would just like to point out that, while you and Robert seem to agree that Scala has problems, the way you would like to solve those problems is exactly the thing that Robert said is the cause of Scala being a bad language: people with PhDs introducing academic concepts into the language which are not intuitive to typical developers with a Java background.

Maybe we can take this to mean that Scala strikes a nice balance between academic and business oriented. In Belgium we say a good compromise is one where all parties are equally unhappy.

Kind regards,

Jasper

Op dinsdag 7 maart 2017 00:09:59 UTC+1 schreef Hossein:

[Seyed H. HAERI (Hossein)]

Hi Jasper,

As a matter of fact, I am not trying to solve any problems here.
Indeed, for me, Scala is nothing beyond a research language. And, just
because it has some truly impressive industrial use cases, I don't
consider it of an industrial significance. What I agreed with Robert
about Scala was that the language is too hard to learn for it to win a
good market share ever. When it comes to discussing language design
issues, however, academic terminology is inevitable or precision will
be lost.

Cheers,
--Hossein

> I would just like to point out that, while you and Robert seem to agree that
> Scala has problems, the way you would like to solve those problems is
> exactly the thing that Robert said is the cause of Scala being a bad
> language: people with PhDs introducing academic concepts into the language
> which are not intuitive to typical developers with a Java background.
> Maybe we can take this to mean that Scala strikes a nice balance between
> academic and business oriented. In Belgium we say a good compromise is one
> where all parties are equally unhappy.

P.S. Being based in Brussels, I can cheerfully testify it that you
Belgians are indeed skilful when it comes to the art of coexist. :)