The JVM situation

7,527 views
Skip to first unread message

Tony Morris

unread,
Jun 12, 2013, 10:54:33 PM6/12/13
to scala-fu...@googlegroups.com
Hello,
I have a question. I hope you might be able to help me out. The question
can be broadly phrased, "what can be done about the JVM situation?", but
what do I mean by this?

I have some strong opinions that influence any potential answer I might
have. Maybe you don't quite agree and so it changes the question up a
bit, but here they are anyway:

On Scala:

1) Scala is the best available programming language for the JVM.
2) Scala does not do Java (legacy library) inter-operability very well.
3) Scala is not a well-designed language, but is at least better than
other not-so-well-designed JVM languages.
4) Scala is unlikely to be influenced into adhering to better language
design principles. That is, the poor design is likely to continue into
the future.
5) The Scala compiler cannot be worked on easily. Specifically, if
language improvements were to come into consideration, it would be
difficult to implement them with the current compiler.

On "the JVM situation":

5) The so-called library support offered by the JVM is often over-stated
in its utility.
6) The JVM is a platform that is best avoided if possible e.g. a new
project without any commitment would be best served by avoiding the JVM
due to all of its penalties and almost zero benefits.

There exist projects that cannot avoid using the JVM. This situation is
what I will call the JVM situation.

It has been argued to me that the extent to which the JVM cannot be
avoided is often over-stated. That is to say, a project participant may
claim, "but I cannot avoid the JVM!" to which an appropriate response to
such a hasty remark is, "bullshit, think harder." I do not hold a strong
position on this argument and I would like others' opinions on whether
it has merit.

Supposing that the argument fails, we are left with a not-so-rare
situation of having to use the JVM. However, Scala is not very well
designed. What to do about this? Assuming you had a lot of time to
address this problem, I see a number of possibilities:

a) Construct a JVM language that is well-designed with good legacy
library interoperability.
b) Construct a Scala-compliant compiler with extensions to accommodate
language improvements and optimisations. This would also address the
speed issue with scalac as a consequence. It would also offer better
external tool support that integrates into the compiler.
c) Concede the aforementioned argument. You are not bound to the JVM as
much as you formerly claimed. Use a language like Haskell or O'Caml and
reap the benefits.
d) Concede that you are simply doing the best you can with available
tools. Use Scala -- at least you're doing better than the guy down the
road using Java or "Scala as if it were Java."

There are possibly others options I have not thought of. Each of these
has some amount of required effort ranging from nothing to "lots." This
is important to providing an answer. In other words, which response to
the JVM situation would you consider is the best use of effort?

Thank you for any insights you might have.

--
Tony Morris
http://tmorris.net/

Brian McKenna

unread,
Jun 12, 2013, 11:19:28 PM6/12/13
to scala-fu...@googlegroups.com
I definitely agree that the JVM is over-valued.

I would not like to see b) or d) taken as solutions. We either need to
invest in a good language on the JVM or not the JVM at all. I'll take
the latter as much as possible.

Now, there's many things that can be approved on the state of the art
(e.g. Haskell) - a sane standard library, non-crazy parser, good
records, easy package management, a real module system, etc. We could
do slightly better than Haskell and much better than what's on the
JVM. I think we both know that's the "right" way to go if we had spare
time/effort.

So how long would it take?

There is also the JavaScript problem, perhaps you could solve that one
for me? ;)
> --
> You received this message because you are subscribed to the Google Groups "scala-functional" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

Ivan Meredith

unread,
Jun 12, 2013, 11:29:08 PM6/12/13
to scala-fu...@googlegroups.com
The one thing I like about the jvm is that I can compile and package code on one platform and deploy to another without worrying about anything but the fact that the jvm is installed.

Another nice thing about the jvm/java/scala tooling is that it seems to work better than just about anything else I've used. I've had multiple issues with cabal - one i think was because i had the 64bit version of haskell trying to build a lib (again a non-issue on the jvm).


Vincent Marquez

unread,
Jun 13, 2013, 3:43:32 AM6/13/13
to scala-fu...@googlegroups.com
Hello all,

While I'm new to the community and functional programming, I thought it might be useful for someone to argue for the JVM.  While the JVM has its share of problems, and Scala seems to be collecting plenty of its own,  I think there are enough quality libraries that exist for it maintain it's draw. 

Netty is a fantastic piece of non- trivial engineering that I've made good use of in multiple projects (and currently in my OSS scala lib).  I'd actually be surprised if there was comparable performing network libs in Haskell or other platforms (please correct me if I'm wrong).   I've also made good use of JVM SIP (VoIP) libraries that most likely have had tens of thousands of man hours invested into them.  

Plenty of large JVM libraries are large and over engineered due to bad programmers and java's language deficiencies, but there are some that are enormous and complex because they implement some really fat RFCs.   These types of infrastructure level libs are extremely helpful for creating the big, visible projects OSS languages need to grow.  

Tooling is another area that benefits from the JVM.  I won't defend any of the IDEs, but having MAT and YourKit for analyzing memory and performance issues has been invaluable in my experience.  

If I get a vote, I'd say the community should focus on B.  Well written scala code in the functional style (eschewing some of the language's 'tradeoffs') is some of the most pleasant code I've seen.  Unless someone has vast resources backing them, I just can't see A gaining enough momentum at this point.  I'd be interested to be proven wrong though, and will be happy to assist in any way with A or B, if either end up being viable.  

--Vincent

Chris Marshall

unread,
Jun 13, 2013, 4:15:48 AM6/13/13
to scala-fu...@googlegroups.com
I'm going to argue for d) here. 

It's quite plainly obvious that constructing a serious language is a prodigious undertaking. Who would do it? Why? Who would put a roof over their head and food in their babes' mouths for the years it would take? Even if you started now, it would probably be 2016 before you had something stable. No, I think a) is out

b) well, the above argument goes for b) as well. You have to construct a compiler for a notoriously "complex" language and then you have to keep this compliant as scala evolves. I might say that this would presumably take as least as much effort as is being put into scalac, which appears to be a lot, judging from scala-internals

c). Maybe you *are* bound to the JVM. "Boss, can we move off the JVM?" "No". 

Chris

Tobias Hammerschmidt

unread,
Jun 13, 2013, 4:25:30 AM6/13/13
to scala-fu...@googlegroups.com
Maybe CAL (https://github.com/levans/Open-Quark) could be a starting point for a)?

Tony Morris

unread,
Jun 13, 2013, 4:49:51 AM6/13/13
to scala-fu...@googlegroups.com
On 13/06/13 18:25, Tobias Hammerschmidt wrote:
> Maybe CAL (https://github.com/levans/Open-Quark) could be a starting
> point for a)?
> --
> You received this message because you are subscribed to the Google
> Groups "scala-functional" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to scala-function...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
CAL does not do legacy interop well.

Hugh Gilmore

unread,
Jun 13, 2013, 6:10:49 AM6/13/13
to scala-fu...@googlegroups.com
Scala has its problems. It's not ideal. But, what are the attributes of the ideal language and its concomitant ecosystem? Please enlighten. All I read is criticism of the current state of art.

Gregory Tucker

unread,
Jun 13, 2013, 8:37:19 AM6/13/13
to scala-fu...@googlegroups.com
My background is operations management, and I have only recently joined the Scala community. So far I like what I see, but you raise some interesting questions for me. Please be assured these questions are from a naive starting place of inquiry, and not meant to inflame, so please don't flame me as an idiot (though i would deserve it... :-) 

1) How is Scala not a "well-designed" language? 

2) What is a "well-designed" language? Haskell? Why don't you just switch to Haskell? I have been told by a Haskell developer that its code is faster and much more elegant looking with fewer parentheses, etc. 

3) What's keeping people on the JVM? Libraries and platform interoperability and maybe GC? 

4) A little off-topic but I just started learning Ruby for another project, and it seems more "clever" but less elegant than Scala, for example the += operand or using = at the end of class names. By the standards of a "well-designed" language above, is Ruby even below Scala? (Ignore for now Ruby is dynamically typed...) 

Regards,
Greg

David Landis

unread,
Jun 13, 2013, 10:30:10 AM6/13/13
to scala-fu...@googlegroups.com
e.) If one devoted a fraction of the time it would take to construct a new language to working on the existing Scala compiler, then I expect a lot of improvements could be implemented that would benefit many Scala users. Even if one concedes that certain areas are hopelessly broken, there are likely other areas in which useful improvements could be made. 


Hugh Gilmore

unread,
Jun 13, 2013, 10:45:57 AM6/13/13
to scala-fu...@googlegroups.com
Could someone enlighten me about what is "hopelessly broken" in scala. I've been programming "idiomatic" scala professionally for almost 3 years and, while the scala development stack has many flaws, I can't really call anything in scala "hopelessly broken". If this thread is going to be any benefit to the unenlightened like myself, the cognoscenti should share their insights so that we all may achieve satori. 

Oscar Vargas Torres

unread,
Jun 13, 2013, 11:50:14 AM6/13/13
to scala-fu...@googlegroups.com
>> There exist projects that cannot avoid using the JVM. This situation is what I will call the JVM situation.
I am facing this (like the "Boss" situation oxbowlakes mentions).

>> Scala is not very well designed. [...] Assuming you had a lot of time to address this problem
*I* don't have enough background to design a programming language. Assuming *I* had the necessary expertise, it would take me years to get something stable, mature and widespread. As oxbowlakes said,  "Who would put a roof over their head and food in their babes' mouths for the years it would take? Even if you started now, it would probably be 2016 before you had something stable. No, I think a) is out". I have the same opinion.

>> c) You are not bound to the JVM as much as you formerly claimed. Use a language like Haskell or O'Caml and reap the benefits.
This is the best choice for *me*.

If *you* are planning to invest your valuable time on a good programming language design, maybe the JVM should be avoided. I *guess* Odersky chose the JVM because he would have more chances his language would become mainstream. If your most important concern is "correctness" and a "good design" ... *I* would be worried to get, as many as possible, "competent" people working with me (an isolated effort might be praiseworthy, but the tasks seems overwhelming for one person --if you want something better than Haskell, for example--).

Money is always a concern (at least for me), and there should be money supporting the development of the language (business + investigation is more than just investigation). Time is never enough...


>> d) Concede that you are simply doing the best you can with available tools.
This is why I am using Scala. It seems to be gaining momentum, and there is money and business around the language (typesafe?), so there are more chances for Scala to survive (even if there are pitfalls in the language, compiler, JVM, etc).

I would like to invest my time learning and using a language that is "widespread" (Haskell, for example), but I would be less motivated to invest my time on a language that is known to just a dozen of people. However, my mind is always excited by "correctness", "good design", "useful and higher level abstractions".

In short, you would need help from talented people, and money, to get a "better language".

Razvan Cojocaru

unread,
Jun 13, 2013, 12:16:50 PM6/13/13
to scala-fu...@googlegroups.com
Just a quick first thought - it's not all about just libraries, but
environment.

For instance, the Eclipse+XText is a very powerful environment built on top
of the JVM, allowing you to define DSLs at will and comes with a lot of
bells and whistles, content assist etc... there's also PaaS services that
run JAR files etc.

I think the only two real choices are a) and d). You simply have a lot more
flexibility in the JVM environment and they're not exclusive.

Brian McKenna

unread,
Jun 13, 2013, 1:36:43 PM6/13/13
to scala-fu...@googlegroups.com
I love this talk about money. I think people are greatly
over-estimating the resources needed to make a good JVM language. We
don't need venture capital.

I'll give an estimate for a usable prototype:

* 1 week to make a Hindley-Milner, strict, functional JVM language
* 3 days to add laziness via InvokeDynamic (requires Java 8)
* 2 days to add type-classes
* 2 days to add a sane FFI

Is there something I'm missing? I'd much rather use the prototype than
Scala for a lot of things, almost makes it worth the 2 week
investment.

Yorick Laupa

unread,
Jun 13, 2013, 1:45:48 PM6/13/13
to scala-fu...@googlegroups.com
I think you've forgot higher-rank polymorphism. While not being too difficult to implement, add it to Hindley-Milner takes more than a week IMO.

Sent from my iPhone

Oscar Vargas Torres

unread,
Jun 13, 2013, 2:07:59 PM6/13/13
to scala-fu...@googlegroups.com
I love this talk about money. I think people are greatly
over-estimating the resources needed to make a good JVM language.
We don't need venture capital.

Ok. I do not doubt you guys are fantastic coders and very knowledgeable (personally I could learn a lot from you, and I bet I am not as talented as you are, neither I have your experience. But I was not talking about another prototype language. I was talking about making your language used by thousands of people. Wouldn't you need more resources to get your language in the road to become mainstream?

Ryan LeCompte

unread,
Jun 13, 2013, 2:13:45 PM6/13/13
to scala-fu...@googlegroups.com, scala-fu...@googlegroups.com
Regarding Ruby vs. Scala, you might be interested in my recent scala days talk:


Ryan

Brian McKenna

unread,
Jun 13, 2013, 2:30:12 PM6/13/13
to scala-fu...@googlegroups.com

I do not value 1000s of users or a mainstream language. I value being able to solve problems with capable teams and capable tools.

--

Oscar Vargas Torres

unread,
Jun 13, 2013, 3:02:11 PM6/13/13
to scala-fu...@googlegroups.com

I do not value 1000s of users or a mainstream language. I value being able to solve problems with capable teams and capable tools.

Then we might be talking about different (valid) concerns. Tony Morris (that raised this "JVM situation" conversation) is who may clarify goals. For example, I didn't understand if the proposed language "has to be" on top of the JVM or can use some other existent platform (on top of Haskell's??), or even start from scratch.

What are the language design principles that would guide development?
Correctness, functional language, "good" legacy library interoperability (could you point out some problems with Scala - Java interoperability? I would like to learn from your expertise), what else?

Does the possible number of users matters? (for those involved in the design and implementation of the language)
(If it doesn't, then legacy library interoperability wouldn't matter neither). For example, I would like to see a version of Scala that could totally forget about Java. Then you could have a more beautiful (useful?) language.

What is the intended audience for the proposed language?

Chris Marshall

unread,
Jun 13, 2013, 3:13:51 PM6/13/13
to scala-fu...@googlegroups.com, scala-fu...@googlegroups.com
Presumably the company you work for are ok with you writing their systems in a language you knocked up in a fortnight and which only a handful of other people know? <--- although this is a question mark, this is not really a question

And I thought Tony mentioned interoperability with Java. Will you have this? How will subtyping in Java interact with HM?

I didn't do a CS degree and know next to nothing about language design, complexity and implementation. But suggesting that you can do it in a few weeks seems to be optimistic - just by looking at how it's panning out for other languages (Scala, kotlin, Ceylon etc). Cue Tony spitting out his tea.

When I mentioned money, I merely meant that, if this would take one person a year, that person must be able to support themselves for a year.

Chris

Tobias Hammerschmidt

unread,
Jun 13, 2013, 3:48:20 PM6/13/13
to scala-fu...@googlegroups.com
Hi Tony,

I'm not advocating CAL here, I just stumbled upon it some years ago and I remembered it when reading your questions. But anyway I'm interested in your ideas around legacy interop or as Brian McKenna wrote "sane FFI" - for instance how would you handle the ever present nullness problem? Autowrap everything in Option monad?

Back to your original questions - as you might recognized by now I'm tending towards option a) ;-) If there are too many problems in scala itself to ever be fixed (can't really judge here as a scala newby) then just leave it behind and start something fresh. Also the decision for a specific target platform shouldn't be based on the available libs alone (though this of course an important part) but also on ease of portability. For example when developing an application for Linux, OS X and Windows I want to minimize the amount of code changes required to make it run on the different operating systems. This is where the JVM really shines IMHO.

Brian McKenna

unread,
Jun 13, 2013, 3:58:27 PM6/13/13
to scala-fu...@googlegroups.com
For people that don't get what Tony means by "good interoperability".

To get correctness in a language that talks to Java, we'd have to wrap
things up. Almost every call to Java should look like this:

pop :: IO (LinkedList a) -> OptionT (EitherT IO) Exception a

That is it could return null (let's make that a value), it could throw
an exception (let's make that a value) and could do arbitrary IO.

Let's make that a type alias:

type Java a = EitherT (OptionT IO) Exception

So:

pop :: IO (LinkedList a) -> Java a

Now we can map over that data, do safe things with it and have better
guarantees that our programs do the right thing, since the types are
no longer lies and errors are explicit.

You'd probably also want a way of saying String#length() is pure.

Brian McKenna

unread,
Jun 13, 2013, 4:00:42 PM6/13/13
to scala-fu...@googlegroups.com
I made two mistakes there. Should look like:

pop :: IO (LinkedList a) -> OptionT (EitherT IO Exception) a
type Java a = OptionT (EitherT IO Exception) a

John Nilsson

unread,
Jun 13, 2013, 4:21:12 PM6/13/13
to scala-fu...@googlegroups.com

What function on the jvm would not have an exception in the return type?

BR
John

simon.ochsenreither

unread,
Jun 13, 2013, 4:27:42 PM6/13/13
to scala-fu...@googlegroups.com
From my POV, as someone who is not good at producing code, I don't think creating a new language will help.

There is so much more to it than implementing a compiler.
Have a look at Kotlin or Ceylon, they have barely anything yet, but their marketing is excellent.
Have a look at Dart: It's hardly an improvement over JavaScript, but people are buying the PR.
Have a look at Haxe: The amount of supported platforms is incredibly impressive, but no one is using it.

In that sense, I think creating a new language will not help, but might even be harmful. Creating the message of "even 'hardcore' Scala people are not happy with the language and started to build their own one instead" is certainly increasing the chances that you won't end up with Scala (with all its faults) in 2020, but with Java 8.5 or Kotlin 0.7.

Regarding the language:
I'm extremely happy with the progress we made in 2.11 to get rid of stupid, plain broken or dangerous stuff in the language or the libraries. Are we there yet? Not by any measurement!
This doesn't mean we have to give up, we just need people continuously pushing things into the right direction.
I think that things like the effect system, union types, the work on a sound foundation of the typesystem and macros show that there is willingness to improve and change the language, even if it painful.
This is ALL about politics. One has to learn to act when the timing is right and stop to annoy people otherwise. Considering that I'm NOT payed by anyone and do this in my free time, I think I'm very happy with the results.

Regarding the JVM:
Maybe some people are aware of it, maybe some are not: I have spent the last weeks and months on testing and fixing Scala to run on Avian.
If you want proper tail calls, runtime support for continuations, native compilation and embedding, this is the place where you want to be!
If you want better support for laziness, better performance for functional idioms, runtime support for value types, runtime specialization, this is the place where you want to contribute!
The right to fixing the JVM is not some completely new runtime, but a JVM adapted to our needs. This way, all the existing code keeps working while we can build better abstractions for us.

Regarding Java:
Honestly, I don't care.

So please, if you are unhappy with Scala and don't think you can influence the language in the way you like, please contribute to Avian!

Imho, what we need is no revolution, but a continuous evolution.

Marimuthu Madasamy

unread,
Jun 13, 2013, 7:17:04 PM6/13/13
to scala-fu...@googlegroups.com
I think that instead of creating a new JVM language, why not contribute to and evolve existing languages like Yeti (impure, strict, functional) or Frege (pure, non-strict, functional, Haskell-like for JVM)? Both the projects are very active but the community is very small. With these languages, being on the JVM, we have a nice option to choose between Scala, Yeti, Frege or Clojure for a particular job.

I am really interested in Frege and have done a few programs. Here are some examples:

Native declaration for java.lang.Integer.parseInt:

pure native parseInt Integer.parseInt :: String -> Int

We can call the function like:
main _ = println $ parseInt "23"

Here is an impure native function declaration:

native currentTimeMillis java.lang.System.currentTimeMillis :: () -> IO Long

Accessing Java Map and HashMap:

data JMap k v = mutable native java.util.Map where
   
native get :: JMap k v -> k -> IO (Maybe v)
   
native put :: JMap k v -> k -> v -> IO ()
   
data
HashMap k v = mutable native java.util.HashMap where
   
native new :: () -> IO (HashMap k v)

Here the "get" method on Map might return null so we can wrap that in "Maybe" which tells Frege to automatically handle null and return with Nothing in case of null.

Here is how you can reference trim method from Java String (a pure function):
pure native trim :: String -> String

So my conclusion? For now I would recommend Scala in my job but would choose Frege for my projects. I would love to hear about the problems with Yeti and Frege too and maybe we can help address those.

Tony Morris

unread,
Jun 13, 2013, 7:26:37 PM6/13/13
to scala-fu...@googlegroups.com
On 13/06/13 18:15, Chris Marshall wrote:
I'm going to argue for d) here. 

It's quite plainly obvious that constructing a serious language is a prodigious undertaking. Who would do it? Why? Who would put a roof over their head and food in their babes' mouths for the years it would take? Even if you started now, it would probably be 2016 before you had something stable. No, I think a) is out

I wish to make one point clear. I have no illusions about the amount of effort that would be required to create a decent JVM language.

Where we might disagree is the disparity between these two points:
1) That which is the most practical language (Scala) for the JVM available now
2) That which is possible

I may not have been clear in the original post, but I wish to point out that it is my opinion that the distance between points 1) and 2) is huge.

It is also my opinion that Scala cannot close this gap for technical and social reasons. Let's stick with the technical reasons. Why wouldn't I just contribute to scalac? Because I am well-convinced that it is both easier and with a higher pay-off to simply start again and use a language more suited for compiler-writing (not Scala).




b) well, the above argument goes for b) as well. You have to construct a compiler for a notoriously "complex" language and then you have to keep this compliant as scala evolves. I might say that this would presumably take as least as much effort as is being put into scalac, which appears to be a lot, judging from scala-internals

I think this would be considerably less effort than a) but with less of a pay-off. This is why I proposed it.



c). Maybe you *are* bound to the JVM. "Boss, can we move off the JVM?" "No".

I do not accept this argument. I am immune to bullying and I encourage others to do same. This is a digression, but an interesting one.
-- 
Tony Morris
http://tmorris.net/

Tony Morris

unread,
Jun 13, 2013, 7:27:38 PM6/13/13
to scala-fu...@googlegroups.com
I would love to provide a useful summary in answer to this question. But I think the best answer is simply to write it don't you? I am wondering if others have similar aspirations or insights.

Tony Morris

unread,
Jun 13, 2013, 7:35:04 PM6/13/13
to scala-fu...@googlegroups.com
On 13/06/13 22:37, Gregory Tucker wrote:
My background is operations management, and I have only recently joined the Scala community. So far I like what I see, but you raise some interesting questions for me. Please be assured these questions are from a naive starting place of inquiry, and not meant to inflame, so please don't flame me as an idiot (though i would deserve it... :-) 

1) How is Scala not a "well-designed" language?

The best summary answer I can give here is simply that Scala ignores a considerable amount of language design research, especially that which has occurred in the last 20 years. It suffers as a result. A more detailed answer would go into each aspect of that research. Perhaps others on list can help with an answer here, because I might end up pulling my hair out if I go into too much detail.



2) What is a "well-designed" language? Haskell? Why don't you just switch to Haskell? I have been told by a Haskell developer that its code is faster and much more elegant looking with fewer parentheses, etc.

Yes I use Haskell a lot. While Haskell has a lot of "WTF were they thinking!?", I also think the distance between where we are at and what is possible is not as large. We could complain about Haskell's records or modules all day, but what is the practical cost? Is it reasonable to start again? Some might argue yes, but I am not in that race for now.




3) What's keeping people on the JVM? Libraries and platform interoperability and maybe GC?

Well I think this is the most interesting question of all. I do not think it is libraries, because there simply aren't many useful libraries for the JVM. There are a *lot* of libraries, but their utility is questionable.

As I said earlier, it has been argued against me that what is "keeping people on the JVM" is nothing more than an illusion. Those people might claim "libraries" or similar, but I am told this is often hyperbole and/or misdirection. The claimant is under the illusion that the JVM is providing benefit where no such thing exists.

I am sitting on the fence with this position for now. I simply do not know. Comment appreciated!



4) A little off-topic but I just started learning Ruby for another project, and it seems more "clever" but less elegant than Scala, for example the += operand or using = at the end of class names. By the standards of a "well-designed" language above, is Ruby even below Scala? (Ignore for now Ruby is dynamically typed...)

Yes off-topic, but my opinion is that our children are going to blame us for Ruby and we should start apologising now.

--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Tony Morris

unread,
Jun 13, 2013, 7:36:29 PM6/13/13
to scala-fu...@googlegroups.com
I hope I didn't imply that Scala is hopelessly broken. It isn't. I have been using it since around 2006. It's just hit both a technical and social limit and that limit is very low in the greater scheme. I like to aim high. I cannot help it.
-- 
Tony Morris
http://tmorris.net/

Gabriel Claramunt

unread,
Jun 13, 2013, 7:40:16 PM6/13/13
to scala-fu...@googlegroups.com
Yes, in less than a month you can have a pretty decent functional language up and running, but you'll still have to write the libraries for that. 
At least I can think of: String manipulation, IO, common data structures, monads et al.  
I don't think is that easy to come up with performant and mature implementations for those in a couple of weeks (I'm no superstar, so it might be my lack of imagination :D )
Still, very interesting, to the wish list I would add a deadly simple and smooth FFI so I can reuse existing libraries.


--
Gabriel Claramunt

Tony Morris

unread,
Jun 13, 2013, 7:40:30 PM6/13/13
to scala-fu...@googlegroups.com
--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
To be clear, I despise popularity contests. Scala is in some trouble because of this degenerate aspiration. SIP-18 is a perfect example of an attempt to further popularity, while providing zero (or less than zero) benefit to users of all persuasions. Just recently[1], having forgot SIP-18 existed myself, I had to explain to a class of ~12 people why they are seeing the implications of it. This digression was costly, especially that we were making such excellent progress otherwise.

The popularity aspiration is for losers in my opinion.

[1] http://www.yowconference.com.au/lambdajam/FPscala.html

Tony Morris

unread,
Jun 13, 2013, 7:41:36 PM6/13/13
to scala-fu...@googlegroups.com
I would rather a language knocked up in a fortnight that few people know, than a language knocked up over a few years that even fewer people know (Java, Scala).
-- 
Tony Morris
http://tmorris.net/

Tony Morris

unread,
Jun 13, 2013, 7:44:23 PM6/13/13
to scala-fu...@googlegroups.com
It's only a thought at this point and you are on the right track, but it would be a little more detailed than simply wrapping everything in Option. After all, can you imagine how clumsy that would be?

There are various monad stacks, such as Option[_], OptionT[IO, _] and then you might throw in exceptions. This idea alone would be clumsy to use, so taking it further so that the burden is zero (a requirement) would require significant library support. Thankfully a lot of this library support has been thoroughly investigated already.
--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Tony Morris

unread,
Jun 13, 2013, 7:47:07 PM6/13/13
to scala-fu...@googlegroups.com
On 14/06/13 09:17, Marimuthu Madasamy wrote:
I think that instead of creating a new JVM language, why not contribute to and evolve existing languages like Yeti (impure, strict, functional) or Frege (pure, non-strict, functional, Haskell-like for JVM)? Both the projects are very active but the community is very small. With these languages, being on the JVM, we have a nice option to choose between Scala, Yeti, Frege or Clojure for a particular job.

I am really interested in Frege and have done a few programs. Here are some examples:

Native declaration for java.lang.Integer.parseInt:

pure native parseInt Integer.parseInt :: String -> Int

We can call the function like:
main _ = println $ parseInt "23"

Here is an impure native function declaration:

native currentTimeMillis java.lang.System.currentTimeMillis :: () -> IO Long

I want to do better than this. Just because it is not referentially-transparent does not mean it goes in IO. The IO type constructor is "all bets are off." But it's not true here. After all, we happen to know that currentTimeMillis does not read/write the file system. Where is the type denoting this?

--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Mark Hibberd

unread,
Jun 13, 2013, 7:55:59 PM6/13/13
to scala-fu...@googlegroups.com
To be clear, I despise popularity contests. Scala is in some trouble because of this degenerate aspiration. SIP-18 is a perfect example of an attempt to further popularity, while providing zero (or less than zero) benefit to users of all persuasions. Just recently[1], having forgot SIP-18 existed myself, I had to explain to a class of ~12 people why they are seeing the implications of it. This digression was costly, especially that we were making such excellent progress otherwise.

The popularity aspiration is for losers in my opinion.

[1] http://www.yowconference.com.au/lambdajam/FPscala.html


So this prompts the obvious question: if better is the goal and not popularity, then why limit yourself to building a language on a platform that is actively hostile to your goals.

I am as quick, or quicker then most, to criticise scala, but with a few exceptions, but my biggest issues in scala come from the platform, for example the fact that I can write the obviously correct and straight forward solution and be hostage to stack overflow exceptions is unacceptable (the counter is that scala at least provides enough tools to help me address this, but it is not "better" that I need to deal with them). 

Based on this I disagree with the concept of building a new JVM language (weakly disagree anyway, it would be fun). If you want better, get off. If you _need_ the jvm, and not in the bullying sense, then scala actually has merit, and I disagree that there are technical reasons that prevent progress. I can see obvious progress with many smaller wins:
 - Better/faster/more robust compiler
 - Better ADT support, that lets me get away from boilerplate to avoid subtypes
 - A better framework for dealing with java (or even scala as java code), being forced to use an FFI to do these things is desirable, but just having one that I can use if I choose to insulate myself from that insanity is useful
 - Better standard library by simply making it really small (or optionally). Let libraries solve the problems.
 - Start working towards extensions to actually turn off the real problems that are just bugs (as opposed to SIP-18), so things like unit implicit conversion etc...

Anyway, you really want better, get off the JVM, you want better then Scala on JVM I think there is plenty of progress to be had - and I think that is enough for the limited time you _have_ to use the JVM.

Mark.

Mark Hibberd

unread,
Jun 13, 2013, 8:06:18 PM6/13/13
to scala-fu...@googlegroups.com
On Fri, Jun 14, 2013 at 3:36 AM, Brian McKenna <br...@brianmckenna.org> wrote:
I love this talk about money. I think people are greatly
over-estimating the resources needed to make a good JVM language. We
don't need venture capital.

I'll give an estimate for a usable prototype:

* 1 week to make a Hindley-Milner, strict, functional JVM language
* 3 days to add laziness via InvokeDynamic (requires Java 8)
* 2 days to add type-classes
* 2 days to add a sane FFI

Is there something I'm missing? I'd much rather use the prototype than
Scala for a lot of things, almost makes it worth the 2 week
investment.


I agree the effort required is a distraction, but I need to ask in what case is your prototype better than dropping down to a non-jvm language that already exists and has these (and isn't going to have performance, and correctness issues to deal with because of the JVM)? 

I think I know the answer, but I remain highly skeptical that the imposed JVM restriction is really meaningful.

Cheers
Mark

Mark Hibberd

unread,
Jun 13, 2013, 8:09:04 PM6/13/13
to scala-fu...@googlegroups.com
And apologies for the grammar. Urghh.... rant -> send is not that great of an idea.

John Nilsson

unread,
Jun 13, 2013, 8:19:46 PM6/13/13
to scala-fu...@googlegroups.com
"Scala ignores a considerable amount of language design research, especially that which has occurred in the last 20 years"
I would be very interested in pointers to particular papers. A bit of a language designer aspirant myself.

In defense of Scala though,
a) Oderskey et. al has produced it's own fair share of research
b) I'm sure there has to be some trade off between studying research and focus energy on actually implementing stuff. I know I spend far to much time studying and thinking and too little coding and experimenting. At the other extreme you see people like Bret Victor who I have a feeling gets very interesting things done precisely by not spending time reading papers or otherwise interact with the world around us.

When it comes to designing something with high aspirations I do think a few weeks for a prototype and then experimental iterations from there is exactly how you end up with a pile of things and compromises not really thought out all the way, turning out to interact with each other in not precisely desirable ways. But then again, I also believe it's impossible to foresee exactly how everything will impact the design without iterating over a prototype, so the alternative is probably to not create anything at all. Thus I guess we're all doomed to recreate a language that is "not a well-designed language, but is at least better than
other not-so-well-designed" should we attempt to construct a "language that is well-designed", and that is how progress is made. So even with an urge to replace them, a humble appreciation and respect of previous attempts is probably in order.

As an example of an effort to reach "well-designed" nirvana I've found very educational to follow have a look at David Barbours progress towards implementing Awelon[1], (hint, it's has been more than 2-3 weeks in the making)

[1] http://awelonblue.wordpress.com/

BR,
John

Marimuthu Madasamy

unread,
Jun 13, 2013, 8:23:11 PM6/13/13
to scala-fu...@googlegroups.com
After all, we happen to know that currentTimeMillis does not read/write the file system. Where is the type denoting this?

Isn't this typed the same way in Haskell also? That doesn't make it correct but I would like to know how you would represent this otherwise? Apart from this, Isn't Frege or Yeti a viable option for "The JVM situation" if we take option 'a'?

John Nilsson

unread,
Jun 13, 2013, 8:36:41 PM6/13/13
to scala-fu...@googlegroups.com
One approach would be to not allow arbitrary importing of functions from any scope and instead implement something like the object-capability model
[1] http://en.wikipedia.org/wiki/Object-capability_model

BR,
John

Sébastien Bocq

unread,
Jun 13, 2013, 8:49:48 PM6/13/13
to scala-fu...@googlegroups.com
If people over this mailing list think they can design a better programming language, I don't see any reason why not to just go for it or at least discuss the idea. For me, it raises tons of interesting questions and as far as I know there has never been such community driven initiative before. How do we start and how do we structure the effort? Write a basic language spec, code and iterate? Should we target LLVM? In which language should the compiler be written in?

It doesn't need to be fancy. Take Google Go for example. Although it is very boring research wise (dixit Rob Pike at OOPSLA 2012), there are some good lessons to be learned. It is not the most sophisticated language out there yet it provides some feature that are very appealing to C programmers. It compiles fast and its syntax is simple enough to ease tool support and language evolution (e.g. 'go fix'). So given we don't want to completely ignore the last 20 years of research in functional programming languages and we are not being rewarded for fanciness, what is the minimal set of features that matter to the average functional programmer to get the job done today?


2013/6/14 Tony Morris <tonym...@gmail.com>



--
Sébastien

Lars Hupel

unread,
Jun 14, 2013, 1:26:46 AM6/14/13
to scala-fu...@googlegroups.com
> There is so much more to it than implementing a compiler.
>
> [�]
>
> In that sense, I think creating a new language will not help, but might
> even be harmful. Creating the message of "even 'hardcore' Scala people are
> not happy with the language and started to build their own one instead" is
> certainly increasing the chances that you won't end up with Scala (with all
> its faults) in 2020, but with Java 8.5 or Kotlin 0.7.

I wholeheartedly agree with you.

> *Regarding the language:*
> I'm extremely happy with the progress we made in 2.11 to get rid of stupid,
> plain broken or dangerous stuff in the language or the libraries. Are we
> there yet? Not by any measurement!
> This doesn't mean we have to give up, we just need people continuously
> pushing things into the right direction.
> I think that things like the effect system, union types, the work on a
> sound foundation of the typesystem and macros show that there is
> willingness to improve and change the language, even if it painful.
> This is ALL about politics. One has to learn to act when the timing is
> right and stop to annoy people otherwise. Considering that I'm NOT payed by
> anyone and do this in my free time, I think I'm very happy with the results.

I for one are very grateful for the work you do. It's much appreciated.

> Imho, what we need is *no revolution, but a continuous evolution*.

Yes.

Alexandru Nedelcu

unread,
Jun 14, 2013, 3:14:49 AM6/14/13
to scala-fu...@googlegroups.com
On Fri, Jun 14, 2013 at 2:35 AM, Tony Morris <tonym...@gmail.com> wrote:
As I said earlier, it has been argued against me that what is "keeping people on the JVM" is nothing more than an illusion. Those people might claim "libraries" or similar, but I am told this is often hyperbole and/or misdirection. The claimant is under the illusion that the JVM is providing benefit where no such thing exists.

Hi Tony,

(I'm attaching your personal email address, as I'm still in approval for scala-functional and so this message will likely not end-up on scala-functional if it doesn't get approved)

I started using Scala in production only because it's a JVM language.
Let me give you a list of things I need for my current project:

- a capable web server that doesn't choke and die on 10,000 requests per second per server instance
- a virtual machine that's able to handle most HTTP requests in under 15 millisecs, with 100 ms being the upper limit
- real multi-threading support, with a good memory model and the ability to pick the right abstraction for doing concurrency and parallelism
- a good garbage collector that takes care of short-lived junk efficiently (specific to FP) and that only stops the world for compacting - seriously, people saying that the JVM is overestimated, never had to deal with GC issues
- a good profiler that helps me track down dead-locks, memory leaks and that helps me fine tune the memory allocation patterns and the various GC parameters
- the profiler should also be attachable to a remote host running in production
- a good IDE that helps me navigate and read other people's source code
- Async I/O that abstracts over the seriously broken and incompatible APIs of all major operating systems, including select, poll, epoll, kqueue, AIO and WinSock
- Memcached, DynamoDB, Cassandra, MySQL and PostgreSQL clients, preferably based on async I/O
- an HTTP client based on async I/O
- communication with remote actors by means of RabbitMQ or similar
- Futures/Promises for dealing with asynchronicity, preferably that doesn't leak on flatMap ;-)
- a JSON parser that doesn't represent a bottleneck on 60 MB/sec of incoming HTTP requests ... I know that writing JSON parsers is for some people the perfect weekend project, but all of them suck for my use-case, which is why I gave up and just went with the lexer in Jackson
- already built libraries for dealing with natural language processing / machine learning / scientific computing
- the ability to write Hadoop Map-Reduce jobs, or an alternative for Hadoop
- the ability to do graphics (Processing.org) or to build quick GUIs for small utilities

The stuff I mentioned above is just the bare minimum of what I need in my day to day work. It would be nice if Scala were more like Haskell and less like Java. But speaking about the language and the libraries, instead of fancy type systems I would be rather happy if I had:

- the ability to compile Scala to Javascript or to rely on it to continue working for Android

I'm not joking. Having beauty or expressiveness in a programming language is not an end-goal in itself. The end-goal is to build stuff that you can deliver to users. And solving problems that really do make a difference to users, like the brokenness of Javascript that leads to bloat and unpredictable failures, is way more important than having rank-2 polymorphism or lazyness or what-not. The ideal is to have both of course, but in absence of the ideal, I would like to be a happier developer today, not in 10 years from now.

Kotlin sucks. But instead of stressing about the soundness and expressivity of the type system, its devs made it so that Kotlin compiles to Javascript. Clojure core devs did the same with ClojureScript. This means I'm actually considering mixed Scala and Kotlin or ClojureScript projects. Because I don't know how much the quality of my life will degrade if I don't have the ability to work efficiently with Functors and Type-Classes, but avoiding Javascript would sure as hell make me happy.

--
Alexandru Nedelcu
https://bionicspirit.com

Alexandru Nedelcu

unread,
Jun 14, 2013, 4:38:52 AM6/14/13
to scala-fu...@googlegroups.com
On Thu, Jun 13, 2013 at 8:36 PM, Brian McKenna <br...@brianmckenna.org> wrote:
I'll give an estimate for a usable prototype:

* 1 week to make a Hindley-Milner, strict, functional JVM language

Dude, I'm in awe of your abilities. It takes me more than 1 week to read a light book or to build a shitty blog, let alone design and build the frontend and the backend of a Turing-complete programming language with Hindley-Milner type-inferencing.
 
* 3 days to add laziness via InvokeDynamic (requires Java 8)

I'm curious as to how does InvokeDynamic help for laziness? Maybe I'm missing something, but are you seriously thinking of regenerating bytecode for memoization or something like that? Because that would be an awful idea and I hope you're only thinking of the benefits of garbage-collectable method-handles, but that's less of a benefit unless you really need to generate bytecode dynamically or to do custom method dispatching that doesn't conform to the default single-dispatch resolution that the JVM does. I fail to see how laziness is helped by InvokeDynamic here. Also, InvokeDynamic is part of JDK 7, why do you need Java 8 for it?

A lazy value is most efficiently modelled with a C/C++ Union. Say for a lazy integer, you test if the first bit is either 0 or 1 and depending on that you can say if the integer is a memoized value or a method reference. This is why Ints in Haskell only have a guaranteed 30-bit signed precision, or why Ocaml's integers have 31 bits of precision. If this technique is used everywhere, it also solves the issue of conservative garbage collectors, like Boehm's, that couldn't distinguish otherwise between a stack-allocated integer and a reference - an issue that Go used to have last time I checked, leading to really ugly memory leaks on 32bits systems.

So back to the point at hand, both the CLR and the JVM are unsuitable for modelling lazy values, because both lack the low-level Union type and you can't say to neither of them "consider this Int a method reference and jump to it". To represent a lazy value, you need a Boolean that says if a value was memoized or not, plus a function reference and an Integer in which to store the value once it's computed. And on the JVM at least, there are no instances in which you can avoid boxing the value. So this means higher-memory consumption and boxing, which is overkill if you have laziness everywhere, like Haskell does.

Microsoft's research labs tried implementing Haskell on top of CLR. Later they gave up and implemented F# instead. This happened primarily because of laziness (purity is workable with enough effort and tricks, laziness is not).

This isn't to say that avoiding these virtual machines is a good idea, because then you'll also have to build a runtime. Then you'll end up with other issues. Lazy values also have to have correct behaviour in multi-threaded systems. Even if the act of computing the values is a pure process, preserving the substitution model, you can have performance issues when too many threads try reading it and then compute it at the same time. Which means you need some form of synchronization when "necessary". But then you get into the overhead of boxing and synchronizing on those lazy values, which means you'll need some form of escape analysis to track down values that can get stack-allocated or that don't need synchronisation, either in the compiler or at runtime.

Also, if you start with a strict language, you cannot make it lazy after the fact. A strict language can only evolve towards one where laziness is optional, such as Scala.

So yeah, good luck with implementing laziness in 3 days after 1 week of Hindley-Milner :-)

Chris Marshall

unread,
Jun 14, 2013, 4:51:08 AM6/14/13
to scala-fu...@googlegroups.com
Marimuthu above mentions a language called Frege. Looking at the commit history, it appears that there were active commits for around 6 months. Again, this does not tally well with the assertion that it is a couple of weeks' work. Possibly the assertion was made with puffnfresh's tongue in his cheek, in which case, ignore the fact that I'm banging on about it.

I realize you have little time for Gavin King and the Ceylon project but one of the reasons I felt this was doomed to failure was the economics of their time-to-market problem, which I would couch as this:

  People are going to need a *very* compelling reason to use your language, which means that, if you're not going to ship it for another 2 years, it had better have stuff in it that a) people have been crying out for, and b) does not already exist.

Ceylon (IMHO) fails this test miserably. People are not crying out for union types and a decidable type system and the other "benefits" (such as they are) like closures/functions already exist in JVM languages [*]. So I would say that, if this is the direction you are going down (i.e. writing your own language), you should think seriously about whether it passes these tests. Maybe it does.

As for the "bullying" - it is manifestly not bullying for the person who employs you to lay down certain rules about what you can and cannot do. After all, they might be left holding the baby when you decide to up sticks and leave for a shiny new job, or win the lottery, or get run over by a bus.

Chris

[*] - actually their module system looks like their killer feature, but I wonder if this is enough
 

Ingo Wechsung

unread,
Jun 14, 2013, 5:31:49 AM6/14/13
to scala-fu...@googlegroups.com


Am Freitag, 14. Juni 2013 01:47:07 UTC+2 schrieb Tony Morris:
On 14/06/13 09:17, Marimuthu Madasamy wrote:

Here is an impure native function declaration:

native currentTimeMillis java.lang.System.currentTimeMillis :: () -> IO Long

I want to do better than this. Just because it is not referentially-transparent does not mean it goes in IO. The IO type constructor is "all bets are off." But it's not true here. After all, we happen to know that currentTimeMillis does not read/write the file system. Where is the type denoting this?

Actually, reading the current time is input/output.
It doesn't matter if the hardware (or the VM) implements this through an I/O port, or a global register.

That being said, Frege (like Haskell) has the ST monad for doing local stateful computations. Here, in contrast to IO, where the state is global, the type system can make sure that the state cannot esacpe its local context. 

Francis Njenga

unread,
Jun 14, 2013, 5:31:12 AM6/14/13
to scala-fu...@googlegroups.com
Reinventing the wheel is always a fools errand, but the first wheel was square not round,
so thank God for fools.

A compiler imbued with as much research as has been made since the 1950's
would be a milestone for programmers, business and computing.

2016 is not that far in my opinion. A great design and community driven implementation would eventually
catch on. I say this knowing that for the several languages that have caught my fancy(scala included),
there's always that moment when someone goes like... "there's this one catch... " which goes on to
"Oh, and another thing..."
Kore Francis Njenga
Running and Walking are only breaths apart.

Alexandru Nedelcu

unread,
Jun 14, 2013, 6:27:53 AM6/14/13
to scala-fu...@googlegroups.com
On Fri, Jun 14, 2013 at 11:38 AM, Alexandru Nedelcu <m...@alexn.org> wrote:
I'm curious as to how does InvokeDynamic help for laziness? Maybe I'm missing something, but are you seriously thinking of regenerating bytecode for memoization or something like that? Because that would be an awful idea and I hope you're only thinking of the benefits of garbage-collectable method-handles, but that's less of a benefit unless you really need to generate bytecode dynamically or to do custom method dispatching that doesn't conform to the default single-dispatch resolution that the JVM does. I fail to see how laziness is helped by InvokeDynamic here. Also, InvokeDynamic is part of JDK 7, why do you need Java 8 for it?

I take that back. Wasn't aware of ConstantCallSite, which is in the next iteration of JSR 292 (thanks Brian McKenna):
https://www.java.net//jsr-292-goodness-lazy-singleton-pattern-12

Damn that's cool.

Ingo Wechsung

unread,
Jun 14, 2013, 8:25:24 AM6/14/13
to scala-fu...@googlegroups.com


Am Donnerstag, 13. Juni 2013 19:45:48 UTC+2 schrieb Yo Eight:
I think you've forgot higher-rank polymorphism. While not being too difficult to implement, add it to  Hindley-Milner takes more than a week IMO.

Took me at least 3 weeks to read and understand SPJs "Practical type inference ..." paper.

Lars Hupel

unread,
Jun 14, 2013, 8:45:13 AM6/14/13
to scala-fu...@googlegroups.com
> * 1 week to make a Hindley-Milner, strict, functional JVM language
> * 3 days to add laziness via InvokeDynamic (requires Java 8)
> * 2 days to add type-classes
> * 2 days to add a sane FFI

To give some context for these numbers: A couple of years ago I was part
of a team of 5 students to write a compiler for some subset of Caml with
JVM backend. Took us two weeks, albeit with a very limited feature set
(e.g. pattern matching, data types, working HM type system and
inference; but no separate compilation, no laziness, no modules, no
operator "overloading").

In summary, I think Brian's estimate could work, but I doubt that a
practical language would be the result � as often, the devil is in the
details.

simon.ochsenreither

unread,
Jun 14, 2013, 9:22:13 AM6/14/13
to scala-fu...@googlegroups.com
Additionally, the hard part of building a language is not the implementation.

Oscar Vargas Torres

unread,
Jun 14, 2013, 10:27:05 AM6/14/13
to scala-fu...@googlegroups.com
I've been reading comments that say something like "Do not start another language". If you (and others) are going to get something better (technically speaking) than Scala, I would be really happy to use your language.

Sincerely,
oscarvarto

Brian McKenna

unread,
Jun 14, 2013, 10:39:20 AM6/14/13
to scala-fu...@googlegroups.com
Just wanted to point out the goal for creating a new language is not
to get thousands of users or make a direct Scala competitor in 2
weeks.

The goal for creating a new language is to get a good foundation.
Scala is not a good foundation: just try to work on the compiler, it's
insanely hard to do anything with it. A smaller language would be a
much better place to work.

I want to be able to say "Scala needs X" or "Scala's Y is awful, let's
remove it" but those are not easy. Macros don't help much.

So advocating Scala because of "incremental improvement" is not very
consistent. The point is that we can't easily improve Scala (I'd fork
it today if I could). We need something else to incrementally improve.

On 14 June 2013 07:22, simon.ochsenreither
<simon.och...@gmail.com> wrote:
> Additionally, the hard part of building a language is not the
> implementation.
>

simon.ochsenreither

unread,
Jun 14, 2013, 1:44:34 PM6/14/13
to scala-fu...@googlegroups.com
Hi!


Just wanted to point out the goal for creating a new language is not
to get thousands of users or make a direct Scala competitor in 2
weeks.

I don't want to dispute your motivation, maybe I just don't get it.
(My general rule is that I only work on stuff which has a positive impact on a reasonable amount of people.)

I don't get the complaints about Scala's Java interop. On the one side it is argued "interop with Java sucks" and on the other side it is argued "there aren't any useful Java libraries which one might want to use". Then why waste time with it?!

Regarding the complaints about the JVM: If you think scalac is too complicated to change, then Avian is everything scalac is not: The implementation is simple, easy to understand and concise. You can sit down and implement all the stuff you want NOW, without reinventing the wheel. Why aren't you all doing that?


The goal for creating a new language is to get a good foundation.
Scala is not a good foundation: just try to work on the compiler, it's
insanely hard to do anything with it. A smaller language would be a
much better place to work.

Of course Scala is really complicated in that department, but seeing all the great stuff like the effect plugin, continuations, the JavaScript plugin, union types, InSynth, the new pattern matcher, ... it seems like a lot of people are able to deal with it.


I want to be able to say "Scala needs X" or "Scala's Y is awful, let's
remove it" but those are not easy. Macros don't help much.
So advocating Scala because of "incremental improvement" is not very
consistent. The point is that we can't easily improve Scala

I'm certainly not one of those smart geniuses like th rest of this mailing list, but I think I have had reasonable success doing exactly that.

 
(I'd fork it today if I could). We need something else to incrementally improve.

That seems to have worked out well the last time people tried it ... not.
In fact, "we will do X, but from scratch and better in everything!" seems to be a symptom of failed software projects.

Note: I don't dispute that you could build a prototype which would be better than Kotlin or Ceylon from day one, but I think there are tons of options which would be more productive and beneficial to more people:
If you think you can't work on Scala, improve the runtime instead. If you don't care about the runtime, why not just use/adapt one of the existing languages of the ML family?

Bye,

Simon

Brian McKenna

unread,
Jun 14, 2013, 2:51:40 PM6/14/13
to scala-fu...@googlegroups.com
The value of existing Java libraries is overstated but that doesn't
mean non-existant. Good interoperability would be useful.

Avian is a Java VM, not a Scala compiler.

I have no idea how Paul Phillips is capable of doing so much with
scalac but I'm sure he'd agree that it's incredibly hard to work with.
Many people write large Java systems, that doesn't mean it's easy.

And it sucks that Scala didn't just incrementally improve Java. That
would have been the secret to success. Typesafe should stop
reinventing wheels. Scala is following the symptom of a failed
software project.

On 14 June 2013 11:44, simon.ochsenreither

Meredith Gregory

unread,
Jun 14, 2013, 4:44:51 PM6/14/13
to scala-fu...@googlegroups.com
Dear All,

My attention was mysteriously drawn to this thread!

Implicit in Brian's estimates is that the language is functional.

i'm curious: how many believe that functional is the correct foundational model for a language designed to meet today's computing requirements?

Best wishes,

--greg

P.S. In the interest in full disclosure, i don't have that belief and it stems from the core mathematics. Berry's theorem (Theorem 14.4.8 in (Barendregt, 1984), the broader discussion begins at pg 375 in my copy) says that the lambda calculus is fundamentally sequential. Of course, as with all computationally complete models we can build parallelism, concurrency and distribution on top, but then we will always be battling expressivity considerations. And, the model will not fit well with a VM underneath that natively supports these features.

i submit that are computational models enjoying the following properties that are better suited to today's computing requirements
  • a simple, core compositional mathematical presentation enjoying both operational and denotational semantics, as well as a proper Curry-Howard orientation 
  • fundamentally concurrent
  • support for reasoning about time and space complexity within the model
  • enjoying rich types that include standard structural considerations (as found in functional settings) as well as behavioral considerations, including but not limited to
    • expressing concurrency constraints, such as guaranteeing deadlock freedom
    • expressing resource constraints, such restricting 
    • expressing security and information flow constraints
  • fits well with existing language design wisdom -- especially monadic structuring of code
  • enjoys a modular account of reflection
i submit that its these properties that most of the successful functional programmers employ and exploit and enjoy, rather than elements specific to lambda, itself.

Further, note not only does lambda not enjoy a fundamental account of concurrency phenomena, it also does not enjoy a fundamental account of resources. This is why most of the original complexity work was done in alternate models (such as Turing machines). It's also at the heart of the old quip: a Lisp programmer knows the value of everything and the cost of nothing. It was only after linear logic and the discover of linear lambda calculi that we had a way to reason about space resources in lambda. Modern functional languages are far from enjoying a linear lambda foundation. 

Apart from direct connection with a long tradition of functional language design, wisdom and experience (which is quite a lot), the main thing i see that would be lost in striking out in a direction influenced by the models i'm talking about is any immediate importation of the work being done on dependent typing. This is a hard choice, but i believe we need a major reset in terms of our thinking about types, right now. The structuring of types and effects could be given a much more expressive setting in which to explore the design space. And, because of the fact that the models i have in mind do enjoy Curry-Howard oriented denotational semantics, there is a path to integrate work on dependent typing.

Note also that the actor model (which is quite a different thing from what people are calling "actors" in the Scala and wider industrial settings) does not enjoy either a well established and well understood presentation of an operational or denotational semantics. Further, it suffers from the principal port limitation which is a major limitation in many modern concurrent and distributed problems. Note this critique comes from someone who was so fond of the actor model that he spent many years of his life building a VM, compiler, interpreter for a language organized around it and then built several major industrial applications on top of said language to verify its efficacy and understand its limitations.

i'm always interested when a group is potentially seriously interested in building a new language. i wonder how much of the impulse comes from minor irritation with the languages available, and how much comes from an interest in fitting the very heart of the languages design principles to the actual challenges we face in today's world. Of course, continuous evolution is good. But, it takes a long time to get from a creature that was organized around doing one thing at a time to one that -- in its very DNA -- is organized around coherent and coordinated concurrent action. Sometimes even great nature benefits from mutation! ;-)
--
L.G. Meredith
Managing Partner
Biosimilarity LLC
7329 39th Ave SW

Leif Warner

unread,
Jun 14, 2013, 9:24:58 PM6/14/13
to scala-fu...@googlegroups.com
My 2 cents - what I like about being on the JVM is interop with other langauges, namely calling into a wealth of existing libraries.  If I didn't feel that was necessary I'd just be writing Haskell or Idris or something.  Idris can target the JVM - why not just use that?  Its Java FFI can be improved through type providers or something.  You can perform arbitrary computation at compile time, so I don't see any barriers to that.  Also, I don't share in viewing laziness as a "must-have".  Optional laziness, yes, but not laziness by default.  I don't think it makes that much of a difference in writing programs, and it adds non-trivial complications when targeting the JVM.
-Leif

simon.ochsenreither

unread,
Jun 15, 2013, 6:57:04 AM6/15/13
to scala-fu...@googlegroups.com
Avian is a Java VM, not a Scala compiler.

... and it would allow you to address all the issues you have with the JVM in the first place, without having to waste time to reinvent the GC wheel, the JIT wheel, the codegen wheel, the platform abstraction wheel, the memory model wheel, the runtime services wheel,  ...

 
I have no idea how Paul Phillips is capable of doing so much with
scalac but I'm sure he'd agree that it's incredibly hard to work with.
Many people write large Java systems, that doesn't mean it's easy.

Sure, writing software is hard. I just don't think the alternatives are more appealing.


And it sucks that Scala didn't just incrementally improve Java. That
would have been the secret to success. Typesafe should stop
reinventing wheels. Scala is following the symptom of a failed
software project.

Wow, that's getting a bit melodramatic, isn't it?
Success has always been a matter of having luck with timing and good marketing, NEVER about language quality. Otherwise, please explain PHP.

Anyway, I'm not seeing anymore how this thread will produce anything worthwhile.
I have given enough pointers to projects which would have a measurable improvement on the status quo.
Those interested in actually improving stuff instead of just whining about it, can shoot me an email. I'm happy to get them started.
Apart from that, I'll focus again on fixing and shipping code, because I think this bike-shedding will lead to nothing.

huynhjl

unread,
Jun 15, 2013, 9:54:49 AM6/15/13
to scala-fu...@googlegroups.com

In other words, which response to 
the JVM situation would you consider is the best use of effort? 
 
a) Construct a JVM language that is well-designed with good legacy
library interoperability.
 
If you have a very very clear idea of what that language is and you know how to overcome the tendency of the JVM to throw stack overflows, then that is a great option. On the other hand, if the language itself has to be designed and discussed, then it is a lot of effort. Or if it will lead to a language that is impractical to use because I have to do some contorsions for my programs to avoid stack overflows then it is not a best use of effort to create such a language. 

Examples of having a very clear idea of the language is, you know what it is, you just have to communicate it.
Examples of JVM limits with respect to stack overflows and related contorsions to avoid them: the trampoline monad. 

d) Concede that you are simply doing the best you can with available
tools. Use Scala -- at least you're doing better than the guy down the
road using Java or "Scala as if it were Java."

Nothing wrong with this concession. And it does not have to be forever...

Jean-Laurent

etorreborre

unread,
Jun 16, 2013, 6:31:58 AM6/16/13
to scala-fu...@googlegroups.com
Hi Greg,

Your PS could have easily been put in the main message!

One thing especially grabbed my attention: the actor model. I had a look at Akka recently and I felt that this model of programming was making thing really difficult in terms of reasoning and probably hard in terms of maintenance because the "business" logic seemed very tangled to the "infrastructure" logic.

What would be, in your mind, a better approach? Some kind of asynchronous CSP (if there's such a thing)?

Thanks,

Eric.

Alexandru Nedelcu

unread,
Jun 16, 2013, 2:40:11 PM6/16/13
to scala-fu...@googlegroups.com
On Sat, Jun 15, 2013 at 1:57 PM, simon.ochsenreither <simon.och...@gmail.com> wrote:
And it sucks that Scala didn't just incrementally improve Java. That
would have been the secret to success. Typesafe should stop
reinventing wheels. Scala is following the symptom of a failed
software project.

Wow, that's getting a bit melodramatic, isn't it?
Success has always been a matter of having luck with timing and good marketing, NEVER about language quality. Otherwise, please explain PHP.

Simon, the quoted opinion is indeed melodramatic and wrong, however language designers would do well to learn from PHP, because it was neither marketing or luck that contributed to its success.

PHP simply rode the success of the Apache web server, that was in need of a scripting language. mod_php was designed to be that scripting language. It worked great in a shared hosting environment because of the one process per request model. The barrier to entry was extremely low - you could simply open a static HTML file and add dynamic behaviour to it by simply adding stuff withing <?php blocks. And remember that VPS instances where quite expensive a few years backs.

So really, the success of PHP came from it being available by default on cheap, shared hosting and from having a really low barrier to entry.

The proposition of building a better Java is hopelessly broken, because there's nowhere you can go. A better Java already exists and that language is C#. For many of us C# simply does not exist because it's tied to Windows, but back in early 2000 Microsoft was dominating the desktop, the only smartphones worth considering were running Windows Mobile and Microsoft was increasingly winning the server-side too. Microsoft had all the marketing muscle it needed. C# looked poised to be the next Java. We all know how that turned out.

This is why I believe languages that are only a marginal improvement over Java are poised to fail, because Java is like a freight train that survives and grows on inertia alone and the benefits of picking up a language that's only marginally better is outweighed by the advantages of sticking to a language that's dozens of times more popular. This is also why Scala is less popular than it should, because its biggest problem is one of image - it's pretty hard to show to people why another language is better (the Blub paradox). Scala also has the disadvantage that it attracted many idealists that criticise the language, but really, it doesn't look good when a language's (vocal) community is not satisfied with it.

When it comes to both Java and PHP, there is one lesson that language designers must learn and not forget - the language is important, but it's less important than everything else combined. And we can all learn to disagree on this point, however those who do not learn from history are doomed to repeat it.

simon.ochsenreither

unread,
Jun 16, 2013, 3:33:24 PM6/16/13
to scala-fu...@googlegroups.com
The proposition of building a better Java is hopelessly broken, because there's nowhere you can go. A better Java already exists and that language is C#. For many of us C# simply does not exist because it's tied to Windows, but back in early 2000 Microsoft was dominating the desktop, the only smartphones worth considering were running Windows Mobile and Microsoft was increasingly winning the server-side too. Microsoft had all the marketing muscle it needed. C# looked poised to be the next Java. We all know how that turned out.

This is why I believe languages that are only a marginal improvement over Java are poised to fail, because Java is like a freight train that survives and grows on inertia alone and the benefits of picking up a language that's only marginally better is outweighed by the advantages of sticking to a language that's dozens of times more popular. This is also why Scala is less popular than it should, because its biggest problem is one of image - it's pretty hard to show to people why another language is better (the Blub paradox).

My strategy has been to make Scala less dependent on Java/Oracle and create a future where we might have a better runtime regardless of what Oracle decides to implement.
I have already bootstrapped the whole Scala distribution on top of it, verified that proper tail calls work. The only thing remaining before I look into adding features to the runtime are a few failing tests which I need to fix.
The work on having a native implementation of BigInt goes into the same direction, btw.

It just annoys me when people claim that a) Scala can't be improved and b) we are stuck with the JVM, when I'm happily shipping code which shows that both claims are wrong.


Scala also has the disadvantage that it attracted many idealists that criticise the language, but really, it doesn't look good when a language's (vocal) community is not satisfied with it.

Imho, that's a good thing. My experience is that language communities where no one criticizes the lanugage are either quite authoritarian and toxic (Go) or lack any basic knowledge about how things could be potentially improved (NodeJS, PHP).


When it comes to both Java and PHP, there is one lesson that language designers must learn and not forget - the language is important, but it's less important than everything else combined. And we can all learn to disagree on this point, however those who do not learn from history are doomed to repeat it.

I think we agree. The point I was making was that PHP's success was not related to its language design, but by the ecosystem around it and external factors.

Meredith Gregory

unread,
Jun 16, 2013, 6:12:50 PM6/16/13
to scala-fu...@googlegroups.com
Dear Eric, Jim, and All,

Thanks for both or your replies! i was truly mostly interested in how many people feel that functional is the right computational model on which to base a new language. That was my main message. The rest was full disclosure. There were a couple points i should correct. One was the line "its these properties that most of the successful functional programmers employ and exploit and enjoy, rather than elements specific to lambda, itself." Obviously, the functional model is not fundamentally concurrent. My position was really, "it's a subset of these properties... ." Further, a model with all of these properties would make them even more successful! ;-)

As for your question about actors and candidate models -- i agree that the way actors are materialized in AKKA makes it hard for me to reason. But, that may be just me, or that i haven't read any proper documentation about them. i could give specific examples, if anyone is interested. As for candidate models, here's what i've learned in the past 20 years since my deep dive into mobile process calculi.
  • With the exception of the Join Calculus, calculi derived from Milner's π-calculus (of which there are several asynchronous flavors) have a tough time with fork-join patterns. Join suffers because it bundles too much in the way of control constructs into a single pattern.
  • Calculi derived from the Ambient Calculus are too darned expressive. Being able to do leader-election algorithms in one-liners means you can distribute this over the Internet without consent from the fire-wall keepers. They gave their consent to HTTP because it was stupid and inexpressive (and then it was too late to take it back).
  • Looking at the success of the monadic approach to providing denotational semantics for all of the features of mobile process calculi, in a nicely modular fashion, it occurred to me to turn things around and begin with the monad and then see about building up language features on an as-needed basis.
In order to give examples of what i mean, let me set up some background. We think of a port, conceptually, as our access to a (typed) stream of data. Hence, the monadic API is perfectly suited to input from a port. Thus, x?( y )P (read wait on the port x for input, bind that to y in the process expression P, and then run P) becomes for( y <- x ){ P }
  • Note that just within the monadic API we can do fork-join: for( v <- u; y <- x ){ P }. This is way simpler than the corresponding encoding in π-calculus, and yet doesn't bundle every control construct like join calculus. Note, also, this clearly defeats the principal port limitation of actors.
  • With monadic composition (either distribution laws or monad transformers) we can encode choice. So, u?( v )P + x?( y )Q can be encoded 
for( w <- choose( u, x ) ){ w match case Left( v ) => P; case Right( y ) => Q }

Not quite as succinct, but certainly still on par and clearly readable.
  • This can all be integrated with fair-interleaving and back-tracking on streams, ala Kiselyov, et al.
Output can be done in the usual way: x!( y ) becomes x ! y. i can give lots more if there is interest. The main point is that by taking as the centerpiece of the language construct the core semantic device underpinning all the success denotational accounts of mobile process calculi, we get fine-grained control on what we want to expose as the surface area.

Finally, note that, this construct fits perfectly with Scala's existing type system. KVDB uses a delimited continuation-based interpretation to provide a typed channel implementation of a language akin to the Applied π-calculus that is currently used in Protegra's Protunity Business Matching Network.

Best wishes,

--greg

Vincent Marquez

unread,
Jun 16, 2013, 6:30:04 PM6/16/13
to scala-fu...@googlegroups.com
In an effort to steer the thread back on track, I'd like to explore choosing or at least discussing the options Tony (correctly IMO) laid out for the community.  


a) Construct a JVM language that is well-designed with good legacy
library interoperability.

I haven't seen too much discussion of this.  However, my personal wish list would be a subset of Scala's features (no OO, vars, etc) with some additional safety guarantees, so personally I would think it would make more sense to ...

 
b) Construct a Scala-compliant compiler with extensions to accommodate
language improvements and optimisations. This would also address the
speed issue with scalac as a consequence. It would also offer better
external tool support that integrates into the compiler.


Brian McKenna's suggestions looked really good: 



Now we can map over that data, do safe things with it and have better
guarantees that our programs do the right thing, since the types are
no longer lies and errors are explicit.

This gives us a lot more safety than Scala does.   Java can also block in addition to side effect (is the former a type of the latter? probably) so we may want a Future thrown in there too, but more or less this would rock if it could be implemented.  Could a theoretical compiler (extension?) enforce this by annotating classes(modules?) known to be safe and pessimistically enforce wrapping of calls on types that don't?   

You'd probably also want a way of saying String#length() is pure.

Agreed, but again, probably better to be pessimistic so at worst this may just be some elbow grease.  If you take Brian's ideas, get rid of var, make every class sealed by default, don't allow IO arbitrarily, the end result is getting closer to a language *I* would be quite happy to use.  What does the community think?  Is this a minor, marginal improvement over scala that isn't worth the effort?  

c) Concede the aforementioned argument. You are not bound to the JVM as
much as you formerly claimed. Use a language like Haskell or O'Caml and
reap the benefits.

I think enough people have convinced us that when this is a viable option, it's worth exploring, but it isn't always viable.  I still haven't been informed of a high performance NIO haskell lib or well tested SIP library.  Even if a language could be designed in weeks or months time, I'm sticking to the argument that there are certain libraries that will take orders of magnitude longer.

--Vincent




 
d) Concede that you are simply doing the best you can with available
tools. Use Scala -- at least you're doing better than the guy down the
road using Java or "Scala as if it were Java."

There are possibly others options I have not thought of. Each of these
has some amount of required effort ranging from nothing to "lots." This
is important to providing an answer. In other words, which response to

the JVM situation would you consider is the best use of effort?

Thank you for any insights you might have.
--
Tony Morris
http://tmorris.net/

Meredith Gregory

unread,
Jun 16, 2013, 9:20:57 PM6/16/13
to scala-fu...@googlegroups.com
Dear Vincent,

Tony's dismissal of CAL is too glib. It would take considerably less time to fix CAL's legacy interop issues that to build an entirely new language. CAL is essentially Haskell for the JVM. In micro benchmarks it has been shown to beat GHC. A brand new language won't be at that level of maturity for a very long time. That's one of the reasons i took a position as principal architect for SAP -- at the BusinessObjects office where CAL was developed, right after SAP bought them -- co-located with the remainder of the CAL team. i was attempting to resuscitate that effort, but SAP couldn't see the gem they had in their possession. Regardless, if you really want to base your language on a functional model, you could definitely do worse than CAL.

Beyond that, sense is that choosing the computational model on which you base your language design is right on point. What model do you think works best?

Best wishes,

--greg

Vincent Marquez

unread,
Jun 17, 2013, 1:44:06 AM6/17/13
to scala-fu...@googlegroups.com
Beyond that, sense is that choosing the computational model on which you base your language design is right on point. What model do you think works best?


It's interesting you ask because it seems you've made a distinction between functional and monadic as models for basing a language.  I say this only because being very new to functional programming, (scala is the first 'functional' language I've used) they are one and the same in my mind, I can't ever imagine using the former without the latter.  

I'm not sure I have the math background to articulate what 'model' I think is best, but what I want most out of my next language (or if the community chooses to fork scala) is rigorous enforcement of referential transparency.  From my very humble view of things, Brian's suggestions for tweaking scala seem the fastest way to get there on the JVM with interop.  

You also mentioned concurrency.  They *only* sane way I've found to implement concurrency in my many years of writing multithreaded apps has been monadically: using futures and promises to manage asynchronisity, immutable data and the State monad to 'mutate' data , and non blocking atomic operations (cmpexg) to actually share data.  

I do not know anything of CAL other than what has been mentioned on this thread, but I will do some research in my free time, along with looking into some of the other suggestions you had. Thanks for the response, and eager to see this thread continue and what comes of it. 

--Vincent

Alexandru Nedelcu

unread,
Jun 17, 2013, 2:33:26 AM6/17/13
to scala-fu...@googlegroups.com
On Thu, Jun 13, 2013 at 11:27 PM, simon.ochsenreither <simon.och...@gmail.com> wrote:
Regarding the JVM:
Maybe some people are aware of it, maybe some are not: I have spent the last weeks and months on testing and fixing Scala to run on Avian.
If you want proper tail calls, runtime support for continuations, native compilation and embedding, this is the place where you want to be!
If you want better support for laziness, better performance for functional idioms, runtime support for value types, runtime specialization, this is the place where you want to contribute!
The right to fixing the JVM is not some completely new runtime, but a JVM adapted to our needs. This way, all the existing code keeps working while we can build better abstractions for us.

Hey Simon,

So I want to play with Scala on Avian to see how it works, but my Google-fu is failing me.
Is this a separate project? Is there a Scala-optimized Avian VM? Can you provide me with a link?

Thanks,

etorreborre

unread,
Jun 17, 2013, 2:49:49 AM6/17/13
to scala-fu...@googlegroups.com

Meredith Gregory

unread,
Jun 17, 2013, 7:55:48 PM6/17/13
to scala-fu...@googlegroups.com
Dear Vincent,

Thanks for that!

It's interesting you ask because it seems you've made a distinction between functional and monadic as models for basing a language.  I say this only because being very new to functional programming, (scala is the first 'functional' language I've used) they are one and the same in my mind, I can't ever imagine using the former without the latter.

That is exactly the sort of information i need to gather. Different people come to a modern functional language, like Scala or Haskell, by different paths, and it's challenging to know what sort of view they form of these languages and what they mean when they say 'functional'. Your comments are really, really useful data to someone like me who wants to put programmers in touch with ideas, architectures and code that could serve them.

To answer a question implicit in your comment: monads come from category theory. They can be formulated in a meaningful and substantive way completely outside of the functional setting. In fact, category theorists did so before there were languages like Scala or Haskell -- or even Lisp for that matter. And, functional programming languages, like Lisp or Clojure, can be formulated completely independently of monads. What makes a programming language functional is that it is built from the lambda calculus. The lambda calculus can be interpreted in category theory. But, category theory is much, much bigger, and knowing how to formulate monads outside of the lambda calculus setting can significantly enrich one's effective use of monads in programming. Monads are wonderful as a structuring technique. But when we talk about language design, what is it, exactly, that we are structuring?

Specifically, the use of futures, promises, asynchrony and state management can be brought together in coherent packages that make sense as an independent model of computation. By 'independent' i mean independent of Scala, of the JVM and independent of the lambda calculus. Here's a teeny-tiny toy example. i will define a VM made to run processes -- which i will denote by P, Q, R, ... . These processes can implement functions, but are not functions. Only a limited set of them 'return' anything. The difference is something like this: an encryption scheme inputs a message in a given format and produces a message in a new format. That is a function. The sliding window protocol, by way of contrast, has no return type. It is a process which provides a means to guarantee a property of a communication channel -- namely that messages arrive exactly once and in order. That's the difference between a process and a function. Functions can be usefully thought of as a subset of processes where the process behavior outputs a result. Not all processes have outputs. On to the definition of the VM.

First, we define process code. This is the code processed by the VM.

P, Q ::= 0 // the stopped process: it does nothing
          | x?( y )P // this process is blocked on the channel x waiting for input which it will bind to the variable y in the process code P and then run P
          | x!( Q ) // this process will asynchronously send the code of the process Q on the channel x
          | P | Q // this process represents the concurrent composition of P and Q, meaning P and Q run concurrently
          | *x // dereference -- all channels represent the code of some process! this gets the code for the process back and begins running it
x, y ::= @P // reference -- this says explicitly that all channels represent the code of some process and taking the "address of" a process P -- i.e. get its code -- is how you get a channel

Now, this code makes syntactic distinctions that don't make any real difference to execution. For example, if P|Q means P and Q run concurrently, then certainly this is no different than Q|P. Similarly, if 0 is the stopped process then running the stopped process concurrently with P is the same as running P. Further, just as in Scala def f( x ) = M is the same as def f( y ) = M{ y/x } where M{ y/x } means replace x with y in every occurrence in M, a rule we use when we effect certain refactorings in Scala, we have the same relationship in this model: x?( y )P is the same as x?( z )( P{ z/y } ). This last equivalence of variable re-naming is called alpha-equivalence. The more general collection of equivalences identified here gets bundled into one equivalence called structural equivalence. In symbols

=S= denotes the smallest equivalence containing alpha-equivalence that makes ( P, |, 0 ) into a commutative monoid, i.e.
  • P | Q =S= Q | P
  • P | ( Q | R ) =S= ( P | Q ) | R
  • P | 0 =S= P
Finally, we have the transition rules of the VM, taking it from a state, S, to a new state S'.

comm: x?( y )P | x!( Q ) -> P{ @Q / y }
par: P -> P' => P|Q -> P'|Q
struct: P =S= P', P' -> Q', Q' =S= Q => P -> Q

There -- that's the complete specification of the VM. You can implement the VM in less than an hour in your favorite programming language, if you know what you're doing. Clearly, we can realize this VM directly on top of the JVM. The computational model this VM realizes is Turing complete. Anything you write in Scala can be written is this VM.

You can dress this language up in syntax familiar to the Scala programmer via the following compiler, M( p : Process )( e : Map[Name,Channel] )( r : Map[Name,Process] ) : ScalaLikeSyntax

M( 0 )( e )( r ) = { }
M( x?( y )P )( e )( r ) = for( y <- e( x ) )M( P )( e )( r )
M( x!( Q ) )( e )( r ) = e( x ) ! M( Q )( e )( r )
M( P | Q )( e )( r ) = spawn{ M( P )( e )( r ) }; spawn{ M( Q )( e )( r ) }
M( *x )( e )( r ) = r( x )

Note that when an expression of the form @P is encountered you have to update e and r in recursive calls to M. 

Using the formulation of the previous message you can put nice sugar around the use of spawn and package it up in for-comprehensions. Thus, for example, u?( v )P | x?( y )Q becomes for( v <- u; y <- x ){ spawn { P }; spawn { Q } }. Or, if we agree that inside the curly braces of this kind of comprehension the semi-colon means run concurrently, then expression is represented by for( v <- u; y <- x ){ P; Q }. ( Note, i wrote M going in this direction to give a hint about how to implement it in Scala. You could just as easily have gone the other direction, taking this VM as the target for a more Scala-like language.)

To my eyes this is a very nice core language. It has a lot going for it by comparison to the core language for Haskell. It should also be compared to the calculi considered as the underpinning of Scala. This one has concurrency and reflection built in. None of the ones i've seen for Scala do.

We can also give a category theoretic interpretation of this VM without ever mentioning the lambda calculus. Thus, it is specifically not a functional language. Programmers don't write functions. They write processes. But, it can enjoy a rich type system. With these types we can do things like
  • ensure deadlock-freedom of code
  • express and ensure constraints on information flow in code -- all of the object-capability language features -- we can do these
It also enjoys all kinds of other benefits. For example, we can reason both formally and in an automated or semi-automated fashion about when one process is substitutable for another. We can use the technique of bisimulation to reason (once and for all) about whether the VM will behave the same with the process code P as with the process code Q. Even though this is only semi-decidable (namely, this technique may not always terminate because of the halting problem) we have a rich set of behaviors for which this is extremely valuable.  

Please bear in mind, i'm not specifically arguing for this particular VM. i just used it as a toy to get the point across. Having something like this is more or less like having a blueprint for your language. If you want to build a real language, having a blueprint like this is vital. That's how Haskell began. That's how Scala began. That's how OCaml began. Etc. Each blueprint begins with a computational model. i'm really asking the people on this thread: what's the blueprint would they consider as they basis for their new language. If it's the lambda calculus, why? If it's not the lambda calculus, what is it? 

It's perfectly fine to say something like: Scala is my blueprint or Haskell is my blueprint. That's how most evolution happens. However, if we want to support programmers programming in today's world, i believe that beginning from a model that has certain properties built in from the beginning is possibly a better foundation.

Best wishes,

--greg

etorreborre

unread,
Jun 17, 2013, 8:10:19 PM6/17/13
to scala-fu...@googlegroups.com
Hi Greg,

To add to the discussion and answer your question, the Concurrent Pattern Calculus is probably the computation model I would like to have as the core of my language:


PS: what about "Monadic Design Patterns for the Web"? I think we haven't had an update for a long time now.

Tony Morris

unread,
Jun 17, 2013, 8:26:42 PM6/17/13
to scala-fu...@googlegroups.com
On 15/06/13 03:44, simon.ochsenreither wrote:
Hi!

Just wanted to point out the goal for creating a new language is not
to get thousands of users or make a direct Scala competitor in 2
weeks.

I don't want to dispute your motivation, maybe I just don't get it.
(My general rule is that I only work on stuff which has a positive impact on a reasonable amount of people.)

I don't get the complaints about Scala's Java interop. On the one side it is argued "interop with Java sucks" and on the other side it is argued "there aren't any useful Java libraries which one might want to use". Then why waste time with it?!

Let me help you understand the position. You may disagree with it ultimately, but hopefully you can understand the motivation for it.

We probably agree on the following fact: Scala is currently the best language available today for the JVM and inter-operability with legacy libraries.
We probably disagree on the following fact: Although Scala is the best available, it is overall not very good and is likely to stay that way. It is not improving and every indication is that lack of improvement is a primary goal. I do not intend to de-motivate you or anyone with this position, but it is one I hold nonetheless. I have come to learn that others agree similarly or in whole.

Let me first explain what motivates inter-operability. Yes, there are very few useful libraries in Java, but only a few. Remember when Hibernate and Spring was all the fuss, then finally everyone figured out that they are not worth the air they occupy? Speed that analytical process up a bit and observe the current state of libraries; both Java and for Scala. You will likely reach a similar conclusion; there is a *lot*, but not much at all useful. That is all this position is -- an extension of a conclusion that eventually gets worked out. In the meantime, it is socially-unacceptable, blasphemous even, to suggest that Hibernate or Spring are rubbish, but they are right?

However, this scarcity of useful JVM libraries doesn't mean we can throw the shackles off altogether. Someone is one day (already has) going to ask me to write a WebSphere AS (servlet) application. In reality and more than likely, this may be an extremely bad idea -- the requesting party does not truly need nor want such a thing, but to what extent is it true that we should push back? This was the open question -- I do not know the answer to this with any significant degree of confidence.

Supposing then that there is some non-zero requirement of the need to inter-operate with WebSphere AS (legacy Java library), to what extent should we do this? How much of that library should we use? I have been in this position a number of times, including the implementation of WebSphere AS itself. The answer is: avoid it as much as you can and fortunately, you can often do this to a large extent and bypass all the nonsense. However and unfortunately, this does not remove the burden of JVM library inter-op altogether. Somewhere I am going to call out to the servlet API or some other silliness. This does not mean I am burdened with all the other garbage that WebSphere attempts to impose on the exploited analyst, but I am at least burdened with having to inter-operate on the JVM even if it is just a teensy little bit.

Everything else provided by WebSphere AS in terms of library support (to be clear, I am using WebSphere as the example), I can bypass simply by writing an appropriate library in a short amount of time and instead of otherwise wasting an enormous amount of time using the "provided libraries." We all know how that works out right? I have lost count of the number of times I have been in this position: "Oh look, some Java developer(s) have written a load of unusable garbage. It is quicker both short and long term to ask the intern (or even do it myself) down the hall to write a proper library and all I need to do is provide a bit of guidance from peers." I want to make this process easier; I want better Java inter-op, since even though in practice, it may be a very thin "shim" to the legacy library, this mechanism propagates throughout library code.

How often have you chosen to run with an existing Java library and wrapped all the null returns in Option? Not all of them, since performance, right? Or maybe not that specific method because it is clumsy to work with. Or maybe not there because you are going to unsafely unwrap (Option#get) anyway to inter-operate with that other silly method. Or maybe this or that or exception or side-effect or blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah.

Why are you even asking these laborious, mechanical questions? How much effort do you spend on answering them? This is not your job -- it is the job of tools and of Java inter-operability. Therefore, Scala does not do this well.




Regarding the complaints about the JVM: If you think scalac is too complicated to change, then Avian is everything scalac is not: The implementation is simple, easy to understand and concise. You can sit down and implement all the stuff you want NOW, without reinventing the wheel. Why aren't you all doing that?

Because I cannot do all the stuff I want. Not even close. Ultimately, it is a false economy, that's why.




The goal for creating a new language is to get a good foundation.
Scala is not a good foundation: just try to work on the compiler, it's
insanely hard to do anything with it. A smaller language would be a
much better place to work.

Of course Scala is really complicated in that department, but seeing all the great stuff like the effect plugin, continuations, the JavaScript plugin, union types, InSynth, the new pattern matcher, ... it seems like a lot of people are able to deal with it.

I contend that no, a lot of people are not able to deal with it (the consequences of Scala's shortfalls alluded to here-in). Indeed, very few are able to deal with it. I watch this happen before my very eyes with remarkable consistency. It might be my bias, but I see it in person, on mailing lists, on forum posts and I very rarely see any counter-examples. Convince me otherwise.




I want to be able to say "Scala needs X" or "Scala's Y is awful, let's
remove it" but those are not easy. Macros don't help much.
So advocating Scala because of "incremental improvement" is not very
consistent. The point is that we can't easily improve Scala

I'm certainly not one of those smart geniuses like th rest of this mailing list, but I think I have had reasonable success doing exactly that.

On a personal note, call me a dick head or something less insulting and playful, but please do not call me a genius.

Totally off-topic I know, but I am feeling a bit vocal right now. It's this cold weather you see.



 
(I'd fork it today if I could). We need something else to incrementally improve.

That seems to have worked out well the last time people tried it ... not.
In fact, "we will do X, but from scratch and better in everything!" seems to be a symptom of failed software projects.

Scala cannot incrementally improve for a very long list of technical and social reasons. I am not going to waste any more time hoping otherwise. I have already spent a lot of time doing that.


Note: I don't dispute that you could build a prototype which would be better than Kotlin or Ceylon from day one, but I think there are tons of options which would be more productive and beneficial to more people:
If you think you can't work on Scala, improve the runtime instead. If you don't care about the runtime, why not just use/adapt one of the existing languages of the ML family?

Bye,

Simon
--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Tony Morris

unread,
Jun 17, 2013, 8:33:02 PM6/17/13
to scala-fu...@googlegroups.com
I think a functional model is correct in that it is most useful.

Note however, that functional does not imply other things:
* not imperative e.g. Haskell is pure-functional and one of the most practical imperative languages available
* not concurrent -- it is possible, desirable even, to implement concurrency using functional programming

What might you propose in opposition?
-- 
Tony Morris
http://tmorris.net/

etorreborre

unread,
Jun 17, 2013, 10:04:04 PM6/17/13
to scala-fu...@googlegroups.com
I think that what Greg is saying is that if you equate "functional programming" with "lambda calculus" then you essentially have sequential computations. 

Then there are other calculi, such as the pi-calculus, which subsume the lambda calculus and are better suited for concurrent programming (because you have new tools to reason about concurrency properties).

In that sense I guess that Haskell is lambda-calculus + some concurrency primitives (forkIO + MVars) but I don't know exactly what kind of concurrent calculus this characterises (a quick run through the literature seems to indicate that this is a subset of the pi-calculus).

E.

Meredith Gregory

unread,
Jun 17, 2013, 10:38:50 PM6/17/13
to scala-fu...@googlegroups.com
Dear Tony,

Thanks for your questions!

i have already identified several alternate proposals implicitly and explicitly. See my last two posts (or the postscript below). i'm also intrigued by Eric's suggestion of the (concurrent) pattern calculus. i've had my eye on that one for quite some time. i think a nominal version of the concurrent pattern calculus might very well have legs. More generally, many flavors of nominal rewrite systems enjoy the desiderata i outlined in my initial post on this thread. To reiterate, these are the desiderata i pay attention to
  • a simple, core compositional mathematical presentation enjoying both operational and denotational semantics, as well as a proper Curry-Howard orientation 
  • fundamentally concurrent
  • support for reasoning about time and space complexity within the model
  • enjoying rich types that include standard structural considerations (as found in functional settings) as well as behavioral considerations, including but not limited to
    • expressing concurrency constraints, such as guaranteeing deadlock freedom
    • expressing resource constraints, such restricting 
    • expressing security and information flow constraints
  • fits well with existing language design wisdom -- especially monadic structuring of code
  • enjoys a modular account of reflection
It's well understood that if one's model is Turing complete then encoding various features (such as concurrency) is possible. To me that is a distraction. The point is expressivity relative to the features one wants to support out of the box.

For purposes of discussion, to me functional means built on top of some flavor of the lambda calculus. If it doesn't mean that to you, then i would request a statement of what functional means to you (or others on this thread) -- such as the one Vincent gave. That was brilliant in its honesty and directness. 

Monads are not specific to functional computing. It is perfectly possible to base a language on any of the π-calculus, the join calculus, the ambient calculus, the pattern calculus or a nominal rewrite system and provide support for monadic structuring of language features and user-defined features, data types and control structures. This arises from having a category-theoretic interpretation of the model. None of this ties such a language design to the lambda calculus or a Cartesian closed category structure. (In fact, these are liabilities when it comes to reasoning about complexity -- unless the CCC structure arises as an extensional collapse of an underlying linear structure.) So, monadic doesn't mean functional and functional doesn't mean monadic.

Looking specifically at the lambda calculus, Berry's sequentiality theorem lets us know that the lambda calculus is about sequential computation. So, that foundation doesn't help one reason about concurrency. There's no proof techniques, logics or types that derive from the model that help one:
  • reason about concurrency
  • adjudicate language design features
  • provide basic language support -- such as compile-time type-checking -- to support concurrency
These will come from outside the model. A similar argument can be made for reasoning about complexity. So, what purpose is the model serving?

To me the model is the blueprint for the language. It defines what will be easy to express and what will be hard. It also defines what will be easy to reason about in programs and what will be hard. i submit that what we want to be easy to express and to reason about in modern day programs unavoidably includes concurrency and complexity. i also believe that shortly after that the pressures to manage code at higher and higher levels of abstraction make a modular account of reflection a must have rather than a nice to have.

Best wishes,

--greg

P.S. i reproduce here the toy model that i described in a previous email (written at the same level of abstraction as a standard presentation of the untyped lambda calculus).

Specifically, the use of futures, promises, asynchrony and state management can be brought together in coherent packages that make sense as an independent model of computation. By 'independent' i mean independent of Scala, of the JVM and independent of the lambda calculus. Here's a teeny-tiny toy example. i will define a VM made to run processes -- which i will denote by P, Q, R, ... . These processes can implement functions, but are not functions. Only a limited set of them 'return' anything. The difference is something like this: an encryption scheme inputs a message in a given format and produces a message in a new format. That is a function. The sliding window protocol, by way of contrast, has no return type. It is a process which provides a means to guarantee a property of a communication channel -- namely that messages arrive exactly once and in order. That's the difference between a process and a function. Functions can be usefully thought of as a subset of processes where the process behavior outputs a result. Not all processes have outputs. On to the definition of the VM.

First, we define process code. This is the code processed by the VM.

P, Q ::= 0 // the stopped process: it does nothing
          | x?( y )P // this process is blocked on the channel x waiting for input which it will bind to the variable y in the process code P and then run P
          | x!( Q ) // this process will asynchronously send the code of the process Q on the channel x
          | P | Q // this process represents the concurrent composition of P and Q, meaning P and Q run concurrently
          | *x // dereference -- all channels represent the code of some process! this gets the code for the process back and begins running it
x, y ::= @P // reference -- this says explicitly that all channels represent the code of some process and taking the "address of" a process P -- i.e. get its code -- is how you get a channel

Now, this code makes syntactic distinctions that don't make any real difference to execution. For example, if P|Q means P and Q run concurrently, then certainly this is no different than Q|P. Similarly, if 0 is the stopped process then running the stopped process concurrently with P is the same as running P. Further, just as in Scala def f( x ) = M is the same as def f( y ) = M{ y/x } where M{ y/x } means replace x with y in every occurrence in M, a rule we use when we effect certain refactorings in Scala, we have the same relationship in this model: x?( y )P is the same as x?( z )( P{ z/y } ). This last equivalence of variable re-naming is called alpha-equivalence. The more general collection of equivalences identified here gets bundled into one equivalence called structural equivalence. In symbols

=S= denotes the smallest equivalence containing alpha-equivalence that makes ( P, |, 0 ) into a commutative monoid, i.e.
  • P | Q =S= Q | P
  • P | ( Q | R ) =S= ( P | Q ) | R
  • P | 0 =S= P
Finally, we have the transition rules of the VM, taking it from a state, S, to a new state S'.

comm: x?( y )P | x!( Q ) -> P{ @Q / y }
par: P -> P' => P|Q -> P'|Q
struct: P =S= P', P' -> Q', Q' =S= Q => P -> Q

There -- that's the complete specification of the VM. You can implement the VM in less than an hour in your favorite programming language, if you know what you're doing. Clearly, we can realize this VM directly on top of the JVM. The computational model this VM realizes is Turing complete. Anything you write in Scala can be written is this VM.

Just to be clear, to load a VM all you need to do is write down the code of a process. Here's one

x?( y )0 | x!( 0 ) 

this reduces in a single step to the stopped process, 0

Here's a more complex computation

Loading the VM with the state: 
x?( y )x?( z )y!( *z | *z ) | x!( 0 ) | x!( u?( v )0 ) | @0?( w )*w
-> comm + par
x?( z )@0!( *z | *z ) | x!( u?( v )0 ) | @0?( w )*w
->  comm + par
@0!( u?( v )0 | u?( v )0 ) | @0?( w )*w
-> comm
u?( v )0 | u?( v )0

and the VM halts

Meredith Gregory

unread,
Jun 17, 2013, 10:49:02 PM6/17/13
to scala-fu...@googlegroups.com
Dear Eric,

Thanks for your query about MDP4tW! Funnily, Bill Venners and i are meeting this week to plan the next update!

Your summary of my position is excellent! i would refine it thusly.

i identified an abstract set of properties i think are important in a model of computation to support modern computing challenges. This -- in my own estimation -- is perhaps the most important contribution i made to the thread.

Your assertion about Haskell is almost correct. SPJ and i discussed the foundations of Haskell ages ago. He told me that when he began he chose a flavor of system F as the core. Since then, people have glommed onto Haskell all manner of features -- including concurrency.

It is perfectly possible to give an account of Haskell + X (for some feature X) post facto -- such as modeling it in π-calculus or a bi-category built on top of a symmetric monoidal object, etc. Since the thread is about a greenfield effort, i was opening dialogue about what sort of model would support modern language features -- from the get go and hence provide a trajectory for the language that gives it long life and positive impact.

Best wishes,

--greg

Tony Morris

unread,
Jun 17, 2013, 10:54:14 PM6/17/13
to scala-fu...@googlegroups.com
On 18/06/13 12:49, Meredith Gregory wrote:
> Your assertion about Haskell is almost correct. SPJ and i discussed
> the foundations of Haskell ages ago. He told me that when he began he
> chose a flavor of system F as the core. Since then, people have
> glommed onto Haskell all manner of features -- including concurrency.
Amusingly, I had the same discussion with SPJ in December 2011 when he
was in Brisbane. He seemed to hold a strong position that any new
language should compile to the same flavour of System-F.

Meredith Gregory

unread,
Jun 17, 2013, 10:58:35 PM6/17/13
to scala-fu...@googlegroups.com
Dear Tony,

LOL! Of course, success breeds success! (And, my next band will be called The Beatles! ;-)

Best wishes,

--greg


--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Tony Morris

unread,
Jun 17, 2013, 11:40:20 PM6/17/13
to scala-fu...@googlegroups.com
This is a very interesting point regarding language foundation. I have not investigated it enough to answer what is my own burning question: is it possible that one model subsumes another? I know the answer is, "probably not", however, if not, then to what extent must sacrifices be made between each? Can it be minimised? Can we pay a small amount in sacrifice for a large gain?

For example, I previously alluded to an oft-repeated false dichotomy regarding functional (lambda calculus) versus imperative programming. In my experiences, these two are often made out to be in contention, but it is simply not true[1]. I often like to point out that Haskell does imperative programming better than the other guys (Java, C, Scala). It sounds like an obtuse thing to say, solely for the purpose to elicit provocation, however, I contend that it is a very valuable insight -- once it is realised how and why it is true, much fruit can be had.

I wonder if there are other valuable insights with regard to that which you mention and I suspect so given the foundational under-pinning. In any case, I would have to work more out to better form solid opinions and insights with respect to this.

[1] http://apocalisp.wordpress.com/2011/05/30/imperative-vs-functional-programming/

Jonathan Leonard

unread,
Jun 18, 2013, 12:05:51 AM6/18/13
to scala-fu...@googlegroups.com
If you're not happy with the choice of languages for any VM the solution is simple:  cross-compile a language you are happy with to target that particular VM.

Specifically I think F# and this http://xmlvm.org/clr2jvm/ would be a nice pairing.

Other options include a JVM backend for LLVM (which you can feed Haskell [http://llvm.org/pubs/2008-06-CompilingHaskelltoLLVM.pdf]):

and this looks promising too:

Regards,
Jonathan Leonard


On Wednesday, June 12, 2013 7:54:33 PM UTC-7, Tony Morris wrote:
Hello,
I have a question. I hope you might be able to help me out. The question
can be broadly phrased, "what can be done about the JVM situation?", but
what do I mean by this?

I have some strong opinions that influence any potential answer I might
have. Maybe you don't quite agree and so it changes the question up a
bit, but here they are anyway:

On Scala:

1) Scala is the best available programming language for the JVM.
2) Scala does not do Java (legacy library) inter-operability very well.
3) Scala is not a well-designed language, but is at least better than
other not-so-well-designed JVM languages.
4) Scala is unlikely to be influenced into adhering to better language
design principles. That is, the poor design is likely to continue into
the future.
5) The Scala compiler cannot be worked on easily. Specifically, if
language improvements were to come into consideration, it would be
difficult to implement them with the current compiler.

On "the JVM situation":

5) The so-called library support offered by the JVM is often over-stated
in its utility.
6) The JVM is a platform that is best avoided if possible e.g. a new
project without any commitment would be best served by avoiding the JVM
due to all of its penalties and almost zero benefits.

There exist projects that cannot avoid using the JVM. This situation is
what I will call the JVM situation.

It has been argued to me that the extent to which the JVM cannot be
avoided is often over-stated. That is to say, a project participant may
claim, "but I cannot avoid the JVM!" to which an appropriate response to
such a hasty remark is, "bullshit, think harder." I do not hold a strong
position on this argument and I would like others' opinions on whether
it has merit.

Supposing that the argument fails, we are left with a not-so-rare
situation of having to use the JVM. However, Scala is not very well
designed. What to do about this? Assuming you had a lot of time to
address this problem, I see a number of possibilities:

a) Construct a JVM language that is well-designed with good legacy
library interoperability.
b) Construct a Scala-compliant compiler with extensions to accommodate
language improvements and optimisations. This would also address the
speed issue with scalac as a consequence. It would also offer better
external tool support that integrates into the compiler.
c) Concede the aforementioned argument. You are not bound to the JVM as
much as you formerly claimed. Use a language like Haskell or O'Caml and
reap the benefits.
d) Concede that you are simply doing the best you can with available
tools. Use Scala -- at least you're doing better than the guy down the
road using Java or "Scala as if it were Java."

There are possibly others options I have not thought of. Each of these
has some amount of required effort ranging from nothing to "lots." This
is important to providing an answer. In other words, which response to
the JVM situation would you consider is the best use of effort?

Thank you for any insights you might have.

Meredith Gregory

unread,
Jun 18, 2013, 12:08:10 AM6/18/13
to scala-fu...@googlegroups.com
Dear Tony,

i believe we can have our cake and eat it, too! This is the point that Moggi and Power and others were making about the power of monads in modular language design. We can take some simple, but powerful core and then mix and match language features into that core using monads as a structuring technique for language features. So, you can calculate

lambda-calculus + concurrencyMonad + RX + LINQ + ...

or you can calculate

π-calculus + LINQ + ... // π-calculus subsumes most of the interesting concurrency monads and eventing

or 

concurrent-pattern-calculus + LINQ + RX + ...

This is why i would first shift the discussion to what challenges need the most support -- i.e. what are the desiderata of the model. Then, consider different factorizations and how they support expressing and reasoning about code. Then consider how that fits with a type system and compilation.

Actually, the three models above arise as nominal rewrite systems. It is well known that rewrite systems have a monadic presentation and nominal functionality also has a monadic presentation (see Cheney's Scrap Your Nameplate). So, you could potentially contemplate an entirely monadic foundation. 

Taking it a step further, based on the recent work i did with SpecialK/KVDB and various dev teams, i'm beginning to amass data that having the for-comprehension as the centerpiece of the language is not terrible. There are challenges, but it doesn't suck worse than anything else i've tried. ;-) 

And, when you think about it, after Cantor mathematicians reformulated mathematics on top of the set comprehension.  Furthermore, making the comprehension the centerpiece of SQL was a very good idea. So, there's both theoretical and practical precedent for this approach.

Best wishes,

--greg

Tony Morris

unread,
Jun 18, 2013, 12:24:36 AM6/18/13
to scala-fu...@googlegroups.com
On 18/06/13 14:08, Meredith Gregory wrote:
>
> Taking it a step further, based on the recent work i did with
> SpecialK/KVDB and various dev teams, i'm beginning to amass data that
> having the for-comprehension as the centerpiece of the language is not
> terrible.
I don't know about this.

What about comonad comprehensions, applicative comprehensions or arrow
comprehensions? I tend to use this interface more so in practice:

trait ApplySemiComonad[F[_]] {
def extend[A, B](f: F[A] => B): F[A] => F[B]
def ap[A, B](f: F[A => B]): F[A] => F[B]
}

Also, if monad-comprehensions are so important, are you not compelled to
fix Scala's implementation?
* No common trait hierarchy specifying values on which comprehensions
may run.
* The final call to map in a comprehension causes all sorts of
unnecessary pain.

Mike Stay

unread,
Jun 18, 2013, 12:29:09 AM6/18/13
to scala-fu...@googlegroups.com
One interesting approach to language design that I came across recently is Daira Hopwood's Noether language.
On slide 3, she gives twelve nested languages; at the top, the language is very weak, but one can make very strong guarantees about its behavior.  As she adds levels, the language becomes more powerful, but the guarantees one can make about the behavior weaken.  It would be nice to be able to simply verify that a subroutine is written in a particular sublanguage and immediately be able to know some strong bounds on its authority.

Meredith Gregory

unread,
Jun 18, 2013, 12:31:09 AM6/18/13
to scala-fu...@googlegroups.com
Dear Tony,

i'm still internalizing the comonad. The monad-comonad relationship is deeper than i originally contemplated, by a longshot. Any light you can shed on this would be gratefully appreciated.

A new language would certainly be one way to fix the problem with Scala's for-comprehension. ;-)

Best wishes,

--greg


--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Dylan Just

unread,
Jun 18, 2013, 1:19:01 AM6/18/13
to scala-fu...@googlegroups.com
Scala does some awesome stuff, but the existence of subclassing will always hold it back.

I would suggest contributing to, or forking one of the few projects that appear to be on track.

a. Idris - a dependently-typed ML-style language that can compile to C/binary, JavaScript or Java
b. Fjord - F# on JVM
c. Frege - Haskell-like language on JVM. If you don't like the FFI approach, fork the language or contribute to it.

In terms of pure language, Idris seems to be ahead of any other ML-style language, including Haskell. The fact that Idris can compile to those 3 platforms is just huge and those efforts should contributed to.

Idris shows immense amount of promise and needs some serious loving from our community.

Alexandru Nedelcu

unread,
Jun 18, 2013, 2:15:25 AM6/18/13
to scala-fu...@googlegroups.com
On Tue, Jun 18, 2013 at 8:19 AM, Dylan Just <dy...@techtangents.com> wrote:
Scala does some awesome stuff, but the existence of subclassing will always hold it back.

I really like this trait of Scala. It complicates things surely, as suddenly you need to think about co/contra-variance. But many things are just easier this way. Haskell has a conceptual elegance in its simplicity by simply not having subtyping, but the gymnics you have to pull for downcasting or for heterogenous collections in case you need them are just hacks bolted on. I also like for example the representation of Sets or Maps as functions. Can Haskell do that?
 
b. Fjord - F# on JVM

I really hope you're not thinking of F# as a model for what Scala must be. I especially love their usage of "static" methods for operator overloading, plus the *special* type restrictions you need for static methods to make List.sum work, or how you need to work with two type systems that were shoved in the same language and their interactions.

Simon Ochsenreither

unread,
Jun 18, 2013, 6:07:55 AM6/18/13
to scala-fu...@googlegroups.com
Hi Tony,

Why are you even asking these laborious, mechanical questions? How much effort do you spend on answering them? This is not your job -- it is the job of tools and of Java inter-operability. Therefore, Scala does not do this well.

You might be interested in the work the Kotlin people are doing. They try to infer at least nullability.

Anyway, I think interop in Java is fundamentally broken. The Java libraries are and were designed without any thought about the things we care about. They will still be designed that way in ten years.
By definition, the types inferred for them will be terrible, but because the authors will never see those signatures, they won't improve their code.

I just don't see a reasonable benefit/cost ratio here, compared to a lot of other things I could do in the mean time.
Yes, it would be nice, but there are probably 100 other things which I would do first.


Because I cannot do all the stuff I want. Not even close. Ultimately, it is a false economy, that's why.

Well, you need to pick a runtime on day or the other, except if you never want to run your language.

 
I contend that no, a lot of people are not able to deal with it (the consequences of Scala's shortfalls alluded to here-in). Indeed, very few are able to deal with it. I watch this happen before my very eyes with remarkable consistency. It might be my bias, but I see it in person, on mailing lists, on forum posts and I very rarely see any counter-examples. Convince me otherwise.

I've regularly sit down in workshops and make notes about the issues people hit while learning the language. Most of the popular issues are fixed already, with the rest hopefully going into 2.11.

 
Scala cannot incrementally improve for a very long list of technical and social reasons. I am not going to waste any more time hoping otherwise. I have already spent a lot of time doing that.

I'm quite happily doing that right now. My trick is doing one thing at a time.

Bye,

Simon

simon.ochsenreither

unread,
Jun 18, 2013, 6:16:31 AM6/18/13
to scala-fu...@googlegroups.com

In terms of pure language, Idris seems to be ahead of any other ML-style language, including Haskell. The fact that Idris can compile to those 3 platforms is just huge and those efforts should contributed to.

Idris shows immense amount of promise and needs some serious loving from our community.

I think I met the guy who is working on the JavaScript backend now a few years ago at the 27C3 in Berlin. Occasional Scala user, as far as I remember. His twitter name is @raichoo, blog: http://raichoo.blogspot.de/2013/01/towards-dependently-typed.html

Sébastien Bocq

unread,
Jun 18, 2013, 7:36:28 AM6/18/13
to scala-fu...@googlegroups.com
Like Scala depends on Java/JVM, Haskell can't completely get rid of C/*nix. Although Haskell enjoys better designed general purpose libraries, some seasoned Haskell programmers have already done the job of wrapping low-level C libraries into FFI, which represents a more complex task than wrapping null into an Option. Although Haskell offers a really nice programming model, as good as it is, its STG machine, which is a rather complex beast written mostly in C, and the fact that its user-level thread implementation is built into the runtime, highlight to me some shortcomings of the platform towards systems programming. I know there is work on this like the Disciplined Disciple Compiler [1] and Habit [2], which both implement some strict dialect of Haskell, but I don't know their status.

To reply to Greg's inquiry on what programming models work best, in my opinion, a nice feature of a hybrid language like Scala is that it makes it easy to wrap strict and impure functions into lazy and pure DSLs without switching language or runtime. This is my experience with Molecule [3]. It shows that it is possible to design a self hosting concurrent DSL and runtime that combines lazy streams with a concurrent programming model based on the pi-calculus on top of an impure language (Scala) and a strict runtime (JVM). The end result is prettier than you think. Its type-safe streaming channels support AttoParsec like combinators directly on top of the JVM NIO interfaces, it handles better termination than the alternatives and the performance of its runtime scheduler is competitive with Go in multicore benchmarks when there are more than one thread [4].

So based on my own experiences, I'd say the ideal programming language should:
- be able to self host its runtime because it must be good at systems programming (e.g. C/Unix),
- be homoiconic to ease the implementation of DSLs that can tune into the compiler (e.g. Lisp family languages),
- have Curry-Howard orientation for correctness (e.g. ML, Haskell),
- support ease of reasoning about time and space complexity
- come with good support for namespace and versioning at source and runtime level to ease maintenance, reuse and modularity (e.g. modula-3, smalltalk, JVM).

Unfortunately, I'm afraid that design a single programming language with all these features is still research today.

Cheers,
Sébastien

[1] http://disciple.ouroborus.net/
[2] http://hasp.cs.pdx.edu/
[3] https://github.com/molecule-labs/molecule
[4] https://github.com/sbocq/multicore-benchmarks#latest-results

runarorama

unread,
Jun 18, 2013, 1:18:29 PM6/18/13
to scala-fu...@googlegroups.com

On Thursday, June 13, 2013 1:36:43 PM UTC-4, Brian McKenna wrote:
I'll give an estimate for a usable prototype:

* 1 week to make a Hindley-Milner, strict, functional JVM language
* 3 days to add laziness via InvokeDynamic (requires Java 8)
* 2 days to add type-classes
* 2 days to add a sane FFI


As someone who has in fact worked on a new language for the JVM, a prototype really is that quick to write. But writing something that isn't painful to work in takes more like a year. This includes adding rank-n types, records, kind polymorphism, etc. You have to also understand that you would be working at the cutting edge of PL research, so you spend a lot of your time reading papers.

Making a language that is enjoyable to work in takes another year. So I think that if you had two years to spend on this, full time, you could come up with something truly amazing. You need money, but I submit that you could do it by simply saving up enough working in the salt mines to live comfortably for 2 years, and then taking time off to actually do it.

I was just talking to Rich Hickey about how he managed to find time to create Clojure. The answer was exactly this. He took a couple of years off work and spent them working on Clojure.


Josh Suereth

unread,
Jun 18, 2013, 1:22:12 PM6/18/13
to scala-fu...@googlegroups.com
I love salt!


Meredith Gregory

unread,
Jun 18, 2013, 1:26:56 PM6/18/13
to scala-fu...@googlegroups.com
Dear Sebastien,

That's very nice! There's a lot of overlap in your desiderata and mine. i like the self-hosting and homoiconic properties!

Best wishes,

--greg


--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Meredith Gregory

unread,
Jun 18, 2013, 4:28:51 PM6/18/13
to scala-fu...@googlegroups.com
Dear Runar,

That's almost exactly my experience with GScheme, Rosette and Highwire. A new language takes a full two person years -- almost independent of the underlying computational foundation. There's a lot of basic stuff that you just can't get around. 

For example, at the end of the day you have to support arithmetic. No matter what your model you still have to parse arithmetic expressions and then reformulate them in the native computational model -- whatever that may be. Getting that right and organizing it to be satisfyingly generic and getting it situated with respect to typing takes time and thought. Decisions like these still plague Scala and that's way past 2 person years.

It's things like this that have me building embedded languages in a setting that provides a decent backdrop and good legacy interop. But, the energy and idealism reflected in those estimates of Brian's might be just what it takes to begin the next stage of evolution. ;-)

Best wishes,

--greg


--
You received this message because you are subscribed to the Google Groups "scala-functional" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-function...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Oscar Vargas Torres

unread,
Jun 18, 2013, 10:01:57 PM6/18/13
to scala-fu...@googlegroups.com

Hello Greg!

I would say mathematicians with the Computer Science (theoretical and practical) background you have are not that common. I would love to see a practical implementation of a "correct foundational model for a language designed to meet today's computing requirements". And if this proposal is supported by several (as much as possible) talented people, I think the final product would be better than if it's just one person working on it.

Please, continue sharing your opinions and knowledge with the Scala community (or the community of that "better" language). No one knows everything, so it might be helpful to get help from other knowledgeable people (and I am sure there are some talented people interested in getting something better: Tony Morris, just to mention one).


Ziad Hatahet

unread,
Jun 19, 2013, 12:18:43 AM6/19/13
to scala-fu...@googlegroups.com
I am curious to know what the folks here think of the Rust programming language (rust-lang.org). It is influenced by several languages, including: Haskell, ML, and C++. It compiles to native code as well.

Sven Efftinge

unread,
Jun 19, 2013, 2:53:21 AM6/19/13
to scala-fu...@googlegroups.com


On Thursday, June 13, 2013 7:36:43 PM UTC+2, Brian McKenna wrote:
* 1 week to make a Hindley-Milner, strict, functional JVM language

If you want a language seamlessly compatible with existing Java libraries, you have to
support the Java type system, including generics, overload resolution, primitives etc.

This takes months and if you want decent type inference it takes even longer.
 

Sven Efftinge

unread,
Jun 19, 2013, 3:22:11 AM6/19/13
to scala-fu...@googlegroups.com
If you really want to support seamless interoperability with legacy Java APIs you *need* to support sub typing. In fact it's best to have as little differences in the type system as possible. 
If you want to have even better support for Java APIs than Java, you need to look out for idioms and find way to support them in your language. That is what Xtend does. It let's you write code directly against legacy Java APIs which is much more concise than in  any other language. It can be compared to the Scala equivalent when you written and thrown in glue code and implicits.

And it has really cool support for macros and a great IDE!

So a) is done : http://xtend-lang.org :-)

Tony Morris

unread,
Jun 19, 2013, 3:23:13 AM6/19/13
to scala-fu...@googlegroups.com


On 19/06/2013 5:22 PM, "Sven Efftinge" <sven.e...@googlemail.com> wrote:
>
> If you really want to support seamless interoperability with legacy Java APIs you *need* to support sub typing. In fact it's best to have as little differences in the type system as possible. 

Who says?

> If you want to have even better support for Java APIs than Java, you need to look out for idioms and find way to support them in your language. That is what Xtend does. It let's you write code directly against legacy Java APIs which is much more concise than in  any other language. It can be compared to the Scala equivalent when you written and thrown in glue code and implicits.
>
> And it has really cool support for macros and a great IDE!
>
> So a) is done : http://xtend-lang.org :-)
>

Runar Bjarnason

unread,
Jun 19, 2013, 7:25:09 AM6/19/13
to scala-fu...@googlegroups.com
I don't think you want to embed Java's type system directly. What you want is a foreign function interface that is painless to use.

Also don't think I want seamless interoperability with Java APIs. I very much want a seam because calling into Java is unsafe in the general case.




--
You received this message because you are subscribed to a topic in the Google Groups "scala-functional" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scala-functional/kvbaE6cAzqM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scala-function...@googlegroups.com.

Sven Efftinge

unread,
Jun 19, 2013, 7:51:59 AM6/19/13
to scala-fu...@googlegroups.com
On Wednesday, June 19, 2013 1:25:09 PM UTC+2, runarorama wrote:
I don't think you want to embed Java's type system directly. What you want is a foreign function interface that is painless to use.

At the end of the day you need to write code against these APIs and usually the designer had Java clients in mind.
I cannot prove it, but I haven't seen any JVM language (besides Java and Xtend) without any Java interop issues. 
 
Also don't think I want seamless interoperability with Java APIs. I very much want a seam because calling into Java is unsafe in the general case.

Are you referring to nullability of references?
Sure, that's one thing you cannot just change as it's part of the contract of Java APIs.

I guess its ok or even desirable to have a safe interop mode with some foreign language's library if you just call out to it rarely. 
But if you want to live in the Java world and use the JDK and other Java APIs all day than
your code suffers from even small interop gotchas. 
Changing e.g. nullability or whether generics are done on declaration or use side causes interop problems.
 
It is loading more messages.
0 new messages