Interesting ideas about concurrency and mutable state from Pony lang

[Renato Athaydes]

Hi,

I was reading about a new language called Pony (terrible name, but do not let it mislead you, very interesting stuff) and became fascinated by some of its ideas after having a look at its documentation and paper.

Specifically, I was intrigued by "guarantees", which seem to allow Pony to guarantee deadlock-, race-condition-free code, and allows safe, if limited, sharing of mutable state in parallel code (sounds fantastic? Please read the paper and try to disprove their claims!).

This would kind of go against the common belief that you can only have safe, truly parallel code by using 100% immutability or locks. It's a big deal, in my view.

The language itself has some common things with Ceylon. Given that in Ceylon, "variable" is just an annotation, I thought that perhaps it would be interesting to add something like "guarantees" to Ceylon as well, as new annotations.

Sounds possible??

Other ideas that I liked from Pony include language-level support for Actors (which support "behaviors", similar to "methods" but not quite the same), support for both structural subtyping and nominal subtyping (might be a good way to have something similar to type classes in Ceylon?), and the fact that it integrates seamlessly with C.

Here's the marketing page: http://ponylang.org/

Just dropping the ideas here! Any comments?

Renato

[David Hagen]

support for both structural subtyping and nominal subtyping

In my opinion, repudiating structural subtyping was one of the best design decisions of Ceylon. That refactoring is guaranteed to work is a beautiful thing.

This would kind of go against the common belief that you can only have safe, truly parallel code by using 100% immutability or locks.

I am pretty sure the common belief is still true. Pony has actors, and I am not familiar with an implementation of actor mailboxes that does not involve locks. I'll guess it still has locks, it just wraps them to make them harder to break. Of course, it is easy to emulate locks (and deadlocks) with actors, unless their actors are really different from what I am used to.

and the fact that it integrates seamlessly with C

It seems that every language made in the last fifteen years runs on either the JVM or the LLVM.

[Renato Athaydes]

In my opinion, repudiating structural subtyping was one of the best design decisions of Ceylon. That refactoring is guaranteed to work is a beautiful thing.

Having BOTH is the main thing here. I can definitely see how I would like some types, sometimes, to be treated uniformly without changing them.
 

This would kind of go against the common belief that you can only have safe, truly parallel code by using 100% immutability or locks.

I am pretty sure the common belief is still true. Pony has actors, and I am not familiar with an implementation of actor mailboxes that does not involve locks. I'll guess it still has locks, it just wraps them to make them harder to break. Of course, it is easy to emulate locks (and deadlocks) with actors, unless their actors are really different from what I am used to.

Pony actors are really very different to what you and I are used to. I am disappointed you felt appropriate to comment without looking into it first (you claim yourself "I'll guess it still has locks" and that's wrong). I had already linked to the docs, and from there you could get to the paper that describes the model in detail.
 

and the fact that it integrates seamlessly with C

It seems that every language made in the last fifteen years runs on either the JVM or the LLVM.

I believe there are many examples of newer languages that are becoming really popular, yet *can* run natively. See:

* Julia
* Rust
* Nim
 
I do like having a VM, but that depends on what kind of application you're developing, of course.

[David Hagen]

On Saturday, May 9, 2015 at 2:49:56 AM UTC-4, Renato Athaydes wrote:

In my opinion, repudiating structural subtyping was one of the best design decisions of Ceylon. That refactoring is guaranteed to work is a beautiful thing.

Having BOTH is the main thing here. I can definitely see how I would like some types, sometimes, to be treated uniformly without changing them.
 

Unfortunately, we cannot have both what you want and what I want, because by having both, refactoring can no longer be guaranteed.

 

This would kind of go against the common belief that you can only have safe, truly parallel code by using 100% immutability or locks.

I am pretty sure the common belief is still true. Pony has actors, and I am not familiar with an implementation of actor mailboxes that does not involve locks. I'll guess it still has locks, it just wraps them to make them harder to break. Of course, it is easy to emulate locks (and deadlocks) with actors, unless their actors are really different from what I am used to.

Pony actors are really very different to what you and I are used to. I am disappointed you felt appropriate to comment without looking into it first (you claim yourself "I'll guess it still has locks" and that's wrong). I had already linked to the docs, and from there you could get to the paper that describes the model in detail.
 

I read that docs paper in totality. This line is pretty suggestive of a locking mechanism in the implementation: "Note that if you have a variable referring to an actor then you can send messages to that actor regardless of what capability that variable is."

Indeed, here is a block of code from messageq_push in messageq.c in the Pony source:

pony_msg_t* prev = (pony_msg_t*)_atomic_exch(&q->head, m, __ATOMIC_RELAXED);

bool was_empty = ((uintptr_t)prev & 1) != 0;

prev = (pony_msg_t*)((uintptr_t)prev & ~(uintptr_t)1);

_atomic_store(&prev->next, m, __ATOMIC_RELEASE);

 

I am no expert in C locking, so I could be mistaken that actors in Pony wrap locks. If this not a lock, I am curious to know how this works.

and the fact that it integrates seamlessly with C

It seems that every language made in the last fifteen years runs on either the JVM or the LLVM.

I believe there are many examples of newer languages that are becoming really popular, yet *can* run natively. See:

* Julia
* Rust
* Nim
 
I do like having a VM, but that depends on what kind of application you're developing, of course.

From julialang.org: "Julia’s LLVM-based just-in-time (JIT) compiler combined with the language’s design allow it to approach and often match the performance of C. "

From Rust on Wikiedia: "The self-hosted compiler uses LLVM as its backend."

You are correct that Nim appears to compile to C++ first.

[Renato Athaydes]

On Saturday, May 9, 2015 at 2:49:56 AM UTC-4, Renato Athaydes wrote:

In my opinion, repudiating structural subtyping was one of the best design decisions of Ceylon. That refactoring is guaranteed to work is a beautiful thing.

Having BOTH is the main thing here. I can definitely see how I would like some types, sometimes, to be treated uniformly without changing them.
 

Unfortunately, we cannot have both what you want and what I want, because by having both, refactoring can no longer be guaranteed.

Do you have some evidence showing that it is impossible to safely refactor code in the presence of structural subtyping? And that obviously does not affect the safety of refactoring when using nominal subtyping... the two do not intersect!
 

I read that docs paper in totality. This line is pretty suggestive of a locking mechanism in the implementation: "Note that if you have a variable referring to an actor then you can send messages to that actor regardless of what capability that variable is."

The reason I posted this here was not to argue whether or not Pony uses locks... I expected people to actually accept that at first, and then try to understand how that may be possible by reading the paper (if you can find a hole in the theory, please contact the author and kindly inform him about how his research is rubbish). And from the theory described in the paper, see if it would be possible to implement something similar in Ceylon. It is really ridiculous to be accusing the author of being a liar like you're doing without any basis for it except preconceived ideas about how actors are implemented and a C snippet which has absolutely nothing to do with locks as far as I can tell (perhaps you're assuming that atomicity requires locking?).

From julialang.org: "Julia’s LLVM-based just-in-time (JIT) compiler combined with the language’s design allow it to approach and often match the performance of C. "

From Rust on Wikiedia: "The self-hosted compiler uses LLVM as its backend."

You are correct that Nim appears to compile to C++ first.

The point I made was that you can call C (and its libraries) from Pony without jumping through hooks (like in Java) and I like that.... you can also call C directly from Rust and Julia code.... using the LLVM (which is not a VM in the same sense as the JVM is) does not in any way make that harder. If you didn't notice, ponyc (the compiler) uses LLVM as well.

Let me make one thing clear: I am not an advocate of Pony!!!

I am an advocate of Ceylon! But I think the Pony concurrency model in particular could be of great use for Ceylon and other new languages (but I may be wrong because of the differences in the memory model and Ceylon's dependency on the JVM and JS VM). The other things I mentioned just in passing as I saw some novelty in them.

[Philippe Lhoste]

Pony looks interesting, but when I see "64-Bit installers for Windows 7, 8, 8.1 and 10 will be available soon." I stop there... :-)
Of course, Go had a similar issue, and finally caught up.
But well, if I need a natively compiled language with no null, no garbage collection and no manual memory management, I would rather look again at Rust. It matures and soon will be at v.1.0.

[Steven Sagaert]

Specifically, I was intrigued by "guarantees", which seem to allow Pony to guarantee deadlock-, race-condition-free code, and allows safe, if limited, sharing of mutable state in parallel code (sounds fantastic? Please read the paper and try to disprove their claims!).

This would kind of go against the common belief that you can only have safe, truly parallel code by using 100% immutability or locks. It's a big deal, in my view.

Actually AFAIK this has been possible for a long time with dataflow style programming/variables. See the Oz programming languages. Clsoe to home: Groovy implements dataflow concurrency (next to other styles) in its GPars library.  

[Renato Athaydes]

Hi Steven,

Nice mention of GPars! It's a pretty cool library.

However, I must say you missed the point of Pony tags: they are part of the type system. You get your concurrency to "type-check"!
Comparing that with GPars is really missing the point: Groovy by default does not type-check, and obviously has no way of telling you if you misuse some construct... even though Groovy with GPars seems much nicer to use than Java's synchronized and related constructs, and it is based on the same principle of limiting where you can read/write variables as Pony, it is nevertheless completely up to the developer's discipline to ensure the code's concurrency behaves correctly... whereas with the model used in Pony, the type checker does that and you can't, even if you try, cause a deadlock or race condition on a program that compiles!

[Steven Sagaert]

Well I didn't miss the point: my comment was on the statement that it's possible to write safe concurrent code without locking or everything immutable and the answer to that is yes and not only in Pony. The fact that Pony checks this at compile time is nice and an extra safety blanket.

[Steven Sagaert]

Other new  AOT statically compiled languages that do some concurrency related checking at compile time are Rust & Nim.  In fact after a quick glance of the Pony doc section "passing and sharing" it sounds rather similar to Rust.

On Wednesday, May 20, 2015 at 8:14:23 PM UTC+2, Renato Athaydes wrote:

[Renato Athaydes]

Hi,

The way Nim and Rust do concurrency is also interesting, but nothing revolutionary and definitely not at all the same as Pony.

You see, Nim just makes sure you use your locks correctly at compile time:

http://nim-lang.org/blog/concurrency2.html

Rust goes further, but implements concurrency at the library level (though in a quite powerful way):

http://doc.rust-lang.org/book/concurrency.html

But please notice there's a big difference between these models and Pony's... Rust and Nim's model are both lock-based (the links above make that patently clear), whereas Pony's is lock-free... It forces you to organise your data flow such that the compiler can tell you if you get it wrong without having to recur to locks to achieve that.... it does not make you use mutex, like in Rust, or explicit locks as in Nim.

I think that there's a chance that Pony's model could be the right choice for supporting concurrency in Ceylon without the JVM-specific constructs for that... all we would need is to be able to add some annotations (the guarantees) that get checked by the compiler.... my question: is it possible for a library to do that (add compiler time checks on annotated variables)? Or perhaps a compiler plugin? That could be like a language extension, which could be pretty cool.