ANN: Fez - an fsharp to core erlang compiler experiment

[Karl Nilsson]

Hi,

The last couple of months I've been experimenting with a backend to the fsharp compiler that compiles down to the "core erlang" language and from there to beam byte code. 

The aim is only to cover the functional subset of the fsharp language at this point and to investigate what it is like to use an ML when targeting the beam and where the pain-points are. So far it has gone reasonably well. A fair bit of the core language and a bit of the core library has been covered as well as spawning of processes as well as message sending and (selective) receiving but it is very much an experimental prototype. I've tried to go wide rather than deep so there will be holes in most areas.

https://github.com/kjnilsson/fez

One can look at [1] to see what kind of programs can be written at this point and at [2] to read some half coherent implementation notes.

[1] https://github.com/kjnilsson/fez/blob/master/test/basics.fs

[2] https://github.com/kjnilsson/fez/blob/master/thoughts.md

Hope someone finds it interesting. Any feedback/engagement/discussions are very welcome.

Cheers

Karl

[Warren Young]

On Jun 7, 2017, at 1:24 PM, Karl Nilsson <kjni...@gmail.com> wrote:
>
> The last couple of months I've been experimenting with a backend to the fsharp compiler that compiles down to the "core erlang" language and from there to beam byte code.

Why not translate IL to BEAM instead?

It’s probably initially a lot more work, but I suspect it’s one of those cliff-climbing sorts of problems, where once you climb the cliff, things become a lot easier.

If you compile the F# app statically, this may allow you to use a lot of existing .NET framework and C# code.

> [1] https://github.com/kjnilsson/fez/blob/master/test/basics.fs

With all due respect to the work you’ve already done, that looks like low-hanging fruit to me: the parts of F# where the mapping to Erlang is mainly a matter of syntax differences, rather than semantic or behavioral differences.

Let’s talk about the hard bits now:

> [2] https://github.com/kjnilsson/fez/blob/master/thoughts.md


Quoting that now…

> Mutation (<-) could possibly be supported using the process dictionary.

As I understand it, the way Elixir handles this is by creating a different immutable variable under the hood each time you modify the value. The later versions of that value are said to “shadow” the earlier ones, with only the most recently seen version accessible at any given time.

> High-level object programming could possibly be supported using processes and or behaviours.

Or, do it the way so many other OO features have been strapped onto procedural languages after the fact: methods are just freestanding functions that happen to accept a hidden “this” or “self” parameter as their first parameter, and the function implicitly dereferences the “this” reference on each member access.

That could be a reference to an Erlang record in this case.

(Examples are Perl, Lua, R, Tcl, Lisp, and C++’s ancestor C-with-Classes.)

> First cut - no objects.

One of the reasons I prefer F# over Erlang is that it’s a truly multi-paradigm language. One-size-fits-all is a fairy tale.

If the only purpose of this project is to provide yet another syntax sugar coating atop Erlang, that already exists. I think it would be better if this actually brought new functionality to the table.

Other hard bits:

1. How do you implement Erlang atoms in F#? Without atoms, how do you do traditional Erlang-style message passing?

2. Do modules work yet? If so, does that imply that you have automatic function arity detection and declaration? Do you obey visibility keywords? (public, private, etc.)

3. It’s been quite a while since I wrote any Erlang, but my recollection is that currying could only be done with hackery in Erlang. Do you translate F#’s automatic currying into Erlang somehow?

4. Do you have any hope of implementing units-of-measure?

5. Does you current translation method preserve the distinction among numeric types? That is, does the translation allow unwanted mixture of integer sizes or integer and FP, or is that taken care of by the F# compiler front end, so that the translated AST is always semantically correct in the F# sense?

That’s enough grief for now. :)

[Karl Nilsson]

See inline.

On Wed, 7 Jun 2017 at 21:13 Warren Young <tan...@west.etr-usa.com> wrote:

On Jun 7, 2017, at 1:24 PM, Karl Nilsson <kjni...@gmail.com> wrote:
>
> The last couple of months I've been experimenting with a backend to the fsharp compiler that compiles down to the "core erlang" language and from there to beam byte code.

Why not translate IL to BEAM instead?

IL -> BEAM is an attractive idea. The runtimes are very different and I suspect if it was easy/possible someone would have tried it by now. My guess is it will be very painful. How would IL represent a process for example?

 

It’s probably initially a lot more work, but I suspect it’s one of those cliff-climbing sorts of problems, where once you climb the cliff, things become a lot easier.

If you compile the F# app statically, this may allow you to use a lot of existing .NET framework and C# code.

> [1] https://github.com/kjnilsson/fez/blob/master/test/basics.fs

With all due respect to the work you’ve already done, that looks like low-hanging fruit to me: the parts of F# where the mapping to Erlang is mainly a matter of syntax differences, rather than semantic or behavioral differences.

Let’s talk about the hard bits now:

> [2] https://github.com/kjnilsson/fez/blob/master/thoughts.md


Quoting that now…

> Mutation (<-) could possibly be supported using the process dictionary.

As I understand it, the way Elixir handles this is by creating a different immutable variable under the hood each time you modify the value.  The later versions of that value are said to “shadow” the earlier ones, with only the most recently seen version accessible at any given time.

IIRC elixir does not do "true" mutation but rather shadowing, just like fsharp. It is possible elixir has some macro-magic to simulate while loops (for example) but I find there is little value in those constructs apart from trying to make stuff go faster.

 

> High-level object programming could possibly be supported using processes and or behaviours.

Or, do it the way so many other OO features have been strapped onto procedural languages after the fact: methods are just freestanding functions that happen to accept a hidden “this” or “self” parameter as their first parameter, and the function implicitly dereferences the “this” reference on each member access.

Yes that is how I would approach it but it isn't a priority.

 

That could be a reference to an Erlang record in this case.

(Examples are Perl, Lua, R, Tcl, Lisp, and C++’s ancestor C-with-Classes.)

> First cut - no objects.

One of the reasons I prefer F# over Erlang is that it’s a truly multi-paradigm language.  One-size-fits-all is a fairy tale.

If the only purpose of this project is to provide yet another syntax sugar coating atop Erlang, that already exists.  I think it would be better if this actually brought new functionality to the table.

Other hard bits:

1. How do you implement Erlang atoms in F#?  Without atoms, how do you do traditional Erlang-style message passing?

DUs translate to tagged tuples for example where the DU name (it is actually another tuple of type, name) becomes the "tag". So you can define the atoms you want using a discriminated union with single cases.

 

2. Do modules work yet?  If so, does that imply that you have automatic function arity detection and declaration?  Do you obey visibility keywords?  (public, private, etc.)

FSharp modules translate to erlang modules directly

 

3. It’s been quite a while since I wrote any Erlang, but my recollection is that currying could only be done with hackery in Erlang.  Do you translate F#’s automatic currying into Erlang somehow?

Yes, that was certainly one of the harder bits. In brief, I flatten any declarations then wrap them in funs at the call site. So far that seems to work but I'm sure there will be nasty edge-cases.

 

4. Do you have any hope of implementing units-of-measure?

I haven't tried it. It is possible units-of-measure have no actual representation in the type AST (Which I translate from). If so it may just work.

 

5. Does you current translation method preserve the distinction among numeric types?  That is, does the translation allow unwanted mixture of integer sizes or integer and FP, or is that taken care of by the F# compiler front end, so that the translated AST is always semantically correct in the F# sense?

This part is a bit hand-wavey right now any ints become erlang ints etc. :) //TODO

 

That’s enough grief for now. :)

Thank you for your feedback! appreciated.

 

--
--
To post, send email to fsharp-o...@googlegroups.com
To unsubscribe, send email to
fsharp-opensou...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/fsharp-opensource
---
You received this message because you are subscribed to the Google Groups "F# Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fsharp-opensou...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Greg Young]

I have already answered on twitter. Will put here as well. The
IL->BEAM question is a good one. Also how do you handle the reloading
of a BEAM process while running given the type system. This is a hard
issue that the Pony guys were working on. There is a reason why the
messaging system is late bound in erlang.

> --
> --
> To post, send email to fsharp-o...@googlegroups.com
> To unsubscribe, send email to
> fsharp-opensou...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/fsharp-opensource
> ---
> You received this message because you are subscribed to the Google Groups
> "F# Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to fsharp-opensou...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Studying for the Turing test

[Karl Nilsson]

I'll also reply here as well :)

Fez is really just a "transpiler" (hate the word). It translates the typed fsharp AST to core erlang (https://www.it.uu.se/research/group/hipe/cerl/). Ultimately it's a tool for generating beam files, so if you can write types to describe your live update you should be able to generate the appropriate beam code (one day that is).

[Greg Young]

Shipping a full new BEAM file will require IIRC a full update and not
allow a live update (don't quote me on that but I am 80-90% sure).
When dealing with a running cluster the ability to change a process at
runtime is quite useful. This is why I think erlang has taken the
position of late-binding between processes but offering a type system
within a process (dialyzer).

[Karl Nilsson]

I'm not an expert in release upgrades (although currently an erlang developer by day we don't actually do live upgrades) but I do believe you ship beam files as part of it. There are other ways to change code on the fly in erlang but the fundamental unit is typically a module (i.e a beam file).

Dialyze is really just a tool to derive a degree of typing over some erlang code and has no effect on the compiled artifacts. As fez only translates from one AST to another the actually type information is mostly erased.

[Greg Young]

I will have to give a try and see how to break it :)

[Karl Nilsson]

Thank you! Feel free to report anything that breaks and I will work on that next.

[Karl Nilsson]

Hi All,

The first 0.1.0 alpha release is now out if anyone is interested in trying fez without having to use the source directly. There are generic builds for windows, osx and linux.

https://github.com/kjnilsson/fez/releases/tag/v0.1.0-alpha.1

Enjoy

Karl