A bit of philosophy and some design principles

[Robert Virding]

There was a bit of discussion on the IRC about the CL module and where LFE gets its impulses from, that it doesn't try to stand on its feet. There are things in it from scheme and CL but most of it is based on Erlang and the features it provides and the type of systems you build using it.

The base of LFE rests directly on what the underlying Erlang VM provides and this determines what LFE can do and how it works. It is based on language features like modules, pattern matching, immutable data and how functions work. This is what defines LFE. I don't think that LFE tries as hard as Elixir to hide this background.

This, for example, is why LFE is a lisp-2 not a lisp-1 as it is a better fit for how Erlang handles functions. On top of this there is a set of convenience macros which were originally more scheme inspired but became more CL inspired when LFE became a lisp-2. They are just a better fit. Of course this doesn't make LFE a scheme or a CL as there are many things in both scheme and CL which LFE can't do because of the underlying Erlang VM like mutable data and the handling of nil/() and symbols with values, plists and function bindings [*].

An alternative would have been more inspired by clojure which shares some properties with Erlang. However they are fundamentally different in many ways so it would mainly have been using clojure's naming conventions. Also clojure's concurrency model is very different from Erlang/LFE so its way of building systems is also very different. You get the feeling, at least I do, that it is based on a central thread of execution where we you can run things in parallel, but there is this central thread. This is very un-Erlangy as Erlang/LFE systems typically don't have a central thread.

This gets us, finally, to the CL module. It is just a library containing many of the standard CL library functions which gives you the possibility of writing in a CL style. Not everything can be included, for example the nXXXX functions which mutate data, and some features don't mesh well with Erlang/LFE. For example equating nil/() and predicates which in Erlang/LFE return true/false while in CL are truthy and return nil/() and anything else [**]. It is in no way a fundamental part of LFE and is just an add-on. If anyone feels inclined to do a similar module for clojure then I will definitely consider including it. Again it would just be an add-on. These could easily be included in the base release.

Packages like flavors, which I did because they are interesting and I think fun, should probably not be part of the base. This wou|d also apply to things like LFE CLOS (if anyone decided to do it) and LFE clojure-like protocols. They are interesting and useful in themselves but not things I would consider part of the LFE base. A set of these should probably kept in a standard place to make them easily accessible for everyone. I would like to keep the base relatively simple, clean and "basic", a "lean, mean, fighting machine" if you will. It is all too easy to add things, even sensible and useful things, and end up with a bloated mess. I really want to avoid this, hence keeping the base simple and clean.

In one sense we don't really have an "LFE machine" when we run LFE in that we need to start things up. From the Erlang POV it is almost just a set of libaries. Flavors work like this, you just need to the flavors libraries to use and no servers or such are started. This limits what you can do but makes it very easy to include LFE, you don't need to start it up. I don't know if this is the way we should go, maybe we should have an "LFE machine". Ideas around this very welcome.

That's about all for now. It has already become much longer than originally planned and if I don't stop now I will never get it out.

Please comment!

Robert

[*] Though I have worked out a beautiful/ugly/cunning hack how we could do some of this.
[**] The boolean issue has not yet been resolved.

[Feng Hou]

Robert, great write-up and thanks for sharing your thoughts here!

I'd like to share a few comments from a newcomer's perspective. 

First, I'd like to say LFE community should encourage usage of library *implementation* by Elixir community and their ecosystem. Make it as seamless as possible from LFE. For example, Elixir has a nice Stream library for lazy list and file io, lots of consistent functions working with |> macro (which LFE already borrowed from clojure as -> ). However, right now in order to use those Elixir functions we need to quote long module name like 'Elixir.Stream... they clutter the code, and lose the conciseness. Ideally, I'd like to just do (import (prefix Elixir.Stream stream)), but it's NYI as of now :( Robert, any plan to implement it :)

A good side effect by doing that, I hope, LFE community (esp. leaders like you and Duncan) sends a message to Elixir community that LFE *wants* to leverage and share ecosystem mutually (LFE already does an excellent job w.r.t vanilla Erlang/OTP, hence I'm emphasizing Elixir here because somehow Elixir managed to gain huge mindshare in such short period).

On the flip side, when LFE borrows good ideas from either CL or Clojure, and implement them as libraries. We should also encourage Elixir community to use them, build stuff on top of them. By doing that, maybe even drag 10% of them to LFE :) here I'd like to add my 2c in terms of CL/Clojure influence to library design and implementation. First, I'd like to see function/data interface to be more consistent with Erlang/Elixir culture/conversion (for example, proplist as list of 2-tuples, not CL list of cons). Second, Keep macro minimum where it's absolutely necessary so that Elixir just port a thin macro layer but share the same function/data substrate. Last, maybe even avoid hyphens as in CL/Clojure tradition since they have to be single quoted atoms on Elixir/Erlang side.

Not sure these make sense or not. by hey it's all about participation :)

Regards,

Feng

[H Durer]

Thanks Robert for that perspective on the history and philosophy of lfe.
I don't actually think we are that far apart in our thinking.

From what I see where we maybe differ is
1. I'd prefer to get rid of the cruft collected during the history and generally offer one way of doing things rather than multiple
2. I'd be far more happy to not follow the old lisps (scheme and cl) and instead go our own lfe way and if in doubt rather borrow from modern lisps like clojure.

Simplicity is important to make learning the language easier.  Yes, you yourself only need to learn one variant but if you want to read other people's code you suddenly have to know both progn and begin, mref and map-get, ...

Following an existing lisp helps if the learner can then transfer knowledge from the other language.  But since neither cl nor scheme match well that doesn't really happen.
cl is great -- but the good bits are clos, generic functions, keywords etc., i.e. all the stuff we don't get. What is old and crufty, the "secret handshake language" of car, cdr and their friends, the extra parens in let, etc. -- all that is what we get.

I am sure I don't understand all the decision that went into it, all I can say is that to me as basically an outsider coming in to lfe, the language feels weird at times.


BTW: While thinking about this, I tried to see what there actually is in syntax in lfe and compiled a list of predefined things I could find: https://gist.github.com/hdurer/b2850329e3d444ce2df8  The comments there maybe help understand where my problems are.


Also: A big +1 to looking towards the Elixir community.  I don't have good visibility of the ecosystem but to me it certainly seems like Elixir is the big thing in Erlang land that if not bigger yet is likely to eclipse Erlang itself.  Support to make use of their modules would be quite interesting.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.
To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at http://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

[Feng Hou]

lfe (import (from (..) (rename ...)) already supports calling elixir library quite nicely, only that doesn't work with hot module reloading, also add some boilerplate code in defmodule clause. I hope (import (mod prefix ...)) will resolve these minor inconvenience. 

I actually don't mind car, cdr as long as they are mostly for implementing macros. But it's bad for lfe if large part of any lfe library api are macros. or some CL style data structures leak out module public functions. They create barriers to reuse and compose libraries across languages (Elixir has macros, but not sure they are as powerful as lfe). 

Imagine Robert's Flavors becomes lfe's killer library that attracts adoption and contribution from elixir community...how cool is that!

[arp...@cryptolab.net]

On 2015-10-21 21:54, Feng Hou wrote:
> lfe (import (from (..) (rename ...)) already supports calling elixir
> library quite nicely, only that doesn't work with hot module reloading,
> also add some boilerplate code in defmodule clause. I hope (import (mod
> prefix ...)) will resolve these minor inconvenience.

This is wonderful, thanks for sharing that. It opens a whole new game
for me :3

> I actually don't mind car, cdr as long as they are mostly for
> implementing
> macros. But it's bad for lfe if large part of any lfe library api are
> macros. or some CL style data structures leak out module public
> functions.
> They create barriers to reuse and compose libraries across languages
> (Elixir has macros, but not sure they are as powerful as lfe).

One thing I wanted to mention is that as far as I know, Elixir macros
are
hygenic while LFE's are not (according to the guidelines). I might be
very
wrong though. I agree we should clean up all the missing pieces
according
to the documentation to match the game and be able to do wonders at
compile/run time.

[arp...@cryptolab.net]

On 2015-10-19 20:03, Robert Virding wrote:
> There was a bit of discussion on the IRC about the CL module and where
> LFE
> gets its impulses from, that it doesn't try to stand on its feet. There
> are
> things in it from scheme and CL but most of it is based on Erlang and
> the
> features it provides and the type of systems you build using it.
>

> [...]

>
> That's about all for now. It has already become much longer than
> originally
> planned and if I don't stop now I will never get it out.
>
> Please comment!

This is what I needed to understand the rationale and properly take a
stand
about what needs to be in the LFE codebase and what would be "cool to
have".

Thanks for being clear!