LFE-based Web Development

[Robert Herman]

I am a total beginner with LFE with some experience in Scheme and Lisps. I do not make my living coding, so I would like to put together and learn a platform for future (fun & educational) development, with the far chance of employment.
I have a Linode that I would like to setup my main blog, a game server, and some other side projects.
Elixir / Phoenix would be my choice for the backend, if it weren't for my love of Lisp. I cannot seem to find any examples or recommendations for such an ecosystem with LFE. I thinking LFE/Cowboy/Postgres, and Exemplar (or mabye the Elm language for the front end HTML, CSS and JS stuff).
I would like to host a 'world' for educational gaming to take place. I am not sure in what or how that will be written, but I do know concurrency is a must and robustness for a smooth end user experience.
Given this is my toy project, what suggestions would you have for the components of this general setup? LFE is a priority for me! Thanks.

Rob

[Duncan McGreggor]

Hey Robert, great to hear from you -- and even better to hear how intrigued you are with LFE :-)

There is a sample web app for LFE + YAWS + Exemplar + Bootstrap + Docker here:
 * https://github.com/oubiwann/docker-lfe-yaws-sample-app

In short, if you have Docker installed on your system, you can run this command right now:
 * docker run -p 5099:5099 -t oubiwann/lfe-yaws-sample-app:latest

And then open up http://localhost:5099/ in your browser and see the Bootstrap front end served by YAWS, written in LFE. There was a blog post about this in November (based upon an older blog post with more details):
 * http://blog.lfe.io/tutorials/2015/11/28/2110-lfe-yaws-docker-update/

Note, however, here there be dragons: there is *nothing* polished about that sample app.

The LFE community doesn't have such a solution for the web like Elixir's Phoenix. It would be nice, though :-) Robert Virding and I have discussed this over the course of the past few years. More recently, the Phoenix project rekindled this topic ... but it's a big one and there's a lot to do.

There are several in-progress LFE projects that would support an effort like this:

 * lroutes (a refactoring of lfest for composable routes, a la Clojure's Compojure)

 * a series of lmug projects for composable middleware that are web-server-agnostic, a la Clojure's Ring (similar perhaps to Elixir/Phoenix's "Plug")

 * a ledis (Redis) client which is a lot of fun to use for web projects (I've done a couple myself)

Other than ledis, the other projects aren't really ready for use. If you really want to use LFE for a web project, I'd focus on YAWS + lfest + Exemplar for now. I think there are examples of other web servers having been used to create LFE web apps. One or two might be somewhere in the lfex github org ...

If you end up enjoying hacking on LFE + lfest, etc., to the extent that it inspires you to do more with LFE + web, and you would also like to contribute to a web framework for LFE (even if just in the form of submitting ideas), you can start by creating tickets here:
 * https://github.com/lfex/dragon

So that you're aware ahead of time, here are some opinions I hold about web frameworks :-)

 * I should be able to write an extremely complicated web application just as easily as a simple app -- in other words, a framework should not be so "specialized" or opinionated as to prevent me from doing one or the other. Generally, this means I like web "frameworks" that are very simple and can be essentially treated more like libraries.

 * The Clojure community has gotten this as close to perfect as I've ever seen: Ring. Ring also happens to be one of the best implementations of a web middleware framework I've ever seen -- strongly FP and very, very simple.

 * MVC is not a good fit for the web.
 * "Event-driven" frameworks and "Reactive" frameworks don't need to be emulated for LFE web apps; Erlang/OTP (and thus LFE/OTP) already have a solution for many web use cases: behaviours and "green" callbacks.

 * After years of painful experience, some time ago I came to the conclusion that ORMs are a terrible idea. Ultimately they make a project harder to maintain and tend to seriously degrade application performance at the data layer (for dubious "developer" benefit).

 * For many of the above bullet points, it's better to use general best practice software engineering and project/source code organization to keep a web application's UI, data access, and behaviours loosely coupled -- better to provide instructions on how to think about these than to mandate a shoehorned solution.

 * I have nothing against templating :-) Exemplar has suited my needs, but there are lots of improvements it could get and it definitely doesn't provide the ease of UI reuse that template "engines" tend to do. Still, some clever hacks/functions/macros might get it there ...

 * Rather than tout a project's feature "secret sauce" and marketing spin, I'd rather tout the *language* features provided by LFE/OTP and any libraries that make building web apps as consistent and as painless as possible. However, providing a collection of go-to libraries, tutorials, sample web apps, and best practices under the umbrella of a "web framework" is probably a pretty good idea.

That might be a nice rough draft for the Dragon manifesto ;-)

Hope this helps, or is at the very least entertaining!

d

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.
To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

[Lorenzo Moriondo]

Hi Rob,
I am trying to do the same. Starting with Elixir and building a edu-gaming platform (I start from some years of Python developing) as oet project. If you are interested to collaborate, I would be glad to share my ideas and listen to yours.
Big fun.
Bye.

[Eric Bailey]

I’ve grown quite fond of elli (https://github.com/knutin/elli) lately. It takes a bit of a different approach than other Erlang webservers, but I think I prefer it.

If you’re looking to use Cowboy with LFE, I’ve ported the examples here: http://quasiquoting.org/lfe-cowboy-examples

As Duncan mentioned, Exemplar is great for generating HTML and can be used with any web framework (YAWS, Cowboy, Elli, etc). His email is a gold mine of links/info, too.

Eric

[Duncan McGreggor]

Quick follow-up: I'm currently working on a minimal LFE web app howto for starting from zero.

Eric's going to look into making a rebar3 plugin that codifies this.

More soon,

d

[Robert Herman]

Thanks Duncan and everyone else!

Duncan, looking forward to the 'starting from zero' write-up! I sure could use it.

Eric, elli looks interesting (Game analyics), and I will check out your Cowboy examples today.

Lorenzo, are you going to do it in LFE vs. Elixir? I have sort of put a bar on learning Elixir for now, so I can focus on LFE. I think there will be more jobs in Elixir/Phoenix, but I am not seeking employment at the moment. I'd like to hear what you are trying to implement. I am starting small, very small. I moved to a village in Java, Indonesia and wound up teaching the local children. Then creating some apps for their phones. The idea is tie my apps to a server (Linode in Japan), that can allow them to play and learn together online, and eventually branch out to other villages and towns. Basic maths, English, music and art. Everyone has a smart phone even if half of the houses have dirt floors around me!

I am reading Gene Sher's 'Handbook of Neuroevolution Through Erlang', but I am not an Erlang programmer (yet). I was thinking I would try and duplicate the Erlang programs in LFE, sort of what Corey Haines has done in Erlang and Elixir a few years ago: https://github.com/coreyhaines/handbook-of-neuroevolution to better learn the language.

Rob

On Tuesday, March 8, 2016 at 6:40:39 AM UTC+7, Duncan McGreggor wrote:

Quick follow-up: I'm currently working on a minimal LFE web app howto for starting from zero.

Eric's going to look into making a rebar3 plugin that codifies this.

More soon,

d

On Mon, Mar 7, 2016 at 1:16 PM, Eric Bailey <ifsixw...@gmail.com> wrote:

I’ve grown quite fond of elli (https://github.com/knutin/elli) lately. It takes a bit of a different approach than other Erlang webservers, but I think I prefer it.

If you’re looking to use Cowboy with LFE, I’ve ported the examples here: http://quasiquoting.org/lfe-cowboy-examples

As Duncan mentioned, Exemplar is great for generating HTML and can be used with any web framework (YAWS, Cowboy, Elli, etc). His email is a gold mine of links/info, too.

Eric

On Mar 3, 2016, at 11:24 PM, Robert Herman <rpjh...@gmail.com> wrote:

I am a total beginner with LFE with some experience in Scheme and Lisps. I do not make my living coding, so I would like to put together and learn a platform for future (fun & educational) development, with the far chance of employment.
I have a Linode that I would like to setup my main blog, a game server, and some other side projects.
Elixir / Phoenix would be my choice for the backend, if it weren't for my love of Lisp. I cannot seem to find any examples or recommendations for such an ecosystem with LFE. I thinking LFE/Cowboy/Postgres, and Exemplar (or mabye the Elm language for the front end HTML, CSS and JS stuff).
I would like to host a 'world' for educational gaming to take place. I am not sure in what or how that will be written, but I do know concurrency is a must and robustness for a smooth end user experience.
Given this is my toy project, what suggestions would you have for the components of this general setup? LFE is a priority for me! Thanks.

Rob

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.

[Robert Herman]

I am assuming 'oet' is supposed to be 'pet' project?

[Robert Herman]

Duncan,

I just noticed, and maybe I saw and forgot before, that you do have a directory on Github for neural-lfe based upon Sher's book I mention below. Are you working on that? I'd hate to duplicate it, but I would like to see how you would do it in LFE!

Rob

[Duncan McGreggor]

Oh, I meant to email you back about that -- thanks for the reminder!

Yeah, I'm *really* interested in neural simulations (and AI in general) -- and this book caught my attention when it came out, needless to say :-) Robert Virding was doing some reading from it and we'd talked about converting some of the examples over to LFE. It's one of those projects that got back-burnered :-)

That being said, as soon as I read your email I had thought we should make a Github team ... so let me do that.

[minutes passing ...]

Okay, I've created an LFE AI Team on github (in the lfex org). I've also moved that repo over there from my personal org. I never pushed code up to it, but I *think* I have something on a workstation at home -- I'll have to look that up when I'm there tonight.

Robert (Herman): what's your github user id? I'll add you to the team...

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

[Duncan McGreggor]

Another update:

 * Got a basic into page done last night, but didn't get to the actual tutorial

 * Part of the intro page was listing a batteries-included DB solution ... I decided to go with Mnesia for now: thus began a yak-shaving fest

    - I updated the mnesia-tutorial repo, bringing all the deps up to the latest

    - started planning a mneisa and qlc wrapper for more intuitive use by web devs

    - updated the mnesia-tutorial docs and tried to push, running into compatibility issues with Github's latest gh-pages release (and migration to Jekyll 3)

    - this lead down the rabbithole of our current docs situation (see the other thread) and the plans we're making for a Big Update; I've now had to re-evaluate the use of gh-pages, etc.)

    - after getting the mnesia-tutorial updates published, I created a mnesia wrapper library and project and pushed it

 * More yak to come:

    - will finish mnesia-tutorial

    - will create a copy of that tutorial for the new wrapper

    - will add this wrapper as a component of Dragon

    - will finish "Getting Started" doc for Dragon

[Robert Herman]

My github user id is rpherman

Thanks, Duncan! Like I've written, I was looking forward to seeing your code, since I am but a beginner in LFE (and Erlang for the most part). I hope to be able to lower my learn/contribute ratio as quickly as possible with the help of the LFE community. I was reading about neural networks in the 1990s over my head, but I stayed with it, and did the exercises in Common Lisp back then. Mark Watson's book 'Common LISP Modules: Artificial Intelligence in the Era of Neural Networks and Chaos Theory' back in 1991 started it all for me. I have to go dig out of storage, since I see it going for over $85 US on Amazon now. Mark replied to me on HN that he found one cheaper before the price craze, since he lost his original manuscript! It might be easier for me to do those exercises than Sher's book for now.

Is there a mission statement for LFE? I know it is Robert's project, but is it going to be stable for production if it is already not there? I am not a coder by trade, but it motivates me knowing my knowledge could be flipped into a job some day.

Rob

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-erlang+unsub...@googlegroups.com.

[Duncan McGreggor]

On Wed, Mar 9, 2016 at 8:26 PM, Robert Herman <rpjh...@gmail.com> wrote:

My github user id is rpherman

Thanks, Duncan! Like I've written, I was looking forward to seeing your code, since I am but a beginner in LFE (and Erlang for the most part). I hope to be able to lower my learn/contribute ratio as quickly as possible with the help of the LFE community.

We'll do whatever we can to help :-)

One thing I like to tell people when they come to LFE or Erlang is that the classic "hello world" example (in LFE's case, (io:format "Hello, World!") is not actually accurate in the spirit of what those examples attempt to do: show the basic syntax and usage of a language. I content that the "basic syntax and usage" of LFE is inseparable from OTP -- if you're not using LFE (or Erlang) for distribution, it's like paying for a Lamborghini and just leaving it in the garage :-)

A "real" hello-world in LFE would be a gen_server that accepts a "hello" message and return "world" (for example). Maybe it's time I did a blog post on this ...

No overkill or anything (e.g., not implementing the full behaviour of gen_server), rather to give a sense of "this is how the language works." In other words "this is a very different world from most languages" and by extension for LFE, most Lisps.

 

I was reading about neural networks in the 1990s over my head, but I stayed with it, and did the exercises in Common Lisp back then. Mark Watson's book 'Common LISP Modules: Artificial Intelligence in the Era of Neural Networks and Chaos Theory' back in 1991 started it all for me. I have to go dig out of storage, since I see it going for over $85 US on Amazon now. Mark replied to me on HN that he found one cheaper before the price craze, since he lost his original manuscript! It might be easier for me to do those exercises than Sher's book for now.

If there are others in the LFE community interested in working on this (such as yourself!) I will be much more inclined to give this some time. There is a factorial relationship between fun and the number of participants (we'll, maybe for small values of participants ... !)

 

Is there a mission statement for LFE?

Yes! Mission Trek Impossible :-) "Our mission (should we choose to accept it (and we do)) is to explore a new Lisp, go where no Lisps have gone before."

I know it is Robert's project, but is it going to be stable for production if it is already not there?

There are several startups that have used/are using LFE in production. I've used it at various companies in production. It compiles to .beam, and is 100% Erlang Core compatible, so as much as Erlang is production-ready, so is LFE.

 

I am not a coder by trade, but it motivates me knowing my knowledge could be flipped into a job some day.

Yeah :-) I've been hired both 1) to do LFE, and 2) because I knew LFE and those skills were needed (and appreciated).

 

d

Rob

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

To post to this group, send email to lisp-flavo...@googlegroups.com.
Visit this group at https://groups.google.com/group/lisp-flavoured-erlang.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

[Duncan McGreggor]

Tiny correction:

On Thu, Mar 10, 2016 at 12:57 PM, Duncan McGreggor <dun...@mcgreggor.org> wrote:

On Wed, Mar 9, 2016 at 8:26 PM, Robert Herman <rpjh...@gmail.com> wrote:

 

I am not a coder by trade, but it motivates me knowing my knowledge could be flipped into a job some day.

Yeah :-) I've been hired both 1) to do LFE, and 2) because I knew LFE and those skills were needed (and appreciated).

Because *they* knew -- not me! 

This is significant, since there is a growing deep appreciation for the power and effectiveness of Erlang/Elixir/LFE at solving particular problems at scale and reliability ... and general experiences in solving these problems are in high demand, even if the tech is not the same as used when we solved them with BEAM languages.

d

[Robert Virding]

Just some short comments.

On Thursday, 10 March 2016 03:26:21 UTC+1, Robert Herman wrote:

My github user id is rpherman

Is there a mission statement for LFE? I know it is Robert's project, but is it going to be stable for production if it is already not there? I am not a coder by trade, but it motivates me knowing my knowledge could be flipped into a job some day.

My mission statement is probably what I say in some of the LFE presentations:

- LFE is a proper lisp based on the features and limitations of the Erlang VM

- LFE coexists seamlessly with vanilla Erlang and OTP

- Runs on the standard Erlang VM

It is production quality and is stable enough for serious production use and has been for quite a while. The only reason it isn't way past 1.0 is that I suffer from a Jag Ska Bara[*] syndrome (I'm just going to ...) but I'm working on it. :-) I will continue supporting and working with the core system and some core libraries for the foreseeable future. Others, including Duncan, are working on libraries and tools for it so there is an active community.

I think an important thing here is what Duncan was hinting at: if you have worked with one of the languages in the erlang eco-system, LFE/erlang/elixir, then crossing over and using another is relatively simple. I would the hardest step is getting used to building system in this environment as it is different from what many are used to. It is this way of system building systems which is the reason behind the highly concurrent, fault tolerant, and scalable systems for which Erlang is renown. So in this sense LFE is "safe".

Robert

[*] This is the classic swedish tv cartoon series "Alfons Åberg" where in this case Alfons is "jag ska bara" before he is ready for day-care. The whole cartoon series is wonderful but only in swedish as fas as I know.

[Robert Virding]

There were a lot of blank lines there which just appeared.

[Duncan McGreggor]

Oh, that cartoon was fantastic :-) We just watched it ...

"Jaaaag ska baaaara!"

(And, I'm sorry Robert, I just don't believe that you have that much LFE tape wrapped around you ;-) I think you have that much LFE lined up like railroad tracks, though ... speaking of which ...).

--
You received this message because you are subscribed to the Google Groups "Lisp Flavoured Erlang" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lisp-flavoured-e...@googlegroups.com.

[Unix One]

On 03/10/2016 10:57 AM, Duncan McGreggor wrote:
> On Wed, Mar 9, 2016 at 8:26 PM, Robert Herman <rpjh...@gmail.com
> <mailto:rpjh...@gmail.com>> wrote:
> I was reading about neural networks in the 1990s over my head, but I
> stayed with it, and did the exercises in Common Lisp back then. Mark
> Watson's book 'Common LISP Modules: Artificial Intelligence in the
> Era of Neural Networks and Chaos Theory' back in 1991 started it all
> for me. I have to go dig out of storage, since I see it going for
> over $85 US on Amazon now. Mark replied to me on HN that he found
> one cheaper before the price craze, since he lost his original
> manuscript! It might be easier for me to do those exercises than
> Sher's book for now.
>
>
> If there are others in the LFE community interested in working on this
> (such as yourself!) I will be much more inclined to give this some time.
> There is a factorial relationship between fun and the number of
> participants (we'll, maybe for small values of participants ... !)
>

Hello, I just wanted to chime in that I'm very interested in this topic,
but I'm afraid I'm not much help right now. I'm relatively comfortable
with Erlang/OTP, but a complete newbie in LISP/AI/NN. I'm currently
chewing through some introductory LISP material, then Peter Norvig's
"Paradigms of AI Programming" is up next, before I start applying all of
this to NN and LFE.

IOW, I've got long ways to go, unless anyone suggests a more efficient
route.

[Duncan McGreggor]

Cool! I think that's a great route, btw :-) PAIP is one of my all-time favourite Lisp books. Probably my second-favourite AI book (his AIMA being the Rigveda of all AI books ...)

Btw, it's pretty awesome hearing about the AI stuff that folks want to do on LFE ... this is getting me pretty excited :-)

d

[Unix One]

On 03/11/2016 08:07 AM, Duncan McGreggor wrote:
> Cool! I think that's a great route, btw :-) PAIP is one of my all-time
> favourite Lisp books. Probably my second-favourite AI book (his AIMA
> being the Rigveda of all AI books ...)

Great, thanks for validating the plan.

[Duncan McGreggor]

For those who are interested in this, there are a slew of new Dragon milestones added to the project. Dates are rough, and subject to change as contributors join the project.

[Robert Herman]

That's all good, and thanks. Now, I need to learn me some LFE! I've updated my dual boot linux notebook, so I can put Windows issues aside for now.

Rob

[Robert Herman]

You had me at the Lisp in LFE ;)

Good enough for me, Robert, thanks!