[erlang-questions] Erlang in the browser

[Mike Oxford]

Forget javascript. Forget Flash. Forget java/Go/Dart/whatever.

Is anyone working on a stripped-down erlang VM to be run in a browser,
maybe as a plugin?

Download the .beam and run it instead of a script/jar.

httpc for ajax-style work, sockets already in place for websocket stuff.
Dupe the DOM and fire up V8 via port to handle LEGACY javascript sites.
Cross platform due to portability of beams, VM already compiles to all
of the significant platforms.
async actor model with lightweight processes and micro-heaps ... other
code looking to go here but Erlang already there

"fe" used for frontend, thinking that maybe it's a more extensible
name for anything "not web"

fe:update_item(TheDOM, button1, "new text") when is_dom(TheDOM). %% or
"button1" to save atoms
fe:ajax(MyCallback, URI).
etc.

It sounds like many of the building blocks are there...anyone already
moving on this?

-mox
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Jack Moffitt]

> Forget javascript.  Forget Flash.  Forget java/Go/Dart/whatever.

In the browser environment you can't forget about security. Erlang has
basically no security model to speak of, and would probably make a
poor browser language in its current form.

However, I agree with you that much of it's semantics would work very
well for the domain.

As an intermediate step, you might experiment with BEAM on NaCl. It's
limited to Chrome, but it should be enough to see if the idea has
legs.

jack.

[Daniel Dormont]

I don't use Windows much anymore, but does Erlang have an adapter to the Windows Scripting Host? If so, you could in principle support <script language="escript"> and run Erlang or more likely a modified escript directly in the browser. I get the impression this is not a recommended approach these days, and it's IE-specific obviously, but it's an amusing idea. Something more akin to Java/Flash is more likely viable.

You'd probably not want to use httpc, at least not by default. The nice thing about Java and Flash is that they can transparently use the browser's proxy settings and the browser's cookies - even if they themselves don't get permission to view those cookies directly - when making HTTP requests.

To be at all useful you'd need to interface with the DOM pretty deeply, and coming up with a library as capable as jQuery will be an exciting challenge for sure! I don't know much about Flash, but in Java the main way this is done is to just call Java from Javascript or vice versa, which might be a good short-term solution while the Erlang DOM library is being built out.

Sounds fun, at any rate.

dan

[Witold Baryluk]

On 10-25 13:36, Mike Oxford wrote:
> Forget javascript. Forget Flash. Forget java/Go/Dart/whatever.
>
> Is anyone working on a stripped-down erlang VM to be run in a browser,
> maybe as a plugin?

Yes :) Even without plugin.

>
> Download the .beam and run it instead of a script/jar.
>
> httpc for ajax-style work, sockets already in place for websocket stuff.
> Dupe the DOM and fire up V8 via port to handle LEGACY javascript sites.
> Cross platform due to portability of beams, VM already compiles to all
> of the significant platforms.
> async actor model with lightweight processes and micro-heaps ... other
> code looking to go here but Erlang already there
>
> "fe" used for frontend, thinking that maybe it's a more extensible
> name for anything "not web"
>
> fe:update_item(TheDOM, button1, "new text") when is_dom(TheDOM). %% or
> "button1" to save atoms
> fe:ajax(MyCallback, URI).
> etc.
>
> It sounds like many of the building blocks are there...anyone already
> moving on this?
>

I have working BEAM to JS translator and small VM In JS to run erlang programs.
It have a scheduler and everything, but few things are still missing.
Especially obsucre opcodes, like binary handling.

About your example I have built a small framework with very similar
syntax, which allows handling DOM from erlang running in this VM.

There is many other advantages of this model, and because I have some
time to finish this project I will publish it shortly.

Feel free to contact me.

--
Witold Baryluk

[Witold Baryluk]

On 11-27 16:12, Yurii Rashkovskii wrote:
> Hi Witold,
>
> This is very interesting. When do you think you will be publishing it?
>
> Yurii.

In fact I just pushed it to github

https://github.com/baryluk/erljs

It is mess however, and I will work in next week on it.
Just doing few other small Erlang related projects,
and want to finish them before cleaning up.

I just decided it is better to do git commit -a
and push it, than perform multiple commits
of changes code base accumulated. It is better sure
for historical reasons, but I will probably not done it in sensible time.
(anyway git doesn't track modification times, so it even
doesn't have historical value).

Most important files are

client side:
erljs.jss
erljs_datatypes.js
erljs_scheduler.js
erljs_vm.js
erljs_control.js

server side:
erljs.erl

Few changes (XHR+websocket communication with server processes,
and BEAM verifier for example, which isn't really needed,
it just helps me figure out,
if my assumptions about BEAM are correct in actual BEAM code,
but it maybe be also helpful, if would like to for example
load untrusted modules from outside) are queued to be pushed soon.

I will first move files to proper subdirectories,
and clean up testsing infrastructure (there are currently
2 separate testing systems, and common_test is not yet supported
properly).

Then I will implement fully binaries, because this is one of the most missing
feature, which needs to be done, to have most of stdlib, compiler,
etc. modules to work (most of them works on lists, but sometimes
binaries are used, and for completnes it must be done first).
Binaries are hard to implement mainly because of lots of
opcodes involving them, as well lack of built-in float<->binary
conversion in JS (I needed to build IEEE754 conforming parser/generator
for this, cool but awfull).

Then I will fix exception handling (it works, but stacktrace
is slightly different than in BEAM VM, but I would
like to have them exactly same, so I could be sure
that all functions behaves in same way both in BEAM and in erljs).

After that I will try to put all code into managable
namespace, to not polute global symbol table in JS,
(this will also simplify running multiple copies of Virtual Machine).

After that probably on-demand module loading will be done.

I will also look at Joe trick with parsing distributed binary terms
using JavaScript.

There is many things to be done on horizon, for example
utilizing WebWorkers transparently to use multiple cores
automatically.

My approch (direct interpretation of opcodes)
have some overhead, but considering how it simplifies
writing of non-blocking code in browser, and how composable it is
comparing to JavaScript or hack which tries to solve problem
(like yield, continuations, events, etc. etc), we can just use
multiple processes, and block on receive, or perform computations
without worring about webbrowser getting freezed, or
some events being queued (like button press, or delivery of
server response for other requests).

In the future, if erljs_vm.js will be complete, I would like
to squeeze beam code and big-switch over opcodes,
to make it more space efficient, and do some research on JIT compilation
from BEAM to JS (it is possible, but quite hard mainly due tail-recursion,
and non-blockignes, being both contradicting in JS and Erlang,
lack of goto in JS makes it even harder ;D - however as I said it is possible,
and will provide substantial performance gain in the future).

When developing whole erljs I do not use any BEAM machine
internals, so I do not need to use EPL license,
or whetever. I just guess how BEAM works by inspecting it output,
and correlating it with BEAM decompiler listings.
Unfortunetly it is needed because BEAM code have no official documentation,
as well Erlang have not strictly and formally defined semantics.
However, doing reverse enginering is possible, and I have
substantial progress (even phash which looks to be giving
random values, have been somehow been done).

Before rushing into code, you should wait slightly,
as I want to really clean up it first,
and clarify licensing issues of all code.

Thanks for your interest.

I have a small presentation about erljs in pdf,
and will publish it soon.

You can try some really old version (not in sync with what
is in github now), http://smp.if.uj.edu.pl/~baryluk/erljs/erljs.html

You can run there some unit tests, event tracing, do some event handling
on DOM as well use simple REPL, i.e. lists:merge([2,7,11],[1,5,13])
(expression must be of the form, module:function(literal1, literal2, ...),
it is restriction, because I do not have yet full parser on client side,
but allows lot of fun).

[Raoul Duke]

wow.

[Fredrik Svahn]

There is also http://svahne.github.com/browserl, a small script which
I wrote just for fun.

Short version: it is an Erlang emulator in javascript, which can load
and run an Erlang/OTP system from a tar file with ordinary beam files.
It can also load and execute individual beam files. It will run common
test and successfully complete some OTP test suites (the test suites
are not loaded in the example page, though). Files, binaries, ets and
floats are work in progress, at the moment they are only supported to
the extent needed to boot the system and run common test.

BR /Fredrik

[Tony Rogvall]

This is not a "small script", this is an ultra cool "small script" :-)

I like!

/Tony

"Installing applications can lead to corruption over time. Applications gradually write over each other's libraries, partial upgrades occur, user and system errors happen, and minute changes may be unnoticeable and difficult to fix"

[Joe Armstrong]

Wow - ultra interesting.

Just out of curiosity - how do you represent lists and tuples in JS?
Are they represented
as native JS objects or as objects on the beam stack and heap? And if
they are native JS objects what is the storage overhead of a
list-cell?

/Joe

[Max Bourinov]

That is very cool indeed.

Do you have any use-cases where we can benefit from it in our projects? Would be great to load ur users with some Erlang in their browsers :-)

Best regards,

Max

[Tony Rogvall]

On my wish list is distributed Erlang over web sockets.

Toss in a built in module for creating and manage HTML DOM elements. 

But a web socket "driver" could get us far enough :-)

/Tony

[Max Bourinov]

So, the idea is to run distributed Erlang in browsers over the Internet?

Erlang web-based distributed computation network. That is very intresting indeed. Don't know how to use it at the moment because I never think about using Erlang this way, but I think cool ideas will come :-)

Best regards,

Max

[Fredrik Svahn]

Hi Joe,

The Erlang list [1, 2, 3] is represented as js native objects {value:
1, next: {value: 2, next: {value: 3, next:nil}}}. The size of a string
represented as a list is 40 bytes per character in Chromium on my
x86_64 machine (I do not know if the size would be smaller on a 32 bit
machine). The corresponding number for beam is 16 bytes/char for
x86_64, I believe, so it is more wasteful as expected. I have
considered mapping it to js arrays but haven't done any measurements.
This is just a small script so I did not want to write my own GC, also
mapping to native objects is better if I later would like to make it
possible to compile performance critical beams to native javascript
code.

The Erlang tuple {1,2,3} is represented as the js array [1, 2, 3].
Binaries are mapped to js strings, and atoms to numbers.

BR /Fredrik

[Jakob Praher]

Hi Fredirk,

nice work!

Regarding representing data types. It might be worth to look at emscripten (https://github.com/kripken/emscripten).

Alon Zakai (the author) uses native memory layout for data structures. Native memory is represented as a arrays (optionally separated between primitive types) in Javascript.
According to his evaluations this is a very efficient representation technique. One could also investigate in taking the existing implementation of erl and compile it on LLVM to be used by emscripten.

For interfacing with Javascript (the downside of the data structures as native blobs) ints and floats can be used as is, but other data structures are just slieces in arrays. So it also depends on how much interaction is happening between erlang and javascript for this to be more efficient or not.

See: https://github.com/kripken/emscripten/wiki/Interacting-with-code

In preamble.js there are functions for setting and getting values on/from the heap. For instance the function intArrayFromString that shows how the conversion between a Javascript string and the heap C-like null-terminated char array is done.

Cheers,
Jakob

PS: For my resarch I have prototyped a Prolog interpreter in Javascript (for educational reasons only). In this implementation I used Javascript arrays for functors. I also thought about marking arrays as being either native arrays or functors.

[Joe Armstrong]

On Tue, Jan 31, 2012 at 11:56 AM, Fredrik Svahn <fredri...@gmail.com> wrote:
> Hi Joe,
>
> The Erlang list [1, 2, 3] is represented as js native objects {value:
> 1, next: {value: 2, next: {value: 3, next:nil}}}. The size of a string
> represented as a list is 40 bytes per character in Chromium on my
> x86_64 machine (I do not know if the size would be smaller on a 32 bit
> machine).

How do you know that it is 40 bytes/character? - in fact how do you know the
internal size of any JS object. if I say x = {abc:123, def:"xyz", ...}
etc. how big is this
object in memory - there must be some hidden pointers associated with
x, but without reading
the code for the JS emulator it would be difficult to say how large it is.

> The corresponding number for beam is 16 bytes/char for
> x86_64, I believe, so it is more wasteful as expected. I have
> considered mapping it to js arrays but haven't done any measurements.
> This is just a small script so I did not want to write my own GC, also
> mapping to native objects is better if I later would like to make it
> possible to compile performance critical beams to native javascript
> code.

If memory were a problem you'd need to write the GC and use the beam
memory model ..

/Joe

[Björn-Egil Dahlberg]

On 2012-01-31 11:56, Fredrik Svahn wrote:
> The Erlang list [1, 2, 3] is represented as js native objects {value:
> 1, next: {value: 2, next: {value: 3, next:nil}}}. The size of a string
> represented as a list is 40 bytes per character in Chromium on my
> x86_64 machine (I do not know if the size would be smaller on a 32 bit
> machine). The corresponding number for beam is 16 bytes/char for
> x86_64, I believe, so it is more wasteful as expected. I have
> considered mapping it to js arrays but haven't done any measurements.
> This is just a small script so I did not want to write my own GC, also
> mapping to native objects is better if I later would like to make it
> possible to compile performance critical beams to native javascript
> code.

Very cool project Fredrik. You obviously have too much time on your
hands right now =)

Btw, If memory is a problem, wouldn't it be better to represent lists as
some sort of (Phil Bagwell) VLists? I guess you have already thought
about it since you did measurements on it before for erlang.

// Björn-Egil

[Fredrik Svahn]

> How do you know that it is 40 bytes/character? - in fact how do you know the
> internal size of any JS object. if I say x = {abc:123, def:"xyz", ...}
> etc. how big is this
> object in memory - there must be some hidden pointers associated with
> x, but without reading
> the code for the JS emulator it would be difficult to say how large it is.

40 bytes is what the Chromium heap profiler claims that the shallow
size of the list is. The retained size for my example string
represented as a 144 character list is 5.63 KB which is more or less
exactly 144 chars * 40 bytes. A little bit of testing shows that in
Chromium the size of an object seems to be 24 bytes + 8 bytes/property
which matches the 40 bytes I saw. However I missed to take into
account that the value of the property needs to be stored as well. If
the value of the property is e.g. a javascript string the property
value and its' memory consumption is also shown in the profiler*, but
in case it is a number the size of the property is not shown. The size
of a number seems to be 16 bytes according to separate measurements.
The retained size of 144 chars represented as a JS array is 1.17 KB.

IIRC from a blog post I read about Firefox memory optimizations, the
figures above should be in the same ball park for FF.

> If memory were a problem you'd need to write the GC and use the beam
> memory model ..

An empty window in Chromium has a heap size of 5.8 Mb. With the emu
loaded it consumes 11.5 Mb which isn't bad considering 66 modules are
loaded and started in the example. If optimizing for memory is
interesting (might be for running on mobiles) I would go with
optimizing the code storage array into a typed array first.
Unfortunately this isn't supported yet on a number of browsers
(including the Android one), which is why I only use them in the
loader. Optimizing the loading time on mobile browsers would also come
higher on my list of possible optimizations.

First make it work at all, then optimize, right? ;-)

BR /Fredrik
* Strings are probably a very bad example since they might be stored
outside the heap in Chromium...