[erlang-questions] Erlang on RumpRun Unikernel

[Neeraj Sharma]

Hi,

I did an initial port of Erlang on the RumpRun unikernel (https://github.com/rumpkernel/rumprun-packages/tree/master/erlang) in September last year. While the experience was enthralling, there were after thoughts which remained unanswered. I wonder what Erlang experts think regarding running Erlang on unikernel. RumpRun unikernel is an great project which (in my view) opened possibilities to design in some unique ways while shifting aware from the traditional operation system based design. There are some nuances like multi-threading instead of providing multi-process (no fork), but I Erlang does play nice (at least for the most part) with it. Needless to say the energy spent and my lack of understand on Erlang internals ensured that the project is suited for pet projects rather than production (or any serious use).

My choice Erlang is a bit biased primarily it being my first experience to functional programming language and a long history of working in the telecommunications industry (though not using Erlang in production as much I'd wanted to). The language is awesome for many use cases though this email is primarily looking at its play with unikernels. The language blew me away with the ease and core language features for meeting complex requirements like (though not limited to) scalablity, availability and soft-real time behavior (not to mention the VMs capability to magically load the system resources evenly) which takes a lot of effort in implementing in traditional programming languages.

My initial motivation to attempt the port was to look at role of Erlang (which pretty much does most of stuff a traditional OS+utils would provide to regular applications) in microservices architecture. In my opinion the choice of RumpRun unikernel makes a lot of sense in this respect rather than rely on traditional operating system architecture. Having said that, I would like to hear opinions from Erlang experts as to whether the marriage of Erlang with Unikernel has a bright future :)

Thanks for your time,

Neeraj

[Grzegorz Junka]

Sorry for my ignorance, how does it differ from http://erlangonxen.org/ (apart from its ability to run on bare metal without Xen)? Does rumprun support ARM processors/boards?

IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem as IoT. They look great on paper as technologies, but it's hard to find an application in which they would be useful. They might be just waiting for that great idea which will allow them to break through and blow your mind.

I could imagine an application which scales nearly linearly with the amount of nodes you allow it to use. When the capacity or speed needs to be increased, you just provision new nodes, each one a simple ARM module running Erlang, and you add it to the cluster. The application then uses the new node and distributes a bit of the running load to it.

It's not impossible from a technical point of view but it's hard to imagine why a company would want to do that instead of moving the system to a new server with more CPU/RAM. In any case it's great that such an option exists and we can use it to try things. Anyway, thanks for bringing it up, I wouldn't have known Rumprun otherwise.

Grzegorz

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Radoslaw Gruchalski]

And what would you do with the old server?

Put in the box on a shelf?

Horizontal scalability is a thing.

–
Best regards,

Radek Gruchalski

ra...@gruchalski.com
de.linkedin.com/in/radgruchalski

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.

[Neeraj Sharma]

On Wed, May 25, 2016 at 12:15 AM, Grzegorz Junka <li...@gjunka.com> wrote:
>
> Sorry for my ignorance, how does it differ from http://erlangonxen.org/ (apart from its ability to run on bare metal without Xen)? Does rumprun support ARM processors/boards?

There are a number of differences if you care about the internals,
which may or many not interest all users. Having said that any comment
I make here will have subjectivity and my bias towards RumpRun
unikernel over the Ling (aka erlangonxen) project. I will give you my
opinion and reasons for choosing RumpRun, but the official
documentation for those projects can only do justice for any detailed
comparison. Although dated I remember that Ling did a number of
customization to get Erlang to work (one of them being playing around
with the binary format beam). Additionally, the system software is to
get only Erlang to work hence highly customized. The RumpRun unikernel
on the other side is a generic library operating system which has a
very strong focus towards POSIX compliance to allow existing code to
run as much possible. Additionally, the drivers for RumpRun is from
NetBSD which does infuse confidence. The Erlang port changes very
little (for example uses the patch for running epmd in pure Erlang and
such) to run it on RumpRun. I don't know how they compare in terms of
performance, since I never benchmarked either so cannot comment on
that.

Some references for official documentation on the RumpRun and Rump
Kernels project.

* http://rumpkernel.org/
* https://github.com/rumpkernel/wiki/wiki
* https://github.com/rumpkernel/rumprun
* https://github.com/anttikantee/rumpkernel-book

>
> IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem as IoT. They look great on paper as technologies, but it's hard to find an application in which they would be useful. They might be just waiting for that great idea which will allow them to break through and blow your mind.
>
> I could imagine an application which scales nearly linearly with the amount of nodes you allow it to use. When the capacity or speed needs to be increased, you just provision new nodes, each one a simple ARM module running Erlang, and you add it to the cluster. The application then uses the new node and distributes a bit of the running load to it.
>
> It's not impossible from a technical point of view but it's hard to imagine why a company would want to do that instead of moving the system to a new server with more CPU/RAM. In any case it's great that such an option exists and we can use it to try things. Anyway, thanks for bringing it up, I wouldn't have known Rumprun otherwise.

I agree that lack of such an showcase application does make things
complicated and restrict adoption.

-Neeraj

[Eric Merritt]

Comparing this to erlangonxen is really comparing apples and oranges.
Erlangonxen is not the erlang system as put out by Ericsson running on
xen. Its a one-man reimplementation of the beam interpreter running on
xen. Whether that is good or bad is an open question, but you can' t
really compare it to what Neeraj is doing, which is the actual erlang
system running as a microkernel.

As to why a company would want it: Everything running on a server that
isn't required by the service it is running is an opportunity for
either failure or additional surface area to attack. If I can run,
just my erlang service and nothing else but what it depends on
directly and indirectly, that is a huge win for me.

[Lyn Headley]

On Tue, May 24, 2016 at 11:45 AM, Grzegorz Junka <li...@gjunka.com> wrote:

> IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem
> as IoT. They look great on paper as technologies, but it's hard to find an
> application in which they would be useful. They might be just waiting for
> that great idea which will allow them to break through and blow your mind.
>

erlang on xen has what I think is a great application: super-elastic
clouds. Imagine a web site with no server. When it gets a request, it
launches the server, and serves the request. Think how cheap this
would be.

[Grzegorz Junka]

On 24/05/2016 21:52, Lyn Headley wrote:
> On Tue, May 24, 2016 at 11:45 AM, Grzegorz Junka <li...@gjunka.com> wrote:
>
>> IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem
>> as IoT. They look great on paper as technologies, but it's hard to find an
>> application in which they would be useful. They might be just waiting for
>> that great idea which will allow them to break through and blow your mind.
>>
> erlang on xen has what I think is a great application: super-elastic
> clouds. Imagine a web site with no server. When it gets a request, it
> launches the server, and serves the request. Think how cheap this
> would be.

OK, nice idea, but where would be the web server running then?

[Radoslaw Gruchalski]

People do that today, to a certain extent, with docker / containers.

–
Best regards,

Radek Gruchalski

ra...@gruchalski.com
de.linkedin.com/in/radgruchalski

Confidentiality:
This communication is intended for the above-named person and may be confidential and/or legally privileged.
If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately.

[Richard A. O'Keefe]

On 25/05/16 8:38 AM, Eric Merritt wrote:
> As to why a company would want it: Everything running on a server that
> isn't required by the service it is running is an opportunity for
> either failure or additional surface area to attack. If I can run,
> just my erlang service and nothing else but what it depends on
> directly and indirectly, that is a huge win for me.

Also everything running on a server that isn't required is limiting the
amount
of memory available for useful work (paging is not your friend) and wasting
electric power.