[erlang-questions] Process migration/ Process state extraction/ injection

[Tyron Zerafa]

Hello everyone,

   Does Erlang support process migration? Meaning, is there a way in which I can stop a running process (A) on one node and resume it on another completely different node? 

If this is not supported, one way to go around it is to extract the process state before suspending it and somehow inject it into the new process on the remote node. Does Erlang provide such functionality?

Any tips and suggestions on how one can extract/ inject the process state from/ into processes?

- Thanks in advance

[Paul Barry]

Hi Tyron.

For my Masters thesis (way back in 2003), I looked at 'mobile agents'
as a technology area, and implemented a 'strategy' I came up with in
Perl. What I did was look at how Perl's debugger handled a Perl
"process", then I used some of the debugging techniques to query the
process state, extract & bundle it all together, then ship the code &
the state to the new host... this worked reasonably well, and I was
able to prove my point. It was/is a bit clunky, but it works! :-)

I've an old(ish) website describing all of this here:
http://glasnost.itcarlow.ie/~scooby/

So... my suggestion would be to look at how some of Erlang's debugging
tools to see if you can attain inspiration...

Good luck.

Paul.

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>



--
Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul....@itcarlow.ie
Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland.
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Anders Nygren]

This is not exactly what You are asking about but take a look at
distributed applications in the documentation,
http://www.erlang.org/doc/design_principles/distributed_applications.html

Also check the archives for "takeover", there has been some
discussions in the past about this that I think can be helpful for
You.

/Anders

[Motiejus Jakštys]

Hi,

there is no easy general way to do it. But, if you said what exactly you
are trying to achieve, there might be an other, unexpected, easy way to
do what you want.

If you gave us a higher-level description, we might help you better.

Motiejus

[Tyron Zerafa]

Hey,

    My primary intention is to implement strong mobility of processes. Let's say that process A is running on Node 1, I want to be able to suspend this process, transfer it to Node 2 and resume such process from where it halted. I believe that in order to achieve such I need to somehow preserve the stack trace, memory and other info. 

--
Best Regards,

Tyron Zerafa

[Gleb Peregud]

On Tue, Jul 31, 2012 at 8:11 PM, Tyron Zerafa <tyron....@gmail.com> wrote:
> Hey,
> My primary intention is to implement strong mobility of processes. Let's
> say that process A is running on Node 1, I want to be able to suspend this
> process, transfer it to Node 2 and resume such process from where it halted.
> I believe that in order to achieve such I need to somehow preserve the stack
> trace, memory and other info.

This task is non-trivial and will require a lot of work with ERTS and
whole Erlang VM. Things you have to handle are:
- stack
- heap
- binary refs
- monitors
- links
- ets ownership
- port ownership
- messages sent to old process pid?
- replacing old pid with new in other processes?
- replacing old process with "replay" process?

And I'm sure that those are not all details which will have to be handled.

Cheers,
Gleb

[Tyron Zerafa]

Hey again,

    I am thinking that a weaker form of such mobility might suffice for my project, but I am not sure. In this weaker form, it is enough to transfer the code and execution can restart again. I need this primary to implement the code-on-demand and remote-evaluation architectures. I do not believe that this is so complex in Erlang, in fact if I remember correctly some functionality already exists for such. 

On Tue, Jul 31, 2012 at 8:15 PM, Gleb Peregud <gleb...@gmail.com> wrote:

On Tue, Jul 31, 2012 at 8:11 PM, Tyron Zerafa <tyron....@gmail.com> wrote:
> Hey,
>     My primary intention is to implement strong mobility of processes. Let's
> say that process A is running on Node 1, I want to be able to suspend this
> process, transfer it to Node 2 and resume such process from where it halted.
> I believe that in order to achieve such I need to somehow preserve the stack
> trace, memory and other info.

This task is non-trivial and will require a lot of work with ERTS and
whole Erlang VM. Things you have to handle are:
- stack
- heap
- binary refs
- monitors
- links
- ets ownership
- port ownership
- messages sent to old process pid?
- replacing old pid with new in other processes?
- replacing old process with "replay" process?

And I'm sure that those are not all details which will have to be handled.

Cheers,
Gleb

[Ali Sabil]

Hi Tyron,

You could implement process migration yourself, I remember seeing
something like that implemented in ejabberd:
https://github.com/esl/ejabberd/blob/master/apps/ejabberd/src/p1_fsm.erl#L557

The idea is to spawn another process in the remote node, and use a
handover protocol to coordinate the migration.

Best,
Ali

[Philippe McLean]

What goals are you trying to achieve? many problems can be solved with another level of indirection, but a robust approach to error handling and process creation might give the same or better results.

[Daniel Dormont]

One of the first things I did in Erlang was implement node migration for a certain type of Ejabberd process similar to the one below, but less general and I entirely failed to implement message relaying. I had a few things going for me:

- the process was a gen_fsm, so there was no mystery about how to obtain its state at a given moment in time. In fact the module in question already had handle_sync_event capable of setting or getting the state as a whole (normally bad from a design standpoint I suppose, but it was helpful here)

- the process was created by a simple_one_for_one supervisor but had no other links or monitors

- the process did not own any ports or ETS tables

- no other processes kept direct references to it. Messages sent to it were immediately preceded by a Mnesia lookup to get its PID.

So all I had to do was spawn the process on the new node, push its gen_fsm state, update the Mnesia table to point to the new PID, and kill the old one. There was still a risk of losing messages; a relay would be the right way to handle that I think.

Dan

[Michał Piotrowski]

Hi,

I'm currently working on so-colled proc_mobility behavior. It is a prat of my thesis which aims to add agent migration to eXAT platform. proc_mobility doesn't depend on eXAT platform so can be used in whatever you want. Here is link to the repository https://github.com/michalwski/proc_mobility/tree/over_tcp. The brach "over_tcp" has more features than master. Unfortunately I haven't write any documentation so far, so feel free to ask if you think that proc_mobility can be useful.

Best regards,

Michał Piotrowski