supervision and auto restarted actors

[Julien Ammous]

First I have to say the current state of the documentation just made me stop using it altogether, that's one reason I hate having documentation separated from the codebase (wiki here),
if not carefully taken care of it almost always ends up end up completely disconnected from the code itself and you have no idea what is still true and what is not which is in my opinion even worse
than having no documentation :(

Anyway I am trying to find out how to spawn an actor which will be restarted on errors, after some exploration I came up with: https://gist.github.com/schmurfy/d3ba8cdbc4c1f943cdb2#file-supervised_actor-rb

What I expected was a new actor to be spawned and hello printed again but what I get is just:
[:create]
[:hello]

What did I do wrong ?

[Petr Chalupa]

I am really sorry about the current state of the documentation. We know about it and we are trying to improve it, but unfortunately there is not always enough time :/ But I finally have time today so I'll try to improve at least actor documentation. 

Yeah we agreed some time ago to move the wiki to the code but we are still in the process of migrating.

I've posted a comment there https://gist.github.com/schmurfy/d3ba8cdbc4c1f943cdb2#comment-1326340, I hope it helps.

--
You received this message because you are subscribed to the Google Groups "Concurrent Ruby" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concurrent-ru...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Schmurfy]

Thanks :)

This project is really great but after keeping it aside for a couple of months I am now completely lost and having to dig into the code each time I want to play with a construct instead of working on what I would actually want to build is frustrating, it will be much better once the documentation is updated ^^

I have to wrap my head around some of the constructs but I still think this gem has a great potential :)

Last time I checked the actress was being merged but since then I have only followed from afar, I am curious about the current state of supervision trees, is there a way to implement them or is there just a one level supervision for now ? (I mean actors spawning supervised actors which in turn might spawn supervised actors with different strategies on errors)

[Petr Chalupa]

Hello Julien,

thanks for the question about the supervision trees! It turned out that I failed to provide key events to support building it manually (as I intended before I provide behavior for it in the gem). So this PR https://github.com/ruby-concurrency/concurrent-ruby/pull/180 contains fixes to allow the supervision-trees and here you can find example https://github.com/ruby-concurrency/concurrent-ruby/blob/actress/doc/actor/supervision_tree.out.rb (part of the same PR). Hoping that helps and thanks again for letting me know.

Petr

On Thu, Oct 30, 2014 at 2:29 PM, Schmurfy <schm...@gmail.com> wrote:

After your anwser I tried to play with supervision, what I wanted to achieve is this:

I have one actor called "master" which will get spawned first with "supervise: true" so it is restarted on failure, this part works, then I want to add a bunch of actors which will be spawned after the master but need to be linked to it and restarted if the master is (maybe after a crash).

What I tried is to pass the master reference as an argument when spawning the other actors and in their initialize method to link to the master with "master << :link", it works but they receive the :reset message in their on_message() method instead of being restarted, I tried with :supervise but with the same result.

How can I achieve that ?

I added my current test file to the gist: https://gist.github.com/schmurfy/d3ba8cdbc4c1f943cdb2#file-linking-rb

On 30 October 2014 08:48, Petr Chalupa <chalu...@post.cz> wrote:

Hi, about supervision trees: they are not fully baked e.q. a failing parent will not pause all its children. But you can build multiple levels of supervising and add what is need for your use-case in the actor or by adding new Behavior. I would like to fix this for v0.8.

[Schmurfy]

Nice to see this working but that's not exactly what I was trying to do, in my scenario the listener actors are spawned outside of the master and I wanted to "attach" them to the master so they would be restarted when needed.

The idea was that the master is spawned on application start but listener(s) can join in later, I suppose I could implement a specific message telling the master to spawn the child but if the listener can just tell the master to supervise it feels less convoluted.

PS: I like what the idea of auto generated results in the examples, that's a nice touch :)

[Petr Chalupa]

Thanks for your kind words :) 

​Ah, so in that​ case master could updated to accept listeners he should supervise:

```ruby

class Master

  # ...

  def on_message(msg)

    case msg

    when Concurrent::Actor::Reference

      msg << :supervise

      # ...

    end

  end

end

```

then it can be sent to master

```ruby

external_listener = Listener.spawn(name: 'external_listener')

master << external_listener

```

and it'll become supervised. It won't restart automatically (in the example) though, some code needs to be added to keep the reference and to restart the supervised listener which is not an child in `on_event` method. 

I'll add an array of supervised actors to the Supervising behaviour soon.

Petr

[Schmurfy]

your suggestion would work, when you say it won't restart automatically is it something that will become possible with your changes or am I doing something that is outside the scope of library ?

The need to spawn the child inside the master "context" seems needlessly limiting if mandatory.

[Petr Chalupa]

Restarting supervised actor (child or not) is possible in the `actress` branch. In both cases they need to be restarted manually form `on_event` method. I would like to provide some reasonable default behavior for both cases in future (v0.8). The actor does not have to be child of the superviser but the message `:supervise` has to come from the supervising actor right now, since the superviser actor is determined by `envelope.sender` in the supervised actor. I hope it helps.

[Schmurfy]

it helps, and the fact that the code is actually readable helps too (after celluloid that's certainly a breeze), I really like the simple design choices and it works well.

How does linking works with the actor implementation ?

I played a little with Erlang some time ago and if I remember well any actor (in Erlang terms) can decide to link itself to another one and handle the broadcasted event as needed with one of the options being to die when the linked actor died.

Maybe what I am looking is closer to that than supervision although they are related.

In my case I could spawn both the master and the listeners as supervised actors directly (if I get that correctly they will be supervised by an anonymous global supervisor) and then the listener could link itself to the master to receive its events and then restart when the master does, does it make sense and if yes is it possible that way ?

[Petr Chalupa]

I've polished only a little bit, see https://gist.github.com/schmurfy/d3ba8cdbc4c1f943cdb2#comment-1335847

On Tue, Nov 11, 2014 at 2:19 PM, Schmurfy <schm...@gmail.com> wrote:

my bad, my last attempt was 2 days ago and I gave up unable to achieve it but right now while trying to build a gist I managed to do it as it looks :p

I would appreciate if you could have a look to see if it could be done better: https://gist.github.com/schmurfy/d3ba8cdbc4c1f943cdb2#file-linking-rb

On 11 November 2014 13:40, Petr Chalupa <chalu...@post.cz> wrote:

Do you have a gist pls?

On Tue, Nov 11, 2014 at 11:31 AM, Schmurfy <schm...@gmail.com> wrote:

I made some attempts but I still fails at creating this:

the master is spawned as supervised (by the root actor)

each listener is spawned as supervised (by the root actor) and are given a reference to the master to link to it

if the master dies for whatever reason the listener should be restarted too and link again to it.

If I get your previous answer correctly this is possible but I just can't figure out how, none of my attempts resulted in the listeners being restarted.

My idea behind this is that listener subscribe to specific messages so if the master (which sends them the messages) dies they should be restarted to subscribe again to ensure they receive the messages, in that case I could possibly do it differently but that's also a good exercise.

On 6 November 2014 10:01, Petr Chalupa <chalu...@post.cz> wrote:

Thanks! I am glad you like it and I appreciate your nice words.

The linking works as you've described. Linked actor gets the events and can react accordingly. Supervisor is linked and on top of that it tells the supervised actor what to do on error. Supervised actor pauses on error and waits for message from supervisor, it can be :terminate!, :resume!, :reset!, or :restart!.

Yep that makes sense and it's possible. The global actor is Actor.root and it resets any supervised actor. This is not configurable, to use different strategy an user has to spawn and use different supervisor.

Petr

[Schmurfy]

thanks :)

[Petr Chalupa]

This was very good discussion, thanks. I try to extract some of it to actor FAQ.