OTP noob: combining Supervisor and GenServer?

[Chris Keele]

I'm still pretty new to OTP, and I'm embarking on creating my first real OTP application.

I'm probably conflating concepts here, but I think I want a Supervisor that behaves as a GenServer too. Correct me if I'm wrong, or suggest how to implement it if I'm right? Here's my situation.

Similar to the example in Programming Elixir, I have a supervisor tree:

The worker can receive commands to perform stateful work; the stash allows the worker to be restarted with its previous state on error. I want the entire tree above to behave as a single unit consumable through other applications, for example, the entire tree should be treated as a worker in a pool.

So the pool could spin up 5 of these trees, and if any component fails aside from the main supervisor it will be restarted properly. If the main supervisor fails the pool will start a new one.

When the pool receives work, it seems like it should send it directly to my main supervisor, since it shouldn't be aware of the unit's internal infrastructure. So each supervisor should also be a genserver that passes work down to its children until it hits the actual worker, right?

I also want is to be able to kill the worker if it takes too long. This post on stackoverflow suggests doing this by having a parent start a :timer on a handle_call, and terminate the worker if the :timer returns first. In my tree this suggests the subsupervisor should have a handle_call, further reinforcing the idea that it should also be a genserver.

Is this the right way to do this? Is there a concept I'm missing? How does one properly rig up a supervisor as a genserver too? Is the following valid OTP?

defmodule My.Supervisor do

  use Supervisor.Behaviour

  use GenServer.Behaviour

  def start_link(configuration) do

    :supervisor.start_link(__MODULE__, configuration)

  end

  def init(configuration) do

    children = [

      worker(My.Worker, configuration, shutdown: :brutal_kill)

    ]

    supervise(children, strategy: :one_for_one)

  end

  def handle_call({ :work, unit_of_work }, _from, configuration) do

    # Somehow delegate to worker, run timer, etc?

    { :reply, result, configuration }

  end

end

At a glance it seems strange because 

a) There's no :genserver.start_link call

b) I have no idea if supervise returns the expected { :ok, starting_state } in init necessary for a genserver to be started

Please advise!

[José Valim]

You can't share the same module because both supervisor/genserver exactly because of b). Both require init to be implemented and they expect different results from it.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Chris Keele]

I figured as much. :/

I suppose the 'subsupervisor' should be a genserver instead. Call it a Runner. Each time it receives work it could call start_link to spin up a new Worker alongside a timer, sending it state from the stash. That feels righter. I suppose pool would start the main supervisor and get back the pid of the of the Runner to send work to.

In that setup, it feels a little strange having the Worker be a fully-fledged GenServer, when all it needs to do is invoke a single 'perform work' function and has no need to persist afterwards... All it needs to be is a function invoked in new process, that gets created in Runner's handle_call. The Runner can terminate that process early if the :timer returns first... Does that sound like an appropriate structure?

[Saša Jurić]

I didn't understand your problem completely, but this seems like a bunch of concepts put together.

Let me try to address different aspects of the problem, in hope I got the gist of your problem right:

1. Keeping state.

If I understand correctly, the only purpose of the "stash" is for the worker to put the state in it once it's done transforming the state, so that you can restore that state in case of restart, right? If so, I think ETS would be a better fit for that problem. You'd still need one owner process for the table, but that can be a high level process, possibly even a top level supervisor.

2. Pooling

If you need to do pooling, take a look at devinus poolboy (https://github.com/devinus/poolboy). Besides being a possible out of the box solution, it also may illustrate possible approaches even if the library is not a direct fit. From what I can tell, poolboy starts a supervisor from the gen_server. I also think I did similar thing somewhere in my own code once, but I'd have to look for it. The point is that supervisor shouldn't really contain any logic other than supervising. However, you can start the gen_server, and then underneath (in the gen_server init) start the supervisor. The parent gen_server would be something like the controller, or an interface to communicate with the supervisor and its children.

3. Timeouting

When you issue a :gen_server.call, you can specify the timeout. You can then catch the resulting error, and brutally kill the gen_server process using Process.exit with the reason :kill

This entire workflow is best put in the interface function inside a module responsible for the worker implementation.

[Chris Keele]

Thanks Saša, this has helped me disentangle my understanding of OTP somewhat! I'm doing a lot more reading and things are starting to come together a little. Can you recommend any other elixir librarys whose implementations I could refer to for direction?

[Sasa Juric]

On Nov 4, 2013, at 8:59 PM, Chris Keele wrote:

> Thanks Saša, this has helped me disentangle my understanding of OTP somewhat! I'm doing a lot more reading and things are starting to come together a little. Can you recommend any other elixir librarys whose implementations I could refer to for direction?

Unfortunately, nothing particular comes to mind. Perhaps something from meh's github repositories. Yurii (yrashk on github) also has some more complex libraries. It's probably best that you post a question, once you're stuck.