I am building an Erlang based system that is composed of several
applications. In principle, the system consists of a main application
and a number of "plug in applications". The latter have modules
implementing behaviours defined by the main application. In this
particular case, should I have all the "plug in applications" as
library applications, and access them from the main application based
on some explicit configuration? Or should all the applications have
its own application modules starting own supervisors and then
registering self to the main application? The main thing confusing me
is the recommendation to have single supervision tree for the entire
system.
> I am building an Erlang based system that is composed of several
> applications. In principle, the system consists of a main application
> and a number of "plug in applications". The latter have modules
> implementing behaviours defined by the main application. In this
> particular case, should I have all the "plug in applications" as
> library applications, and access them from the main application based
> on some explicit configuration? Or should all the applications have
> its own application modules starting own supervisors and then
> registering self to the main application? The main thing confusing me
> is the recommendation to have single supervision tree for the entire
> system.
There are good reasons for taking either approach, but if you're in
doubt, I'd suggest the second option: each plugin is an "active"
application (i.e. specifies a mod in the .app file) that registers
itself as needed with the main application.
The advantage to this approach is that you're further separating the
components of your system:
- The main application doesn't have to know about the plugins at
startup, which simplifies its configuration and removes the "wiring"
logic that it would otherwise need
- Plugins can be added and removed by adding and removing OTP
applications in the release (i.e. you use boot files to specify which
plugins are run for a given Erlang process)
The OTP application abstraction in Erlang is very powerful in this
respect, as long as you design your applications to be as independent
as possible from one another. You can add new functionality to a
system by the simple act of starting a new application!
Your only real design consideration then is how the plugin apps
register with the main application. This is a good spot for a
gen_event, or pub/sub broker model, where you use an intermediary
process to broker messages/requests between the main application and
the plugins.
Thank you for the comments. It sounds reasonable to go with active
applications. One aspect I forgot to mention is that the main
application should start its own processing only when all the required
plugins are ready. The plugin applications act as backend processors.
Maybe that could be solved in more abstract way: the main application
could be configured with names of processes (or incoming messages) to
wait for before processing is started. Although I am not sure if such
an approach is "erlangish", I was deep in java for last ten years.
On Sat, Nov 10, 2012 at 6:15 PM, Garrett Smith <g...@rre.tt> wrote:
> On Sat, Nov 10, 2012 at 1:11 AM, Karolis Petrauskas
> <k.petraus...@gmail.com> wrote:
>> Hello,
>> I am building an Erlang based system that is composed of several
>> applications. In principle, the system consists of a main application
>> and a number of "plug in applications". The latter have modules
>> implementing behaviours defined by the main application. In this
>> particular case, should I have all the "plug in applications" as
>> library applications, and access them from the main application based
>> on some explicit configuration? Or should all the applications have
>> its own application modules starting own supervisors and then
>> registering self to the main application? The main thing confusing me
>> is the recommendation to have single supervision tree for the entire
>> system.
> There are good reasons for taking either approach, but if you're in
> doubt, I'd suggest the second option: each plugin is an "active"
> application (i.e. specifies a mod in the .app file) that registers
> itself as needed with the main application.
> The advantage to this approach is that you're further separating the
> components of your system:
> - The main application doesn't have to know about the plugins at
> startup, which simplifies its configuration and removes the "wiring"
> logic that it would otherwise need
> - Plugins can be added and removed by adding and removing OTP
> applications in the release (i.e. you use boot files to specify which
> plugins are run for a given Erlang process)
> The OTP application abstraction in Erlang is very powerful in this
> respect, as long as you design your applications to be as independent
> as possible from one another. You can add new functionality to a
> system by the simple act of starting a new application!
> Your only real design consideration then is how the plugin apps
> register with the main application. This is a good spot for a
> gen_event, or pub/sub broker model, where you use an intermediary
> process to broker messages/requests between the main application and
> the plugins.
What makes the plugins "pluggable"? If there's no uncertainly about
what might be registered, it might make more sense to define the
workers as children of a supervisor in your main app.
If you're breaking these into separate apps to manage complexity, then
I suspect you're looking at your option 1 -- library style apps that
are required by the main app.
If it's not know ahead of time what the plugins are, what is the
trigger for processing? Is it a certain number of plugins? A total
worker capacity?
On Sat, Nov 10, 2012 at 4:02 PM, Karolis Petrauskas
<k.petraus...@gmail.com> wrote:
> Thank you for the comments. It sounds reasonable to go with active
> applications. One aspect I forgot to mention is that the main
> application should start its own processing only when all the required
> plugins are ready. The plugin applications act as backend processors.
> Maybe that could be solved in more abstract way: the main application
> could be configured with names of processes (or incoming messages) to
> wait for before processing is started. Although I am not sure if such
> an approach is "erlangish", I was deep in java for last ten years.
> Karolis
> On Sat, Nov 10, 2012 at 6:15 PM, Garrett Smith <g...@rre.tt> wrote:
>> On Sat, Nov 10, 2012 at 1:11 AM, Karolis Petrauskas
>> <k.petraus...@gmail.com> wrote:
>>> Hello,
>>> I am building an Erlang based system that is composed of several
>>> applications. In principle, the system consists of a main application
>>> and a number of "plug in applications". The latter have modules
>>> implementing behaviours defined by the main application. In this
>>> particular case, should I have all the "plug in applications" as
>>> library applications, and access them from the main application based
>>> on some explicit configuration? Or should all the applications have
>>> its own application modules starting own supervisors and then
>>> registering self to the main application? The main thing confusing me
>>> is the recommendation to have single supervision tree for the entire
>>> system.
>> There are good reasons for taking either approach, but if you're in
>> doubt, I'd suggest the second option: each plugin is an "active"
>> application (i.e. specifies a mod in the .app file) that registers
>> itself as needed with the main application.
>> The advantage to this approach is that you're further separating the
>> components of your system:
>> - The main application doesn't have to know about the plugins at
>> startup, which simplifies its configuration and removes the "wiring"
>> logic that it would otherwise need
>> - Plugins can be added and removed by adding and removing OTP
>> applications in the release (i.e. you use boot files to specify which
>> plugins are run for a given Erlang process)
>> The OTP application abstraction in Erlang is very powerful in this
>> respect, as long as you design your applications to be as independent
>> as possible from one another. You can add new functionality to a
>> system by the simple act of starting a new application!
>> Your only real design consideration then is how the plugin apps
>> register with the main application. This is a good spot for a
>> gen_event, or pub/sub broker model, where you use an intermediary
>> process to broker messages/requests between the main application and
>> the plugins.
Your questions allow me to understand the problem better. Thanks
:) Maybe the terms "master application" and "plugin application" is
not totally appropriate. In java world it would be:
master application = application server,
plugin application = application (or user application).
The application server governs all the user applications, provides
some infrastructure and delegates some work to them. I would like to
have ability to add those applications freely, but once particular
user application added, the application server should proceed with its
own processing only when the added application is available.
Maybe another way to solve this is to reverse the runtime
dependency: "the user applications take tasks from a queue provided by
the application server" instead of "the application server delegates
tasks to user applications". I need to think about that more. The
requirement for the solution to support cluster operation is not
making things clearer :)
On Sun, Nov 11, 2012 at 3:16 AM, Garrett Smith <g...@rre.tt> wrote:
> What makes the plugins "pluggable"? If there's no uncertainly about
> what might be registered, it might make more sense to define the
> workers as children of a supervisor in your main app.
> If you're breaking these into separate apps to manage complexity, then
> I suspect you're looking at your option 1 -- library style apps that
> are required by the main app.
> If it's not know ahead of time what the plugins are, what is the
> trigger for processing? Is it a certain number of plugins? A total
> worker capacity?
> On Sat, Nov 10, 2012 at 4:02 PM, Karolis Petrauskas
> <k.petraus...@gmail.com> wrote:
>> Thank you for the comments. It sounds reasonable to go with active
>> applications. One aspect I forgot to mention is that the main
>> application should start its own processing only when all the required
>> plugins are ready. The plugin applications act as backend processors.
>> Maybe that could be solved in more abstract way: the main application
>> could be configured with names of processes (or incoming messages) to
>> wait for before processing is started. Although I am not sure if such
>> an approach is "erlangish", I was deep in java for last ten years.
>> Karolis
>> On Sat, Nov 10, 2012 at 6:15 PM, Garrett Smith <g...@rre.tt> wrote:
>>> On Sat, Nov 10, 2012 at 1:11 AM, Karolis Petrauskas
>>> <k.petraus...@gmail.com> wrote:
>>>> Hello,
>>>> I am building an Erlang based system that is composed of several
>>>> applications. In principle, the system consists of a main application
>>>> and a number of "plug in applications". The latter have modules
>>>> implementing behaviours defined by the main application. In this
>>>> particular case, should I have all the "plug in applications" as
>>>> library applications, and access them from the main application based
>>>> on some explicit configuration? Or should all the applications have
>>>> its own application modules starting own supervisors and then
>>>> registering self to the main application? The main thing confusing me
>>>> is the recommendation to have single supervision tree for the entire
>>>> system.
>>> There are good reasons for taking either approach, but if you're in
>>> doubt, I'd suggest the second option: each plugin is an "active"
>>> application (i.e. specifies a mod in the .app file) that registers
>>> itself as needed with the main application.
>>> The advantage to this approach is that you're further separating the
>>> components of your system:
>>> - The main application doesn't have to know about the plugins at
>>> startup, which simplifies its configuration and removes the "wiring"
>>> logic that it would otherwise need
>>> - Plugins can be added and removed by adding and removing OTP
>>> applications in the release (i.e. you use boot files to specify which
>>> plugins are run for a given Erlang process)
>>> The OTP application abstraction in Erlang is very powerful in this
>>> respect, as long as you design your applications to be as independent
>>> as possible from one another. You can add new functionality to a
>>> system by the simple act of starting a new application!
>>> Your only real design consideration then is how the plugin apps
>>> register with the main application. This is a good spot for a
>>> gen_event, or pub/sub broker model, where you use an intermediary
>>> process to broker messages/requests between the main application and
>>> the plugins.
Yeah, now the correct answer is very clear: it depends :)
Erlang applications will "turn on" a particular set of features. But
they're probably not the right abstraction for increasing "capacity".
E.g. you can't start multiple instances of the same application to do
more work. You can certainly add processes, managed under the auspices
of an application.
If you're willing to skim over some of the low-level technical areas,
I recommend reading the 0MQ "The Guide":
There's obviously quite a bit specific to 0MQ, but that document (soon
to be a published book as I understand!) elaborates a way of thinking
about messaging patterns that might be very helpful in provoking
different questions and ideas for you.
Each of the messaging patterns presented in The Guide can be applied
quite easily within a single Erlang node -- it's all just message
passing.
And if you're so inclined, Erlang + 0MQ makes a very good platform for
distributed systems IMO (i.e. using these patterns across multiple
Erlang OS processes, without relying on distribute Erlang).
On Sun, Nov 11, 2012 at 1:00 AM, Karolis Petrauskas
<k.petraus...@gmail.com> wrote:
> Your questions allow me to understand the problem better. Thanks
> :) Maybe the terms "master application" and "plugin application" is
> not totally appropriate. In java world it would be:
> master application = application server,
> plugin application = application (or user application).
> The application server governs all the user applications, provides
> some infrastructure and delegates some work to them. I would like to
> have ability to add those applications freely, but once particular
> user application added, the application server should proceed with its
> own processing only when the added application is available.
> Maybe another way to solve this is to reverse the runtime
> dependency: "the user applications take tasks from a queue provided by
> the application server" instead of "the application server delegates
> tasks to user applications". I need to think about that more. The
> requirement for the solution to support cluster operation is not
> making things clearer :)
> Karolis
> On Sun, Nov 11, 2012 at 3:16 AM, Garrett Smith <g...@rre.tt> wrote:
>> What makes the plugins "pluggable"? If there's no uncertainly about
>> what might be registered, it might make more sense to define the
>> workers as children of a supervisor in your main app.
>> If you're breaking these into separate apps to manage complexity, then
>> I suspect you're looking at your option 1 -- library style apps that
>> are required by the main app.
>> If it's not know ahead of time what the plugins are, what is the
>> trigger for processing? Is it a certain number of plugins? A total
>> worker capacity?
>> On Sat, Nov 10, 2012 at 4:02 PM, Karolis Petrauskas
>> <k.petraus...@gmail.com> wrote:
>>> Thank you for the comments. It sounds reasonable to go with active
>>> applications. One aspect I forgot to mention is that the main
>>> application should start its own processing only when all the required
>>> plugins are ready. The plugin applications act as backend processors.
>>> Maybe that could be solved in more abstract way: the main application
>>> could be configured with names of processes (or incoming messages) to
>>> wait for before processing is started. Although I am not sure if such
>>> an approach is "erlangish", I was deep in java for last ten years.
>>> Karolis
>>> On Sat, Nov 10, 2012 at 6:15 PM, Garrett Smith <g...@rre.tt> wrote:
>>>> On Sat, Nov 10, 2012 at 1:11 AM, Karolis Petrauskas
>>>> <k.petraus...@gmail.com> wrote:
>>>>> Hello,
>>>>> I am building an Erlang based system that is composed of several
>>>>> applications. In principle, the system consists of a main application
>>>>> and a number of "plug in applications". The latter have modules
>>>>> implementing behaviours defined by the main application. In this
>>>>> particular case, should I have all the "plug in applications" as
>>>>> library applications, and access them from the main application based
>>>>> on some explicit configuration? Or should all the applications have
>>>>> its own application modules starting own supervisors and then
>>>>> registering self to the main application? The main thing confusing me
>>>>> is the recommendation to have single supervision tree for the entire
>>>>> system.
>>>> There are good reasons for taking either approach, but if you're in
>>>> doubt, I'd suggest the second option: each plugin is an "active"
>>>> application (i.e. specifies a mod in the .app file) that registers
>>>> itself as needed with the main application.
>>>> The advantage to this approach is that you're further separating the
>>>> components of your system:
>>>> - The main application doesn't have to know about the plugins at
>>>> startup, which simplifies its configuration and removes the "wiring"
>>>> logic that it would otherwise need
>>>> - Plugins can be added and removed by adding and removing OTP
>>>> applications in the release (i.e. you use boot files to specify which
>>>> plugins are run for a given Erlang process)
>>>> The OTP application abstraction in Erlang is very powerful in this
>>>> respect, as long as you design your applications to be as independent
>>>> as possible from one another. You can add new functionality to a
>>>> system by the simple act of starting a new application!
>>>> Your only real design consideration then is how the plugin apps
>>>> register with the main application. This is a good spot for a
>>>> gen_event, or pub/sub broker model, where you use an intermediary
>>>> process to broker messages/requests between the main application and
>>>> the plugins.
On Sun, Nov 11, 2012 at 4:54 PM, Garrett Smith <g...@rre.tt> wrote:
> Yeah, now the correct answer is very clear: it depends :)
> Erlang applications will "turn on" a particular set of features. But
> they're probably not the right abstraction for increasing "capacity".
> E.g. you can't start multiple instances of the same application to do
> more work. You can certainly add processes, managed under the auspices
> of an application.
> If you're willing to skim over some of the low-level technical areas,
> I recommend reading the 0MQ "The Guide":
> There's obviously quite a bit specific to 0MQ, but that document (soon
> to be a published book as I understand!) elaborates a way of thinking
> about messaging patterns that might be very helpful in provoking
> different questions and ideas for you.
> Each of the messaging patterns presented in The Guide can be applied
> quite easily within a single Erlang node -- it's all just message
> passing.
> And if you're so inclined, Erlang + 0MQ makes a very good platform for
> distributed systems IMO (i.e. using these patterns across multiple
> Erlang OS processes, without relying on distribute Erlang).
> On Sun, Nov 11, 2012 at 1:00 AM, Karolis Petrauskas
> <k.petraus...@gmail.com> wrote:
>> Your questions allow me to understand the problem better. Thanks
>> :) Maybe the terms "master application" and "plugin application" is
>> not totally appropriate. In java world it would be:
>> master application = application server,
>> plugin application = application (or user application).
>> The application server governs all the user applications, provides
>> some infrastructure and delegates some work to them. I would like to
>> have ability to add those applications freely, but once particular
>> user application added, the application server should proceed with its
>> own processing only when the added application is available.
>> Maybe another way to solve this is to reverse the runtime
>> dependency: "the user applications take tasks from a queue provided by
>> the application server" instead of "the application server delegates
>> tasks to user applications". I need to think about that more. The
>> requirement for the solution to support cluster operation is not
>> making things clearer :)
>> Karolis
>> On Sun, Nov 11, 2012 at 3:16 AM, Garrett Smith <g...@rre.tt> wrote:
>>> What makes the plugins "pluggable"? If there's no uncertainly about
>>> what might be registered, it might make more sense to define the
>>> workers as children of a supervisor in your main app.
>>> If you're breaking these into separate apps to manage complexity, then
>>> I suspect you're looking at your option 1 -- library style apps that
>>> are required by the main app.
>>> If it's not know ahead of time what the plugins are, what is the
>>> trigger for processing? Is it a certain number of plugins? A total
>>> worker capacity?
>>> On Sat, Nov 10, 2012 at 4:02 PM, Karolis Petrauskas
>>> <k.petraus...@gmail.com> wrote:
>>>> Thank you for the comments. It sounds reasonable to go with active
>>>> applications. One aspect I forgot to mention is that the main
>>>> application should start its own processing only when all the required
>>>> plugins are ready. The plugin applications act as backend processors.
>>>> Maybe that could be solved in more abstract way: the main application
>>>> could be configured with names of processes (or incoming messages) to
>>>> wait for before processing is started. Although I am not sure if such
>>>> an approach is "erlangish", I was deep in java for last ten years.
>>>> Karolis
>>>> On Sat, Nov 10, 2012 at 6:15 PM, Garrett Smith <g...@rre.tt> wrote:
>>>>> On Sat, Nov 10, 2012 at 1:11 AM, Karolis Petrauskas
>>>>> <k.petraus...@gmail.com> wrote:
>>>>>> Hello,
>>>>>> I am building an Erlang based system that is composed of several
>>>>>> applications. In principle, the system consists of a main application
>>>>>> and a number of "plug in applications". The latter have modules
>>>>>> implementing behaviours defined by the main application. In this
>>>>>> particular case, should I have all the "plug in applications" as
>>>>>> library applications, and access them from the main application based
>>>>>> on some explicit configuration? Or should all the applications have
>>>>>> its own application modules starting own supervisors and then
>>>>>> registering self to the main application? The main thing confusing me
>>>>>> is the recommendation to have single supervision tree for the entire
>>>>>> system.
>>>>> There are good reasons for taking either approach, but if you're in
>>>>> doubt, I'd suggest the second option: each plugin is an "active"
>>>>> application (i.e. specifies a mod in the .app file) that registers
>>>>> itself as needed with the main application.
>>>>> The advantage to this approach is that you're further separating the
>>>>> components of your system:
>>>>> - The main application doesn't have to know about the plugins at
>>>>> startup, which simplifies its configuration and removes the "wiring"
>>>>> logic that it would otherwise need
>>>>> - Plugins can be added and removed by adding and removing OTP
>>>>> applications in the release (i.e. you use boot files to specify which
>>>>> plugins are run for a given Erlang process)
>>>>> The OTP application abstraction in Erlang is very powerful in this
>>>>> respect, as long as you design your applications to be as independent
>>>>> as possible from one another. You can add new functionality to a
>>>>> system by the simple act of starting a new application!
>>>>> Your only real design consideration then is how the plugin apps
>>>>> register with the main application. This is a good spot for a
>>>>> gen_event, or pub/sub broker model, where you use an intermediary
>>>>> process to broker messages/requests between the main application and
>>>>> the plugins.
