[erlang-questions] gproc and gen_event

[Andrew Berman]

Hello,

I am trying to set up a pub-sub type situation where I have a function called which does some stuff and at the end broadcasts a message.  I would like for multiple handlers to be able to receive that message and do something with it.  I'm currently using gen_event, but have been reading about gproc as well.  Can anyone explain why they'd use one over the other for pub/sub?

Thanks,

Andrew

[AD]

Andrew,

 I posted about something similar on my blog http://blog.dberg.org/2012/04/using-gproc-and-cowboy-to-pass-messages.html.  I cant speak to gen_event personally, I havent used it that much, but I think its more of a use case for logging errors and such centrally versus a messaging system but I could be off here.

-A

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Garrett Smith]

gen_event is useful when you need to support "pluggable" functionality
and you can't tolerate the overhead of Erlang message passing.

gen_event runs all handlers in a single process. If one of those
subscribers misbehaves, it's unceremoniously removed.

To give you an idea:

Eshell V5.8.5 (abort with ^G)
1> error_logger:info_msg("say hello").

=INFO REPORT==== 18-May-2012::16:13:25 ===
say hellook
2> error_logger:info_msg([{"say", "goodbye"}]).
ok
3> error_logger:info_msg("anyone?").
ok

error_logger uses gen_event -- tty support is a handler. Bugs, as the
one above, result in silent changes to your program!

While you can use add_handler_sup to get notification of a handler
delete, I find the whole facility easy to get wrong and so I avoid it.

But it's not hard to create a publisher in yourself. e2 [1] has
something you can steal from, or just use:

https://github.com/gar1t/e2/blob/master/src/e2_publisher.erl

Here's a silly example of a "broker" that decorates published messages
with some tag:

https://github.com/gar1t/e2/blob/master/examples/pubsub/src/broker.erl

The advantage of pubsub using Erlang messages over gen_event is that
you can implement more natural supervision -- and you're not going to
get silent, mysterious breakage. The price is that you're sending
Erlang messages. If you can't afford that (e.g. perhaps high
throughput logging) gen_event is better.

Garrett

[1] http://e2project.org/

[Andrew Berman]

Ok, cool, thanks Garrett.  

Do you have any experience using gproc for pub/sub?

Nice email address, btw :-)

Andrew

[Michael Truog]

Hi Andrew,

gen_event is an OTP behavior for responding to events, like logging, database drivers, etc. (http://www.erlang.org/doc/design_principles/events.html).  The only different part is the supervisor wants "dynamic" in the child spec (http://www.erlang.org/doc/design_principles/sup_princ.html#id71666).

gproc is a way of having centralized storage among Erlang nodes.  However, if you read here http://erlang.org/pipermail/erlang-questions/2010-April/051030.html , you will discover:
"As stated before, gen_leader is a bit clumsy in environments where nodes are dynamically added and removed. Changing the participants of a gen_leader cluster is fairly difficult. The gen_leader_revival project tries to merge a number of existing modifications to gen_leader to make it more flexible." (i.e., where gen_leader is the guts behind gproc).  So, if you don't have netsplits, then gproc should be fine, but I believe mnesia fits into that category as well.

pg2 is one you have ignored, which replicates process groups among the collected nodes, storing them in ETS.  I chose an approach based on pg2, but without ETS in CloudI (http://cloudi.org), here: https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi/src/list_pg.erl .  This makes more sense with CloudI focusing on the AP part of the CAP theorem, whereas gproc is more focused on CA.  I find the approach in CloudI better, since it avoids global state and that is the main point of using Erlang (with its Actor model).  However, I am very biased, since I wrote it.

Regards,
Michael

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Ulf Wiger]

On 19 May 2012, at 01:38, Andrew Berman wrote:

Ok, cool, thanks Garrett.  

Do you have any experience using gproc for pub/sub?

Gproc does do pub/sub fairly well, but if I may, the main purpose of 

gproc initially was to serve as a form of "property index" for processes.

It had dawned on me that complex signaling applications tended to

end up with a lot of code that basically provided different sorts of 

mappings to allow you to locate the right processes. Having so many

different mappings complicated debugging, and obscured many 

aspects of the architecture.

When we started using (the predecessor of) gproc in some Ericsson

projects, there was significant code reduction. The nice thing was that 

the developers picked it up themselves; there wasn't really any central

decision to go with it.

One of the things I like about gproc is that it allows you to publish the

'interaction features' of a process: registered aliases say much about 

what services a process can perform, and properties illustrate its 

dependencies on other processes. The registry can be queried with

select operations or QLC, or simply with gproc:i() from the shell.

In short, there are many fine ways of doing pub/sub, but if you use 

gproc consistently for this sort of thing, and similar things, it can help

clarify runtime aspects of your system, making it more transparent and

easier to debug.

This also explains why gproc relies on a central registry (which is a

challenge in the global case). The registry itself is much of the point.

Of course, for a distributed pub/sub, you don't really need to turn on 

the distributed parts of gproc. Running a local gproc instance on each

node, you can publish like so:

publish(Event, Data) ->

   rpc:eval_everywhere(

      gproc, send,

      [{p, l, {?MODULE, Event}}, {?MODULE, Event, Data}]).

BR,

Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.

http://feuerlabs.com

[Andrew Berman]

Awesome, thanks for the explanation Ulf!  I'm going to give it a shot.

--Andrew

[Andrew Berman]

Thanks for this!