[erlang-questions] What's the most efficient method to broadcast a message to multiple (100Ks) of processes?

[Zvi]

Hi,

what's the most efficient method to broadcast a message to multiple
(100Ks) of processes?

The most naive way is:

[ Pid ! Msg || Pid<-Pids ].

less naive method:

[ spawn(fun() -> [Pid ! Msg || Pid<-PidsSublist] end) ||
PidsSublist <- partition(Pids, N) ].

where partition(L,N) splits list L into N sublists.

Any ideas for better method?

thanks,
Zvi

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questio...@erlang.org

[Joel Reymont]

http://groups.google.com/group/erlang-programming/browse_thread/thread/1931368998000836/b325e869a3eea26a

Basically, raise priority to high around your sending loop and then get it back to normal right after.

On Oct 7, 2010, at 8:01 PM, Zvi wrote:

> Hi,
>
> what's the most efficient method to broadcast a message to multiple
> (100Ks) of processes?
>
> The most naive way is:
>
> [ Pid ! Msg || Pid<-Pids ].
>
> less naive method:
>
> [ spawn(fun() -> [Pid ! Msg || Pid<-PidsSublist] end) ||
> PidsSublist <- partition(Pids, N) ].
>
> where partition(L,N) splits list L into N sublists.
>
> Any ideas for better method?

---
http://twitter.com/wagerlabs

[Eric Newhuis (personal)]

Um, are your processes in the same Erlang node (OS process) or are they distributed? Are they distributed on the same network segment or over the internet? What kinf of ping times do you get from the worst-case host?

If distributed then the only correct answer is to use hardware multicast and write your own binary packet readers. ;-) Or maybe there is already some library for that. Hm. I recall gen_tcp or something perhaps setup as an IP multicast reader/writer. Memory doesn't serve me well from a life I led over 15 years ago.

Seriously if you really do have 100,000 clients then you may need to say something about message size, frequency, bursting, etc., before this could be better answered. Also I worry about Erlang's networking kernel sending heartbeats over the net. Is this all local or are we talking internets?

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions