On Tue, Oct 23, 2012 at 7:22 AM, Joshua Muzaaya <joshm...@gmail.com> wrote:
> Yes,i tries using the random method, but because requests are so
> frequently many, you find the erlang random generator skewing results
> making around 50/60 requests hitting one Node while others are waiting.
> Another thing is that, i am not using gen_servers at the Web Server layer.
> I am using yaws web server and for each connection, yaws spawns a process,
> this process communicates with Mnesia Nodes to query for data. But the
> connections are so many and i wanted to scale the application horizontally,
> adding more web servers and more mnesia Nodes. I came to think of a load
> balancing middle ware, abstracting my processes from knowing where the call
> has hit ( i.e on what mnesia node the call has hit). This middle ware
> ensures that requests are load balanced across my Mnesia DBs.

> That is the background of the problem. Its a real-time Web Notification
> system, plugged into a major intranet Management System. However, clients
> are many, and yaws is sustaining 30,000 concurrent connections at low
> peaks. I am a software engineer in one of the telecommunications companies
> in Africa. I keep running into a few memory problems on single node yaws
> server, so i need ti add more web servers to assist. Also, mnesia sometime
> will get *** Too many DB Tables ** when requests are too many and too
> frequent. I changed everything to use dirty operations and when i by-passed
> the transaction manager, things improved a bit.

> I need fellow erlangers to think of a load balancing algorithm in such a
> situation. Do you think a process dictionary like GPROC would be so useful
> ? i was kinda thinking about it last night, but i wonder how i would apply
> it in this case.

> Having one gen_server to decide where the request may go, might alos slow
> down the application as all requests will have to go through that
> gen_server.

>   <http://www.linkedin.com/pub/muzaaya-joshua/39/2ba/202>
> Designed with WiseStamp -
> <http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350...>Get
> yours<http://r1.wisestamp.com/r/landing?u=cf16262215eb8784&v=3.11.21&t=1350...>

> On Tue, Oct 23, 2012 at 2:53 AM, Yogish Baliga <bal...@gmail.com> wrote:

>> One option is to run proxy gen_server on each Mnesia box and register
>> these gen_server pids with pg2. Now you can do load balancing on pg2
>> processes based on message queue length as described here

>> http://dev.lethain.com/load-balancing-across-erlang-process-groups/

>> When I used this method in my last project in Erlang, it gave better
>> result than normal round robin. Under very low load, all requests were
>> redirected to single Mnesia instance.

>> http://dev.lethain.com/load-balancing-across-erlang-process-groups/

>> -- baliga

>> On Mon, Oct 22, 2012 at 2:22 PM, Paul Peregud <paulpere...@gmail.com>wrote:

>>> May you specify why load balancing should be based on time and can not
>>> be random? Have you implemented random load balancing? Has it proved
>>> to be insufficient?

>>> On Mon, Oct 22, 2012 at 9:51 AM, muzaaya_joshua <joshm...@gmail.com>
>>> wrote:
>>> > Building from this question (
>>> http://stackoverflow.com/q/5339329/431620 ),
>>> > imagine an application with N Erlang Web Servers, and N/2 Mnesia
>>> Database
>>> > Nodes. The set up is such that the Web Servers, each, runs on its own
>>> > hardware server (say HP DL385), and each Mnesia Instance, runs on its
>>> own
>>> > hardware Server as well.

>>> > Web Servers make rpc:call/4 calls to the back end (the Mnesia DB
>>> Servers).
>>> > The Data is all replicated across all the Mnesia instances. Now, you
>>> want to
>>> > have the calls being made to the Database servers, MULTIPLEXED, more
>>> > precisely ( by TIME), on each Web Server, so that some kind of LOAD
>>> > BALANCING is attained.

>>> > If Web Server A makes a connection to Mnesia Instance 3, it cannot
>>> make the
>>> > next connection to the same Instance. All Database Nodes need to be
>>> kept
>>> > busy and not having any one of them idle while the others are working.
>>> The
>>> > Load balancing Algorithm should not be random, but should be aimed at
>>> > balancing the load on the Database Servers.

>>> > Qn 1: Come up with your load balancing strategy, in such a situation.
>>> Also,
>>> > please show with some sample illustrative code, how you would
>>> implement this
>>> > strategy.

>>> > Qn 2: If a Mnesia Instance goes down, how would your load balance
>>> Algorithm
>>> > adapt to the changes in the cluster ?

>>> > Qn 3: Is there any Erlang library aimed at load balancing of Erlang
>>> Servers
>>> > working within the same system, and calling each other via rpc:call/4 ?

>>> > --
>>> > View this message in context:
>>> http://erlang.2086793.n4.nabble.com/Load-balancing-multiplexing-rpc-c...
>>> > Sent from the Erlang Questions mailing list archive at Nabble.com.
>>> > _______________________________________________
>>> > erlang-questions mailing list
>>> > erlang-questi...@erlang.org
>>> > http://erlang.org/mailman/listinfo/erlang-questions

>>> --
>>> Best regards,
>>> Paul Peregud
>>> +48602112091
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questi...@erlang.org
>>> http://erlang.org/mailman/listinfo/erlang-questions

> --
> *Muzaaya Joshua
> Systems Engineer
> +256774115170*
> *"Through it all, i have learned to trust in Jesus. To depend upon His
> Word"
> *