Multiple servers listening on same-port.. would they eat each other messages ?

[Ashish Negi]

I am making a chat-application. To scale and use all cores, i am creating multiple servers which are listening on same port given by infrastructure for listening. Every information is in memory.

However, is it possible that lets say.. A and B are chatting each other and have information in process1 and X and Y are chatting and have information in process2.

My question is :

1. Do programs compete for each message ? If a new message comes from A, Can it go to process2 which have no information about A or B.

Or does it depend upon something like if i use simple http messages above problem would happen but if i use web-sockets it would not happen.

I am using node.js, though i consider it to be a language independent problem.

so, code is like :

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }
} else {
  http.Server(function(req, res) { ... }).listen(8000);
}

Should i make only one listening-process which would act like intelligent load-balancer that would send the message to right process ?

[Ryan Schmidt]

On Jan 19, 2015, at 12:44 AM, Ashish Negi wrote:
>
> I am making a chat-application. To scale and use all cores, i am creating multiple servers which are listening on same port given by infrastructure for listening. Every information is in memory.
>
> However, is it possible that lets say.. A and B are chatting each other and have information in process1 and X and Y are chatting and have information in process2.
>
> My question is :
>
> • Do programs compete for each message ? If a new message comes from A, Can it go to process2 which have no information about A or B.
> Or does it depend upon something like if i use simple http messages above problem would happen but if i use web-sockets it would not happen.
>
> I am using node.js, though i consider it to be a language independent problem.

When you use node's cluster module, the master will listen on the port. If a connection is established, node will hand that connection to one of the children. Which child it hands the connection to varies. With node 0.11 and later, it will be round robin. With node 0.10 and earlier, it is left to the discretion of the operating system; often, this means the same (first) worker is reused unless it is busy.

To share information between your children, store any important state information in a central database, e.g. redis, not in each child's memory.

[Jimb Esser]

It is worth adding that, though every http connection will get delivered to an (effectively) arbitrary process, causing messages over http to go to arbitrary processes, WebSockets messages, since they are delivered over a persistent connection, will all continue to be delivered to the same process.  So, assuming all of your target platforms support WebSockets (and by this, I mean actual WebSockets, not something like socket.io which may emulate socket message delivery over non-persistent connections for old clients), for certain kinds of applications, a lot of the usual complexity and overhead around tracking user sessions in a database can be completely eliminated if it is acceptable that a session is exactly equal to a WebSocket connection.

That being said, if your application involves chatting, and you want to be able to chat between arbitrary clients, you're going to need inter-process communication of some kind anyway, and your life may be easier in the long run if you just deal with arbitrary messages from arbitrary clients arriving on arbitrary processes (not having any in-memory server state has all sorts of great side effects for system reliability, restartability, debugability, etc).

  Jimb Esser