Pubsub Redis concerns for MMO game

[hd nguyen]

Hi all,


I have a concern when using pubsub mechanism of redis in my real application.


I built a MMO game with 5 servers (NodeJs) and use pubsub of Redis to broadcast game state between servers.


But I concern which is the best practice to utilize this mechanism efficiently with a lot of users connect and interact simultaneously?

I would love to hear your ideas about following problems:

1/ Should I create NEW channel for each action or just ONE channel for ALL actions? 

Ex: user1 connect to server1, user2-server2,...userN-serverN

When user1 move/fight and lost his health, those action should be broadcast to other servers, so each time want to broadcast I should use: 

pub1.publish("pub1", "action data sent from pub1"); //just one channel for ALL actions

OR

pub1.publish("pub1_actionID", "action data sent from pub1"); // each channel for EACH action

2/ And in each servers we create a client to listen:

//subscribe ALL channels from ALL OTHER servers

 client = redis.createClient(port, host);

client.subscribe("pub1");

client.subscribe("pub2");

....

client.subscribe("pubN");

client.on("message", function(channel, message){

  console.log(host + ":" + channel + ": " + message);

});

As above snippet code, each server should be a client to listen ALL CHANNELS published by ALL other servers.

But I concern whether I should use just ONE client to listen ALL channels or ONE client for EACH channel?

//each subscribe for each channel

client1 = redis.createClient(port, host);

client2 = redis.createClient(port, host);

client1.subscribe("pub1");

client2.subscribe("pub2");

....

client1.on("message", function(channel, message){

  console.log(host + ":" + channel + ": " + message);

});

client2.on("message", function(channel, message){

  console.log(host + ":" + channel + ": " + message);

});

...

3/ Could you tell me how many connections redis can handle maximum in a second? (each time we createClient() we should create a connection to Redis server, right? Does it take a lot of time and overhead?) 

4/ Any tool to evaluate performance of pubsub mechanism for a MMO game with multiple requests at the same time?

Thanks for your time.

--
Nguyen Hai Duy
Mobile : 0914 72 1900
Yahoo: nguyenhd_lucky

[Tim Smart]

Hello,

I would definitely recommend just adding one listener and doing all your routing
from there. Something like:

redis.subscribe('channel1')
redis.subscribe('channel2')

redis.on('message', function (channel, message) {
// emitter.emit(channel, message)
//
// -or
//
// if (methods[channel]) methods[channel](message)
//
// -or
//
// switch (channel) {
// case 'channel1':
// break
// case 'channel2':
// break
// }
})

The main reason being no matter how many clients you have, they will all
receive the same messages as each other; which means your clients will be
doubling up on emit operations.

Tim

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

[hd nguyen]

Thanks Tim for your suggestion about subscriber part.

Any suggestions for publisher part(one publisher for each ACTION or one publisher for ALL ACTION)? I want to hear about it.

Thanks.

[Tim Smart]

The `npm install redis` client has a (very generalised) throughput of around 50k
messages per second. If you think that will be a ceiling for you, then client
pooling might become an option.

Another alternative is a redis client I made that can roughly double the
performance for small messages. Take a look at
https:/github.com/tim-smart/node-redis if this interests you.

Tim.

[Thomas Blobaum]

In your case if you are just publishing strings/numbers, creating some
network of channels by name could be useful. If you are publishing a
stringified json object, its probably not necessary to also use
different subscription channels because that data can be part of what
you send but you could still make use of them.


Thomas Blobaum
https://github.com/tblobaum

[hd nguyen]

Do we have any tool to measure/simulate a large number of channels publish message simultaneously?

And what do you think about redis pubsub performance in comparing to ZeroMQ pubsub mechanism?

Thanks.

[Angel Java Lopez]

Quick response:

If you are using Socket.IO to communicate clients with servers, you can use redis as a "socket.io store", so the messages that are received by Socket.io in a server, are "broadcasted" to the other Node.js servers that use Socket.IO

No production stressed yet, but for a similar problem, I wrote (broadcast messages to many servers):

https://github.com/ajlopez/SimpleBroadcast

--

[hd nguyen]

@Tim Cash: 

Now I have not made any test to compare redis pubsub and zmq. But with our business we intend to use redis pubsub as ultimate choice, and maybe we'll ignore zmq.

On Mon, Jul 30, 2012 at 2:32 AM, Tim Cash <tim...@gmail.com> wrote:

@hd nguyen

I am also interested in comparing the performance of redis pubsub v zeromq. If you run any kind of bench-marking let us know about your results. I have found https://github.com/JustinTulloss/zeromq.node to be somewhat disappointing when working with PUSH PULL and PUB SUB. I think this may be due to the way node works. With LUAJIT I can PUB SUB millions of messages per second so Node and Redis is off by two orders of magnitude. So node+redis pubsub is similar to node+zeromq in my testing so far. I think there must be something going on with the node zeromq library though.

One option could be to batch up all the messages on the same channel for 10 - 20 milliseconds and send them as one.

Thanks.

>> > nodejs+unsubscribe@googlegroups.com

>> > For more options, visit this group at
>> > http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nod...@googlegroups.com
>> To unsubscribe from this group, send email to

>> nodejs+unsubscribe@googlegroups.com

>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>
>
>
>
> --
> Nguyen Hai Duy
> Mobile : 0914 72 1900
> Yahoo: nguyenhd_lucky
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to

> nodejs+unsubscribe@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to

nodejs+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--
Nguyen Hai Duy
Mobile : 0914 72 1900
Yahoo: nguyenhd_lucky

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--