Scaling/Sharding Juggernaut

14 views

Skip to first unread message

Artem Yankov

unread,

Nov 4, 2011, 3:09:37 AM11/4/11

to juggernaut...@googlegroups.com

What's the current state of things with scaling Juggernaut? If, say, you want to be able to handle hundreds of millions messages and one server is not enough. Has anyone already ran into this problem?

The first idea that comes in mind is to shard by channel_name. Ruby Redis client has a built-in support for consistent hashing and it's not a problem to distribute publish operations among different Redis nodes. But I can't find node.js client with support for consistent hashing.

There are pieces of code that can be combined together in order to implement it though:

http://cowboyrushforth.com/2011-06-03/consistent_hash_ring_in_node-js

http://stackoverflow.com/questions/4483952/nodejs-consistent-hash-library

But I feel like it something that could be done already and I'm just missing it?