How to setup a memcached cluster ?

[kha...@gmail.com]

I have 20 web servers and 4 instance of dynamic cache

How would different client know on which memcached instance is the key
store? Since the client hashing on a Server-A does not know if the
same caching(say on cache server1) has been done by another server
Server-B on another cache server2.

This will lead to duplicacy and evictions

The way I see is that it requires a middle tier setup of hash-key
maintenance, and all client hit to this middle server, which will
determine which of the caching server holds the respective key.

Has anyone having any ready solution for my problem. Help will be
appreciated

[Brian Hawkins]

You may misunderstand the way the hashing works.  You hash some deterministic key, say the client IP and some string value.  From the key the client creates a md5 (or other) hash and then combines that down to an int value.  Then to find the memcached instance it maps to, you take (int % <number of memcached instances>) = which instance.

The formula works such that it does not matter what web server calculates the hash it will always map to the same server. (that is if you've configured the client on each web server with the same memcached instances)

Brian

[Adam Lee]

Have you actually tried it?  

As Brian said, it works on the principal of consistent hashing. The only shared knowledge that client machines need is a list of servers.  As long as they all have the same list and use the same, consistent hashing protocol, then they don't need to share any other information in order to guarantee that they will always put and get the same key to the same server.

--
awl

[Henrik Schröder]

First I got pretty annoyed that someone, yet again, posts a message completely missing the point of memcached since it solves exactly this problem.

But then I googled a bit and checked out the top hits, and it actually doesn't say any of this on either the wikipedia page or the main memcached page. You have to dig quite far into the FAQ, or read long articles on it before anyone actually starts explaining the consistent server selection. I'll try to update the wikipedia page, it would be awesome if someone could update the main memcached page as well. :-)


/Henrik

[Dustin]

I'm interested in your copy. We're planning to revitalize some of
the documentation in coincidence with the imminent 1.4.3 release.

On Nov 6, 6:48 pm, Henrik Schröder <skro...@gmail.com> wrote:
> First I got pretty annoyed that someone, yet again, posts a message
> completely missing the point of memcached since it solves exactly this
> problem.
>
> But then I googled a bit and checked out the top hits, and it actually
> doesn't say any of this on either the wikipedia page or the main memcached
> page. You have to dig quite far into the FAQ, or read long articles on it
> before anyone actually starts explaining the consistent server selection.
> I'll try to update the wikipedia page, it would be awesome if someone could
> update the main memcached page as well. :-)
>
> /Henrik
>