Re: Problems when one of the memcached server down

[Henrik Schröder]

Which client are you using, there's at least three .Net clients?

Either way, see if you can turn off failover in the client, that's what causing this behaviour. Alternatively, you can switch to my .Net client, which doesn't do failover at all:

http://code.google.com/p/beitmemcached/


/Henrik

On Thu, Mar 3, 2011 at 22:53, Evil Boy 4 Life <lior...@gmail.com> wrote:

Hi,
I use the .net clients and 2 servers at 2 different machines.
When one of the memcached servers is down, I try to set an item to the
cache (According the hashing algoritem of the client, this item should
stored the inactive server!) and this item get stored at the active
server.
After I set the item, the second server get back to work. Now if I'll
try to get this item I won't succeed, because the client will searche
the item at the second server (According the hashing algoritem).

What can i do to solve this problem???
(To set the item again at the second server isn't a solution, because
I don't want to store any item at 2 servers)

[Geoffrey Hoffman]

I couldn't easily find any memcache .net documentation, but a user
contributed note on the php memcache doc page says:
http://php.net/manual/en/book.memcache.php

The memcache changelog for 3.0.5 has the comment:
"Added method MemcachePool::findServer(key) which returns the server a
key hashes to"

I bring this up because knowing (storing elsewhere) the server storing
your key allows to handle the case when your key may no longer exist
there and reload it from the database or file.

Your client and or your server memcache/d versions may not provide
these features.

Also read the other thread `Replication ?`

[Matt Ingenthron]

Hi Lior,

On 3/3/11 1:53 PM, Evil Boy 4 Life wrote:
> Hi,
> I use the .net clients and 2 servers at 2 different machines.
> When one of the memcached servers is down, I try to set an item to the
> cache (According the hashing algoritem of the client, this item should
> stored the inactive server!) and this item get stored at the active
> server.
> After I set the item, the second server get back to work. Now if I'll
> try to get this item I won't succeed, because the client will searche
> the item at the second server (According the hashing algoritem).

Are you referring to the Enyim client? I believe there's a separate
mailing list for that client, but I am not sure.

> What can i do to solve this problem???
> (To set the item again at the second server isn't a solution, because
> I don't want to store any item at 2 servers)

This seems like something more fundamental is wrong, or I miss the
question.

Keep in mind, having two servers isn't a situation where they are
active/inactive, it's a collection of two servers. Each adds to the
"pool" of cache available. It's normal for a given key to hash to one
server or another, depending on the key.

Are you saying that in a normal, no failures kind of situation, you
cannot get and then set the same key?

Matt

[Evil Boy 4 Life]

Thanks guys, I turned off the failover in the client...
In fact, I didn't know that there are several .net clients and now I
get confused, because I don't know which one is better (now I use
memcacheddotnet)
Do you know if one of the .net clients support replication in any way?

Thanks again!

[Les Mikesell]

Memcache isn't supposed to do replication - it's a cache that you should
be able to replenish from the real data source. What happens when one
server fails depends on the client hash mechanism. Some people want the
load to be rebalanced evenly over the remaining servers, some don't.
Either way, the next attempt to use a failed server should detect it is
down and retrieve the data from the backing DB.

--
Les Mikesell
lesmi...@gmail.com

[Adam Lee]

it is, however, possible to support "replication" at the client level, if a bit out of band for memcached. we (Fotolog), for example, at one point wrote our own client that would set data in multiple servers and then get from only one of them. it was a read-heavy environment with a small dataset that easily fit completely into RAM on each server, so it actually worked. not saying that it is a good idea, just that as possible.

anyway, it has since been replaced in our system with TokyoTyrant, which solved the problem much better for us... and it (or something similar) very well might in this case as well.

awl

[Henrik Schröder]

That client works, but it's a pretty bad java port and isn't maintained. You're probably better off using any of the other two clients: BeITMemcached or Enyim. I know my client doesn't support replication, and I don't think the Enyim one does either. And as other people have suggested, if you want replication, you should probably look at some other solution than memcached.


/Henrik Schröder

[SivaRam M]

Apologies for intervening here. I'm using memcahce-php client and there is this call called getstats() which returns stats of a server if it is up an runnig

and returns false if there is something really bad happens to server.

My question is while hashing the key to a particular server . Will our consistent hash function consider this failed server .

I just want to know what backup plans are available if a server is down. 

[Les Mikesell]

On Thu, Oct 17, 2013 at 1:08 AM, SivaRam M <siva...@gmail.com> wrote:
>
> Apologies for intervening here. I'm using memcahce-php client and there is
> this call called getstats() which returns stats of a server if it is up an
> runnig
> and returns false if there is something really bad happens to server.
> My question is while hashing the key to a particular server . Will our
> consistent hash function consider this failed server .
> I just want to know what backup plans are available if a server is down.

The backup plan should always be for the client to fetch the data from
a persistent store on any cache miss. I may have this backwards, but I
think the clients with consistent hashing will keep the same hash
regardless of the server state and continue to have hash misses for
whatever percentage of servers are down at any time. Ones that don't
have consistent hashing will adjust the ring to the number of working
servers, losing some current data but letting the clients update it in
the new location on the first access. Obviously, the more servers
you use the less the impact of losing one will be.

--
Les Mikesell
lesmi...@gmail.com