Load-balanced web-servers + memcached

[TheJonathan]

Hi all,

I'm really excited to start using memcached. I'm currently using in-
memory caches in my ASP.NET+IIS6 application on 2 load-balanced
servers, but of course that isn't a shared memory between the 2
servers. So if key "test1" is cached on web1, web2 has no way of
getting to it. Enter memcached.

One question I had, since I have those 2 servers (and 1 separate db) I
want to install the memcached server on both of them since the
database is the bottleneck, not the RAM on the servers. If I do that,
how does memcached figure out which server has the cache key
efficiently? I am missing something? For example, say web1 is
configured to use memcached on localhost+web2, and web2 is configured
to use memcached on localhost+web1. If a key is created on web1 or
web2, will it always be stored in the same cache?

Thanks for your help!

--Jonathan

[Ray Krueger]

Yes
http://www.socialtext.net/memcached/index.cgi?faq

[Kevin Amerson]

The client should use a hashing algorithm on the key, and if both instances of memcached are in the available pool of servers, then once hashed, it will decide on the server and then always go to that server, unless of course it is taken out of the pool of available servers.

You can test this out more easily by spinning up several instances of memcached on a server and configuring and working with your client.

Which client are you using?

Kevin

On Thu, Oct 30, 2008 at 12:58 PM, TheJonathan <mit...@gmail.com> wrote:

[Henrik Schröder]

When you setup the client in your web app, you give it the same config on both machines, for example that memcached exists on "web1" and "web2" in your case. It's important that you name the servers in the same way and in the same order on all machines, otherwise the clients will distribute your keys differently, and you don't want that. (I know this is true for my .Net client, BeITMemcached, and I think it's true for the other .Net clients as well) Basically, you feed the memcached client an identical config in all your web apps.

If you do that, then yes, requests for the same key will always end up at the same server, no matter where the request originates. (This is kinda the whole point of memcached. If it didn't, it would be useless. :) )


/Henrik Schröder

[TheJonathan]

I think I get it. So if I'm passing in the array of memcached IPs via
a web.config, make sure they are in the same order on all load-
balanced machines. e.g.
web1 = localhost, web2
web2 = web1, localhost

@Kevin: Is the hashing algorithm you mentioned something I would have
to implement myself, or a feature? I was going to use the enyim
Memcached client in .NET 2.0 http://www.codeplex.com/EnyimMemcached/
but I could be swayed. I was also going to try out the session state
provider http://www.codeplex.com/memcachedproviders

On Oct 30, 3:54 pm, "Henrik Schröder" <skro...@gmail.com> wrote:
> When you setup the client in your web app, you give it the same config on
> both machines, for example that memcached exists on "web1" and "web2" in
> your case. It's important that you name the servers in the same way and in
> the same order on all machines, otherwise the clients will distribute your
> keys differently, and you don't want that. (I know this is true for my .Net
> client, BeITMemcached, and I think it's true for the other .Net clients as
> well) Basically, you feed the memcached client an identical config in all
> your web apps.
>
> If you do that, then yes, requests for the same key will always end up at
> the same server, no matter where the request originates. (This is kinda the
> whole point of memcached. If it didn't, it would be useless. :) )
>
> /Henrik Schröder
>

[Simone Busoli]

On Thu, Oct 30, 2008 at 9:26 PM, TheJonathan <mit...@gmail.com> wrote:

web1 = localhost, web2
web2 = web1, localhost

This would be better:

web1 = web1, web2

web2 = web1, web2

This way, you can keep the same configuration on all clients.

[Ray Krueger]

> I think I get it. So if I'm passing in the array of memcached IPs via
> a web.config, make sure they are in the same order on all load-
> balanced machines. e.g.
> web1 = localhost, web2
> web2 = web1, localhost
>
> @Kevin: Is the hashing algorithm you mentioned something I would have
> to implement myself, or a feature? I was going to use the enyim
> Memcached client in .NET 2.0 http://www.codeplex.com/EnyimMemcached/
> but I could be swayed. I was also going to try out the session state
> provider http://www.codeplex.com/memcachedproviders
>

The client library will handle hashing your key for you. In the case
of Enyim, it seems to use Ketama hashing (which is a good thing).
From the website...
"supports consistent hashing for keys: a specific item goes to a
specific server every time. (based on libketama,
http://lists.danga.com/pipermail/memcached/2007-April/003834.html)"

Essentially get/set with the key of your choice and the library will
take care of the rest.

[Kevin Amerson]

Exactly, yes, we use the Enyim client and it has the hashing built in.  You could sub it out if you wanted to, but it works well out of the box ;).

[Clint Webb]

Seconded.  I would definately NOT use localhost in your configs, or your scripts.
Be specific and you will save yourself some confusing problems.

--
"Be excellent to each other"

[TheJonathan]

Thanks for everyone's help! I just switched over my sessions across
the load-balanced servers to memcached and it worked perfectly. I
think the pages even load a little faster without the extra hits to
the database on every page. Can't wait to see how this affects
performance during peak traffic.

Cheers,
Jonathan

On Oct 30, 10:17 pm, "Clint Webb" <webb.cl...@gmail.com> wrote:
> Seconded.  I would definately NOT use localhost in your configs, or your
> scripts.
> Be specific and you will save yourself some confusing problems.
>

> On Fri, Oct 31, 2008 at 5:27 AM, Simone Busoli <simone.bus...@gmail.com>wrote:

[Simone Busoli]

memcached is extremely fast. We're using it as a second level cache, level one being in-memory cache (System.Web.Caching.Cache), and we can hardly notice any performance difference.

[Henrik Schröder]

On Thu, Oct 30, 2008 at 9:26 PM, TheJonathan <mit...@gmail.com> wrote:

I think I get it.  So if I'm passing in the array of memcached IPs via
a web.config, make sure they are in the same order on all load-
balanced machines.  e.g.
web1 = localhost, web2
web2 = web1, localhost

As long as you use the same names everywhere you will be fine. If you were using my client, it can take either "host", "ip", "host:port" or "ip:port" when you setup the list, and it will then use those strings that you supply as the basis for the server mapping algorithm, so those strings have to be the same in all clients. You could for example configure it with "localhost", "localhost:11211" and "127.0.0.1", and it would treat that as three different servers, although the lookup for all three returns the same ip and port.

@Kevin: Is the hashing algorithm you mentioned something I would have
to implement myself, or a feature?  I was going to use the enyim
Memcached client in .NET 2.0 http://www.codeplex.com/EnyimMemcached/
but I could be swayed.

This is not the .Net client you are looking for, you want use my client: http://code.google.com/p/beitmemcached. You are feeling very sleepy, and very swayed! :-)

Yes, all memcached clients implement a hashing algorithm for you, that's not something you need to worry about. Some clients let you plug in your own in the rare event that you would need that.


/Henrik Schröder

[TheJonathan]

On Oct 31, 6:48 am, "Henrik Schröder" <skro...@gmail.com> wrote:
> This is not the .Net client you are looking for, you want use my client:http://code.google.com/p/beitmemcached. You are feeling very sleepy, and
> very swayed! :-)

Must...use...BeIT Memcached. *drool*

To be fair, I'm currently using Enyim's client built-in to this .NET
HTTP Provider release (http://www.codeplex.com/memcachedproviders/
Release/ProjectReleases.aspx?ReleaseId=10468) but I'll definitely try
out BeIT for the rest of my caching needs. I'm the madman behind
ScribbleLive.com so we need all the juice we can to hold-up to the
traffic spikes around Apple Keynotes ;)

@Simone: I'm using System.Web.Caching.Cache now for a few small
lookups that don't require consistency across my load-balanced
servers, so I might stick with that too but use memcached for the
larger database operations.

[Clint Webb]

Are you sure?  When hashing the key, to determine which server to send it to, on some machines it will put key 'foo' on server web3, but on web3 it will put it on localhost?

It just doesn't seem like good practice to me to access a member of a cluster by using localhost, when other clients will be accessing it differently.   Especially if you do like I do and have a single config file that is rsynced to all the cache servers.

I'm not sure I see what actual gain you get by using localhost.

[TheJonathan]

I followed everyone's suggestions in here and didn't use localhost
after-all. web1 and web2 both use the server settings: 192.168.1.1
and 192.168.1.2 (in that order). Works like a charm ;)

On Oct 31, 9:55 am, "Clint Webb" <webb.cl...@gmail.com> wrote:
> Are you sure?  When hashing the key, to determine which server to send it
> to, on some machines it will put key 'foo' on server web3, but on web3 it
> will put it on localhost?
>
> It just doesn't seem like good practice to me to access a member of a
> cluster by using localhost, when other clients will be accessing it
> differently.   Especially if you do like I do and have a single config file
> that is rsynced to all the cache servers.
>
> I'm not sure I see what actual gain you get by using localhost.
>
>
>

> On Fri, Oct 31, 2008 at 7:48 PM, Henrik Schröder <skro...@gmail.com> wrote:

> > On Thu, Oct 30, 2008 at 9:26 PM, TheJonathan <mitc...@gmail.com> wrote:
>
> >> I think I get it.  So if I'm passing in the array of memcached IPs via
> >> a web.config, make sure they are in the same order on all load-
> >> balanced machines.  e.g.
> >> web1 = localhost, web2
> >> web2 = web1, localhost
>
> > As long as you use the same names everywhere you will be fine. If you were
> > using my client, it can take either "host", "ip", "host:port" or "ip:port"
> > when you setup the list, and it will then use those strings that you supply
> > as the basis for the server mapping algorithm, so those strings have to be
> > the same in all clients. You could for example configure it with
> > "localhost", "localhost:11211" and "127.0.0.1", and it would treat that as
> > three different servers, although the lookup for all three returns the same
> > ip and port.
>
> > @Kevin: Is the hashing algorithm you mentioned something I would have
> >> to implement myself, or a feature?  I was going to use the enyim

> >> Memcached client in .NET 2.0http://www.codeplex.com/EnyimMemcached/