distributed memcached

[Tzury Bar Yochay]

Hi,

I got the following servers layout.

Couple of servers, with a load-balancer in the front, On each server I
have my own server application and mySql with replication.

I want to use memcached as the mechanism to share the clients sessions
between the servers.

I want to make sure that whatever I cache on one machine would be
cached on the other as well.

How can I do that in memcached?

thanks for your time and advise.

[Brian Moon]

what language?
what session manager?

Brian.
--------
http://brian.moonspot.net/

[Tzury Bar Yochay]

On Jun 30, 10:03 pm, Brian Moon <br...@moonspot.net> wrote:
> what language?
python
> what session manager?
do not use one but looking for a solution
I am not even sure if memcached would be the right tool for the task.
I actually want to make memcached redundant which according to the faq
it is not applicable.
So I guess I am going to set a session table in the database and
memcache it.

[Tzury Bar Yochay]

there are so many distributed/redundant session managers out there
but all of them are web oriented. cannot find anything generic enough
to use in my project

[Joseph Engo]

I think in most cases, memcache is the wrong place to put sessions.

[Josef Finsel]

In almost every case, memcached is the wrong place to put sessions.

If the session lives somewhere else and you are just using memcached as a true cache of the session, then you're fine. If not, and something happens to memcache, suddenly no one can log in. 

--
"If you see a whole thing - it seems that it's always beautiful. Planets, lives... But up close a world's all dirt and rocks. And day to day, life's a hard job, you get tired, you lose the pattern."
Ursula K. Le Guin

http://www.finsel.com/words,-words,-words.aspx (My blog) - http://www.finsel.com/photo-gallery.aspx (My Photogallery)  -http://www.reluctantdba.com/dbas-and-programmers/blog.aspx (My Professional Blog)

[Brian Moon]

> I am not even sure if memcached would be the right tool for the task.
> I actually want to make memcached redundant which according to the faq
> it is not applicable.
> So I guess I am going to set a session table in the database and
> memcache it.

If persistence is important, that is the right approach. We use
memcache for sessions at dealnews, but they are not ecommerce related or
mission critical information. It's not worth the DB write for us.

Brian.

[Tzury Bar Yochay]

> If persistence is important, that is the right approach.  We use
> memcache for sessions at dealnews, but they are not ecommerce related or
> mission critical information.  It's not worth the DB write for us.

High availability is the most important thing for us. Not persistence.
If memcached is not the solution can anyone point out about other
possible solutions?
What would be the best way to have a session saved in more than one
server?

[David Rolston]

That is not a good solution in my opinion.  Session data is typically temporal.  Do you really want to bog down your entire site in order to replicate all the session data?  When you state "High Availability" what exactly does that mean to you?  What are the use cases? 

Again, we don't know what language you're talking about, so it's hard to give you specifics without that information. 

One solution that hasn't been mentioned is to have the session data live on a robust NFS store -- Netapps are particularly well known in this area.  Netapps basically implement RAID 6, and will provide you great reliability, depending on what you want to pay.  You can iscsi or san connect them and get blazing performance.  Depending on the size of your session data set, you could also use a cluster database like the mysql NDB. 

[Henrik Schröder]

Well, that is not how memcached works, it's not replicated. If you have a cluster of memcached servers, and you store an item, it will be stored on one of the servers chosen semi-randomly, not all of them or some of them. However, the item is still available to all machines connected to the cluster, because each will know on which memcached server a specific item is located.

Also, I'm really curious as to why you have a replicated mysql slave on each webserver? It sounds like it would just cause a lot more problems than it solves?


/Henrik

[Henrik Schröder]

If availability is more important than persistence, memcached is exactly the right solution.

What we do in one of our projects is that when a session starts, we put a session object in memcached for that user. On each pageview for that user, we then fetch the session object from memcached, update it, and set it back. We also update the DB, but no more often than every 60 seconds, since persistence is nice, but not critical. When a session ends, we remove the session object from memcached (it also has an expiry time of 15 minutes), and that's about it.

As an added bonus, given a range of userids, we can answer the question "which of these users are online?" simply by doing a multi-get to memcached and see who has a session object and who hasn't.

And again, why do you want to save the session info in more than one place? Why is that important? You said it yourself that availability is more important than persistence?


/Henrik

[Xaxo]

You need to either write your own memcached client or use one of the
memcached proxies out there. Each memcached instance can only do get/
set/replace => no cluster functions at all (one memcached server
doesn't know about others), managing more than one instance/server is
a matter of the client that you use. So you either need a client that
writes to all servers in your pool simultaneously or you need a proxy,
which does the same.

Momchil

[Henrik Schröder]

Uh, you don't need to write your own client to do that, you only need to write a wrapper class that on set() iterates over all servers and does a set. It's not exactly rocket science... :-)

However, why on earth would you *want* to write something to all memcached servers in your cluster? It's insane, and it completely defeats the purpose of memcached in the first place.


/Henrik

[Josef Finsel]

If you really need high availability and you can run Microsoft products, check out Velocity. It's supposed to be a cache but it supports High Availability, among other things.

[Henrik Schröder]

Wait, what does that even mean when it comes to a LRU cache? It's a cache, if one of the servers in your cluster goes down, you'll get a slightly lower hitrate, but from some sort of systems perspective, you have 100% uptime?


/Henrik

[Syed Ali]

I guess memcache in this scenario is being used as a runtime db...

I know of a few apps doing that

On 7/1/09, Henrik Schröder <skr...@gmail.com> wrote:
> Wait, what does that even mean when it comes to a LRU cache? It's a cache,
> if one of the servers in your cluster goes down, you'll get a slightly lower
> hitrate, but from some sort of systems perspective, you have 100% uptime?
>
>
> /Henrik
>
> On Wed, Jul 1, 2009 at 21:40, Josef Finsel <carp...@gmail.com> wrote:
>

>> If you *really *need high availability *and* you can run Microsoft

>> products, check out Velocity. It's supposed to be a cache but it supports
>> High Availability, among other things.
>>
>>
>> On Tue, Jun 30, 2009 at 4:27 PM, Tzury Bar Yochay
>> <afro.s...@gmail.com>wrote:
>>
>>>
>>> > If persistence is important, that is the right approach. We use
>>> > memcache for sessions at dealnews, but they are not ecommerce related
>>> > or
>>> > mission critical information. It's not worth the DB write for us.
>>>
>>> High availability is the most important thing for us. Not persistence.
>>> If memcached is not the solution can anyone point out about other
>>> possible solutions?
>>> What would be the best way to have a session saved in more than one
>>> server?
>>>
>>
>>
>>
>> --
>> "If you see a whole thing - it seems that it's always beautiful. Planets,
>> lives... But up close a world's all dirt and rocks. And day to day, life's
>> a
>> hard job, you get tired, you lose the pattern."
>> Ursula K. Le Guin
>>
>> http://www.finsel.com/words,-words,-words.aspx (My blog) -
>> http://www.finsel.com/photo-gallery.aspx (My Photogallery) -
>> http://www.reluctantdba.com/dbas-and-programmers/blog.aspx (My
>> Professional Blog)
>>
>

--

--

-Syed Ali

[RogueWarrior]

WOW, this particular thread has been going around and around. The
real question is, do you need memcached for storing session data or
for caching data that is persisted elsewhere?

To me session data is state about the request(s) that is not stored in
a DB. For instance - multi page flows, where the state from the
previous page is not stored in a temp table, is commonly stored in the
HTTP session or in a server side session. In this case, if the data
is lost, the data is NOT able to be reconstructed, as it is not stored
anywhere.

Cached data is data that if lost can be easily reconstructed from
elsewhere.

memcached is NOT a great storage device for session data (unless it is
also stored in a DB, which would then make session data cached data).
Because as
stable as memcached is, the data could be evicted or access to the
server may be lost, because of a bad switch.

So are you storing session data or cached data?

On Jul 1, 3:01 pm, Syed Ali <alisy...@gmail.com> wrote:
> I guess memcache in this scenario is being used as a runtime db...
>
> I know of a few apps doing that
>

> On 7/1/09, Henrik Schröder <skro...@gmail.com> wrote:
>
>
>
> > Wait, what does that even mean when it comes to a LRU cache? It's a cache,
> > if one of the servers in your cluster goes down, you'll get a slightly lower
> > hitrate, but from some sort of systems perspective, you have 100% uptime?
>
> > /Henrik
>

> > On Wed, Jul 1, 2009 at 21:40, Josef Finsel <carpd...@gmail.com> wrote:
>
> >> If you *really *need high availability *and* you can run Microsoft
> >> products, check out Velocity. It's supposed to be a cache but it supports
> >> High Availability, among other things.
>
> >> On Tue, Jun 30, 2009 at 4:27 PM, Tzury Bar Yochay

> >> <afro.syst...@gmail.com>wrote:

>
> >>> > If persistence is important, that is the right approach.  We use
> >>> > memcache for sessions at dealnews, but they are not ecommerce related
> >>> > or
> >>> > mission critical information.  It's not worth the DB write for us.
>
> >>> High availability is the most important thing for us. Not persistence.
> >>> If memcached is not the solution can anyone point out about other
> >>> possible solutions?
> >>> What would be the best way to have a session saved in more than one
> >>> server?
>
> >> --
> >> "If you see a whole thing - it seems that it's always beautiful. Planets,
> >> lives... But up close a world's all dirt and rocks. And day to day, life's
> >> a
> >> hard job, you get tired, you lose the pattern."
> >> Ursula K. Le Guin
>
> >>http://www.finsel.com/words,-words,-words.aspx(My blog) -

> >>http://www.finsel.com/photo-gallery.aspx(My Photogallery)  -
> >>http://www.reluctantdba.com/dbas-and-programmers/blog.aspx(My

[sven bommezijn]

Well, I do not want to kick against any religion, but in grid
computing (like for example Java Spaces) memory is the new harddisk.
Since memory gets cheaper and processors are becoming faster I think
there IS some cause for a memcached like mechanism to be used in a
distributed, replicated way.

I am thinking of a configuration where different memcached servers
maintain a copy of themselves on secondary (or even more) machines.
In the unlikely event of one primary server going down the client just
refers to a secondary server.

Memcached entries might replicate themselves after a put or replace
operation is completed.
Particularly when Memcached is used for sessions it is unlikely that a
get operation will follow in the instant that a put or replace
operation has completed on the same entry.

I realize that this takes up extra network bandwidth and cycles but it
definately offers an interesting mechanism for load balanced
webservers hitting the same cache to retrieve session data. And if I
am not mistaken such an architecture would still be of O(1), that is
it still would grow lineairly with web traffic.

Anyone interested in a discussion about this? seems a nice optional
feature for Memcached.

[John Reilly]

If you are doing this in Java, you can use ehcache server.  Actually, your clients don't need to be java based.  The last time I looked there were no clients for doing the partitioning of requests to servers.  I think there is a diagram and description of this scenario on the ehcache web site.  I'm not using ehcache server myself though, so I can't say how good it is.  I prefer a combination of ehcache and memcache. 

Cheers,

John

[sven bommezijn]

Yep, that ehcache seems pretty sophisticated. However, I like the
simplicity of memcached.

I am not particularly bound to Java, I think Java is a great language
but has disadvantages in memory management and speed.
I was merely referencing the trend of using more in-memory
architectures which make sense these days.
I think that trivial, performance critical tasks are better done using
optimized native code and use high level environments to implement the
business logic.

It also would be great to conform to a well established interface
standard that memcached has already in place.
I found something called repcached which comes closest uptill now...
I need to look into that.

However, I might still be interested to develop some extension to
memcached that implements transparent replication as described
earlier.
If someone sees caveats or has suggestions I am really really
interested...

On Jul 3, 11:38 pm, John Reilly <j...@inconspicuous.org> wrote:
> If you are doing this in Java, you can use ehcache server.  Actually, your
> clients don't need to be java based.  The last time I looked there were no
> clients for doing the partitioning of requests to servers.  I think there is
> a diagram and description of this scenario on the ehcache web site.  I'm not
> using ehcache server myself though, so I can't say how good it is.  I prefer
> a combination of ehcache and memcache.
> Cheers,
> John
>

[Dustin]

On Jul 4, 2:33 am, sven bommezijn <sven.bommez...@gmail.com> wrote:
> Yep, that ehcache seems pretty sophisticated. However, I like the
> simplicity of memcached.

[...]

> However, I might still be interested to develop some extension to
> memcached that implements transparent replication as described
> earlier.
> If someone sees caveats or has suggestions I am really really
> interested...

You seemed to have answered your own question.