Re: a question about memcached

[Brad Fitzpatrick]

[+memcached list]

On Tue, Nov 11, 2008 at 12:09 PM, Lei Gao <gao...@gmail.com> wrote:

Hi Brad,

I am considering integrating memcached tightly with a database, mysql or oracle. I.e. I want the database to actively send updates to the memcached server. One way to do this is to create a memcached client program that gets the data from the database and store them into the memcached server via the client set API.

Some concerns with this approach are 1. the client API seems limited to what I want to do in terms of keeping memcached server in sync. with the database; 2. the client API might be less efficient for loading data onto the server if the in-coming rate is high.

Is it possible to create a memcached server extension to interface directly with the database?

Thanks in advance.

Lei 

On Mon, Jun 23, 2008 at 4:00 PM, Brad Fitzpatrick <br...@danga.com> wrote:

It doesn't.

If that matters for you, don't cache that entity, or set a low expiration timeout.

On Mon, Jun 23, 2008 at 3:52 PM, Lei Gao <gao...@gmail.com> wrote:

Hi Brad,

I am an academic researcher in computer sciences at UT Austin. I have a simple question about memcached. (I got your email address at the end of one of your articles about memcached). How does the memcached handle the situation where the databased is updated from a non-memcached-aware party? For instance, my system admin updates the some database records that are cached in one or more memcached instance, how do those instances refresh themselves? (Another example is when the database is replicated and updates take place on all replicas.)

Thanks,

--
Lei Gao

--
Lei Gao

[Matt Erkkila]

A couple things popped into my mind when i read this, some may apply some may not.

I remember hearing that Facebook actually changed the parsing of MySQL query language to support passing memcache keys across data centers.  This was to prevent an object from being expired/reloaded before the data actually replicated from one data center to another.

So the SQL would look something like: UPDATE users SET password = ('somepass') EXPIRE memcachekey1, memcachekey2

Maybe someone can speak more to this, I think there was a blog post or a presentation released from them regarding this.

Also something to look into would be the MySQL UDF functions for memcached.  Last I saw they were available http://www.tangent.org/, although the site seems to be down for me right now.

Matt

[Aaron Stone]

Sean Chittenden once had a system for updating memcached from
PostgreSQL (or maybe it wasn't updating values, only expiring them?)
called pg_memcached. I don't believe it's been maintained in a long
time, but the idea is very similar.

Aaron

[Edward Goldberg]

You may want to look at mysql_proxy and write a LUA program that does this
update for you.

e.m.g.

[Brian Aker]

Hi!

On Nov 12, 2008, at 5:49 AM, Matt Erkkila wrote:

> So the SQL would look something like: UPDATE users SET password =
> ('somepass') EXPIRE memcachekey1, memcachekey2

You can use the memcached UDF's for MySQL to do the same operation.
You don't need to rewire the parser to get this sort of behavior.

Cheers,
-Brian

--
_______________________________________________________
Brian "Krow" Aker, brian at tangent.org
Seattle, Washington
http://krow.net/ <-- Me
http://tangent.org/ <-- Software
_______________________________________________________
You can't grep a dead tree.

[Brian Aker]

Hi!

On Nov 12, 2008, at 8:33 AM, Lei Gao wrote:

> Thanks for your update. Let's see if I want those UDF's to be
> invoked in my slave db when receiving data changes from the master
> db, would I have to invoke those functions within a trigger?

That is one way to do it. Personally I favor embedding the udf either
in the query itself, or in using a multi-sql issued set and having the
UDF run against a null table (aka blackhole).

> I was looking at the UDF's for mysql but could not find any function
> for invalidating memcached entries. Does it mean one can only delete/
> update existing entries with mysql UDF's?

Invalidation is just deleting the object in the cache. As soon as a
cache miss occurs your application should then just repopulate the
cache.

[Gabriel Sosa]

Hi guys,

I've been following this thread, I like the idea of manage mmc keys
from mysql, my only concern is, since mmc keys are managed by the
client (hash generation, etc), in my case I use PHP how compatible is
the key generation against the UDF mysql functions?
I mean, can I generate some key on PHP and erase it from mysql with
triggers without any problem?

tty for your time!

gabriel

--
Los sabios buscan la sabiduría; los necios creen haberla encontrado.
Gabriel Sosa