Re: Best memcached proxy to use
Guille -bisho-
storage. Something to reduce number of connections of the memcache
servers, and that can be used as multi-layer memcache infraestructure.
--
Guille -ℬḭṩḩø- <bish...@gmail.com>
:wq
Guille -bisho-
I would like to know which memcached proxy is the most recommendable
option. Is there anyone out there using proxies for memcache?
I have read good things about moxi, but looks a bit abandoned
recently, and doesn't sport yet live reconfiguration options. Any
other suggestion?
Thanks a lot!
pub crawler
solution for memcache.
Unsure if Moxi functionality has been upgraded and rolled into
Northscale's latest creation - Membase. (Waiting for docs for membase
and some additional user testing before taking the dive into it).
http://labs.northscale.com/membase/
> --
> You received this message because you are subscribed to the Google Groups "memcachedb" group.
> To post to this group, send email to memca...@googlegroups.com.
> To unsubscribe from this group, send email to memcachedb+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/memcachedb?hl=en.
>
>
Tom Chen
Guille -bisho-
> How many connections are you planning to use with your memcache db boxes?
> Memcache protocol is pretty fast, and the overhead for connections should be
> pretty lite. You might find that you might not need a proxy.
We need... We are using PHP fastcgi on web frontend servers what means
one persistent TCP connection by child to each memcache server. With
hundreds of frontends we are reaching the limit in TCP connections on
the memcache servers and using gigs of memory just in tcp connection
pool memory, thus reducing the memory that could be allocated for
data. We are also trying UDP for reads and non-persistent connections
for writes, but memcache have some constraints (a single process
limits the throughput when using UDP) and we have found the network
card driver to become unstable with heavy UDP traffic.
Guille -bisho-
> Moxi is stable - we use it. Haven't found another more current proxy
> solution for memcache.
I'm unable to find information on the libconflate and management
channel to change the configuration live, without restarting moxi. Are
you using this? Any clue?
> Unsure if Moxi functionality has been upgraded and rolled into
> Northscale's latest creation - Membase. (Waiting for docs for membase
> and some additional user testing before taking the dive into it).
> http://labs.northscale.com/membase/
Looks interesting... Like memcache but with persistent storage and
better management. I wonder if implements also list operations like
redis, but without the scalability problems of redis...
pub crawler
Shouldn't take more than an hour to implement. Moxi is very much
memcached-like and you are already familiar with that.
I'd advise installing Moxi on a spare machine like I assume you are
doing for memcached. Recommend trying to isolate memcached on
dedicated servers always in high activity connection scenarios for
reasons like you have experienced.
This should protect your memcached instances some and give you
headroom and preserve your cached data from failures you might be
experiencing when NIC failures happen as described.
Sounds like each PHP process is making a connection to each and every
memcached server you have for each request. That's massive overhead
and prone to issues.
If you throw Moxi in there your PHP scripts are only going to make one
connection and that is to Moxi and everything is transparent to them.
Great reduction in connections and network activity.
Come back once you have installed Moxi like this and let us know how
it did or didn't help.
pub crawler
> channel to change the configuration live, without restarting moxi. Are
> you using this? Any clue?
Good question. We have never needed this functionality in our environment.
It's per se mentioned in a slide presentation on Moxi:
http://www.slideshare.net/northscale/moxi-memcached-proxy
See slides 18,19 and 20.
> Looks interesting... Like memcache but with persistent storage and
> better management. I wonder if implements also list operations like
> redis, but without the scalability problems of redis...
Membase is very current - in past week release. The wiki and docs are
still formulating. Unsure if they have rolled Moxi functionality into
it, but assuming they have since it seems to build on all that
Northscale has done to date which is inclusive of Moxi.
I'd recommend contacting Northscale on the membase list too about the
Moxi functionality therein - because it appears Membase is very much
the suite on the open source side of the shop and gives management
tools which have been lacking for memcached to date.
Guille -bisho-
> memcached server you have for each request. That's massive overhead
> and prone to issues.
We use persistent connections, so connections are stablished as needed
(not per request), but during the child lifespan the number of
connections grows till one per memcache server, as you pointed out.
When scalling the number of frontends and the number of fastcgi
children is also high, the number of connections are huge.
> If you throw Moxi in there your PHP scripts are only going to make one
> connection and that is to Moxi and everything is transparent to them.
> Great reduction in connections and network activity.
We want to try that approach: one moxi proxy per frontend, to limit to
just one the number of connections a single frontend stablishes with
each mc, instead of the current <number_of_php_children>.
> Come back once you have installed Moxi like this and let us know how
> it did or didn't help.
Sure!