Redis proxies for Redis clusters ?

[George L]

Hi I have started to play with Redis cluster configurations using my own custom written redis-generator.sh shell script i.e. 9x redis node cluster setup https://github.com/centminmod/centminmod-redis/blob/master/redis-cluster-9.md :)

My question is what redis supported proxies are out there besides Twitter's Twemproxy https://github.com/twitter/twemproxy which I already use ?

I found

- Corvus https://github.com/eleme/corvus

- Codis https://github.com/CodisLabs/codis

Any other Redis proxies I should be taking a look at ? :)

cheers

George

[Salvatore Sanfilippo]

I thought no "Redis Cluster" proxy existed, but Convus looks like the
thing you are looking for. I've zero experience with it, discovered it
thanks to your email.
Anyone within the list used Convus maybe?

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to redis-db+u...@googlegroups.com.
> To post to this group, send email to redi...@googlegroups.com.
> Visit this group at https://groups.google.com/group/redis-db.
> For more options, visit https://groups.google.com/d/optout.



--
Salvatore 'antirez' Sanfilippo
open source developer - Redis Labs https://redislabs.com

"If a system is to have conceptual integrity, someone must control the
concepts."
— Fred Brooks, "The Mythical Man-Month", 1975.

[George L]

Hi Salvatore 

I also found Netflix's Dynomite https://github.com/Netflix/dynomite 

So I started benchmarking them Corvus vs Dynomite vs Twemproxy and straight Nginx TCP proxy https://github.com/centminmod/centminmod-redis/blob/master/examples/redis-proxy-benchmarks-160317.md. Not really scientific and well still new to Redis proxy solutions other than Twemproxy :)

Corvus doesn't seem to be fairing that well though.

Interesting to hear what other folks have tried or recommend :)

cheers

George

[The Real Bill]

On Wednesday, March 15, 2017 at 3:50:28 AM UTC-5, George L wrote:

Hi I have started to play with Redis cluster configurations using my own custom written redis-generator.sh shell script i.e. 9x redis node cluster setup https://github.com/centminmod/centminmod-redis/blob/master/redis-cluster-9.md :)

My question is what redis supported proxies are out there besides Twitter's Twemproxy https://github.com/twitter/twemproxy which I already use ?

I found

- Corvus https://github.com/eleme/corvus

- Codis https://github.com/CodisLabs/codis

Are you looking to proxy Redis Cluster, or a proxy which provides a "cluster of Redis servers"?  The distinction is important. Codis, for example does not run Redis Cluster, it is a clustering proxy. I wrote an analysis of it a couple years or so ago at http://www.iamtherealbill.com/2015/04/clusterizing-redis-codis/ which really went into it at the time. Codis requires specifically modified Redis source, so arguably isn't really a true Redis proxy. It also makes some specific choices such as "no automated promoting of slaves when a master fails", which for some people does not meet "cluster" criteria; YMMV.

XCodis is a port of Codis that doesn't rely on modified Redis sources, but consequently lacks some of Codis' more significant features. It also hasn't been touched in a couple years.

For straight TCP pretty much any TCP level proxy, including Nginx and HAProxy, can work, it is primarily a matter of configuration - so long as you aren't talking about Redis Cluster, but a "cluster" of Redis nodes. Dynomite falls under the "collection of Redis nodes" category as well.

Corvus is new w/in the last year so I've not had time to analyze it. Perhaps I can add doing that to my queue for the next few weeks if there is interest. However, I note they only benchmarked get, set, and mset. These are not really what you want to see in a Redis benchmark as it doesn't really measure the capacity of a given install (for starters, in Redis something like 85% of the time in a get call is in the  Linux kernel). I see it also claims to handle cluster management and this brings me to another factor: how reliable is the management? While Twemproxy and TCP process require you either write your own or use Sentinel, when they are inbuilt you really need to consider how, and how well, they handle node failure. For example, for Corvus their benchmarking is suspect to me, and is missing the failure condition.

For example they list their config as 1 corvus server in front of 3 machines, each with 60 instances of Redis, half of each are masters. How the slaves are distributed is not made clear and depending on the test (especially w/Redis benchmark), wholly misleading. Further, the most critical aspect of such an arrangement is what happens when one of those three nodes dies. You now have 30 promotions to make.What would the effect on Corvus performance of a simultaneous failure of 30 masters? I am not picking on Corvus here, merely pointing out the bits conspicuously absent. Indeed, if as I understand it, Corvus lets Redis in Cluster mode handle the promotion and regarding, then it has an easier time for this. But it will still have an impact on that lone Corvus server. Any time I see "benchmark" configurations like that I get leery of the results. Reduce the suite of test to three of the most basic commands, and suspicion breeds. Fundamentally, a Redis Cluster proxy is essentially a connection relay state machine, so the overhead should be pretty lightweight if done right.

Really the primary factor is in determining what your specific functional requirements are. For example if you need to perform operations that don't work in a given proxy/cluster solution that rules out certain options. If all of our data fits in memory and writes don't exceed a single host, perhaps Cluster is overkill and a Redis pod w/Sentinel+TCP proxy is the best solution. Conversely, if you know you will have very high writes and/or more data than fits in memory, that eliminates other types of solutions. Personally, I'd nail down that level of requirement before comparing proxies and "Clustering Redis projects". The time saved in testing solutions that wouldn't meet the requirements can be used for more thorough vetting of ones that do.

Cheers,

Bill

[George L]

Thanks Bill for the insight and info. 

For purpose of just testing and learning what is optimal, I'm actually looking at both proxy Redis cluster and proxy a cluster of Redis nodes solutions and just trying them all out :)

My only real world Redis usage right now is for Wordpress for full page Nginx level Redis caching, Piwik Analytics and Xenforo forum addon for Redis Cache https://xenforo.com/community/resources/zend-redis-cache.3965/ where the Xenforo addon author is building out alot of Redis cache backed addons to improve Xenforo forum performance and scale for things like view counters, flood checks, session cache, deadlock advoidance in mysql etc - including Redis master/slave. So that's the context I'd be testing configs for right now.

[joong dev]

hi.

First of all, sorry to contact you suddenly.

I searched for a redis cluster proxy and came across these mail.

In this email, you were looking for a redis cluster proxy.

Can you let me know if you find a good proxy?

Currently I have found a few proxy(redis-cluster-proxy, corvus, twemproxy.. etc...) but it all seems to be unmanaged.

Do you have any good ideas?

I'm Korean, so now it's time to fall asleep, but I think it's morning where you are, So leave an email like this.

Thank you and have a nice day.

2017년 3월 16일 목요일 오후 3시 40분 59초 UTC+9, George L 님의 말:

[Benjamin Sergeant]

I believe redis-cluster-proxy is the most active one, it's a bit quieter now.

I used it for a while but I've now changed my redis client to be cluster aware so I do not need it as badly as before.

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/a4263489-a5a0-44f5-ad2d-fb1e01e28f65o%40googlegroups.com.

[joong dev]

Thank you comment fastly.

I know redis-cluster-proxy is most active but this proxy don't update cluster infomation after failover. 

And very slow(qps 30% lower than cluster connect directly. Predixy just 2% lower.). 

So I can not use this proxy.

You mean that connect redis client to redis cluster directly? For example, client use library likes go-redis, node-redis etc...

If don't use redis proxy, client connect to all cluster nodes.

And after failover, update cluster infomation in client manually.

I think his is very very uncomfortable.

Why don't you use  some redis proxy?

Why other people don't find redis proxy?

If many developer have found redis proxy, I think redis proxy project more active.

2020년 6월 18일 목요일 오전 9시 18분 45초 UTC+9, Benjamin Sergeant 님의 말:

To unsubscribe from this group and stop receiving emails from it, send an email to redi...@googlegroups.com.

[Benjamin Sergeant]

In case of failure in the cluster, the client will have to update its internal table of connection, that's fine no ?

This shouldn't be the common case.

I didn't know about https://github.com/joyieldInc/predixy / thanks for bringing it up. It seems maintained too.

I searched for XREAD in that code, and couldn't find anything, so that proxy won't support streams well, as special treatment is required for streams to lookup the key so that it can be hashed with crc16  (that's a bit unfortunate btw).

To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/baf8821a-ed48-4cc1-8e56-4c65524e94beo%40googlegroups.com.

[joong dev]

OK Thanks a lot.

I miss check whether predixy doesn't support redis stream. Thanks for bringing it up too.

I think the work "[manually] update its internal table of connection" is problem even if it's easy.

If I can use redis proxy, then I just check proxy info because cluster info is managed by proxy.

So very simple client side.

2020년 6월 18일 (목) 오후 2:25, Benjamin Sergeant <bser...@gmail.com>님이 작성:

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/F8ECE48C-B44B-47F0-9E6A-2147747E3CA9%40gmail.com.

[Benjamin Sergeant]

If I wasn't clear, I mean it's up to the Redis client/library author / maintainer to do this bookkeeping work, not to the application developer.

But you are right that it's always easier when the client can be dumb. There is something called Envoy too which you should look up, it's very active but needs to run on every cluster I believe, so the configuration won't be straightforward -> https://github.com/envoyproxy/envoy

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/CAC7DAiz6gAF6m20r-cZ3XTwMb1Nrce_UP1%2BdtpKbd5YYqs4rHg%40mail.gmail.com.

[slipinski]

Just FYI, per the Redis6 Release notes (https://raw.githubusercontent.com/antirez/redis/6.0/00-RELEASENOTES), the redis-cluster-proxy is the (semi?-)official proxy.  Here's the excerpt: 

 

 * A Redis Cluster proxy was released here:

  https://github.com/artix75/redis-cluster-proxy

That now redirects to RedisLabs github, but still redis-cluster-proxy.

[joong dev]

Thx guys.

@benjamin

Envoy very active. But this is just redis. Not cluster.

As I understand, Envoy has key slot, just redirect request. So they cannot failover and scale in/out.

@slipinski

Yes. I saw that page.

This link(https://github.com/artix75/redis-cluster-proxy) redirect to RedisLab's proxy page.

And the proxy in redirected page is already tested.

I will look for more and think about it.

2020년 6월 19일 (금) 오전 12:40, slipinski <steve.l...@nokia.com>님이 작성:

--

You received this message because you are subscribed to the Google Groups "Redis DB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to redis-db+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/edea566f-fd94-473a-b7b1-837731fb3d01o%40googlegroups.com.

[Benjamin Sergeant]

I think there was a presentation this year at redis conf 2020 about envoy adding redis cluster support.

To view this discussion on the web visit https://groups.google.com/d/msgid/redis-db/CAC7DAixGtN1cHnjh3Eduoeh7EWhnbcz8jc_HKPgpqpeNM9XGKQ%40mail.gmail.com.