Redis cluster auto scaling - Help required

[sha...@peer5.com]

Hello,

Our current setup consists of 3, Quad-Core, 8GB servers.

Each server runs two instances of Redis 3.0.3.

One of the instances is configured as master and the other as slave (Of a different server master).

All of the instances are in the same cluster.

So it looks something like this:

Server 1: Master1, Slave2

Server 2: Master2, Slave3

Server 3: Master3, Slave1

That way we make sure that if any server drops, the cluster won't fail.

The problem is that due to our usage (Tons of concurrent requests), the servers are running at high CPU loads most of the times. (Memory is not an issue, as we are barely using 500MB of it).

We are using Google Cloud Compute Engine, which have the option of autoscaling, which means that when ever the servers cross a specified CPU threshold, a new instance can be created to help with the load.

The thing is, we're trying to find the best way to automatically handle new instances and removed instances, by auto-migrating slots from the living cluster to the new instance.

I understand that there is no such feature currently, but i was wondering if anyone got a chance to write some kind of script that will use redis-trib to figure out the current cluster situation and allocate slots accordingly, maybe something we can run as a cron job or something.

We don't mind about which slot gets where, as all keys are mostly equal and have the same performance issues.

Any help will be greatly appreciated.

[ddorian43]

Can you describe how you are usin it (maybe there is a more efficient way ?) ?

Cool product by the way.