Redis replication

[bbroerman]

What is the approximate bandwidth / load for replication (say, as a
percentage or fraction of the overall transaction usage). Also, has 2-
way replication (between 2 peers instead of master/slave) been
considered?

I am planning on using Redis on a large high-throughput project (60k
simultaneous users out of a total 200M user base) that also requires
high availability..

[Josiah Carlson]

On Wed, Oct 20, 2010 at 11:47 AM, bbroerman <bbroe...@gmail.com> wrote:
> What is the approximate bandwidth / load for replication (say, as a
> percentage or fraction of the overall transaction usage).

Master/slave replication works by initially sending the current dump
to a slave, then sending all commands that are executed on the master
to the slave.

> Also, has 2-
> way replication (between 2 peers instead of master/slave) been
> considered?

Yes, it's called Redis clustering, and it's in-progress.

>  I am planning on using Redis on a large high-throughput project (60k
> simultaneous users out of a total 200M user base) that also requires
> high availability..

Number of users is inconsequential. What kind and how many operations
are you performing per second, over how many keys, with what kinds of
values?

Regards,
- Josiah

[Marcus]

> Yes, it's called Redis clustering, and it's in-progress.

I thought redis-cluster dealt more with proxying commands and
failover? It will also have master-master replication?

[bbroerman]

Ok.

Any idea when clustering will be available?

Statistics... for one cache they're all going to be simple string keys
with string values (of JSON). Key will probably be about 15 chars.
long, and the value will be around 30 - 40 characters long. Probably
about 2,000 to 3,000 writes a second, and most likely 70,000 - 80,000
reads per second. Keys will have a time to live of about 4 to 6
hours.

For another set of caches (for each shard) there will be about 10,000
writes a sec. under max load. Again, mostly short string keys, with
JSON encoded values (some will be around 100 chars in length, others
probably about 250 - 300 characters in length). There will be 3 or 4
lists (queues) that will be pushed onto about 1,000 times a second
from webservices, while having a few daemon processes continually
popping them off if there's anything in the queue... The lists will
all be a simple key name (string) and the values will be 10 - 15
character strings...

One thing I am trying to do, for the 1st cache, is come up with a
replication scheme that will provide almost instantaneous fail-over
(webservice can't connect to the master, so it connects to the slave).
I realize we may lose some data in the switch.... but that's
inevitable.

Now, with the snapshot files, when a failed redis server is brought
back up, will it re-load from the snapshot automatically?

Thanks...


On Oct 20, 3:45 pm, Josiah Carlson <josiah.carl...@gmail.com> wrote:

[Marcus]

I cannot find where it was said, but I remember hearing redis cluster
beta should be out within 6 months. Hopefully someone can confirm

[Josiah Carlson]

On Wed, Oct 20, 2010 at 1:34 PM, bbroerman <bbroe...@gmail.com> wrote:
> Ok.
>
> Any idea when clustering will be available?

When it's done ;) Salvatore is working hard, you can follow him on
Twitter for updates: @antirez .

> Statistics... for one cache they're all going to be simple string keys
> with string values (of JSON). Key will probably be about 15 chars.
> long, and the value will be around 30 - 40 characters long. Probably
> about 2,000 to 3,000 writes a second, and most likely 70,000 - 80,000
> reads per second. Keys will have a time to live of about 4 to 6
> hours.

If you are just using this as a cache, there isn't any reason why you
couldn't just shard this data based on whatever *really fast* hash
function is available for you, with fail-over slaves if a machine
can't contact the master for a particular shard.

> For another set of caches (for each shard) there will be about 10,000
> writes a sec. under max load. Again, mostly short string keys, with
> JSON encoded values (some will be around 100 chars in length, others
> probably about 250 - 300 characters in length). There will be 3 or 4
> lists (queues) that will be pushed onto about 1,000 times a second
> from webservices, while having a few daemon processes continually
> popping them off if there's anything in the queue... The lists will
> all be a simple key name (string) and the values will be 10 - 15
> character strings...
>
> One thing I am trying to do, for the 1st cache, is come up with a
> replication scheme that will provide almost instantaneous fail-over
> (webservice can't connect to the master, so it connects to the slave).
> I realize we may lose some data in the switch.... but that's
> inevitable.

This can be done on the client side. A web tries to connect/use the
master, fails over to the slave. When the client knows it's on the
slave, it occasionally (depending on your requirements) looks for a
particular key on the slave which says "retry the master".

If your webs are smart enough to notify you of a failure,
simultaneously tell your other webs "stop hitting the current master",
*and* cause the slave to stop slaving (woo for runtime config slave
configuration!), then when you have fixed your former master, you can
have *it* slave to your old slave/new master, and everything is
kosher.

> Now, with the snapshot files, when a failed redis server is brought
> back up, will it re-load from the snapshot automatically?

Yes. That's the point of the snapshots. One thing to be wary about
with regards to slaving, autoreconnects, etc.: We have had masters
die, with the slaves trying to reconnect. When the master finally
came up, it started to load it's snapshot, and the slaves got into a
funky state where they would connect, but read/write bad data to the
master and fail slaving, then retry. When in this state, the only
solution seemed to be to kill the slave. If you stop the slave from
slaving while the master is down, and only reconnect the slave to the
master when the master is really up and ready to serve clients, then
you won't run into the issue. I don't know if this has been fixed in
more recent versions of Redis than we are running/have tested (1.3.10
variant for our slaving Redis boxes).

Regards,
- Josiah

> On Oct 20, 3:45 pm, Josiah Carlson <josiah.carl...@gmail.com> wrote:
>> On Wed, Oct 20, 2010 at 11:47 AM, bbroerman <bbroerma...@gmail.com> wrote:
>> > What is the approximate bandwidth / load for replication (say, as a
>> > percentage or fraction of the overall transaction usage).
>>
>> Master/slave replication works by initially sending the current dump
>> to a slave, then sending all commands that are executed on the master
>> to the slave.
>>
>> > Also, has 2-
>> > way replication (between 2 peers instead of master/slave) been
>> > considered?
>>
>> Yes, it's called Redis clustering, and it's in-progress.
>>
>> >  I am planning on using Redis on a large high-throughput project (60k
>> > simultaneous users out of a total 200M user base) that also requires
>> > high availability..
>>
>> Number of users is inconsequential.  What kind and how many operations
>> are you performing per second, over how many keys, with what kinds of
>> values?
>>
>> Regards,
>>  - Josiah
>

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>