atomic master cutover

[Jak Sprats]

Hi,

a while back, Antirez blogged about pre-sharding:
http://antirez.com/post/redis-presharding.html

it then came up that there were some dangers in this approach
http://groups.google.com/group/redis-db/browse_thread/thread/b48f6a9fd664d228/c3b794bab135f632
@ the end of this blog, I suggested a PURGE command.

I implemented the PURGE command, it blocks and waits until ALL slaves
have the same dirty counter as the master and then shuts the master
down.

W/ a correct ha_proxy setup, where request fails over to the slave
when the master goes down, the PURGE command will insure the slaves
are up to date when the master shuts down (there is no race
condition).

EXAMPLE ONE:
1.) start a normal alchemy-server on port 6379
2.) start a slave alchemy-server on port 6380

CLI=./alchemy-cli
$ S_CLI="./alchemy-cli -p 6380 "
$ $CLI SET A A
OK
$ $CLI KEYS \*
1) "A"
$ $S_CLI KEYS \*
1) "A"
$ $CLI DIRTY
(integer) 1
$ $S_CLI DIRTY
(integer) 1
$ $CLI SET B B
OK
$ $CLI DIRTY
(integer) 2
$ $S_CLI DIRTY
(integer) 2
$ $CLI PURGE
OK
$ $S_CLI SLAVEOF NO ONE
OK

SLAVE is NOW the MASTER (the master has been purged & shutdown & the
slave has freed itself)

EXAMPLE 2: SIMULATE LAG
1.) in a seperate window, have the command " $S_CLI DIRTY 1" ready to
fire
2.) rm dump*
3.) start the servers again

$ $CLI DIRTY
(integer) 0
$ $S_CLI DIRTY
(integer) 0
$ $CLI SET A A
OK
$ $CLI DIRTY
(integer) 1
$ $S_CLI DIRTY
(integer) 1
$S_CLI DIRTY 0 # set the slaves dirty count to 0 - simulate lag
$ $CLI PURGE
------------------> at this point the master is asking the slave every
10ms if its dirty count is 1

in the other window:
$S_CLI DIRTY 1 # sets the slave dirty count to 1 -> simulated lag is
over
(integer) 1
-----------------------------> the "$CLI PURGE" will return w/ an
"+OK" and the master will shutdown

I hope this makes sense to people, it is incredibly useful to be sure
that data has been synced between master and slave before promoting
the slave to master.

This also allows alchemy mitosis, where a single alchemy has two
slaves, the master is then PURGEd, and the 2 slaves are then used w/ a
new hashing algorithm (e.g. master used to get all requests, now
new_master1 gets keys%2==0 and new_master2 gets keys%2==1)

- jak