I’ll jump on this discussion with some details that may help to understand how the code works.

There is an overview of how the code work in the win32_cow.c file (included below).

The code defers deletion of objects during the save – if the ref count is going to be 0, the reference is added to a list. After save is done, it decrements the ref count of all objects on the list.

Some objects are modified in place rather than deleted. Typically collection objects such as sets, lists, hash tables. For these objects the code makes a read-only version of them before they are modified. The read-only version is a more efficient storage as it doesn’t need to support modifications. The save code will use the read-only version if it exists –otherwise it uses the normal collection.

The code uses special iterators for the saving code.

    - If the collection being iterated is not modified, it acts as a normal iterator.

    - If a read-only version copy, it iterates on the read-only copy.

   - If a read-only copy is created while the save is in the middle of iterating on the original collector, the iterator is modified to switch to the read-only copy. Locks are used to ensure this is done safely.

As a result, if no updates are done during the save, there is very little extra overhead (no copy).

If some updates are done, there is some copying required, but only for the collection being modified.

If the same collection is modified n times during a save, only the first modification results in a copy.

The special read-only encoding of the collection is used to reduce the cost of allocating, copying and then freeing the copy. A collection with thousands of entries would normally require thousands of allocate and frees. With the read-only encoding, it requires 1. It also uses less memory.

This is the best we came up with to replicate a COW-like behavior without changing drastically Redis code and hence make very difficult any future integration.

Salvatore, Dusan, All: Suggestions and other ideas on how we can make it better are welcome.

Claudio /************************************************************************

* This module implements copy on write to support

* saving on a background thread in Windows.

*

* Collection objects (dictionaries, lists, sets, zsets)

*  are copied to a read-only form if a command to modify the

*  collection is started. This is triggered via lookupKeyWrite().

*

* Objects which are modified in place - ziplist, zipset, etc.

*  are copied before being modified.

* Strings are normally copied before being modified.

*

* In addition deletion of objects is deferred until the save is completed.

*  This is done by modifying the dictionary delete function, and also

*  by modifying the decrRefCount function.

*

* To allow conversion of collections while the save is iterating on them

*  special iterators are used. These iterators can be migrated

*  from their normal mode to iterating over a read-only collection.

*  Locking is used so that iterator can be used from 2 threads.

*  For migration to work properly, only one save at a time may run.

*   (this restriction was already imposed in the Redis code)

*

************************************************************************/

Sent from my Windows 8 PC

From: Dušan D. Majkić
Sent: Monday, April 30, 2012 8:02:15 AM
To: redis-db@googlegroups.com
Subject: Re: Redis for Windows is now stronger

--
You received this message because you are subscribed to the Google Groups "Redis DB" group.
 To post to this group, send email to redis-db@googlegroups.com.
 To unsubscribe from this group, send email to redis-db+unsubscribe@googlegroups.com.
 For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.