Applying eventually consistent on Voldemort

[anderson guo]

Hi, all

As I know, Voldemort is a reference implementation of Amazon Dynamo,
which emphasize the eventually consistent. However, If I'd like to use
Voldemort as the Shopping Cart or Goods ordering system, How could I
ensure the consistency even there are multiple access towards the
share data, like quantity in store and cart items in shopping carts?

BR/anderson

[Geir Magnusson Jr.]

On Jan 20, 2010, at 7:08 AM, anderson guo wrote:

> Hi, all
>
> As I know, Voldemort is a reference implementation of Amazon Dynamo,

Well, it's not really a "reference implementation" in the spec sense. It's just an implementation.

> which emphasize the eventually consistent. However, If I'd like to use
> Voldemort as the Shopping Cart or Goods ordering system, How could I
> ensure the consistency even there are multiple access towards the
> share data, like quantity in store and cart items in shopping carts?

I use it for shopping cart and handling order processing. We carefully designed as well as we could around the consistency issue, but it does actually come up from time to time. You just have to either build around it and accept the limits, or choose another solution.

One solution, which we're considering for a related use, is some kind of 'lock management' that is used along side V, but right now we're still in the handwaving/whiteboard phase...

geir

>
> BR/anderson
> --
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to project-voldem...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/project-voldemort?hl=en.
>
>

[anderson guo]

Hi, Geir

First of all, Thanks for you quick reply.

Second, I don't fully understand what you mean lock management
technology. One mind I learned from Amazon Dynamo paper is make read /
write operation always available, hence lock is evil. Why you
introduce lock in V?

At last, could you give a little hint on what you said carefully
designed for shopping cart and order processing? I think pseudocode is
enough for me to understanding.

BR/anderson

[Dan Ostrowski]

Saying "lock is evil" is nonsense. What you're asking for is your partition tolerant, available data store to be consistent, so that you can do the transactional behavior of selling a product, but according to CAP theorem, that's not possible.

geir, above, is suggesting you lock your resource (such as a product list), briefly, using either, I would imagine, Redis or Zookeeper so that you can be assured you don't oversell. However, this doesn't work, in reality. (An acquaintance of mine wrote about this here: http://aphyr.com/posts/254-burn-the-library).

Amazon has this very problem and, famously, their solution is "coupons!" They just give coupons out whenever their shopping cart system screws up, or that's how they solved it for a while. The solution may be to simply move product management OUT of your Dynamo storage and into something less available, but more consistent like Redis or even a regular DB. Depending on the focus of your site, this might work fine.

Dan

[Lei Gao]

Hi Anderson,

It seems that there are two issues you are facing - one is the centralized inventory and one is the shopping cart items.  I think the solution for each problem can be different.

The system is design of provide high write availability so you never reject a customer request that adds items into the shopping cart. For this purpose, you can configure you store as 2-1-1 and write a customer conflict resolution to always merge the items in a shopping cart when concurrent 'adds' occurs to a shopping cart.

For the global inventory management, you have a couple options. 

• Use 2-1-1 configuration and prepare to back-order for the customer when you sell more than what you have.
• Use 3-2-2 configuration and prepare to update the inventory in the database when inventory decreases more than what you've actually sold. 
• Partition your inventory instead of replicating it for high availability. But you will have to do some additional logic to re-adjust the inventory among partitions over time.

Thanks,

Lei