Counters

[maarten]

Hi,

I have a simple use case where I'd like to use Voldemort: I want to
keep track of a counter per user, with possibly lots of changes.

The easiest way to think of this is as a total running counter of
resources used - like when you're measuring storage and want to add
and subtract deltas. I can see how I can ad delta using put with a
versioned value, and read+retry until I succeed on ObsoleteException,
but afaict there is no way of not getting into an endless loop (or
worse, multiple).

Which leads me to the following conclusions:
1) I am missing something (pretty good chances ;-)
2) Voldemort is not the right tool for this particular use case. Under
high load, what would be (let's define high load as 50k-500k changes/
second).

*Any* thoughts are greatly appreciated.

Thanks, Maarten

PS: I will be logging as well, I just want to keep a running total of
resources used.

[Alex Feinberg]

Hi Maarten,

We actually have an applyUpdate method in StoreClient, which is meant
to simplify doing counters or other updates that require optimistic
locking (which is performed via conditional updates using vector
clocks)

http://project-voldemort.com/javadoc/all/voldemort/client/StoreClient.html#applyUpdate%28voldemort.client.UpdateAction%29

You can specify a maximum number of times to perform a retry (you can
make a reasonable guess based on the number of connections made by all
the clients to the server). You are right in that building high
contention counters is difficult in a distributed system. You could
also use an external tool such as ZooKeeper to obtain a lock on a key
used to hold a counter (in this case, this becomes a pessimistic
lock), but of course that will imply a lower throughput.

- Alex

> --
> You received this message because you are subscribed to the Google Groups "project-voldemort" group.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to project-voldem...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/project-voldemort?hl=en.
>
>

[Maarten Koopmans]

Hi Alex,

Thanks, but I think the rollback is for a complete failure, right?
What I want is applyDelta, where the delta gets added to the value. So
that would mean a loop like:

While !(maxtries && success) {
success = try update
If ! success { getlatest version, add delta to it, ready for next update}
}

If ! success { secondary backlog}

Or does applyUpdate do this already if Iput the getLatestversion and
add delto to that as next try for the updateto the rollback() method

Thanks, Maarten