Introducing HyperDex Warp: HyperDex with ACID Transactions

[Robert Escriva]

We are proud to introduce HyperDex Warp, an add-on to HyperDex that
provides one-copy serializable ACID transactions.

HyperDex Warp provides a true transactional interface. A Warp client
has access to all of the datatypes and key-based operations that
HyperDex offers.

What makes these properties possible is a novel technique called linear
transactions, that build on the value-dependent chaining technique
pioneered by HyperDex.

Evaluation packages of HyperDex Warp are available for immediate
download on Debian 6, Ubuntu 12.04 and Fedora 17. Simply install
"hyperdex-warp" and let your package manager do the rest. More
information about HyperDex Warp is available at
http://hyperdex.org/warp.

Thanks,
The HyperDex Team

[Emin Gün Sirer]

HyperDex core is and will always remain open source. 

Warp, which adds transactions, is an add on. It's a model similar to Riak: Riak core is open, and Basho sells licenses value-added plugins.

On Wed, Feb 6, 2013 at 9:20 AM, Aleksander Bandelj <aleksande...@gmail.com> wrote:

This clearly has enormous potential, but the download page gives me impression that it won't be provided as open source. Correct ?

On Tuesday, February 5, 2013 6:27:46 PM UTC+1, Robert Escriva wrote:

We are proud to introduce HyperDex Warp, an add-on to HyperDex that
provides one-copy serializable ACID transactions.

--
You received this message because you are subscribed to the Google Groups "hyperdex-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hyperdex-discu...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Robert Escriva]

Hi Maxi,

I'm glad you asked this question about ACID and consistency. HyperDex
has always provided linearizability and fault tolerance. These
guarantees are for single operations only. For instance, you could
write the following and be sure that Joe's new balance is 10 more than
the balance before.

c.atomic_add('accounts', 'joe', {'balance': 10})

HyperDex guarantees that such an atomic operation is correct no matter
how many other operations are happening simultaneously.

Transactions extend these guarantees to multiple objects. For instance,
I could transfer $10 from Joe to Tim and charge a $1 transfer fee with
the following code:

jbal = c.get('accounts', 'joe')['balance']
tbal = c.get('accounts', 'tim')['balance']
bbal = c.get('accounts', 'bank')['balance']
jbal -= 10 + 1
tbal += 10
bbal += 1
# <==========
c.put('accounts', 'joe', {'balance': jbal})
c.put('accounts', 'tim', {'balance': tbal})
c.put('accounts', 'bank', {'balance': bbal})

This takes money from Joe and gives it to Tim and The Bank. Take a
moment to think about what could go wrong on the line marked with an
arrow. At this point, what happens if another transaction sneaks in and
adds money to Joe's account, the value we store in the final database
will be wrong. In effect, Joe will lose money.

The key benefit of Warp's transactions is that they remove from you the
burden of thinking about such problems. By wrapping operations in a
transaction, you effectively have the database to yourself. When the
transaction commits, you are guaranteed that the transaction moves your
datastore from one consistent state to the next. You only have to
examine code between "begin_transaction" and "commit_transaction" to
convince yourself that the application is correct.

HyperDex has always made the guarantees of ACID for single key
operations, but it's disingenuous to say such a system is ACID, because
the Application-Level Consistency guarantee doesn't hold up under
concurrency.

HyperDex Warp supports true, client-side, multi-key transactions and
thus provides true ACID guarantees.

-Robert

On Wed, Feb 06, 2013 at 01:30:45AM -0500, Maxi wrote:
> Hi, I guess I'll give you not requested feedback of a typical MongoDB user
> like my dear self ;)
>
> I was under the impression that Hyperdex already supported consistency. So
> I don't know what exactly Warp adds that I would need, I thought Hyperdex
> pretty much already achieved ACID transaction, although we didn't call
> them that but if every request is consistent, then isn't it the same?
> I'll also add what I (mistakenly or not, please correct me) think is the
> main selling point for Hyperdex: That I don't have to worry about setting
> indexes because I can get any object by any of its members back. This is a
> huge gain, because it removes a whole dimension of complexity that is in
> all the other databases, where I need to tune and think about that sort of
> thing and everyone always says how important it is to do indexes
> "properly".
> So maybe I'm wrong, but if Hyperdex can let me forget about that whole
> thing, it's a selling point we haven't even begun to sell to people.

> [1]http://hyperdex.org/warp.
>
> Thanks,
> The HyperDex Team
>

> --
> You received this message because you are subscribed to the Google Groups
> "hyperdex-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hyperdex-discu...@googlegroups.com.

> For more options, visit [2]https://groups.google.com/groups/opt_out.
> �
> �
>
> References
>
> Visible links
> 1. http://hyperdex.org/warp
> 2. https://groups.google.com/groups/opt_out

[Robert Escriva]

Hi Sandeep,

Did you install the libhyperclient-dev package? The headers and
symlinks necessary to link against libhyperclient are in that package.

-Robert

On Sat, Jun 29, 2013 at 10:47:28PM -0700, sanj...@gmail.com wrote:
> Robert,
>
> I just installed Hyperdex-warp on ubuntu linux ("apt-get install
> hyperdex-warp") and can't find any
> header file or library which I must use to invoke the API. Is there a C++
> binding for Warp?
>
> I am assuming Warp requires an API based on the description given at
> http://hackingdistributed.com/2013/02/05/hyperdex-warp/
> and
> http://hyperdex.org/papers/warp.pdf
>
> this is the output of "dpkg". I have installed from Hyperdex from the source
> tarball.
>
> $ dpkg -L hyperdex-warp
> /.
> /usr
> /usr/share
> /usr/share/doc
> /usr/share/doc/hyperdex-warp
> /usr/share/doc/hyperdex-warp/changelog.Debian.gz
> /usr/share/doc/hyperdex-warp/copyright
>
> $ dpkg -L hyperdex-warp-daemon
> /.
> /usr
> /usr/libexec
> /usr/libexec/hyperdexwarp-1.0.rc2
> /usr/libexec/hyperdexwarp-1.0.rc2/hyperdex-initiate-transfer
> /usr/libexec/hyperdexwarp-1.0.rc2/hyperdex-show-config
> /usr/libexec/hyperdexwarp-1.0.rc2/hyperdex-add-space
> /usr/libexec/hyperdexwarp-1.0.rc2/hyperdex-daemon
> /usr/libexec/hyperdexwarp-1.0.rc2/hyperdex-rm-space
> /usr/bin
> /usr/bin/hyperdex-search-stress-test
> /usr/bin/hyperdex-simple-consistency-stress-test
> /usr/bin/hyperdex-replication-stress-test
> /usr/bin/hyperdex
> /usr/share
> /usr/share/man
> /usr/share/man/man1
> /usr/share/man/man1/hyperdex-daemon.1.gz
> /usr/share/doc
> /usr/share/doc/hyperdex-warp-daemon
> /usr/share/doc/hyperdex-warp-daemon/changelog.Debian.gz
> /usr/share/doc/hyperdex-warp-daemon/copyright
> /usr/share/doc/hyperdex-warp-daemon/README.Debian
> /usr/share/doc/hyperdex-warp-daemon/README
>
>
> -Sandeep

> --
> You received this message because you are subscribed to the Google Groups
> "hyperdex-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to hyperdex-discu...@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>