Google Groups

Re: Datomic Database ACID clarification concern

Stuart Halloway Apr 16, 2012 11:42 AM
Posted in group: Datomic
Hi Remy,

Datomic's data functions will address all of your concerns, and then some.

Transactional systems can use optimistic or pessimistic concurrency controls to deal with read-for-update scenarios. Datomic's built-in data functions (compare-and-swap and ensure) enable generic optimistic approaches. User-defined data functions enable pessimistic approaches. Remember: Data functions run inside the transaction, on the transactor.


Stuart Halloway
Datomic Team

Hi Stuart,

   First thank for answering my initial message.
But may be I could have been clearer when I attempted to demonstrate that the Datomic's database needs to abort user transactions more often
than a database providing locking. Let me detail what I mean precisely, comparing the behavior of a locking database and the
Datomic database. (because of the conclusion that follow is important, I think) :

App1 and App2 are 2 concurrent applications, running on separate machines, that attempt to withdraw  800$ from a bank account
containing 1000$.

1) Let's first take an example with a database providing locking :

- App1 is the first to lock in exclusive mode the set of database facts (the bank account), at today time : 9 00 01 

- App1 examines the set of facts to verify if it can withdraw 800$ (this occurs between 9 00 01 and 9 00 50)

- App2 at 9 00 03, try to get an exclusive lock on the bank account. App2 get blocked because at 9 00 03 the exclusive lock
has been granted to App1 and is not released yet.

- App1 finally submits the transaction that is accepted by the database at 9 00 50. The database accepts and applies the App1's transaction
(the bank account now contains 200$), and the database now grants the exclusive lock to App2 that resumes its execution.

- App2, at 9 00 53, access the bank account and see that it cannot withdraw 800$ as the bank account balance is 200$. Thus
App2 know it cannot submit a transaction to withdraw 800$ and then will not submit that transaction and instead will take another
course of action.

Conclusion : Using a database with locking, in the above case, only one transaction has been submitted, none aborted.
2) Now using the Datomic database with the same time sequencing as above.
- App1 gets the set of facts (the bank account) at 9 00 01, from the Datomic database.
- App1 examines the set of facts to verify if it can withdraw 800$ (this occurs between 9 00 01 and 9 00 50)

- App2 at 9 00 03, get the same set of facts from the Datomic database (with no problem as there are no lock to block App2)

- At 9 00 08, both App1 and App2 have the same exact set of facts and both concurrently decide they can withdraw 800$
from the bank account that contains 1000$

- At 9 00 50, App1 submits its transaction request that is queued in the transactor queue.

- At 9 00 51, App2 submits its transaction request that is queued in the transactor queue.

- At 9 00 55, the database processes the App1's transaction and return OK to App1, the bank account contains now 200$

- At 9 00 57, the database is forced to abort the App2's transaction because constraints have been added that force the
transaction to abort when the bank account's value is changed to a negative value. App2 on commit exception will now take
another course of actions.

Conclusion :

The Datomic database needs to abort the App2's transaction which is not the case when using a locking database.

Note also that applications coding is more complex with the Datomic database as you are forced to use constraints checking everywhere
there are potential conflicting concurrent write updates (as shown in the above example).

In the case of a Datomic complex hierarchy of objects (or entities) being modified by one application, it will be impossible
for the application developer to forecast all potential conflicting write updates that could be made by concurrent applications, in
that hierarchy.

Whereas if locking were available, locking the entities hierarchy being modified, the application developer would have solved its
problem with one line of code (locking in exclusive mode the hierarchy so that he/her can make all changes before committing), and
be sure to preserve the ACID properties of the database.

Finally :

What is important is that serialization of write transactions occurs at the database's entity or entities hierarchy level that is
being modified. And the tool to make that (such as locking) must absolutely be provided to the application developer.

Serializing write transactions at the Transactor level is completely insufficient to provide an ACID database easy to use,
even impossible to use in the case of a complex entities hierarchy being updated concurrently by many applications.

I have still a far more obvious general purpose company's application to illustrate that.

Unfortunately this will limit the scope of uses of the Datomic database.

However this is fairly easy to address and implement compared to the tremendous work you have done elsewhere.

If you do not implement that, I fear that many future potential users will consider that as a serious design flaw and will move

Another area that I think of now, that you will probably improve, is to provide a higher level interface to define schema and
transactions so that the notion of entity's ids, temporary's ids, negative numbers used to link entities, disappear from the
application developer 's or user's point of view (not from the Datomic database internals).
If I mention that, it is because I already designed and implemented such a functionality.

But there are many other areas, and new ideas you have brought, that I am very impressed you have done in such a small lapse of time.
Like you, I believe that, as it is today, the Datomic database already has unique advantages in some applications areas. And this
is only the beginning of a bright future.