Liberator State Machine and Datomic Transactions

[Alexander Kiel]

Hi,

I'm about to build a web application which exposes a REST API over a Datomic database. I like to use Liberator to build the REST API. It works fine as long as I only read things. But when it comes to transactions, I have a problem combining the various Liberator decisions with the one single Datomic transaction I can use.

In Datomic the transactor serializes all transactions. One has to send a message to the transactor and it will execute the transaction described by this message. Only code executed inside the transactor can carry out conditional transactions in a consistent manner.

I often need a conditional PUT to prevent my users from the lost update problem. I use etags in conjuction with the If-Match header to implement conditional updates with PUT. Liberator however separates the decision etag-matches-for-if-match? from the put! action. The Datomic transaction will be executed inside the put! action. That means that I have to calculate the etag outside a Datomic transaction. But doing so is not correct. Calculating the etag based on the most current database known to the peer right before sending the transaction is just wrong. One has to calculate the etag based on the database provided inside a transaction function because that is the only database which will stay current until the transaction completes. All other databases could be superseded in the meantime by other transactions.

At the end a critical part of Liberators state machine has to run inside the transactor. Other decisions like exists?, processable? or conflict? are also critical in that sense. World it be feasible to put that part of Liberator in a jar which can than run inside the transactor?

Best

Alex

[Philipp Meier]

Hi,


Am 22.04.15 um 22:48 schrieb Alexander Kiel:

> I often need a conditional PUT to prevent my users from the lost update
> problem. I use etags in conjuction with the If-Match header to implement
> conditional updates with PUT. Liberator however separates the
> decision etag-matches-for-if-match? from the put! action. The Datomic
> transaction will be executed inside the put! action. That means that I
> have to calculate the etag outside a Datomic transaction. But doing so
> is not correct. Calculating the etag based on the most current database
> known to the peer right before sending the transaction is just wrong.
> One has to calculate the etag based on the database provided inside a
> transaction function because that is the only database which will stay
> current until the transaction completes. All other databases could be
> superseded in the meantime by other transactions.

I would use a transaction function that checks and maintains a version
of the entity. Like the built-in function :db.fn/cas but for the entity,
not a single attribute. This is like optimistic concurrency is typically
handled in a SQL database. For your resource implementation that means
it would look up the current version attribute in :etag. In :conflict?
you'd do the actual update with the transaction function which should
fail in case of an interim change and make :conflict? return true.

-billy.
--
Philipp Meier

[Alexander Kiel]

Hi Phillip,

our suggestion to carry out the transaction in :conflict? would mean that there are still two places where the precondition of a matching etag would be checked. I would like to return always a 412 on a failed precondition. Checking in :conflict? a second time would give the client a 409 instead of the 412.

The most correct and simple way would be to issue the transaction as long as :processable? is true. The next decision after :processable? is :exists? which can not decided outside of a transaction. In :exists? one could issue a transaction which decides for 404, 409 and 412 or succeeds. The result can go into the context which will be returned in say :handle-ok as ring-response.

On the other hand, I don't like to abuse Liberator in such a way. What one would need are decisions with more than one outcome. Currently for the decision graph, only true or false is relevant. Having one decision deciding between more than two outcomes is not possible. The next needed thing is a customizable decision graph. If I understand our code correctly, the decision graph is currently defined as a DSL. It would be better to define it as data. If it is defined as data, it would be easy to customize it.

Do you think that my use case is relevant?

Best

Alex

[Philipp Meier]

Am Donnerstag, 23. April 2015 15:50:57 UTC+2 schrieb Alexander Kiel:

Hi Phillip,

our suggestion to carry out the transaction in :conflict? would mean that there are still two places where the precondition of a matching etag would be checked. I would like to return always a 412 on a failed precondition. Checking in :conflict? a second time would give the client a 409 instead of the 412.

Oh I see.

 

The most correct and simple way would be to issue the transaction as long as :processable? is true. The next decision after :processable? is :exists? which can not decided outside of a transaction. In :exists? one could issue a transaction which decides for 404, 409 and 412 or succeeds. The result can go into the context which will be returned in say :handle-ok as ring-response.

On the other hand, I don't like to abuse Liberator in such a way. What one would need are decisions with more than one outcome. Currently for the decision graph, only true or false is relevant. Having one decision deciding between more than two outcomes is not possible. The next needed thing is a customizable decision graph. If I understand our code correctly, the decision graph is currently defined as a DSL. It would be better to define it as data. If it is defined as data, it would be easy to customize it.

Do you think that my use case is relevant?

Yes, absolutely and is also relevant for a "traditional" SQL database. Still my suggestion is to basically the same as before: in :put! check the revision of the resource within the transaction  (datomics db.fn/cas will help) and raise an exception if the database entity changed in the meanwhile. In handle-exception then create a 412 response for that exception. This is the only way to create non-binary workflows in liberators graph and as this is an "exceptional control flow" I think that using an exception is right.

-billy.