Distributed transactions for microservices

[Hristo Stoyanov]

https://tech.ebayinc.com/engineering/grit-a-protocol-for-distributed-transactions-across-microservices/

[Ken Finnigan]

Do you know how it differs from Sagas or Long Running Actions?

On Tue, Oct 29, 2019 at 3:57 PM Hristo Stoyanov <hr.st...@gmail.com> wrote:

https://tech.ebayinc.com/engineering/grit-a-protocol-for-distributed-transactions-across-microservices/

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/3e4cf2c5-f179-4101-9a74-f2ddc44ff317%40googlegroups.com.

[Rudy De Busscher]

That GRIT protocol looks like a variation to the traditional 2PhaseCommit.

And in my opinion, it doesn't work for Long Running Transactions. When you have paid for your concert ticket, your ticket is probably already long gone because during the transactions, read operations still see the 'consistent' values which doesn't take into account the actions which are in progress.

So when 5 people start selecting the last ticket, only one user will be lucky and for the others the rollback needs to be performed.

With Sagas and the MP LRA, once a user has selected the last ticket, the others can't start the process anymore.

My conclusion, not very friendly for end user, resource heavy and very 'sensitive' for distributed failures (for ex what happens if Global Transaction Manager cannot reach one of the Database Transaction Managers when commit decission is made?)

Rudy

On Tuesday, 29 October 2019 21:01:36 UTC+1, Ken Finnigan wrote:

Do you know how it differs from Sagas or Long Running Actions?

On Tue, Oct 29, 2019 at 3:57 PM Hristo Stoyanov <hr.st...@gmail.com> wrote:

https://tech.ebayinc.com/engineering/grit-a-protocol-for-distributed-transactions-across-microservices/

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.

[Ondra Chaloupka]

(My current understanding down here.)

It's different from Saga. Saga is an eventual consistent approach. It hands abort handling of the transaction over to developer. Developer has to create a compensation callback where he writes a logic which restores the state of data to the time when  the Saga had begun.

Saga is an architectural pattern which is useful in broader area of use cases. The GRIT it's meant to be used for distributed transactions over various databases.

E.g. saga pattern (and the LRA implementation) have a good fit to span over several service calls - e.g. when system is integrated with some 3rd party service via API calls.

The GRIT is more comparable to 2PC. The GRIT defines a protocol how driver transaction is managed and how to communicate with database. The 2PC is shaped as the XA protocol specification. The XA is implemented cross over various Java specs like JTA, JDBC, JPA and EJB, plus the support is needed from the database vendor.

In the comparison from what is currently available in Java world the GRIT needs at 1. a different developer approach and at 2. a new type of "JDBC" driver which supports the GRIT protocol.

The GRIT is based on ideas of Calvin (the commercial database on this is FaunaDB[1]). The FaunaDB founders state that 2PC is not the good approach in current times[2]. The benefit of GRIT over 2PC is that It provides a protocol which is non-blocking and it does not require database locks for processing.

at 1.: The 2PC locks the database to provide the data consistency. The programming model is simple - or we are used to it. The developer may call any database update in whatever place in the code. The 2PC manages all that consistency work for you at the end. The GRIT expects the application defines all the database accesses upfront - at the start of the business logic (as the [3] states App starts a global txn with a list of DBs it will involve). The GRIT system is then capable to plan which data is involved and protocol is based on that. I expect e.g. the SQL could be a trouble.

at 2. the current JDBC spec expects use of the XA. The GRIT requires a different handling and a special driver for the database needs to be created. In case of XA the transaction handling needs to be part of the database itself. The GRIT could be an upper abstraction layer which does not need any direct changes in the database vendor code. I would say it's kind of similar to how the CDC disrupts the database write ahead log handling. The CDC access the database write ahead log directly. The GRIT defines its own WAL log that kind of replaces the database one. The database then only consumes the updates from the GRIT log afterwards.

I assume the GRIT and Saga does not compete directly. They can be used jointly - e.g. saga for service to service calls and GRIT as backend for the database transactions.

Still GRIT for database transaction may be easier to use for the database use cases as it does not require any abort logic to be implemented by a developer and the GRIT is that's strictly consistent not only eventual consistent as Saga is. Still GRIT will probably needs a bit of a different way of thinking when the developer writes the application. But e.g. for use of the reactive MSA a different way of the application design is needed as well.

 

Ondra

[1] https://fauna.com/

[2] http://dbmsmusings.blogspot.com/2019/01/its-time-to-move-on-from-two-phase.html

[3] https://tech.ebayinc.com/engineering/grit-a-protocol-for-distributed-transactions-across-microservices/

On Thu, Oct 31, 2019 at 10:04 AM Rudy De Busscher <rdebu...@gmail.com> wrote:

That GRIT protocol looks like a variation to the traditional 2PhaseCommit.

And in my opinion, it doesn't work for Long Running Transactions. When you have paid for your concert ticket, your ticket is probably already long gone because during the transactions, read operations still see the 'consistent' values which doesn't take into account the actions which are in progress.

So when 5 people start selecting the last ticket, only one user will be lucky and for the others the rollback needs to be performed.

With Sagas and the MP LRA, once a user has selected the last ticket, the others can't start the process anymore.

My conclusion, not very friendly for end user, resource heavy and very 'sensitive' for distributed failures (for ex what happens if Global Transaction Manager cannot reach one of the Database Transaction Managers when commit decission is made?)

Even the article does not talk about details I would wonder if the GRIT would be sensitive for distributed failures. If we talk in terms of what is explained I assume it depends on the phase. The GTM may either periodically retry to contact the DTM if the commit was already decided or if the commit was not yet decided there could be a timeout after which abort is chosen. I consider this similar to what is done when the LRA coordinator is not reachable or when the LRA coordinator is not capable to reach particular service with complete/compensate call. Those calls have to be retried until they succeed.

 

Rudy

On Tuesday, 29 October 2019 21:01:36 UTC+1, Ken Finnigan wrote:

Do you know how it differs from Sagas or Long Running Actions?

On Tue, Oct 29, 2019 at 3:57 PM Hristo Stoyanov <hr.st...@gmail.com> wrote:

https://tech.ebayinc.com/engineering/grit-a-protocol-for-distributed-transactions-across-microservices/

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.
To unsubscribe from this group and stop receiving emails from it, send an email to microp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/3e4cf2c5-f179-4101-9a74-f2ddc44ff317%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Eclipse MicroProfile" group.

To unsubscribe from this group and stop receiving emails from it, send an email to microprofile...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/microprofile/fe53ac9a-511f-4b9a-8245-8b95c21b60f5%40googlegroups.com.

[Hristo Stoyanov]

Ken,
Sorry, no, I just came across it, thought it would be relevant to Microprofile. Here is more info:

https://www.infoq.com/news/2019/10/eBay-grit-multidb-transactions