Understanding Sagas and Two Phase Commit

[Nuno Lopes]

As a followup interesting article of Udi about SAGAS, the intent of
this article is to show that there is more to it then orchestrating
When Xed do Y.

Sagas have been used to solve challenges around performing business
transactions across systems in banking for a long time.

Suppose that we transfer an amount of money from an Account A to
Account B.

Take a simple domain service (as of DDD definition)

void TransferMoney(accountIdFrom, accountIdTo, double amount) {
// transaction start
Account A = AllAccounts.get(accountIdFrom);
Account A = AllAccounts.get(accountIdTo);
A.TransferMoneyTo(amount, accountIdTo);
B.TransferMoneyFrom(amount, accountIdFrom);
// end transaction
}

The code above is simple, but it works on the premiss that we can use
cross Aggregate transactions. Suppose account A and account B are of
two distinct banking systems (two different Banks).

The trick is to only Commit the Transaction when both accounts have
Accept it, other wise Reject both. Rejection should be an idempotent
operation. We can easily do that with Sagas.

BankAccountA {
void TransferMoneyTo(transactionToken, toAccountId, amount) {
if (this.HasPendingTransactions) {
throw new BusinessException("Transactions in process");
}

if (availableBalance-amount < 0) {
throw new BusinessException("...");
}

DomainEvent e = new TransferMoneyToAccepted(....);
Apply(e);
Bus.Publish(e);
}

void Apply(TransferMoneyToAccepted e) {
this.HasPendingTransactions = true;
}

void CommitTransferMoneyTo(transactionToken) {
MoneyTransferToAccepted t = this.transactions.get<
TransferMoneyToAccepted>(transactionToken);
if (t == null) ....

DomainEvent e = new MoneyTransferToCommitted(..., this.balance -
t.amount);
Apply(e);
Bus.Publish(e);
}

void Apply(MoneyTransferToCommitted e) {
this.HasPendingTransactions = false;
this.balance = e.balance;
.....
}

...
}

BankAccountB {
void TransferMoneyFrom(transactionToken, fromAccountId, amount) {
if (this.HasPendingTransactions) {
throw new BusinessException("Transactions in process");
}
// some other business logic
DomainEvent e = new MoneyTransferFromAccepted(....);
Apply(e);
Bus.Publish(e);
}

void Apply(MoneyTransferFromAccepted e) {
this.HasPendingTransactions = true;
}

void CommitTransferMoneyFrom(transactionToken) {
MoneyTransferFromAccepted t =
this.transactions.get<MoneyTransferFromAccepted>(transactionToken);
if (t == null) ....

DomainEvent e = new MoneyTransferToCommitted(transactionToken,
this.balance + t.amount);
Apply(e);
Bus.Publish(e);
}

void Apply(MoneyTransferToCommitted e) {
this.HasPendingTransactions = false;
this.balance = e.balance;
}
....
}

class MoneyTransferSaga : Saga<MoneyTransferSaga>,
StartedBy<MoneyTransferToAccepted>,
StartedBy<MoneyTransferFromAccepted>
{
void Handle(MoneyTransferFromAccepted e)
{
this.moneyTransferFromAccepted = e;

if (this._moneyTransferToPending != null)
{
Bus.Send(new CommitMoneyTransferTo(...);
Bus.Send(new CommitMoneyTransferFrom(...);
this.MarkComplete();
}
}

void Handle(MoneyTransferToAccepted e)
{
this.MoneyTransferToAccepted = e;

if (this._moneyTransferFromPending != null)
{
Bus.Send(new CommitMoneyTransferTo(...);
Bus.Send(new CommitMoneyTransferFrom(...);
this.MarkComplete();
}
}

void override TimeOut()
{
try {
if (this._moneyTransferToPending != null) Bus.Send(new
AbortMoneyTransferTo(...);
if (this._moneyTransferFromPending != null) Bus.Send(new
AbortMoneyTransferFrom(...);
} catch (Exception e)
{
.... register failure launch red flag for manual correction.
}

this.MarkComplete();
}
}


Has you can see by the above code, while there are pending
transactions in an Aggregate no other transactions will be processed.
This might be a problem if the external system takes a long time to
accept the transaction. But this what happens when we have distributed
transactions across multiple systems. Both Aggregates will be locked
until the transactions both transactions are accepted done or
canceled. The transaction token works has a handshake. Here is the
TransferMoney domain service.

string transactionToken = GenerateTransactionToken();
Account A = AllAccounts.getAccount(cmd.fromAccount);
A.TransferMoneyTo(transactionToken, ....);
AllAccounts.Save(A);
ExternalAccountService.TransferMoneyTo(transactionToken,
cmd.fromAccount, cmd.toAccount, .....);

In case of failure due to transactions in process you also implement a
retry scheme in the command handler. Say a transaction fails to
pending transactions in an AR, you can simply re-queue the command
until a number is reached and the command is aborted.

So you can actually implement 2PC with Sagas.

To remove Two PCs, what the banking domain does is more or less the
following. Basically we have availableBalance (money that is available
to be used) and an restrictedBalance for each account(money that even
though is part of the over all balance, it can't be used). The total
balance = availableBalance + restrictedBalance. When we accept the
transaction in account A the money is taken from the available balance
and put in the restricted balance (the total balance does not change).
Business rules always run against the available balance. When we
Commit the transaction, the money is then taken the restricted balance
and that is it. If the transaction is Aborted the money is then taken
from the restricted balance and put in the available balance.

This approach allows for none blocking Transactions across Accounts,
yet the process outlined above is exactly the same in terms of what
the Saga.

We can do the same, for almost everything. For instance say that we
aren't dealing with amounts but a simple value such as a Name. We get
the new String and put in an restricted Name. When the transaction is
committed we take the name out of the restricted Name and put in the
available Name.

Thinks can get much more complicated in which case is probably better
defer application of events. That is, we register the event without
applying it. Only when the transaction is committed we apply all the
events with the same transaction token :)

In the rare cases that the above fails, the history of events are very
important and we need to correct the data manually. Say for instance
the AbortMoneyTransferTo or AbortMoneyTransferFrom fails due to some
outofbound problems (say queues get overloaded).

Hope it helps to understand better Sagas and their role in Distributed
Transactions.

Cheers,

Nuno

[@yreynhout]

Sounds a lot like the reservation pattern to me: http://rgoarchitects.bit.ly/5NILfx

[Ramin]

Where can I find Udi's article?

Ramin

[Nuno Lopes]

http://www.udidahan.com/2009/04/20/saga-persistence-and-event-driven-architectures/

Atentamente,

Nuno Lopes
Email: nbpl...@gmail.com
Tel: 914430127

[Nuno Lopes]

It seams to me that the author is describing 2PC. But if Reservation entices creativity to a developer then 2PC so be it.

On another note Udi Sagas look different because mainly events. As such I would not consider this Sagas to be of the same class. Handling an event is not he same as handling a command.

Cheers,

Nuno
PS: Personally I give little credit to the SOA crowd when it comes to effective innovative engineering. A lot of concepts feel ambiguous in their descriptions and when they get more explicit look like complete rip-offs. It looks like engineering to sell products (including conferences) rather then to create new solutions.

[Ramin]

Thanks Nuno!
Ramin

On 19 Apr., 22:44, Nuno Lopes <nbplo...@gmail.com> wrote:
> http://www.udidahan.com/2009/04/20/saga-persistence-and-event-driven-...
>
> Atentamente,
>
> Nuno Lopes
> Email: nbplo...@gmail.com

[Nuno Lopes]

Hi, Yves,

PS: Personally I give little credit to the SOA crowd when it comes to effective innovative engineering. A lot of concepts feel ambiguous in their descriptions and when they get more explicit look like complete rip-offs. It looks like engineering to sell products (including conferences) rather then to create new solutions. - Nuno

I guess yesterday I was in a bad mood and probably was a bit unfair (major site going online last night and finally a good 6 hours sleep). Let me reframe:

I consider SOA mostly to be just another design language describing good old and proven architectures. Much like say the difference between C# and Java. By changing the language we don't change the paradigm.

For instance service orientation is nothing but modular programming techniques (http://en.wikipedia.org/wiki/Modular_programming) applied to modules deployed on the network. The concept of modular programming is language agnostic much as service orientation. 

Calling a service is nothing more then a Remote Procedure Call, wether we use a Web Service, XMLRPC, WCF or a simple TCP/IP UDP connection. Services (as per SOA) is like Modules. The core benefit is not in SOA but in the communication standard (Web Standards).

I try to keep up with all the languages so I can communicate with any audience, but sometimes gets frustrating maintaining multiple grammars in my head.

Conceptually is not much difference between SOA, POA (procedure oriented), same paradigm. OOA is slightly different but is not enough to take the jump. 

Now EDA is a different paradigm altogether from SOA, POA or even OOA since we move from imperative programming to declarative.

For instance, introducing events in Sagas adds IMHO significant semantic changes to the concept (interpretation) that empowers new solutions. The same goes when you introduce Greg's approach to an Aggregate internals and start thinking more about facts rather then commands.

Cheers,

Nuno