Isn't Saga - just a leaked domain logic?

[Burtsev Alexey]

Looking at usages of Sagas in some CQRS architecture style projects, I come to conclusion that is just a leaked part of domain.

Take for example this Saga from MS PNP CQRS sample https://github.com/mspnp/cqrs-journey-code/blob/dev/source/Conference/Registration/RegistrationProcessSaga.cs

 It's Saga.State is just leaked Order.State (AwaitingReservationConfirmation, AwaitingPayment) IMHO. The same situation is seen in other projects. Without any reason people put domain logic and state somewhere outside. It looks like dividing domain logic and state into two objects,

[Rinat Abdullin]

Good observation. I agree.

"Leaked domain logic": is similar to how I perceive implementations of long-running processes that are pushed out of the aggregates and then called "Sagas".

Best,

Rinat

[@yreynhout]

The "problem" is that "sagas" are part of "the domain" (not the artifact most call domain ;-)). Sagas/Workflow/Business Process/Work Decompositioning - and I'm not implying these are all the same thing - are a nice tool in the box to solve problems that involve coordination of time, people and things - regardless of HOW you decide to implement it. What you'll end up with - or should I say SHOULD end up with - , is an argument why you'd rather not put "logic" inside the saga ... a good barometer - albeit in a DDD context - is "How close is the code to my conversation with the domain experts & business process manager?" or "How much refuctor is going to happen if the process changes? How far is it going to ripple?". 

[Nuno Lopes]

Hi,

The concept of Saga needs to be decomposed still in DDD (at least I haven't read an article from DDD community that is complete). From the writings I have the feeling that there are many things that are put under the Saga umbrella that may not map with the concept o "long running business activities".

For instance. Say that you have 3 commands to be executed in parallel, and you want a forth command to be executed once the first three are completed. This is typical control flow in parallel algorithms and it's not necessarily a "long running business process". Some programming languages support logical operators to do this. Some visual workflow solutions (with diagrams) support this kind of operators in their engines and visual languages.

What has baffled the computer scientists for a long time, is how to prove that something is false. For instance, how can we prove that a command wasn't completed in the absence of an exception. It is an NP problem. Negation in languages such Prolog is a process intensive computation. The engineering approach is to define a time interval that when consumed, any non verifiable predicate is considered false until that moment (even though is was not proven as such).

In case of an Purchase Order of the example, the system might cancel the Order out rise alerts for the operator to take action.

Now is this leaked domain logic from the domain model to the environment? Well at least it's a low level implementation of an operator for parallel computation.

The question in my mind is, is it ok to consider this part of the domain model design? I think it can.

Here is what is actually going on in that examples

1. - -> PlaceOrder
2. OrderPlaced -> ReserveSeats
3. SeatsReserved -> PayOrder
4. SeatsReserved and !(OrderPayed) -> UnreserveSeats // need a saga here. Timer for computing the negation.
5. OrderPlaced and !(OrderPlaced and OrderPayed) -> CancelOrder(). // need a saga here. Timer for computing the negation.

As you can see the challenge here more the have not's rather then the have's. That is why a Saga is needed in this case.

So is indeed an operator implemented using a low level technical artifact, a Saga. Yet these artifacts are part of the domain model design. In this case a SAGA is logic.

Hope it clarifies.

Nuno
PS: I omitted the facts correlations for simplification.

[Kelly Sommers]

Nuno,

You are correct about long running business activities. The reason that it feels like Sagas are everywhere is because the term Saga has been misused in contexts where a State Machine or Workflow existed. If you read Garcia-Molina's paper on Sagas you will find that it is very much about long running transactions which have compensating actions. These compensating actions being chainable.

Further more the Saga itself is distributable without centralized state or centralized coordination. This opposes the usage of Saga in this thread (and several others).

What has been discovered is different ways to manage state machines or workflows. I think the goal for this business behavior is to be inside your domain model. If you have a workflow defined outside your domain perhaps that's a smell something is missing in the domain model?

Thanks,

Kell

[Kelly Sommers]

Rinat,

I agree :)

[Scott Reynolds]

Actually Udi defines sagas as the domain logic. Looking at the example below I don't see why an object like order would necessarily care about payments.

---

From: Kelly Sommers
Sent: 16/04/2012 15:15
To: ddd...@googlegroups.com
Subject: Re: [DDD/CQRS] Isn't Saga - just a leaked domain logic?

[Nuno Lopes]

Hi Kelly,

What has been discovered is different ways to manage state machines or workflows. I think the goal for this business behavior is to be inside your domain model. If you have a workflow defined outside your domain perhaps that's a smell something is missing in the domain model?

To be honest, from a practical stand point, discussing if Sagas are part of the domain model or not, albeit important from a conceptual point, It does not really matter so much. I think Udi says his Sagas are part of the domain model as far as I recall. I think it is more helpful to think about what kinds of problems Udi's Sagas solve. IMHO at has been used to solve too many technical problems, some of them might not even be there.

Event then in my mind the question still remains, technically how can we design n parallel activities and their coordination in the domain model using Saga's? I'm trying to propose probably a different solution by analyzing why indeed we need Sagas and TimeOuts. The conclusion I arrive is that we need them to compute negations of facts in a parallel computation in a way that is partially deterministic. By reducing the problem to negation of facts (an operator) It is my believe that we can arrive to cleaner, more domain centric solutions, rather then State Machines. You see, state machines don't resolve negations either so it is not really a solution but an approach.

Going deeper with what I wrote we are lead to conclude that parallel computations where the resolution of negation is based on TimeOuts we are lead to an eventually sound design. This is not very good, let me explain.

After the time interval set in a Saga is consumed the the Order may have been Paid still. So you may end up with a case where we have an OrderPayed and Seats Reserved yet the Order was Rejected! Your state machine worked flawlessly. I agree that the opportunity for it to happen might be small, but I would say that it is really a question of scale. This would be considered a Bug, not so much a case of inconsistency, indeed lack of soundness in  maths, equates to Bugs in programming. In this case it's a very hard to find bug if we don't pay attention to what actually we are building.

The solution I propose is to make these things explicit. Instead of having Sagas sending commands, have these Sagas computing logical negative results and let Aggregates deal with the rest.

1. - -> PlaceOrder
2. OrderPlaced -> ReserveSeats (not a Saga but a simple event Handler)
3. SeatsReserved -> PayOrder (not a Saga but a simple event handler)

4. OrderPayed and SeatsReserved -> CloseOrder (not a Saga but a simple event handler on the OrderAggregate)

Now the Sagas Computing Logical Results

- . OrderPlaced and !(OrderPayed) -> OrderNotPayed // Need a saga here for computing the negation. (Say 10 mins after 10 minutes of not receiving an Order Payed is considered NotPaid).

- . OrderPlaced and !(SeatsReserved) - SeatsNotReserved //   // need a saga here for computing the negation. (Say 10 mins)

Making the stuff consistent and resolving soundness:

5. OrderPayed and OrderNotPayed -> Business Exception. The Order or some other Aggregate may ask the user. (Oracle, NP Problem, Is the Order Payed or Not?). 

6. SeatsReserved and SeatsNotReserved ->  BusinessException The SeatReservation asks the user. (Oracle, NP Problem).

Cheers,

Nuno

PS: I'm also with Reynolds, the example of the Saga given is very bad. It's a Saga that is not resilient to non serialized computations (look at the state transitions and exceptions).

[Kelly Sommers]

Scott,

Garcia-Molina defined saga in 1987 in his paper about Sagas:

http://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf

Sagas contain state machines, so do other concepts in programming. Not every state machine is a Saga however.  What's being discussed here are not Sagas.

This brings up an interesting question in my head though (regardless of what you want to call things) and that is, since a Saga contains compensating actions, are these compensating actions domain behavior in some cases or simply chain-able "undo" operations that are infrastructure concerns?

Thoughts on what these compensating actions are?

Sorry if I'm derailing this conversation. It's put me in an interesting thought chain and thought that question might be useful.

Thanks!

Kell

[Rinat Abdullin]

It would not be the first time Udi Dahan confused or mixed concepts.

Rinat

[Rinat Abdullin]

+1. Original definition of sagas (as in this white paper mentioned by Kelly) is about technical approach to manage long running transactions that involve multiple resource managers (e.g. Databases).

I think, this original definition has nothing to do with domain driven design.

I also personally find that avoiding use of term "Saga" in discussions on DDD/CQRS noticeably reduces amount of confusion.

Rinat

[Nuno Lopes]

Hi,

Sagas contain state machines, so do other concepts in programming. Not every state machine is a Saga however.  What's being discussed here are not Sagas.

I've raid that article before, then and now I fail to see any mention that something to be a Saga needs to be coded has a State Machine.

Also, the article does not say that Saga contain compensating action. As I recall it even says that a Saga might record data that might be used for compensating actions down the road if necessary.

The difference from that to a DDD perspective is simply substitute the word database model to domain model. You might then even find out why Greg and Udi agreed on terminology of Component.

Recook, sell and deliver.

Cheers,

Nuno

[Kelly Sommers]

Nuno,

I don't agree that a Saga is simply orchestration and that a DDD perspective changes the definition in any way based on Garcia-Molina's paper.

I'll refrain from elaborating further since I've derailed this conversation quite enough already :P

Thanks,

Kell

[Peter Ritchie]

Maybe Kelly is thinking of "compensating transactions"...

On Mon, Apr 16, 2012 at 12:52 PM, Nuno Lopes <nbpl...@gmail.com> wrote:

--
"Refactoring with Microsoft Visual Studio 2010": http://bit.ly/c13trs
http://PeterRitchie.com/blog/
http://twitter.com/PeterRitchie
http://facebook.com/Peter.Ritchie/
http://www.linkedin.com/in/PeterARitchie
Skype:Peter.a.Ritchie

[Peter Ritchie]

I think it's pretty clear from the Garcia-Molina,Salem paper that what they're talking about with a "saga" is a long-running transaction that manages implementation details to model a "transaction" within that implementation detail as an atomic operation without blocking other parts of the system from running.

I agree that the RegistrationProcessSaga does not seem like a saga because it dealing strictly with domain information with regard to specific domain states that users in the domain will want to see and talk about.  For example, in the Saga paper, the fact that a database transaction was used for each reservation and that that transaction would block other transactions from reserving seats on the same flight is stricly an implementation detail.  The introduction of a "saga" is something that implements that LLT in light of these implementation details.

The impetus for wanting a "transaction" here may be outside of the domain and related to data consistency.

Cheers -- Peter

[Peter Ritchie]

I think it's just a misunderstanding of what a "saga" is and that the domain logic is just in the wrong place.

Cheers -- Peter

On Mon, Apr 16, 2012 at 6:13 AM, Rinat Abdullin <rinat.a...@gmail.com> wrote:

[Peter Ritchie]

For what it's worth, any domain-specific transactions should be modelled within aggregates--which is partially what they are for--and not introduce a saga.

Cheers -- Peter

[Scott Reynolds]

So ready through this code its trying to connect the different objects of seat, order, billing together. What exactly is the problem we have in the code. I can't imagine people want to have code like order.seat.accept() in the code.

---

From: Peter Ritchie
Sent: 16/04/2012 19:28

To: ddd...@googlegroups.com
Subject: Re: [DDD/CQRS] Isn't Saga - just a leaked domain logic?

[Peter Ritchie]

Law of Demeter issues aside, that doesn't look a lot like DDD to me...  e.g. if order is an aggregate, I wouldn't expect seat to be the root, in which case you wouldn't be able to access seat from outside of order.

If order was the aggregate, it would provide a method (presumably) making the call to accept within it's context of a "transaction"--if it needed it.  (completely guessing at anything implicit you were trying to communicate with "order.seat.accept").  e.g. you may have several methods that may be called and the "last" would would "commit" the transaction...

Cheers -- Peter

[Nils Kilden-Pedersen]

On Mon, Apr 16, 2012 at 11:47 AM, Rinat Abdullin <rinat.a...@gmail.com> wrote:

+1. Original definition of sagas (as in this white paper mentioned by Kelly) is about technical approach to manage long running transactions that involve multiple resource managers (e.g. Databases).

I think, this original definition has nothing to do with domain driven design.

Not sure why you'd say that. DDD, with the concept of ARs, specifically delimits the scope of a single transaction making subsequent transactions necessary, e.g. a Saga. This seems to me to be exactly what the original paper addressed and is probably why it's been introduced into DDD (not sure by whom originally). 

I do agree that the term "Saga" seems to be misused here, often as simply any event listener. I personally use it as close to the original paper as possible, i.e. any situation that involves multiple ARs in a single conceptual transaction, or some sort of set based validation for a single AR, which also involves multiple db transaction.

[Greg Young]

ok I read this conversation and I am really confused. added to backlog a blog post about "sagas" none of this makes sense to me.

--
Le doute n'est pas une condition agréable, mais la certitude est absurde.

[Nils Kilden-Pedersen]

On Mon, Apr 16, 2012 at 3:37 PM, Greg Young <gregor...@gmail.com> wrote:

ok I read this conversation and I am really confused. added to backlog a blog post about "sagas" none of this makes sense to me.

LOL! 

[Rinat Abdullin]

Greg, AFAIK you were promising that post for a few days already :)

Rinat

[Nuno Lopes]

Humm, when we try to say something and no one ask any questions or direct observation quite often what it means is that what we wrote didn't resonate at any level.

The Saga concept is quite simple. It's an approach to solve the problem of transaction with the understanding that either all operations are executed within, or none needs to appear to be executed at all.

1. We can solve this at the data set level, using locking schemes and compensation by copy. The advantage is that this approach is independent from the semantics of ones operation. The disadvantage is that it may not scale as much as wee need to. The issue is further aggravated when we have multiple independent components making the operation and data is distributed on the network.

2. We can solve this using a mix of 1. and semantic compensation. The disadvantage is that ones solution is no longer domain independent, and solution must be found within either the domain. The advantage is that there are far less locking between activities and components can be further distributed.

From Salem white paper:

"To amend partial executions, each saga transaction T, should be provided with a com-pensating transaction C, The compensatmg transaction undoes, from a semantic point of view, any of the actions performed by T,, but does not necessarily return the database to the state that existed when the execution of T"

When we get to "semantics" we get to the domain. It means that the the compensation operation is dependent on the problem domain of ones software solution and design. Hence this is of interest of DDD.

In the chapter about design Saga Kelly writes:

"To identify potential sub-transactions within a LLT, one must search for natural dlvi-sions of the work being performed In many cases, the LLT models a series of real world actions, and each of these actions IS a candidate for a saga transaction"

Furthermore:

"In other cases, it is the database itself that is naturally partitioned mto relatively independent components, and the actions on each component can be grouped into a saga transaction "

You can here substitute the word "Component" by "Aggregate". Or depending on you design , even Component as used by Udi (AC) etc.

"so that each transaction adds the tracing code to one of the components"

The Saga in DDD adds tracing code by calling commands that in turn adds the so called tracing code (events to the log).

This are just a few quotes to show how mind things the Saga concept in the paper is at its core similar to the one we often talk in DDD. The materials of a DDD Saga are different (events, commands, aggregates rather then data sets and database operations), yet in abstract the process of designing a Saga more the same then different.

This might sound strange, but as a programmer trying to automate a business, I've been more often interested in semantics and logic  of a domain rather then software artifacts used to automate them. This by experience allowed me to build programs faster, that worked with less bugs and that are easier to manage. Within this, I've learned that a business process as described by a business person at its core is no different then a program. Yes, things are described using natural language, the semantics of terms variate in context, but the material they work with is the same. (Numbers, rules, people that are worker performing operation, aka business activities). So it comes to me with no surprise what Salem writes:

"However, for many practical apphcatlons It may be as simple (or difficult) as writing the transactions themselves In fact, Gray notes m [Gray8la] that, transactions often have corresponding compensating transactions within the application transaction set"

This means to me, that quite often the business process already have activities encoded that can be used to compensate operations undergone in a failed transaction. We just need to automate them too. Which is the case of the domains I've been working with. In facts, business processes embody the concept of a Saga, yet in a much larger scale and different materials are used to perform activities.

Fast forward to what I've written in the my previous post. In order to understand it we need to look at the core of the transactions concept. And in particular what a command/operation failure means and the reason for it to happen in an environment. Furthermore I've been thinking on how this can be translated to DDD using a logical language with events/facts, commands, business rules etc. Using as base DDD Saga as described by Udi.

We need to notice that unlike an operation, a Saga should be designed never to fail even though the commands/operation it calls may fail . It never rises exceptions (well normally), everything needs do be compensated.

What is interesting to me is considering that act of sending a command never fails due to business rules, neither receiving an event. Udi Saga as far as I see can only fail due to timeout's. For instance, an event takes too long to arrive, beyond the expected interval. In this case the system will compensate assuming that the fact\events never have happened. So it assumes a negation over all the facts and compensate. So what if we instead of having the Saga calling compensation operation it publishes what it assumes for compensation? Say, OrderNotPayed.

If an Aggregate such as PaymentOrder is "listening" these events. aPaymentOrder that knows that has been placed needs then to ask the User, if "I was told the Order was not payed, but as if yet all I knew is it was payed. Should I return the money?". The same goes for the Reservation and Payment. The user of multiple departments will need to talk with each other to resolve the issue. The billing department may need to talk with shipping and sales to resolve the issue.

A good UI may aggregate this events by correlation, so the user not only is notified, about a specific inconsistencies as well as others.

An automatic system could simple compensate. thePaymentOrder could cancel it self and fire a reimbursement activity. PurchaseOrder and the Reservation could also proceed the same way. The nice thing is that compensation is carried out by the Aggregates, and there is no central artifact coordinating  the compensation. Furthermore, this Aggregates never timeout unlike a Saga.

The central point is how negations of facts/events are computed in this approach towards compensation. There is no central coordinator for compensations, each Aggregate will proceed as they see fit considering the contradictions they find. Hence in this case, reducing the Saga's to computing negations while the domain logic still remains in the Aggregate's.

Hope it better explains my drift.

Cheers,

Nuno

[Peter Ritchie]

The "unit of work" pattern and modelling "long-lived-transactions" in light of implementation-details--like database transactions or system distribution--are two different things.  Sagas model dealing with work involved in avoiding blocking out other processes from "atomic" data changes due to implementation-detail transactions by bringing it into application infrastructure code while still maintaining the "atomicity" of changes to involved data within the LLT.

I think, arguably, if you're dealing with an environment where eventual consistency is the norm, I'm not sure I see much value in "Sagas".  The "Unit of work" transaction almost always becomes a domain concept and should be modelled in an Aggregate or, under certain circumstances, a Service.

Cheers -- Peter