Distribution of logic between commands, events and projections

[hothi...@googlemail.com]

When I made my first project with CQRS my assumptions where:

Command are requests that something should happen. They may fail.

Events are notifications that something happened. They cannot fail.

The write model contains the business logic.

The project was all about a complex calculation for a kindergarten. If a child enters/leaves a certain group, changes one of many classifications or ages one year, several complex calculations result in incrementation or decrementation of several statistic numbers.

My first approach was, to have some events like ChildEnteredGroup, ChildLeftGroup, ChildAgesOneYear and let the projections handle the calculations and the updates of the numbers. But this would have put all this complex logic and calculations into the projections, while the write model would have been empty, only containing "command to event" converter methods (LetChildLeaveGroup command fires ChildLeftGroup Event and does nothing else).

This seems unnatural to me and I decided to put all this logic into the write model. So in the end the write model makes all computations and emits events like "ChildsWithXYIncremented, ChildsWithXYDecremented" and projections that only translates this events into crud operations for a database table containing the numbers.

So far so good (I thought).

Now the customer came back to me and told me, that the logic changed. The rules what counts and what not, the ranges and so on where slightly modified.

"No problem" I said. "I just have to change my projections and replay all events and we are done."

But I was wrong. Because the logic and rules are in the write model, and the events only tell about the results, I could not change the logic and rules in the projections and replay all events and get a different result. To achieve this, I would have to change the logic and rules in the write model, deleting all events (!) replaying all COMMANDS(!!) generating new events, causing my projections to create the new output.

I ended with a "RecalculateCommand" that generates compensating events (effectiffly taking each child out of the kindergarten and emit events following the old logic and rules and then reenter every child and emit events following the new logic and rules).

If I had followed my first approach, putting all the calculations and logic into the projections, having a write model that just translates commands into events (with a little bit of validation). It would have been much simpler. Just changing the logic and rules in the projections and replay all events. Ready.

But I refused to put complex logic into the projections because it felt wrong to me. It felt more correct to me to put it into the write model.

For the same reason I think that putting logic and calculacions into an event method is wrong. The calculation should be made in the command method, that then triggers an event with the result, and the event method makes the state changes to my write model (aggregate or entity). But here also it is not possible to change (or correct) the logic later, and after reloading the aggregate have teh new (corrected) results.

So it seems to me that perhaps it is not wrong to put some logic and calculations into the event methods and/or projections. Of course they are not allowed to fail. And it must be logic that I may want to change in the future without worrying about "loosing" the old results. 

So what would have been the right way?

More in general what logic should go into the command method, the event method and into the projections?

Please enlight me. :-)

Best regards,

Holger

[Danil Suits]

I remember making these mistakes!

But I was wrong. Because the logic and rules are in the write model, and the events only tell about the results, I could not change the logic and rules in the projections and replay all events and get a different result. To achieve this, I would have to change the logic and rules in the write model, deleting all events (!) replaying all COMMANDS(!!) generating new events, causing my projections to create the new output.

 

Yup, that's familiar.  I finally reconciled it this way: messages coming from the real world are events not commands.  After all MessageReceived is an event, and it is subsequent to MessageDispatched and MessageComposed.  HttpRequestReceived because HtmlFormSubmitted.  Very broadly, the message is "here's information about the real world".  The "command" part is "record this event in that model".

The model, from these events, can synthesize new information.  I like to use Greg's TradeBook example here (stripped down beyond recognition); orders coming in from the real world are events

OrderPlaced{id:1, BUY}
OrderPlaced{id:2, BUY}
OrderPlaced{id:3, SELL}

And it's the job of the model to synthesize the _trades_, which will depend (in part) on whether last-in/first-out or first-in/first-out is the appropriate policy.  So the model, being the authority for matching orders, gets to decide whether id:1 or id:2 gets paired with id:3.

The real difference between the projections and the write model isn't the complexity of matching the orders; in either case, the operation is just a fold.  The real difference is more about whether the synthesized data is written into an authoritative record (write model)  vs an ephemeral record (projections).

https://abdullin.com/post/ddd-evolving-business-processes-a-la-lokad/

Rinat's discussion of evolving processes hints at some of this; the idea that process automation replaces a human being looking at a view of the data suggests a close kinship between projections (provided so that the human being can synthesize new data) and "aggregates" (so that the write model can synthesize new data).

 

[Dariusz Lenartowicz]

Maybe hybrid version of this?

Record events from real world (emit events) and then process/calculate them in PM (emit new events) and then process new events in projections.

Logic is in PMs. If logic changes then we need to delete projections and events emmited in PM.
Rewrite/Remove-Delete PMs and replay those real world events through new PMs and new Projections.

[hothi...@googlemail.com]

So you say it's completely ok to have complex logic on the read side/projections even if it is (as in my case) the only complex/business logic at all?

So it seems to me that one has to decide for every piece of logic where it should go. Write model or read model.

If future changes of the logic should not affect the current observable state (read model) of the system but only the future behaviour, it should go to the write model.

If future changes of the logic should affect the current observable state (as if the new logic was there from the begin), it should go to the write model. (being aware that it is absolutely possible to keep the old read model with the old logic in parrallel).

Did I get this right?

From your experience, are there situations where it is not obvious which way to go (I bet so) and where there is danger to decide wrong? What methods do you have to prevent or fix it?

[Ryan Platte]

The key is to persist complete information about reality to events. Write model logic is only there to collect the information that goes into the event, and to validate commands in systems that are given a role in deciding whether an event will take place in reality (ordering a ticket for a specific seat). 

Otherwise the read model does nearly all the work.

Practice finding assumptions you're making from the business rules you're presently implementing, and change them wildly. Try to think of changes that would mean your events had to change. If you find one, reconsider your design.

Ryan

[hothi...@googlemail.com]

Now you confused me. From all my readings I got the understanding :

business logic => write model (=more or less DDD domain model) 

read model = materialized queries created by projections which normaly simply translate events into crud operations on the read model.

 

>Otherwise the read model does nearly all the work.

That sounds as if I got it completely wrong. So write model is only for validation if a command is allowed and possible and the complete business logic sits in the read model? That feels wrong.

[Ryan Platte]

First of all, I notice that I was writing about event sourcing, but you were asking about CQRS. So that is probably the core of the confusion I caused. But I'll proceed, as I can clarify in terms of CQRS. (ES is kind of a hopped-up version of CQRS)

There are two models. Both are full-blown models. I think you might be hearing 'read' and 'write' and focusing on those words instead of the presence of the word 'model' with them.

The job of the write model is to faithfully transcribe facts about reality and emit them as events. The job of a read model is to consume those events and make something from them. Yes, that can become CRUD operations on a simplified version of the model, or it can become entries in a graph database or anything at all.

In event sourcing there's not very much going on in the write model that isn't directly aimed at getting data recorded into an event store. In CQRS the write model can take on more responsibility, as you say.

My apologies for the confusion.

Ryan

[Andriy Drozdyuk]

It seems to me that the important thing here is both the activity of children and the rules that govern calculating the statistics.

So it might be beneficial to place these rules into their own aggregate. You can then change or update the way the statistics are calculate, and your projections can be automatically set to "replay" the activity stream whenever a new rule event is detected. I've never done this before, but if "rules" are important, they should be part of the business logic of the application, and therefore an aggregate. Scattering them across process managers, command handlers or projections simply makes things confusing, in my opinion.

Vague example of this can be seen in GoF pattern of Strategy: https://en.wikipedia.org/wiki/Strategy_pattern

[Andriy Drozdyuk]

By the way, thank you for a great example/experience! Very interesting.

[Ben Kloosterman]

I try to ignore BAs for the Write Model  ( unless they are into DDD/CQRS most have CRUD on the brain and  think in "IT business"  eg tables and what goes on the screen ) ...and map the closest to the real world . If its real events , you wont need to change it similar to sensor event processing . 

BL  lives everywhere in the client  , I'm the UI , in the views.. the key point is logic that mutates is what is the most crucial and that lives in the domain . 

"For the same reason I think that putting logic and calculacions into an event method is wrong. The calculation should be made in the command method, that then triggers an event with the result, and the event method makes the state changes to my write model (aggregate or entity). But here also it is not possible to change (or correct) the logic later, and after reloading the aggregate have teh new (corrected) results."

Commands should be light in my case the command is the http message on the wire and the is controller the command handler. Events and Commands should be naked objects they are DTOs ... they should never have logic except for a possible simple validate on command ( which is independent eg NO dependencies and super simple) . The command should have all the instructions needed the Aggregate generates the mutating logic  and writes the result to an event. An event should be a real business thing ..any business person should be able to understand the event ( for an event I do a effective ToString with Guid lookups in reads and that's there audit stream) .  I often have bugs fixed  by replaying events ( especially when read models are in memory) - the event rarely has issues  but the projections often do .. due to changes required by BL eg this part of this view now goes here..   

Don't be afraid to put simple calculations especially aggregation in the view  , If it doesn't mutate it belongs in the read side..Complex queries , reports etc often from of Services in the read side  ..

"Yup, that's familiar.  I finally reconciled it this way: messages coming from the real world are events not commands.  After all MessageReceived is an event, and it is subsequent to MessageDispatched and MessageComposed.  HttpRequestReceived because HtmlFormSubmitted.  Very broadly, the message is "here's information about the real world".  The "command" part is "record this event in that model".

Data Imports have a similar issue.   We have discussed Event Processors creating commands quite a bit on this list . 

Ben

.

[hothi...@googlemail.com]

To have a simple example:

Let us assume I have a kind of account aggregate. It has a command withdraw.

The command is a dto that only contains the amount of money to withdraw.

The account aggregate root class handles the command, checks wether the current amont of the account is greater or equal to the amount to withdraw (business rule) and fails if not.

If the account has enough money we have two possibilities:

1.

The method that handles the command and checks if enough money is there also calculates the remaining amount and raises an event "AccountWithdrawed" with the new amount of money on the account and perhaps (for information) the amount that was withdrawed.

The aggregate root class itself handles the event and updates its current amount.

A projection that want's for some reason to bookkeep the account can process the event and update the read model to the new amount of money.

2.

The method that handles the command and checks if enough money is there does no calculation. It just raises en event "AccountWithdrawed" with the amount that was withdrawed and perhaps (for information) the current amount of money in the account before the command.

The aggregate root class itself handles the event and calculates the remaining amount of money and updates its current amount.

A projection that want's for some reason to bookkeep the account can process the event and also calculate the remaining amount and update the read model to the new amount of money.

Solution 1 puts the complete logic (Calculation) into the aggregate (write model) what feels like the right thing to do for me. It makes the projection easy and without calculations and reduces it to a simple crud operation. On the other hand, if in the future the calculation changes (unlikely in this case of subtraction) I cannot just change my projection and reinitialize it and be done.

Solution 2 repeats the calculation in two places (sounds bad and not DRY). The projection is not merely a simple crud operation but duplicates the (admitted not very complex) calculation. But on the plus side if the calculation changes, I just have to change it in both places and reinitialize the projection an be done.

While in this example solution 1 seems to be correct, I think that solution 2 would have been better for my initial question (here the ugly duplication does not occur because only the projection does it).

The answers of this discussion tell me, that solution 2 is a legitimate solution (and some answers seem to tell me that it is the only correct solution what makes me doubt my understanding of DDD/CQRS/ES).

So to conclude this:

Please tell me if I'm right that solution 1 is the normal way DDD/CQRS/ES takes and i'm not completely of the way.

Please tell me if it is really legitimate to use solution 2 in some cases, even if that means that more business logic is in the projections than in the aggregates.

Please tell me how to decide when to use which solution, if you have more ideas than this:

>If future changes of the logic should not affect the current observable state (read model) of the system but only the future behaviour, it should go to the write model.

>If future changes of the logic should affect the current observable state (as if the new logic was there from the begin), it should go to the read model. (being aware that it is absolutely possible to keep the old read model with the old logic in parrallel).

[Ryan Platte]

Recall as you read my answer that my perspective is event sourcing. From what you've written, I think that understanding that perspective thoroughly will clear up your understanding. Others can discuss this better using CQRS.

In event sourcing, the list of accomplished, past-tense facts is the only source of truth in the system. In particular, a financial account *is* the full list of its transactions, the changes. It is not a list of notes that the current balance is now $X.

Different read models can offer very different pictures from those events. There could be a fraud detection read model for a bank account. As another example, my budgeting software consumes the feed of my bank transaction events and presents quite a different picture than what my bank shows me.

So it is not quite so arbitrary to determine what logic goes into the read vs. write model. They do very different jobs. The write model's job is simply to record the reality of what business facts have taken place. (And when there are commands/requests to validate, it needs its own write-side read model to make the decision of whether to convert those commands/requests into accomplished events.)

With the events in place, we can look at the read model as an entirely separate piece of software. It consumes the events and keeps an updated picture of the account balance, or in the case of the fraud detection read model, the level of suspicion that someone has stolen access to the account.

I think the duplication that's bothering you is the concept that there must be that "is current balance adequate" read model present to validate the command. That is the way to achieve the clean separation of write vs. read that event sourcing provides, which many find quite valuable.

Ryan

[Kyle Cordes]

This bit below is a vital concept, and when we were figuring out how to do CQRS-ES here (the DDD part isn’t vital for this specific topic) effectively, we were initially confused by the dearth of discussion of it in any of the numerous CQRS materials we read everywhere. It is acceptable and sometimes necessary to have one or more read models synchronously kept up-to-date so that they can be used for acceptably sufficient validation in the write side. The good news is: if you are doing a lot of CQRS you’re going to get very good at building read models, so building a few more of them to support command validation is not such a big deal.

Kyle

[Samuel Francisco]

Hi @hothi...@googlemail.com 

I believe that the calculation of the statistics should be in the "Read Side", since the statistics are only interpretations of the facts occurred in the system, such as ChildEnteredGroup, ChildLeftGroup and ChildAgesOneYear. In this case, it seems like a Report.

Only if there are some business rules on the "Write Side" depending on these statistics, for example, if users receive some bonus to be used in the system, based on that calculation. In this case, it would be a business decision if the change in calculation would have a retroactive effect and, if so, a compensatory action would be necessary to propagate this change to the bonuses already taken.

[Danil Suits]

I'm starting to get a sense for where your understanding and mine don't match up, so I want to take another swing at this

From all my readings I got the understanding :

business logic => write model (=more or less DDD domain model) 

read model = materialized queries created by projections which normaly simply translate events into crud operations on the read model.

This doesn't quite match the way that I think about things.

Consider how commands and queries work when we use a single data model (no CQRS yet).  Queries produce answers given some current state; commands modify the current state - or more precisely, they calculate a new state, and then update the "current" reference to point to this new state.

In this design, we can think of handling commands and handling queries as two different _roles_, but those roles are both being played using a single object, with a single reference to current state, and a single data representation of that state.

What CQRS introduces is the idea that instead of one object serving both roles, we separate the responsibilities; an object responsible for implementing commands, with a reference to current state in some representation, and a different object responsible for implementing queries, with a reference to current state in some representation.  Two objects, where once we had one.

The advantage in this design is that we can choose different _representations_ of state, that best serve the use case at hand.  It's the "same" (in a weak sense) state in both cases, but in one we are using a representation that makes _change_ easier to manage, whereas in the other the representation we use is optimized for query.

For example, we might use append only lists of changes as the representation of state for handling commands, but used snapshots built by folding/reducing/aggregating those changes for handling queries.

In both cases, we're using representations of state - it's a common concern between the two, and we should expect to see similar code for interpreting state.

What's unique to handling commands?  Choosing which state comes next.

Folding events is not normally unique to handling commands; the primary reason that we modify "the aggregate" by applying events to it is to ensure that our in memory representation matches what we would get by resuscitating the write model from the durable store.  But an additional benefit is that this also tells us that the in memory representation is being created from the same representation of state that will be used to inform the projections (and therefore, the read models).

"Calculation", to use your example, is just a query that can be answered by state; I don't see any reason why the implementation of that "logic" would be restricted to command use cases.

Does that help?