One aggregate per transaction, with “one” or “multiple” bounded contexts

[Murilo Carlos Cardoso]

In the chapter 8 of the Red Book Vaughn Vernon demonstrated how two aggregates can "talk" to each other with domain events. In chapter 13 how different aggregates in two different bounded contexts can "talk" to each other with notifications.

My question is, why should I technically deal with these situations differently once both of them happen in different transactions? If is it just one or multiple bounded contexts the possible problems wouldn't be the same?

For example, if the application crashes between two domain events in the same bounded context I'll end up with inconsistency as with two bounded contexts. So this is not a safe way to deal with it, right? 

Thanks!

[Greg Young]

Integration through events is *almost always* eventually consistent it's one of the main benefits of the model!

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAH7D42QH3gKXNowiPNMOz8mWYMiHgzp3LMae2uheNuc7A8Vpcg%40mail.gmail.com.

--

Studying for the Turing test

[Murilo Carlos Cardoso]

I could be wrong but "eventually consistent" is different from "real inconsistency".

I mentioned chapter 8 of the Red Book because in the beginning of the chapter, domain events are demonstrated without any kind of persistence for the events and no transitory status in the aggregates (when in the same BC). If anything bad happens between the "conversation" how can I recover? I consider this a real inconsistency. 

My question is about getting a confirmation that, to avoid real inconsistency the steps below are necessary even in the same BC:

(i) Transitional status in the aggregate;
(ii) Persist events before send them (to avoid lose events);
(iii) In the receiving side execute the "domain operation" in an idempotent way when possible. When not possible, deduplicate the event to check if the event had been received before process the domain operation.

Thanks!

(I'm not a english native speaker so I hope you can understand)

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAC9RQtjienwJxqQtQGE0qWbVTxBc%2BTRs%3DCoLYXJ5x4DBM6Jd2w%40mail.gmail.com.

[reda laanait]

domain events are demonstrated without any kind of persistence for the events and no transitory status in the aggregates (when in the same BC). If anything bad happens between the "conversation" how can I recover?

Chapter 8 Domain events, spreading the news to remote Bounded contexts (pp 303-304) handle these points. 

You can find more details about event de-duplication in the implementation part of the chapter

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAH7D42R3N_Un7W%2BfEqFKKMmG%3DQQ7JhTvWj0OAOKK1eKWEnizuw%40mail.gmail.com.

[Chris Richardson]

Murilo,

Life is certainly easier if you can update multiple business objects in an ACID transaction.

If you cannot because 

1. You are following a strict DDD single aggregate per transaction
2. Your framework/database limits you to a single aggregate per transaction
3. Your aggregates are in different services (in a microservice architecture) and you don't/can't use distributed transactions.

Then you must use an eventually consistent approach, which typically relies on the participants exchanging messages, such as events.

A few points about eventual consistency:

1. More generally, I describe it as a Saga (https://microservices.io/patterns/data/saga.html)
2. There are two types of Sagas: choreography-based sagas that use events and orchestration-based sagas that use asynchronous request/reply
3. Events/messages need to be published as part of the "transaction" that updates the aggregate: that's an inherent feature of event sourcing; with traditional persistence you need to use https://microservices.io/patterns/data/transactional-outbox.html
4. You need an idempotent consumer: https://microservices.io/patterns/communication-style/idempotent-consumer.html

I hope this helps.

Chris

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAH7D42R3N_Un7W%2BfEqFKKMmG%3DQQ7JhTvWj0OAOKK1eKWEnizuw%40mail.gmail.com.

--

Learn microservices - http://learn.microservices.io

Microservices application platform http://eventuate.io

Consulting and training http://chrisrichardson.net

[Murilo Carlos Cardoso]

reda laanait, my point is that the approach presented in the book for "remote bounded contexts" should be technically the same for "same bounded context" in order to avoid real inconsistency.  Don't you think? 

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAJg1kZPQzNDA9_GaKPkRmY2Egj-WfVeYYZZR319GWqwiXpzHaQ%40mail.gmail.com.

[Murilo Carlos Cardoso]

Chris, 

Following the Vaughn Vernon recommendation, to achieve a high level of decoupling and single responsibility I should do this: 

1. "You are following a strict DDD single aggregate per transaction".

So, if the aggregates are in the same BC or not (this is my question), I should do this follow the items below avoid real inconsistency: 

1. Events/messages need to be published as part of the "transaction" that updates the aggregate: that's an inherent feature of event sourcing; with traditional persistence you need to use https://microservices.io/patterns/data/transactional-outbox.html;
2. You need an idempotent consumer: https://microservices.io/patterns/communication-style/idempotent-consumer.html;

Right? 

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAA9exRt5sb4eMRRRs_3gi7H_5FHj%2BJ3OoUPbka0NRbyL%2Bu%2Boxw%40mail.gmail.com.

[Tomas Ptacnik]

Hi Murilo,

In my opinion, it doesn't matter if you are following DDD or not or if you are in the same BC or not, whenever you are publishing events you should follow the 

- "safe events publishing" pattern on the publishing side

- "safe events consumption" with an "idempotent receiver" on the consuming side

if you don't, then as you pointed out the "eventual inconsistency" will turn into perpetual inconsistency.

Best

Tomas

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAH7D42RPrucaAuFn%2BNoU3QEEJWnWyZB--FmejU%2BDpWMkXA%3D8rA%40mail.gmail.com.

[Nik]

@Murilo, 

You can still use one single transaction to talk to other aggregates within the same BC using domain events. You just commit the transaction after all domain events have been handled.

Use integration events to forward messages to other bounded contexts in an eventual consistency manner. This, of course, requires more advanced techniques like the outbox pattern, etc.

[Murilo Carlos Cardoso]

Thanks Tomas! This is my opinion too. 

I'm looking for a confirmation or a different point of view.

Thanks for the confirmation.

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAJ7md6Xvp8OFR-iHV-VPh%3D%3DJZ58BPPUgpHxMoUjpUusp3kbVcg%40mail.gmail.com.

[Murilo Carlos Cardoso]

Nik,

The problem with domain events in the same transaction is that I cannot "chain" events. At least I don't know a tool which allows it. 

 In a real world, at least in my experience, it is not rare to chain events across more than two aggregates. Considering CQRS, this becomes impractical. 

Am I wrong?

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/35700f51-8ba2-44a3-a96e-2306444080ben%40googlegroups.com.

[Nik]

Hi Murilo,

I have the aggregate roots holding the domain events to be dispatched and dispatching them before committing the transaction.

In a while(true) loop I get all the agg. roots and the events, clear the event collections on the agg roots and dispatch the events.

In the next iteration of the loop I do the same, checking if there are some new domain events produced, if not I commit the transaction.

I also track the loop iteration count and issue a warning if the chain is too long (too many iterations).

[Nik]

Btw the aggregate roots I get from the Unit Of Work (dbcontext, db session,..).

[Murilo Carlos Cardoso]

Nik,

When I mentioned "chain events" I mean:

Agg A publishes Event A THEN Agg B reacts to Event A and publishes Event B THEN Agg C reacts to Event B and publishes Event C. 

What you have mentioned can deal with this all in the same transaction?  Do you have a reference or some code that I can have a look at?

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/7f51baff-c571-4db2-9d3a-3667cae133c8n%40googlegroups.com.

[Andrew Young]

Tomas,

> then as you pointed out the "eventual inconsistency" will turn into perpetual inconsistency

There is another way you can get perpetual inconsistency. When the consumer of a domain event throws an unexpected exception and doesn't know how to handle it. In these cases, the consumer still needs to move on to the next event because it can't halt processing. From this point on, the system is perpetually inconsistent.

How do you deal with this? Especially on a production environment when you can't bring the system down?

Thanks

Andrew.

To unsubscribe from this group and stop receiving emails from it, send an email to ddd...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAH7D42QH3gKXNowiPNMOz8mWYMiHgzp3LMae2uheNuc7A8Vpcg%40mail.gmail.com.

--

Studying for the Turing test

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ddd...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAC9RQtjienwJxqQtQGE0qWbVTxBc%2BTRs%3DCoLYXJ5x4DBM6Jd2w%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ddd...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAH7D42R3N_Un7W%2BfEqFKKMmG%3DQQ7JhTvWj0OAOKK1eKWEnizuw%40mail.gmail.com.

--

Learn microservices - http://learn.microservices.io

Microservices application platform http://eventuate.io

Consulting and training http://chrisrichardson.net

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ddd...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/CAA9exRt5sb4eMRRRs_3gi7H_5FHj%2BJ3OoUPbka0NRbyL%2Bu%2Boxw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ddd...@googlegroups.com.

[Murilo Carlos Cardoso]

Andrew,

In the consumer side if something goes wrong the message should go to the dead letter queue so it could be reprocessed in the near future (when the bug has been fixed)

If you are not familiar with the concept, just Google for "dead letter queue".

Here is one example in RabbitMQ -> 

https://www.rabbitmq.com/dlx.html

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/2b0963e4-c632-4dbf-af51-45d76db58472o%40googlegroups.com.

[Tomas Ptacnik]

Hi Andrew,

first very important step is to get to know about the problem -> 1. have monitoring and alerting in place.

Step 2. is fixing the problem.

That might be fixing the producer or consumer and then replay event(s). It's highly dependent on the problem -> What was the issue, how, many aggregates were affected.

Best

Tomas

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/2b0963e4-c632-4dbf-af51-45d76db58472o%40googlegroups.com.

[Andrew Young]

So let me see if I understand this. Is the takeaway here that:

1. Assume everything is eventually consistent until monitoring/alerting tells you otherwise
2. There is no general pattern to mitigate situations like this
3. Each situation is different
4. Fixing it is done directly on production whether it is replaying, or SQL script, push code, or whatever else brings the system back to consistency?

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/dddcqrs/2b0963e4-c632-4dbf-af51-45d76db58472o%40googlegroups.com.

[Nik]

@Murilo something like this:

while(true) {

aggRoots = dbSession.entities.aggRoots;

events = aggRoots.events;

if (events is empty) { break; }

aggRoots.clearEvents();

dispatchEvents(events);

}

dbSession.commitTransaction();