How to (not) handle events when replaying events on an aggregate with event sourcing

[Sebastian P.R. Gingter]

Hi,

it seems I have some difficulties understanding some concepts around (or the word "event" might be used wrongly here?).

Let's say I have an aggregate root, and a command that is going to do something with it.

I'd create a new instance of my domain object for that aggregate id.

Then I'd replay all existing domain events from the event store on that instance, which brings it to the state I need.

The command handler now checks if the command can be applied (checking business rules etc.), and if this is the case, this creates an domain event of its own that gets stored and applied to the object too.

Now this new domain event may trigger some logic that goes to an external source, fetches data from there and creates subsequent domain events based on external data.

Now, the question is, how would I prevent this domain event to trigger this logic when I am going to replay it the next time.

Because the event happened, and the logic got applied, and we already got the external data and created other events based on that.

When this would run again, we'd wrongly duplicate events with potentially wrong (different/newer) external data.

At the moment I see three possibilities to solve this, but I am not sure that either would be deemed correct.

First: I would pass a bool to the event handler stating whether this is a replay or not.

This, however, would make my event handlers harder to test, and introduce a lot of

    if (!replaying) {

        // do that now

    }

That somehow doesn't feel right.

Second: I could implement the subsequent domain events as real language-specific events (in my case c# events).

Only after the repository created the aggregate instance and replayed all existing events, the repository would attach all event handlers.

So the event would be raised, but while replaying there would simply be no handler.

I am not sure this is the correct approach, though.

Third: The code to trigger the other events could be placed in the command handler.

That, however, would mean that I would have to separate parts of the business logic. Partly it would go in the domain event handler, partly in the command handler.

What, if I would need to re-use this code?

That would, imho, also violate the "Tell, don't ask" approach, as the command handler would need to make decisions based on the state of the aggregate.

So, what did I miss?

On that, if someone would have a link to an open source example so that I can see how that could be solved, I'd be very happy.

Regards,

  Sebastian

[Vincent van Dijk]

Don’t register that particular event handler when replaying?

Vincent.

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
Visit this group at https://groups.google.com/group/dddcqrs.
For more options, visit https://groups.google.com/d/optout.

[Vincent van Dijk]

After re-reading I think a saga/process manager is the best option. 

Vincent.

On 7 Apr 2018, at 13:01, Sebastian P.R. Gingter <seba...@gingter.org> wrote:

--

[Kijana Woodard]

Here's the where things fall off the rails: " Now this new domain event may trigger some logic that goes to an external source, fetches data from there and creates subsequent domain events based on external data."

This makes no sense to me.

An aggregate is the total understanding of it's part of the domain. There is no "external source" that it needs to consult.

I think what you are looking for is that "something else" handles an event from your aggregate. 

*Then* that "something else" might publish events of its own.

*Then* those events might be result in new commands for your aggregate.

*Then* your aggregate may publish new events of its own.

>  After re-reading I think a saga/process manager is the best option. 

Probably. Yes.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/dddcqrs.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

[Sebastian P.R. Gingter]

Hi Kijana,

thanks for your response.

I think what you are looking for is that "something else" handles an event from your aggregate. 

*Then* that "something else" might publish events of its own.

*Then* those events might be result in new commands for your aggregate.

*Then* your aggregate may publish new events of its own.

Well, in this case my question would be "How do I tell this 'something else' to not handle the event from the aggregate when its replaying.?"

In my case, a new entity (a report) gets introduced to my system. And then at some point later some other user wants to verify that report.

For this I need to import data relevant to that report from an external source to decide whether that report must be rejected (external source has no data => invalid) or if it is okay (data is there, and other information from the report has been verified too => valid). Then another entity (verified report) can be created which references the external data.

Cheers,

  Sebastian

[Stefan Priebsch]

There are many things that need to happen as a consequence to an event. Let's call them side effects. You never want to repeat side effects when sourcing from events. Imagine sending a "welcome" email - you certainly don't want to repeat that every time you source an aggregate.

Create separate event handlers for side effects. They must only run once (for each event), thus the need to keep track of which events they have already "processed". Run a catch-up subscription and make sure that your side effects only receive each event once, then it should be sufficient to keep the id of the last event each side effect handler has processed, and continue from there when the handler is run again.

In case you need the result of the "something else" inside your aggregate, then you would probably want to talk to the external system from the command handler, and document the result in an event that your aggregate consumes.

Hope this helps. Let me know if I need to clarify further.

Regards,

Stefan

[Danil Suits]

how would I prevent this domain event to trigger this logic when I am going to replay it the next time.

Thought experiment: how would you avoid triggering that logic if you were simply reading a snapshot of aggregate state from your database?

To my mind, there's not a lot of difference between "read a representation of current state from the store" and "read a representation of history from the store, and reduce that history to a representation of current state".

My guess is that you would do it by decoupling the side effects from the process of reading the representation.

 Now this new domain event may trigger some logic that goes to an external source, fetches data from there and creates subsequent domain events based on external data.

What's supposed to happen if the fetch of data from the external source fails?  Maybe the external source is temporarily unavailable.  Are you expecting to retry later, or roll back the original command?

 In my case, a new entity (a report) gets introduced to my system. And then at some point later some other user wants to verify that report.

For this I need to import data relevant to that report from an external source to decide whether that report must be rejected (external source has no data => invalid) or if it is okay (data is there, and other information from the report has been verified too => valid). Then another entity (verified report) can be created which references the external data.

 "At some point later" suggests that the SLA is a bit forgiving, which in turn suggests that you are going to retry if there is a failure.  That suggests to me that you want to save the aggregate (commit the transaction) and then trigger the subscriber to wake up and notice the event.  The subscriber can then catch up on the domain events that it has missed, query the external source, and then update its own state with the changes to the world that it has observed.

Key point: there are other ways of triggering the subscriber (like having it also get woken on a schedule, to check for events that it may have missed).

Give Rinat's essay on Evolving Business Processes a read, and think about how you as a human being would coordinate the work to be done.

[Kijana Woodard]

Yes to the two replies above.

Think about it this way: if you're reacting to some external event, do you want to do that more than once? If not, then remember where you are in the event stream and start from there.

To the point about humans, imagine someone sent you an email every day reminding you to submit your monthly time sheet. Once you've submitted it, you'd ignore the future emails. At some point, you'd get annoyed and go to the source and tell them to stop sending the email. You only need it once. You have a permanent stream of all email sent by every external actor in Outlook.

--

You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.

[Daniel Hardt]

First of all i delete my first post, because i wrote it on my phone with the german text recognition activated and it messes with some word. So here ...

You can trigger the external event  for example in the same method, where you do your validation of the command and create the new event, that you want to apply to your state.
You do not trigger your side effect in the method, where you actually alter the state of your aggregate.

How you trigger your side effect is your choice:

Maybe you publish the same event your create for applying to your state to a service bus or an internal domain event handler or something.

Another alternative is like others already said, you subscribe to your eventstore, listen to every event, trigger your side effect and the worker process has to remember what event it already processed. 

The pro of an extra service bus is, that you do not have to remember the last processed event, but if the service bus fails in any way, you maybe lost events for trigger your side effects. 

If you use an event store and a worker that polls for new events, you do not lose any events for triggering, but the worker must remember, which event is processed. 

I hope that helps.

[Philip Macy]

One practical solution I have found was from Martin Fowler https://martinfowler.com/eaaDev/EventSourcing.html  is the "Gateway"

I will quote him 

"

External Updates

One of the tricky elements to Event Sourcing is how to deal with external systems that don't follow this approach (and most don't). You get problems when you are sending modifier messages to external systems and when you are receiving queries from other systems.

Many of the advantages of Event Sourcing stem from the ability to replay events at will, but if these events cause update messages to be sent to external systems, then things will go wrong because those external systems don't know the difference between real processing and replays.

To handle this you'll need to wrap any external systems with a Gateway. This in itself isn't too onerous since it's a thoroughly good idea in any case. The gateway has to be a bit more sophisticated so it can deal with any replay processing that the Event Sourcingsystem is doing.

For rebuilds and temporal queries it's usually sufficient for the gateways to be able to be disabled during the replay processing. You want to do this in a way that's invisible to the domain logic. If the domain logic calls PaymentGateway.send it should do so whether or not you are in replay mode. The gateway should handle that distinction by having a reference to the event processor and checking the whether it's in replay mode before passing the external call off to the outside world.

External updates get more complicated if you are using Retroactive Event see the discussion there for gory details.

Another tactic that you might see with external systems is buffering the external notifications by time. It may be that we don't need to make the external notification right away, instead we only need to do it at the end of the month. In this case we can reprocess more freely until that time appears. We can deal with this either by having gateways that store external messages till the release date, or triggering the external messages through a notification domain event rather than doing the notification immediately.

External Queries

The primary problem with external queries is that the data that they return has an effect on the results on handling an event. If I ask for an exchange rate on December 5th and replay that event on December 20th, I will need the exchange rate on Dec 5 not the later one.

It may be that the external system can give me past data by asking for a value on a date. If it can, and we trust it to be reliable, then we can use that to ensure consistent replay. It also may be that we are using Event Collaboration, in which case all we have to ensure we retain the history of changes.

If we can't use those simple plans then we have to do something a bit more involved. One approach is to design the gateway to the external system so that it remembers the responses to its queries and uses them during replay. To be complete this means that the response to every external query needs to be remembered. If the external data changes slowly it may be reasonable to only remember changes when values change.

External Interaction

Both queries and updates to external systems cause a lot of complication with Event Sourcing. You get the worst of both with interactions that involve both. Such an interaction might be a an external call that both returns a result (a query) but also causes a state change to the external system, such as submitting an order for delivery that return delivery information on that order."