Using domain events to report/track synchronous process progress

[Alexandre Potvin Latreille]

I'm trying to leverage domain events in order to track the progress of a process, but the problem I'm facing is that all raised events are written in the same transaction so if the transaction has to rollback I'd lose progress events too.

For instance, if a DomainService emits the events BeganToApplySomePolicy and PolicySuccessfullyApplied, the BeganToApplySomePolicy event would be lost in case of an exception which would cause the transaction to rollback. From within the domain service, I couldn't just use a try catch statement to swallow the exception because aggregate changes must not be persisted and the DomainService is not aware of the transaction, the application service is.

Is there a practical way of solving this problem? Should I just avoid using domain events for this?

One approach I though of would be to break a domain service process into multiple steps that should be ran by the application service in different transactions, but synchronously. However, even with this approach a failure event would have to be published from the application layer, not the domain.

E.g.

//AppService

function applySomePolicy() {

   

       steps = somePolicy.apply(); //call on domain service

        for step in steps {

            transaction {

                step.apply();

            }

        }

}

[Danil Suits]

BeganToApplySomePolicy

I believe - but I'm not certain - that "To Begin" has the wrong lexical aspect to be useful as an event description, in so far as it doesn't really describe a state change in an entity.  (we would certainly hope that it wouldn't -- after all, if we are only modifying a single aggregate per transaction, then we had best not be modifying both the aggregate and the domain service.

Google suggests that there are some real terms of art at play in the space -- achievements vs accomplishments vs activities.... 

> if the transaction has to rollback I'd lose progress events too.

If that's a problem; which is to say if you have progress events that the business rules in the aggregate are not allowed to veto, that sounds to me like something that should be published asynchronously? message in a bottle, and hope that the recipients don't get confused by the ordering?


 

[Alexandre Potvin Latreille]

You are right that it has more to do with logging than describing state changes, but at first glance events seems to be an interesting logging mechanism as well. I just haven't found a practical implementation that allows both, to log state changes and attempted operations yet.

[Alexandre Potvin Latreille]

Architecture notes:

• It's a web application.
• Not using event sourcing.
• Using a collection-oriented repository without a save method.
• Events are published through a static thread-local event bus.
• A subscriber is registered at the beginning of every request and stores all published events in the event store.
• Transactions are managed at the application service level and the event store participates in the same transaction as the DB.

After thinking more about it, I think that my problem comes from the fact that I'm doing batch processing using domain services and I'm creating/modifying multiple aggregates in a single transaction. I decided to do so because there will rarely be any contention during those processes which runs overnight and it seemed easier to implement because I'm lacking a solid messaging infrastructure for sagas.

 

Therefore, when a failure occurs there may be other aggregates that have been mutated already and I do not want their changes to get persisted, but at the same time I want to raise a failure event that will get persisted. The easiest approach I can think of right now is to actually raise progress/failure events through a different communication channel which would have a subscriber that stores events in a different transaction.

Any other ideas?

[Brad F]

I'd save everything in one transaction. Either loop through all aggregates and add them to the repository, then call one save at the end. Or create a parent aggregate to hold the children and just save that one.

[Frank]

I'm trying to leverage domain events in order to track the progress of a process, but the problem I'm facing is that all raised events are written in the same transaction so if the transaction has to rollback I'd lose progress events too.

The you have simply no domain events. A domain event describes a fact that happened and cannot be taken back. Think of a piece of paper. As soon as you have written something on this paper, you cannot undo this with a technical trick (line time travelling or in your case a transaction). You have to cope with that in terms of business behaviour. Bookkeepers for example do not undo a booking once written into the ledger. Instead they cancel the first one (with the opposite amount and accounts) and do a second one. Then you can see that something happened (one event), was negated (second event) and with a third entry corrected (third event.

For instance, if a DomainService emits the events BeganToApplySomePolicy and PolicySuccessfullyApplied, the BeganToApplySomePolicy event would be lost in case of an exception which would cause the transaction to rollback. From within the domain service, I couldn't just use a try catch statement to swallow the exception because aggregate changes must not be persisted and the DomainService is not aware of the transaction, the application service is.

Exactly. If you want think in domain events you can emit them to a process manager that also receives the "rollback" which is another event that can happen. Then the process manager decides how to compensate the steps in case of a failure, not your technical transaction.

 

Is there a practical way of solving this problem? Should I just avoid using domain events for this?

You should use domain events if it is part of your business domain. Like in the bookkeeping example booking and cancellation are two known use cases in this domain. Rolling back a transaction is not. Ask your domain experts if executing only part of the steps is OK for them. If so, ask them also, what to do if some steps didn't succeed.

Mostly you will come to realise that you need this kind of thinking in a business process that takes some time or some specific order. That is starting your steps, executing your steps and ending your steps can take some time or must be executed in a specific order.

[Alexandre Potvin Latreille]

@Frank I'm not sure to understand what you are suggesting. I understand that event streams are immutable and that to correct the past you need to take compensatory actions, but I'm not sure how that's relevant to my problem. I'm not using event sourcing anyway, so I might have changes that do not generate events.

Imagine a simple import task that reads data from a data source and creates aggregates in bulk. Such import task is a long running process. In order to gather statistics on the imports I may want to publish the following events:

ImportTaskStarted { importer, dataSource }

ImportTaskCompleted { totalImported }

ImportTaskFailed { failureDetails }

The problem is that if I have something like that in a domain service:

  try {

       publish(ImportTaskStarted(...));

       for data in datasource {

           repository.add(createAggregateFromData(data));

       }

       publish(ImportTaskCompleted(...));

   } catch (e) {

       publish(ImportTaskFailed(e));

   }

And an application service that controls the transaction:

transaction.begin();

subscribe('ImportTaskFailed', transaction.rollback);

subscribe('ImportTaskCompleted', transaction.commit);

startImportTask(...);

Then all the events that were raised wouldn't be stored in the DB because they are raised in the same transaction that gets rolled back (I'm using the following event publishing approach with a subscriber that stores all events in an event store). I want to perform a rollback because it is easier to reason about the import task if it succeeds or fail as a whole rather than allowing partial completion.

I think what makes it complicated is that I'm trying to do this in bulk. I though it would be easier, but the problem is avoided if every step happens in it's own transaction. However, that would require more work to rollback an import because it would have to be done through compensating actions (rather than a simple rollback) and these may fail as well leaving the system with a half-complete import for a period of time.

On Friday, September 23, 2016 at 5:11:55 PM UTC-4, Alexandre Potvin Latreille wrote: