Rewrite history: amendments/corrections?

[Alexandre Potvin Latreille]

I maintain a complex case tracking system and I'm often asked to rewrite history after human mistakes were made. For instance, a case could have been closed a month ago with response "A" and I'm being asked to amend the response for "B" as well as the closure date for something else. Other examples could be that the case got assigned to the wrong individual by mistake or a document was uploaded to the wrong case...etc.

It is important to note that the real world is the system of record for most data, not the system itself, but the system still enforces some business rules & processes to help users avoiding mistakes.

This is a problem that exists in many domains and I was wondering if there was a common approach to deal with that? The business people here seems to prefer to entirely rewrite the history rather than adding compensating actions and I'm not sure whether they are right or wrong, but I feel that by rewriting history we introduce the risk of destroying valuable information. Not only that but perhaps some decisions were already taken based off the erroneous information and if these turns out to be investigated in the future then they wont be justifiable given the rewritten history. It also seem slightly more technically challenging to rewrite the past given that future actions are sometimes dependent of past actions as per system rules.

One idea I had would be to allow marking events that have no business value and those would get filtered out when looking at the history, but I think even that could be misleading. Should I actually argue with business people on this one or simply do what they ask?

[Alexander Langer]

> It is important to note that the real world is the system of record for
> most data, not the system itself, but the system still enforces some
> business rules & processes to help users avoiding mistakes.

Indeed.

> This is a problem that exists in many domains and I was wondering if
> there was a common approach to deal with that? The business people here

I don't know of a common approach, but in such cases I commonly argue
for making "corrections" first class citizens in the (software) domain,
since user errors are just too common and can never be avoided.

If you integrate user errors and their corrective measures into your
domain model, you'll get a very clean history and no valuable history
can be destroyed.

I.e., in the example you gave, I'd suggest to add a use-case and
corresponding events to your system to allow certain user groups to
manually reopen a case as a corrective measure, moving the response "A"
to another case. Model this "correction" as a process in your domain
model just as you'd model any other process.

HTH,
Alex

BTW, I submitted a talk on this exact topic to the next DDDEU conference.

[Peter Hageus]

If there’s dependant behaviour/sideeffects in other parts of the system, compensating actions is the only way I’m aware of.

For events that have not yet caused any side effects I’ve used ‘tombstones' in the stream, to point out events that should not be processed. There are a few cases where this works well. (mainly stuff queuing up awaiting some sort of trigger/stimuli)

/Peter

> --
> You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
> Visit this group at https://groups.google.com/group/dddcqrs.
> For more options, visit https://groups.google.com/d/optout.

[Greg Young]

I doubt they prefer to rewrite the entire history, they just like to say that.

With case tracking there are often needs of as-of vs as-at querying abilities. What did we know as of this point vs what did we effectively know at this point even if it came in later.

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/dddcqrs.
For more options, visit https://groups.google.com/d/optout.

--

Studying for the Turing test

[Alexandre Potvin Latreille]

They often argue that the history wouldn't perfectly reflect what actually happened in the real world because in the real world there wasn't necessary a mistake. For instance, perhaps they sent physical files to the right team, but entered the wrong team in the tracking system. With compensating actions the system's history would kind of lie on what really happened in the real world, but by rewriting history it would lie about the previous state of the system.

It seems there are definitely two different competing histories at play. Another common scenario is that they may receive a physical document at a specific date, but took 3 days to actually input that fact in the system. The system records the date the document entry was made in the system as well as the physical received date, but when displaying history entries chronologically we order them according to the date actions happened in the system and I often receive requests from business people to amend the system's event date to match the physical received date. Obviously I'd want to avoid that and I could rather implement special ordering strategies that takes business dates into account, but then you'd have a mix of system-date ordered events and business-date ordered events and that would become even more tangled.

I'm tempted to model the two history lines explicitly, one being the record of all system's actions that were done and the other being a non-immutable history attempting to reflect what happened in the real world as much as possible. I'm not sure it would really be practical though, because it would most likely mean that the system couldn't enforce any rules or processes. For instance, if in their business process a case cannot be considered closed before undergoing review and the system tries to enforce that rule, but what actually happened in the real world is that the case was considered closed and a decision letter already sent to the client at this point we'd have a problem.

I've dealt with these scenarios in the past and was never satisfied by any of the implemented solutions. It seems I can't find any perfect solutions to these problems.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.

Visit this group at https://groups.google.com/group/dddcqrs.
For more options, visit https://groups.google.com/d/optout.

[Greg Young]

"

It seems there are definitely two different competing histories at play. Another common scenario is that they may receive a physical document at a specific date, but took 3 days to actually input that fact in the system. The system records the date the document entry was made in the system as well as the physical received date, but when displaying history entries chronologically we order them according to the date actions happened in the system and I often receive requests from business people to amend the system's event date to match the physical received date. Obviously I'd want to avoid that and I could rather implement special ordering strategies that takes business dates into account, but then you'd have a mix of system-date ordered events and business-date ordered events and that would become even more tangled."

This is an as-of vs as-at query. Common in such systems especially in finance. http://codebetter.com/gregyoung/2014/03/02/event-sourcing-and-postpre-dated-transactions/

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/dddcqrs.
For more options, visit https://groups.google.com/d/optout.

[Alexander Langer]

On 26.10.17 17:11, Alexandre Potvin Latreille wrote:
> They often argue that the history wouldn't perfectly reflect what
> actually happened in the real world because in the real world there
> wasn't necessary a mistake.

I don't agree. In the real, physical world, some person did in fact
entered bad data into a physical computer. That happened.

[Alexandre Potvin Latreille]

@Greg Very interesting.  I'm still not entirely sure how I'd model the process of recording pre-dated actions in practice though, especially when rules come into play. For the example where a document is received physically at an earlier date than when the attempt to record that fact in the system was made it's quite straight-forward. The user could be presented with a "document received on" field that would translate to an "applies" date, but that's a simple one given attaching documents may not have other implications than stating that a document was received at that date. However, what about events that could be involved in business processes where future events might have be influenced (assignments, closures, etc.)?

@Alex That's true, but they could argue that's not an interesting part of the world. I could make a parallel with how I work in Git: when I implement features in their own branch I may often commit half-baked work and have a few "work in progress" commits just for the sake of not losing my work, but before merging the feature I'd cleanup the history because these are irrelevant to the software's history. Those are relevant if you want to track my work as a developer, but not if you want to track the software's evolution. From that perspective I understand the dilemma.