Comments on InfoQ article - a whole system based on event sourcing is an anti-pattern

[alfredo.cr...@gmail.com]

Hello everyone,

I would like to get more context from the experts on this article https://www.infoq.com/news/2016/04/event-sourcing-anti-pattern which summarize this talk from Greg Young.   

I am currently evaluating event sourcing for building an (almost traditional) online lender system. In a nutshell; As a customer, you go to the website and you get loans. As a lender, you need to tweak/adapt your risk policies to make the system sustainable in the presence of defaults. 

I am looking at event sourcing because of the following features highlighted by Martin Fowler in this talk. 

(i) audit trail and 

(ii) distribution support 

My goal is to have event sourcing in the code from the beginning and then distribute/extract services later if I need to. As pointed out by Fowler it is very difficult to add event sourcing at later stages of development (unless you do it for new isolated parts of the system).

In a perfect world different independent parts of the system are writing to the log (say for example the admin of the website and the user-facing website) and everyone can subscribe to the log and build their own models according to their reading needs. 

Now, I don't see exactly why I should cherry pick the places in the system that are better suited to this approach given how general it seems to be on paper. I can see that places where lots of distributed transactions are happening and where conflicts become the norm to be something where I should follow a more classical CRUD approach. But beyond these two cases, why I shouldn't go event sourcing first and leave the classical CRUD style to exceptions?

Thank you for your help, much appreciated

Alfredo

[Kasey Speakman]

Thanks for posting this! Excellent question from my perspective. I watched the video and have a few reactions to some of the points brought up.

First example was making corrections to data. I think he is referring to issues around event-sourcing CRUD data. (Otherwise, just issue another event saying the data was changed?) But CRUD data presents a number of issues. It is awkward to model CRUD as events. Try modeling it in a simple way that permits schema changes... let me know what you come up with in a separate thread. (Seriously, I want to know any good ideas there.) Just using an event log, you cannot do set validation efficiently (e.g. unique email). Naive implementation is to replay all streams of a certain type, collect all email addresses, and see if new email is in list. Obviously that will cease to function (out of memory exception) at some point. You have to make that more efficient by keeping an extra write-side model (aka index), probably in an RDBMS, or just query the read side and accept that you may sometimes get a uniqueness violation.

Second point was: event sourcing analysis is a different style which can be more expensive. I would tend to agree. Mainly because it focuses on behaviors / interactions between entities. This often leads me ask hard questions of domain experts. Their time can be hard to get. It might be hard to code. Sometimes you find that budget constrains you to only implement what you know with limited domain expert access, which is usually just data structure.

Third point was: event source everything is a really really bad idea. Mentions event sourcing system of systems. Not sure I observed the substantiation of this statement. I will say here though, that trying to integrate multiple systems through a single event store is just another version of an Integration Database (or what I call pouring concrete over your software). Greg mentions that EDA is a top-level architecture. My take-away from this distinction is: integrating through Events is a valid strategy, but not through an Event Store.

[alfredo.cr...@gmail.com]

Hi Kasey, thank you for the feedback.

> First example was making corrections to data.

All I know here is that AVRO does seem to help as a tool for that https://www.youtube.com/watch?v=_d4mAi3qkDA&feature=youtu.be

> Second point was: event sourcing analysis is a different style which can be more expensive. I would tend to agree. Mainly because it focuses on behaviors / interactions between entities.

I think it may depend on the domain too. I can imagine for some domains it is better to focus on behaviors as first entities, while for others state/objects are better suited. I don't have any canonical example though, so if you have one that would be awesome. I definitely know that accounting is great for event sourcing :) What is an example of a system that is really bad modeled with event sourcing?

> Third point was: event source everything is a really really bad idea.

> My take-away from this distinction is: integrating through Events is a valid strategy, but not through an Event Store.

That seems fair. Also Martin Fowler pointed out that integrating with systems that are not event sourced is really hard. But if the number of external systems I am working with is fairly small and I own most of the internal architecture, I don't see any big red light for adopting event sourcing as a default.

[Kasey Speakman]

> First example was making corrections to data.

All I know here is that AVRO does seem to help as a tool for that https://www.youtube.com/watch?v=_d4mAi3qkDA&feature=youtu.be

Great video, thanks for linking that!

Re Avro: The speaker in the Event schema section is talking about publishing event schema to external parties. What I'm talking about is your own internal CRUD data structures, assuming you try to even source them. He actually addresses that briefly at the beginning his talk but I was unclear on his explanation. The simplest way to make CRUD events is to put chunks of your CRUD data structure into the events. But when you later have to change your data schema (e.g. new required field), you have to do something about the already-recorded events that have chunks of old schema. Otherwise the events won't replay. One option is to create new events and write an adapter that upgrades old events into the new ones as they are loaded. Another possibility is to make an event for each CRUD property changed. And there are others. Each has trade-offs.

 

> Second point was: event sourcing analysis is a different style which can be more expensive. I would tend to agree. Mainly because it focuses on behaviors / interactions between entities.

I think it may depend on the domain too. I can imagine for some domains it is better to focus on behaviors as first entities, while for others state/objects are better suited. I don't have any canonical example though, so if you have one that would be awesome. I definitely know that accounting is great for event sourcing :) What is an example of a system that is really bad modeled with event sourcing?

It is usually not so cut-and-dry even for a single "system" to be all CRUD or all event sourced. For instance, I have Courses which are used in Registrations. The behavioral (event-sourcing-fits-well) parts of the system do not care about the course's name, just its ID. But it would be folly to develop an application for human use without course names in it. The names are purely CRUD meta data that the user inputs and searches which has nothing to do with behavior, but I need them. There is also user-editable configuration data. This is changed via CRUD operations too, but it affects how the system processes use cases (example: the course's passing score). To develop this system as a homogeneous style (CRUD everything or event-source everything) makes some part of the system ill-fitting. However, to develop each as its own individual subsystem increases operational costs and probably effort-to-market. So depending on business goals, it might be desirable to stick to a single style knowing that I'm creating technical debt around the ill-fitting pieces, which I may have to pay it later. I think the larger point is to make yourself aware of the trade-offs you are making and what you will can do to mitigate them once they start tripping you up. And you seem to be doing that.

[Kasey Speakman]

The end of this talk further addresses some of these issues. Starting at 50 minutes in.

https://www.youtube.com/watch?v=kZL41SMXWdM

[Dariusz Lenartowicz]

How do you communicate other services about changes in CRUD-ed service?
Propably events... Then... How do name those events?

> Second point was: event sourcing analysis is a different style which can be more expensive

Is it different than Aggregate+DomainEvent+ORM solution?
What if I use Vernon style of saving aggregate in database (https://vaughnvernon.co/?p=942)?
Saving aggreagtes in document database (json) has very simillar "problems" like saving events (json), isn't it? :)

[Kasey Speakman]

How do you communicate other services about changes in CRUD-ed service?
Propably events... Then... How do name those events?

Integration techniques other than EDA exist. Who said the CRUD data was a service? I didn't assume that. These choices depend on product goals.

  

> Second point was: event sourcing analysis is a different style which can be more expensive

Is it different than Aggregate+DomainEvent+ORM solution?
What if I use Vernon style of saving aggregate in database (https://vaughnvernon.co/?p=942)?
Saving aggreagtes in document database (json) has very simillar "problems" like saving events (json), isn't it? :)

Analyzing use cases was being described as being more expensive, not storage choices.