Finders - what's for?
Marcin Kuthan
Sławomir Sobótka
Command Stack for performing domain operations
Read stack for reading data for presentation
Now in non-crud applications, there is a significant miss-math:
We try optimize model for
- being supple - 3rd normal form at DB level, buts sucks while reading
in case of cross-table presentation (joins). Upper level (object
model) is good for business operations but sucks as a presentation
model (unnecessary data - irrelevant for screens)
- read fast for reporing-like views- 1st form rocks, no joins
This are just few, very very basic motivators for CqRS.
So finders are responsible for reading stack. In general reading stack
returns DTos because:
- they fit screen needs
- encapsulate domain "shape" - extremely important when remote apps
join our playground.
In general finders should be thin. They just read data and send it to
the caller. Trivial responsibility.
In our sample we tried to present many forms of separation of both stacks.
Strongest form rely on 2 separate DB model - optimized for different
purposes. This doesn not to have to be relational engines even!
Command stack may rely on event sourcing and read on some ultra fas
nonsql.
or You can have 2 models where read model if flaterned do 1st form by
materialized views
or updated b events for domain/application even
Or You can just query for pure sql and pack it to dtos. That is also
ilustrated and thats what mentioned finders do.
WARNING When reading many rows You should not read entities and repack
it to DTOs. That pure evil, besides... db-nazis will kill You for that
sooner or later:)
I know that is presented in may enterprise so called patterns, but
it's hard to imagine something more silly:P
But when reading single DTO... some lazy developer did this what You
ask about: loaded entity and repacked to DTO.
This is very, very basic form of "separation" - rather mental than
technical CqRS.
So just to summarize:
Reading stack is thin, simple, one layer, with no need to repack,
assemble and layering. Just read data and do it as fast as possible.
Good playground for apprenticers.
Sławomir Sobótka
I think it should be explained more straightforward in wiki.
Marcin Kuthan
Sławomir Sobótka
> :-)
General idea is quite simple. Two models (whatever "model" means:)
designed and optimized for two different purposes (uber-model
optimized for both is rather not possible).
But to excite You, I can say that interesting stuff begins when You
introduce Event Sourcing as a persistence technique:P
> I've just figure out that I have been used CQRS several times in my
> projects. External search index plays as de-normalized model of the
> application, optimized for searching. But finders implementation for
> retrieving data from search index is not as easy as "select * from foo" :-(
It is in Your affair to make is as easy and fast as possible:P
As I understand, Your second model was just as index, but actual
business data was fetched from common model...?
I assume that It was designed this way in sake of consistency.
btw: Graph-oriented DB can be used to index data when dealing with
graph problems (ex. recommendations of products by friends, or people
who bought the same as I did...)
In CqRS we can assume that "consistency is eventual" for some/all
data. So in sake of scalability we can approve some staleness.
Therefore we *can* update read model using events send by queues.
Actually all data are stale when leaving transaction, so You may ask
client: how stale is still good enough:)
> For educational purposes I would prepare de-normalized db tables/views for
> finders, get rid of DTOs and return pure collection (perhaps map, or list of
> map) to the UI. Any additional logic in the finders implementation is
> misleading. But if data retrieving is more complex, simple finders are not
> enough and mentioned patterns are welcome.
There is one example presenting "spying" user who removes products
from Cart. This is done by listening to the *application* events
(because we assumed that Cart is application not domain concept). But
the same could be done on Domain Events. So event's data is stored in
denormalized table just to report it fast - just a sample case, but
idea is important.
Marcin Kuthan
> I really like your CQRS explanation, incorporate the definition to the wiki
> :-)General idea is quite simple. Two models (whatever "model" means:)
designed and optimized for two different purposes (uber-model
optimized for both is rather not possible).
But to excite You, I can say that interesting stuff begins when You
introduce Event Sourcing as a persistence technique:P
> I've just figure out that I have been used CQRS several times in my
> projects. External search index plays as de-normalized model of the
> application, optimized for searching. But finders implementation for
> retrieving data from search index is not as easy as "select * from foo" :-(It is in Your affair to make is as easy and fast as possible:P
As I understand, Your second model was just as index, but actual
business data was fetched from common model...?
I assume that It was designed this way in sake of consistency.
btw: Graph-oriented DB can be used to index data when dealing with
graph problems (ex. recommendations of products by friends, or people
who bought the same as I did...)
In CqRS we can assume that "consistency is eventual" for some/all
data. So in sake of scalability we can approve some staleness.
Therefore we *can* update read model using events send by queues.
Actually all data are stale when leaving transaction, so You may ask
client: how stale is still good enough:)
> For educational purposes I would prepare de-normalized db tables/views for
> finders, get rid of DTOs and return pure collection (perhaps map, or list of
> map) to the UI. Any additional logic in the finders implementation is
> misleading. But if data retrieving is more complex, simple finders are not
> enough and mentioned patterns are welcome.There is one example presenting "spying" user who removes products
from Cart. This is done by listening to the *application* events
(because we assumed that Cart is application not domain concept). But
the same could be done on Domain Events. So event's data is stored in
denormalized table just to report it fast - just a sample case, but
idea is important.
Sławomir Sobótka
>> from Cart. This is done by listening to the *application* events
>> (because we assumed that Cart is application not domain concept). But
>> the same could be done on Domain Events. So event's data is stored in
>> denormalized table just to report it fast - just a sample case, but
>> idea is important.
>
> Is it example of mentioned event sourcing?
No, ES is more about "command stack".
In short:
In general ES is about shifting persistence model. In classic approach
we store relational (structural) data, that describe current state.
In ES we store behavioral model - events that occurs on Aggregate. So
there is no current state, just series of behaviors.
Additionally You can project this series of events to any denormalized
read model You want.
Short sample scenario:
Saving data:
1. Aggregate is not stored in relational tables (ex. via ORM).
2. Aggregate just publish events
3. There are at least 2 listeners interested in particular event.
a) Aggregate itself. Because domain methods perform calculations and
fire events. They do not change Aggregate state. Aggregate can listen
to event (one which was just published) and when Aggregate catches
event: looks ad data fields in event and changes own inner state
c) Listener that updates read model (this is case described previously
according do Leaven example). You can have many models (projections)
per Aggregate class - depends what You are interested in and how fast
You want to red.
4. Repo stores in some persistent event store serialized (ex. json)
events that occured on given aggregate (one domain method may fire
many domain events)
Loading Aggregate
1. Repo loads all events associated with concrete Aggregate (by ID)
2. Aggregate is loaded by all events to "replay" all actions in order
to become in current state (the same method that is described in
listener at Saving data 3a)
Read model (Finders) doest not know anything about any events,
aggregates, nothing, This model is designed to read fast.
Marcin Kuthan
Marcin Derylo
W dniu wtorek, 6 września 2011, 15:37:27 UTC+2 użytkownik Marcin Kuthan napisał:
Now, it's clear but new doubts came to my mind ...How to efficiently load aggregates from events? The full event history might be quite long ...
You sure? Maybe that means that your aggregate is just too big and should be split? And the loading of the aggregate from a list of events is blazing fast - applying events to an aggregate is just a matter of copying all/some of the event properties into aggregate internal state, maybe doing some simple operations like adding values etc. You simply don't do any business logic during loading. No logic at all. Just dead simple operations over the internal state of your aggregate.
If you really need your domain to be blazing fast, you can keep it in memory, all of it, can't you? Check out what guys developing LMAX did.
Some sort of save points could be introduced but it will lead to ORM like storage.
If you really have to do it you can create snapshots (Greg suggests doing it asynchronously, as a background process). Use the Memento design pattern for that. After loading the aggregate from snapshot apply only the events that happened after the snapshotting. Nothing close to ORM's level of complexity.
And tell me how to easily check what is the current state of the system (from the domain logic perspective)? Replay the events, look into one of read models? Much worse than looking into regular database.
Marcin Derylo
W dniu wtorek, 6 września 2011, 15:37:27 UTC+2 użytkownik Marcin Kuthan napisał:
How to implement unique constraint checking? Iterate over all events and check one by one?
Nope, for this one you will have to free your mind, Neo ;) Remember eventual consistency.
No matter where you ask this question you will end up being told to check http://codebetter.com/gregyoung/2010/08/12/eventual-consistency-and-set-validation/ . I think it is a good read before discussing this matter in details.
Marcin Kuthan
Marcin Kuthan
Marcin Derylo
- Pretty much every system I've been working on had a requirement for audit log (not really at the beginning, rather a couple of months later, when you already have an established model that don't easily support it). You get that for free, as a side-effect, when event sourcing.
- Enterprise apps often have requirements for some statistics, pie charts etc. More often than not you don't really know upfront what kind of statistics you will need. Not a problem if you get those requirements before your first deploy at client's site. But what to do if we already have some data in the database? Having an event log that carries not only the data itself, but also the user intent of the actions, helps a lot when you have to implement such features. You can create read models for such statistics from the beginning of time by just replaying to them all events you ever stored. This possibility to create any possible structural model of your data is not to be underestimated.
- You can implement your model not thinking about persistence. No more ugly hacks to make ORM work fine with our entities! Probably cleaner code. Of course - you can't just take your current ORM-based model and refactor it into event sourcing - I suppose it would be sub-optimal to the design you could come with if you designed your domain using events from the beginning.
Marcin Derylo
Sławomir Sobótka
Marcin's reply to give You some details.
> Now, it's clear but new doubts came to my mind ...
> How to efficiently load aggregates from events? The full event history might
> be quite long ... Some sort of save points could be introduced but it will
> lead to ORM like storage.
Snapshots, like Marcin said.
Greg who uses ES for algorithmic trading systems runs (I' can
remember, Marcin/Rafał correct me if I'm wrong): 200/300 thousands
transactions per second.
But I think that domain is different in all aspects than in ERP
systems... In ERP i expect hundreds of aggregates and dozens of events
per aggregate. I don't know algorithmic trading domain but I can
imagine dozens of aggregates and hundreds thousands of events per
aggregate.
> And tell me how to easily check what is the current state of the system
> (from the domain logic perspective)? Replay the events, look into one of
> read models? Much worse than looking into regular database.
I add to Marcin's reply that You can think about another read model
(another projection that occurred to be needed) and simply generate it
from events. Because events models behavior - what was done.
And most important is that Your new read model contains projection of
data You have gathered form the beginning of system lifetime (not just
from point in time when You release new version on the production).
> How to implement unique constraint checking? Iterate over all events and
> check one by one?
Very good question sir:)
So you ask if we need to implement by our own all mechanics that are
already provided by all RDBMs out of the box - and they work well.
Indeed, very good question:)
This is one of the main reasons why I personally can not imagine ERP
class system based on ES. But this may be just a problem with my
imagination and lack of experience.
I really hope I have just started a flame:P Unleash hell!