I don't know how Sagas change this, but I've been telling people that most Sagas are indeed Aggregates. Still I don't reckon that the examples I saw of Sagas are easily seen as Aggregates especially since they seem to send Commands which is not at all common in an Aggregate.
Cheers,
Nuno
--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention
I understand what you are saying Greg, I can see in some micro contexts is what is basically required. Yet quite often I find long running transactions to be easily translated to concrete business operations fully represented in the domain model.
Both compensation and reservation are dealt at the business level, not at the architecture IMHO. So if Sagas a triggers of compensation activities, business logic (predicates) is already there. So why hide it from the domain model?
Cheers,
Nuno
"Here’s the strongest indication I can give you to know that you’re doing CQRS correctly: Your aggregate roots are sagas."
Udi seems to bundle a bunch of other technologies/ideas such as ES and
call them CQRS architecture which I think is wrong. IMO he should have
been more specific.
I think I will have to get livewriter setup and actually write a blog
post on the new codebetter system. Much of Udi's post and the ensuing
conversation to me has been very confused in many ways.
1) Sagas should not look like aggregates (in fact this is an
anti-pattern). Sagas should only contain routing logic (aggregates
often contain large amounts of business logic) and have very different
responsibilities than an aggregate. The only time this would be true
is in a very naive system. I guess if you are one of those people that
look at all code as being state machines you could say they look
similar but they have very different constraints/intents.
2) CQRS is being misused as a term. CQRS seems to be being defined as
an all-encompassing thing here when really it is a very simple
concept. Many get huge benefit just by separating reads from writes
and using a thin read layer instead of going through the domain model.
3) Event Sourcing if anything is generally brought in as a cost saving
measure, the assertion that it costs more doesn't make any sense to
me. Perhaps if you take the time to write your own event store as
opposed to using an existing one and use 50% of your project time
optimizing low level code this can make sense to me. EG: You already
have a log of events because you get value off it from a business
perspective and you have a 3nf/no-sql backing model. Keeping just the
events ends up being cheaper and removes code.
My guess is that much of the post is based on something I have seen as
well. People trying to use CQRS + ES on something like say a webpage
for their little sister's diary where it vastly over complicates
things (even doing analysis on such a system is likely vastly
overcomplicating things).
--
Don't know much what message routing is in business terms. But here is IMHO what a Saga may feel in business terms.
It has been said that Sagas should be used in long running business transactions. I tend to think in terms of long running business activities that need to be consistent.
As such I tend to name them as *ing. Ex:
Shipping, Shopping, Selling, Approving, Reserving etc etc etc.
After
- The Purchase Order has been approved AND
- The Purchase Order has been paid for AND
- All Items in the Purchase Order have been reserved in stock AND
- Purchase Order has not been Cancelled
Then
- Items should be shipped.
This for me are business rules. It does not matter if they are then implemented using a finite state machine or whatever
Here are more examples:
Another:
After
- The Purchase Order has been Cancelled AND
- Items have been Reserved AND
- Purchase Order has been paid for a certain amount AND
- Items have been shipped
Then
- Purchase Order cancellation should not be approved.
Another:
After
- The Purchase Order has been Cancelled AND
- Items have been Reserved AND
- Purchase Order has been paid for a certain amount AND
- Items have not been shipped
Then
- Items reservation should be cancelled
- The amount should be refund
The good thing about conjunction is that it is commutative. Meaning that the Order of facts does not matter to the outcome.
How many Sagas are here? They should be in which Bounded Context? Shipping Management, Billing Management, Stock Management ...?
Things can get more complicated. For instance, suppose that the business ships orders partially based on some rules.
For instance:
After
- The Purchase Order has been approved AND
- Items over X amount in the Purchase Order have been reserved in stock AND
- Reserved Items has been paid for AND
- Rest of items fore cast is over a week AND
- Purchase Order has not been Cancelled
Then
- Reserved Items should be shipped.
Cheers,
Nuno
PS: And we are not counting business processes where payment happens within 60 days etc etc. Which basically means that we would need to deal with plafonds.
> If you have separated your code into commands and queries but you
> still have a single underlying data source, I'd call that CQS (or an
> extension thereof) but not CQRS.
> CQRS adds the separation at a data level as well, therefore requiring
> some kind of synchronization mechanism.
You would need to redefine what CQS is in order for this statement to be true.
"It states that every method should either be a command that performs
an action, or a query that returns data to the caller, but not both.
In other words, asking a question should not change the answer. More
formally, methods should return a value only if they are referentially
transparent and hence possess no side effects."
This is not what people are doing when they still use a single
underlying data source.
You are putting forward the claim that people who use the same data
source "aren't doing CQRS" only those who push the specialization back
to the data sources and use events to synchronize the data sources.
This to me sounds very suspect as a definition and I believe is where
many people get the "CQRS is complex" views we have seen in the past.
If I am using the same data source, that means that I have chosen to
separate my interfaces but have decided for these interfaces that I
wish to integrate through the database in your version you integrate
through events. Why would changing the mode of integration of the two
interfaces make it a completely different pattern? Would it be yet
another pattern if I chose to integrate through non-event based batch
operations (not that I would recommend this).
You have also made statements like "If you don't use one-way commands
you aren't doing CQRS". I would disagree with this for the same
reasoning as with the dual data sources... There are lots of separate
things here.
The idea here is that you are taking a bunch of small patterns that
inter-operate with each other ... mashing them together and then
coming up with names for the resulting architectural pattern (which is
fine btw so long as they don't share the same names). I however prefer
to get these core building block patterns described (you can then
describe the resulting system through a series of building blocks). As
an example of these building blocks ... one way commands, CQRS
(described as the separation of reads from writes with the intent of
specializing them separately), event sourcing, all the messaging
patterns, thin read layer, etc.
I don't have any problem with the naming of these larger architectural
patterns (within a given service) but when we use the same pattern
name to describe many things it causes confusion amongst people. The
real value of a pattern is that we can communicate complex concepts
with very few words. We often have words that have multiple meanings
but the meanings tend to have completely different contexts (think car
... it could be an automobile or a car on a train, if we are in the
parking lot of the mall you are 99% sure you know which one I am
talking about. This relationship is not true for the multiple
definitions being used now).
"Very complex domains may thus be broken up into multiple small pieces
to the point where "naive" solutions (as Greg calls them) may be the
simplest thing that could work (for some of them). It is quite likely
that you don't even need a Fowler Domain Model for them. For the rest,
one characteristic that is similar between them (in my experience with
many clients/domains over the years) is that of collaboration. When
you start considering first-one-wins/last-one-wins concurrency is an
indication that you have a collaborative domain. In these, CQRS can
indeed make a lot of sense (as I wrote under the "where should we use
it" heading)."
This is plain DDD. Of course it tends to break things up slightly less
granular than SOA but this is strategic design. My guess would be that
no one is disagreeing because its already accepted and has been for
years.
Per collaborative domains, I can come up with non-collaborative
domains that would still benefit from applying said architectural
ideas (especially event sourcing or storing a historical event log)
although in general I think you will find collaborative domains are a
better match. I don't believe however that there is any direct
causation between collaboration and the use of these patterns.
--
If I gave the impression that I was equating SOA to Web Services that my intent neither the core of my observation.
Cheers,
Nuno
Sent from my iPhone
Udi,
> One of the big differences is that when applying SOA/EDA at the top
> level, entities tend to get vertically partitioned between services.
In my experience, if we apply modular programming principles, functionality tend to be partitioned vertically already. I mean vertically in the sense of vertical business functions (AssetManagement, UserManagement, SiteMapManagement, FileManagement, etc etc). The modules internally separate by more modules etc etc. This is a method to separate concerns at a hight level (modules are defined as namespaces in C#).
One relatively "new" concept that you introduce to SOA and that I've seen/heard being used in architectures such as the one used by Amazon are artifacts similar to the ones you as you define the concept of Autonomous Components. ACs seam to be runtime time functionality that in order to process a call it that don't depend on calls outside its runtime boundary.
I have yet to read a comprehensive methodology around partitioning ones system around fully Autonomous Components (Anyone has any links?) anywhere, and definitely not from SOA writings/books.
> For example, you wouldn't see a customer's email address and their
> shipping addresses on the same entity/class. Correspondingly, a screen
> in the UI would also be a composition/layout of "widgets" belonging to
> different services.
Hum. It depends on how values are used by the domain. For instance, in principle we would need to include the base price of a Product in a Purchase Order. It looks redundant since the price is established in some other Bounded Context that is easily accessible (say by calling a web service, or using some shared view). Yet business rules mandate that the base price should sated as a matter of FACT by the time the order is placed.
Within this we should decide what FACTS are relevant to the business and what facts aren't. To avoid such decision we could use the approach that all FACTs are relevant, that seams to be the case. This has some advantages but may increase complexity, especially when one is not used to the approach.
On another note I would love to read something about what particular aspect of a software architecture Autonomous Components solve.
My impression is that often people equate Autonomy with availability. I believe this is the wrong reading. If process an UI/Screen we require the services of more then one AC, if one is down, the page cannot be served, hence the system is unavailable.
The fundamental law about availability is IMHO redundancy. Within this, there is two types of redundancy:
1) State redundancy. This can be achieved by simple replication, or we can use for instance CQRS, to separate read state from write state. We can scale out the reads in multiple ways.
2) Runtime redundancy. We deploy multiple instances of the same functional module.
We can bring the two together to what we call Autonomous Component and deploy multiple instances of the same AC.
Since ACs don't seam to solve availability problems without redundant instances (separation of reads from writes does) I tend to think they tend to be used to solve performance problems! At runtime ACs seam to equate to processes or threads in gross terms. If I call two ACs I can for sure know that each call will be processed async.
Now IMHO should only be done if we have performance problems in the write side. This problems are usually dependent on bottlenecks around Aggregates. If we have lot's of transaction on the the same Aggregate probably our consistency boundaries are not up to the task. For instance, I can't imagine that there will be lot's of blocking transactions on a standard Order, Shipping Order etc etc. How many business users will make use of such Aggregate along the value chain?
A central point of a component autonomy is to be able process a single call it does not need to call any other AC (or external service). This may potentially make the component itself more available (not the system as a whole). So to this the state of a an external components needs to be replicated to the AC's internal state for reads why not use CQRS for that? Say the AC handles events from external systems feeding it's internal read model.
So there seams to be also a strong connection between ACs and CQRS from a source perspective (feeding the AC internals with external facts). In the end of the day if a system does not need CQRS or a variant of it it does not need ACs.
So should we also avoid AC's most of the time? I get the impression that is the case from your article. But again this is based on may somewhat limited knowledge on these matters.
This are my preliminary thoughts from what you have said and for sure they only show my own lack of understanding of your article.
Cheers,
Nuno
Good clarification. Since more people will read the post than this thread I'm wondering if it might be worth rewording slightly. I just say this because "Your aggregate roots are sagas" definitely says to me that with this approach all ARs are sagas (hence my confusion).
great food for thought, still pondering... :-)
Many people have been getting confused over what CQRS is. They look at CQRS as being an architecture; it is not. CQRS is a very simple pattern that enables many opportunities for architecture that may otherwise not exist. CQRS is not eventual consistency, it is not eventing, it is not messaging, it is not having separated models for reading and writing, nor is it using event sourcing.
...
CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value).
So IMHO CQRS is about having two representations for what is basically the same set of facts, or put in another way, the same set of values aka State.
You really should have a valid reason for that.
Cheers,
Nuno
In other words the interesting stuff is not really the CQRS pattern itself but in the architectural decisions that can be made around it. Don’t get me wrong there are a lot of interesting decisions that can be made around a system that has had CQRS applied … just don’t confuse all of those architectural decisions with CQRS itself.
Hi Laurynas,I know what Greg says about the pattern, I simply don't dig it.It is something like:Problem: ?Solution: "CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value)"
Quite often this means that the "?" can be equal to "Solution".In that same article he writes:"We could apply CQRS to a CRUD based interface (though things like creating separated data models would be much harder)."Where is separated data models is anywhere in the "Solution". Is that part of the solution? If it is part of the solution why isn't it part of the solution description?Again next:"The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events."Again write / read models? Where is anywhere in the solutions description?Again next:"The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events."The fact is, the two "models" seam to be so much into everything about CQRS that defining CQRS without integrating may be the source of a lot of confusion.So in the end of the day, to use CQRS you need to have a valid reason to decide to have two or more representations of the same state. N for reading and one for transactions. Otherwise don't bother. Never mind the interesting properties that may or not be collateral.In the end of the day I think me and Greg are probably saying the same thing. I'm trying to take an algorithmic view over this rather diving in questions of principle.Cheers,Nuno
In that same article he writes:
"We could apply CQRS to a CRUD based interface (though things like creating separated data models would be much harder)."
Where is separated data models is anywhere in the "Solution". Is that part of the solution? If it is part of the solution why isn't it part of the solution description?
I think the biggest differences happen in the domain model when using a single data store.
1) the issues between strucyural and behavioral go away.
2) aggregate boundaries become clearer
3) a lot of non domain things end up in the read model (think repository methods that support paging, sorting, etc) these things often pollute domain models
4) if using an orm in domain model a lot of complexity can go away in terms of doing queries. Often this is the first level of 'specialization, both hit same store but in different ways/models)
Nuno,
To your assertion on services:
"In lame terms each module/service is described as a set of input-ouput operations to which we call Interfaces"
I strongly disagree with that statement. Services are not at all about input/output. What you have described is, at a logical level, a function.
Cheers,
-- Udi Dahan
Seconded
>
>
>
> Cheers,
>
>
>
> -- Udi Dahan
>
>
>
>
>
>
"In lame terms each module/service is described as a set of input-ouput operations to which we call Interfaces"
I strongly disagree with that statement. Services are not at all about input/output.
What you have described is, at a logical level, a function.
To name a few off the top of my head.
Greg
--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention
> 1. Uncluttering your domain.
A domain model is one possible interpretation of the world. Its as algorithmic as it is intuitive since it represents the UL in code.
Albeit it is a good representation from a transaction/linguistic point of view, for presentation for instance it fall short. Quite often people find it not good fit for queries too so much we develop sophisticate Object Mappers for that sole reason. Object navigation is somewhat secondary in the business world ... slicing sets of facts seams to be more useful.
> 2. Change in mindset, helps to better model OO domains.
That is a arguable IMHO. It depends on a lot of factors not directly related to CQRS. I've seen samples of domain models built using CQRS and ES that I don't that are good examples of domain modeling.
> 3. Solid step towards other architecturally useful solutions for example ES, EDA.
Also arguable IMHO. It depends on a lot of factors not directly related to CQRS.
ES for instance, is not so much an architecture decision, but pretty much domain driven. As for EDA, I agree, still the you can do without EDA in a lot of situations.
I can see what you are arguing, but in the end of the day a pattern definition should scoped on what it does and not on what it can potentially do once we add other stuff on top.
If we want to "sell" ideas on top of CQRS then focus the discussion on those ideas and explain why CQRS is necessary to them and not the other way around.
> I think I get where you're going as this *reasoning* can be applied to both full blown eventually consistent systems that apply DDD, CQRS, ES and also could be applied to any kind of other system that just uses CQRS at the object level to separate the domain model from the query data model.
Yes, every time we need more then one representation of state CQRS can be used and probably should be used. If we don't have such need, don't use it.
A very simple example. We had a workflow systems totally build using OO practices (closed vendor solution). The data model, was built to support Dynamic Property pattern and so on.
The result was that for a Task Based UI was fine and dandy (even generated them automatically), but to perform queries across activities and process was extremely slow due to dynamic properties. So we have built a query model and populated it using a ETL approach (every minute). The next step would to populate it using Push (events). This would allow "real time" population, but we haven't don't since it would require us to change the vendor code or data model and the purpose was really optimizing queries. The lag between the state in the domain model and the data in this query model wasn't a nuisance for the use cases at hand. But if we have done that then we could render all the UI using this new read model, in other words, it could be used for any use case, commands would go trough the vendor code and all queries would go through the new query model.
But you see, Commands and Events weren't central to this decision but CQRS helped to form the architecture of this solution.
I believe that in today's Enterprise's (medium to large) a lot of situations like this happen. This is basically a top down approach, from problem to solution in brown field environments.
In green field projects, then we have the liberty to make more profound architectural choices at right at the start including designing a solution towards multiple representation of state. Something that if our workflow vendor followed our lives would be much facilitated, not only in this context but also in other has you can imagine.
A part of the business and infrastructure is being moved to use Sales Force. Again we are facing very similar problems. The problem is not so much in terms of data integration but is actually is in terms of collaboration between bounded context. Without a cross system BUS where SalesForce is an endpoint (of sets of end points) and our systems are other end points ,the only solution is a mix of ETL and simple events store in databases to post process (everything is post processed). On the other side of the tail our internal systems aren't built to publish events for every single business fact.
CQRS is here is still a solution, but not so much a solutions as it is discussed for green field projects, SOA or non SOA. Most of us used to work in enterprises for long terms we have to deal mostly with other people mistakes with the understanding that people do things to the best of their knowledge, that includes vendors, external consultants and experts. The complexity here is different, we have to keep the business running long after short term consultants and vendors are gone.
The kind of reasoning around CQRS that I try to present is more realistic considering this landscape and does not close the doors at all to other things we can put on top towards:
> Solid step towards other architecturally useful solutions for example ES, EDA.
I miss green field projects.
Cheers,
Nuno