How to avoid CQRS

[OlavRask]

Since nobody else posted these here, i thought i would. So did
everybody see these:

http://www.udidahan.com/2011/04/22/when-to-avoid-cqrs/

http://abdullin.com/journal/2011/4/23/use-cqrs-when-constrained-on-developers-and-time.html

and what do you guys think?

[OlavRask]

I don't pretend to have enough practical experience to judge eather
way, but maybe you guys have some thoughts on the initial questions i
am left with:

So far as i understand what Udi recommends (in previous posts as well)
is to have EDA at the top most level, and then CQRS inside each
service. I guess each service could then have multiple bounded
contexts. But are the EDA events the same as the CQRS events then?

Also as far as i understand, in an SOA / EDA architecture something
like a website might be a mashup of several services. In this case it
seems natural to have a CQRS read model be specific to the website
rather then to have seperate read models in each service?

Kind regards
Olav

On 24 Apr., 12:08, OlavRask <olavr...@gmail.com> wrote:
> Since nobody else posted these here, i thought i would. So did
> everybody see these:
>
> http://www.udidahan.com/2011/04/22/when-to-avoid-cqrs/
>

> http://abdullin.com/journal/2011/4/23/use-cqrs-when-constrained-on-de...

[Tom Janssens]

(Warning: opinionated)

I have written a quite extensive post on this newsgroup where I point out that I respect the CQRS principles, but do not fully employ them most of the time: https://groups.google.com/d/msg/dddcqrs/RWd8Z2Q9Wdc/13q23v1SAfEJ

[Nuno Lopes]

I think Udi is simply stating that most often we don't need separate models for reading and writing. I tend to agree.

I don't know how Sagas change this, but I've been telling people that most Sagas are indeed Aggregates. Still I don't reckon that the examples I saw of Sagas are easily seen as Aggregates especially since they seem to send Commands which is not at all common in an Aggregate.

Cheers,

Nuno

[Colin Jack]

> I've been telling people that most Sagas are indeed Aggregates.

 

What Udi actually said was:

 

"Here’s the strongest indication I can give you to know that you’re doing CQRS correctly: Your aggregate roots are sagas."

 

That confused me a bit, I can see sagas as aggregates but where does the all aggregate roots are sagas bit fit in?

[Greg Young]

They may begin to look like sagas but they certainly have different
intent, also you should never really have any business logic in a
saga, only routing logic. Consider an "Instrument" in a trading
domain, doing this as a "saga" there is something really wrong with
your code.

--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention

[Nuno Lopes]

Hi Greg,

I understand what you are saying Greg, I can see in some micro contexts is what is basically required. Yet quite often I find long running transactions to be easily translated to concrete business operations fully represented in the domain model.

Both compensation and reservation are dealt at the business level, not at the architecture IMHO. So if Sagas a triggers of compensation activities, business logic (predicates) is already there. So why hide it from the domain model?

Cheers,

Nuno

[Nuno Lopes]

"Here’s the strongest indication I can give you to know that you’re doing CQRS correctly: Your aggregate roots are sagas."

There in DDD there is no Aggregate without an Aggregate Root, so if a Saga is the former, needs to be the later too. It would be hard for me to imagine a Saga being part of some order Aggregate.

Cheers,

Nuno

[Colin Jack]

Sure, but it's the idea that all your aggregates are sagas (which is
how I read it) that I find difficult to understand.

[Colin Jack]

Yeah agreed, that's why I didn't understand...i feel like I'm missing something.

[stacy]

Apparently we're ALL missing something, except Greg, until Udi
"clarifies" his remarks or you attend his course :)

On Apr 25, 12:08 pm, Colin Jack <colin.j...@gmail.com> wrote:
> Yeah agreed, that's why I didn't understand...i feel like I'm missing something.
>
>
>
>
>
>
>
> On Monday, 25 April 2011, Greg Young <gregoryyou...@gmail.com> wrote:
> > They may begin to look like sagas but they certainly have different
> > intent, also you should never really have any business logic in a
> > saga, only routing logic. Consider an "Instrument" in a trading
> > domain, doing this as a "saga" there is something really wrong with
> > your code.
>

> > On Mon, Apr 25, 2011 at 12:12 PM, Colin Jack <colin.j...@gmail.com> wrote:
> >>> I've been telling people that most Sagas are indeed Aggregates.
>
> >> What Udi actually said was:
>
> >> "Here’s the strongest indication I can give you to know that you’re doing
> >> CQRS correctly: Your aggregate roots are sagas."
>
> >> That confused me a bit, I can see sagas as aggregates but where does the all
> >> aggregate roots are sagas bit fit in?
>

[Žilvinas Šaltys]

+1 for "you attend his course"

Udi seems to bundle a bunch of other technologies/ideas such as ES and
call them CQRS architecture which I think is wrong. IMO he should have
been more specific.

[Rinat Abdullin]

Hm. This all sounds confusing. Here's my personal take.

Saga is an entity used to coordinate some long-running process. As an entity, it is addressable and accessible via some unique identifier. Hence it shares some properties with an aggregate (it can be persisted as such even up to the point of using ES).

Unlike aggregate, saga can subscribe to events and is also aware of the time flow - it can subscribe to timeouts (or rather - send delayed commands to itself), which is essential in mediating business transactions and negotiating complex processes between aggregates.

So.

Aggregate is not a saga (unless you want to complicate your domain a lot).

Saga is an aggregate with a few additional capabilities (or rather lifted constraints).

Rinat

[stacy]

"Saga is an aggregate with a few additional capabilities (or rather
lifted
constraints"

I think your definition of saga is spot on and it stands on its own in
my mind. My confusion is settled a lot by realizing that a saga can be
"implemented" as an aggregate or in other ways as well. Just like we
"implement" cqrs in different ways, doesn't necessarily have to change
the concept of WHAT it is. Like you described so nicely in your post,
the concept alone brings useful simplicity.

[mynkow]

http://www.blogcoward.com/archive/2011/04/24/cqrs-for-dummiesndashan-interlude-ndash-has-it-all-been-a.aspx

[Tom Janssens]

To confuse everyone even further, this is the way I use a saga in my apps:

To me, a saga is nothing more then a conditional event aggregation; based on a range of events (not) occurring in a certain order and/or within a defined timespan, a new event gets raised.
The raised event might get it's data from the consolidated events' properties.

I also consider it a temporal object, i.e. a saga instance gets destroyed once it has raised an event and/or the lifetime timeout occurred.

In it's most simplistic form I would consider it a query on event tables:

create view PaymentOverdueSagaQuery as
select NewGuid as EventId,ob.ArId as OrderId,ob.Date as BilledDate
  from OrderBilledEvents ob
  where ob.Date+30 < @PreviousSagaRunTimestamp
    and ob.Amount >
    (select NVL(sum(op.Amount),0)
       from OrderPaidEvents op
      where op.ArId == ob.ArId)

[Rinat Abdullin]

Yes, saga subscription could be implemented as a query over the stream of events and time events.

Yet without behavior (i.e. publishing messages), this is not a saga. It's merely a read model. 

/Rinat

[Tom Janssens]

Just to clarify my intent : if the query would have a result, the result's records would be published as distinct events.

[Rinat Abdullin]

Ah, that's what you mean by conditional event aggregation. That would work.

Although I would personally try to define saga object as an explicit aggregate, explicitly subscribing to the events. This way it's easier to manage them (i.e. TerminateSagaCommand) or deal with the negotiation logic (which tends to involve maintaining state between incoming events).

Rinat

[Nuno Lopes]

I think the problem of most people is that they give too much credit to a pattern. 

Let me clarify. There are three patterns and best practices that we usually talk about. CQS, CQRS and Event Sourcing (Greg's).

- CQS - CQS is about separating commands and queries in imperative programming. Mainly because it is easy to avoid side effects.

- CQRS is mainly about separating your reads from your writes so that feature tailored data representations to be built to solve resource contention problems. Basically allowing you to make multiple representations of the same domain model for reading. This is similar to MVC if not the same.

- With Greg's Event Sourcing we enter a new programming paradigm on top of OO. If you don't change the mindset from imperative programing to declarative one will not get it right IMHO.

Greg's ES needs CQRS because a collection of facts is usually not the ideal data structure for reads, period. In some case you can do without a separate data structure for reads though. That is why IvieMHO once he told that he could see the interest in CQRS without ES. He comes from a ES paradigm.

Now do we need CQRS to make a simple Shopping Cart? No. I don't need to separate my reads models from my rights for any reason. We could use Greg's ES though, but maybe before deciding for that you should really know why one can benefit from a more declarative programming paradigm.

Cheers,

Nuno

[Tom Janssens]

I agree on the explicit part, I was just trying to explain what a saga is.
For me the "click" came when I distilled it down to the SQL representation.
I have never tried implementing sagas like this IRL, but maybe I just might give it a shot in the future; it might look something like this:

CREATE OR ALTER PROCEDURE PaymentOverdueSaga(PreviousSagaRunTimestamp AS DATE)
AS
BEGIN
  INSERT INTO OrderPaymentOverDueEvents(EventId,OrderId,BilledDate)
  SELECT NewGuid,ob.ArId,ob.Date
    FROM OrderBilledEvents ob
   WHERE ob.Date+30 < @PreviousSagaRunTimestamp
     AND ob.Amount >
      (SELECT NVL(sum(op.Amount),0)
         FROM OrderPaidEvents op
        WHERE op.ArId == ob.ArId
          AND op.Date>ob.Date);  'Added to scope the paid events
END

The advantage is that there is no need for TerminateSagaCommand etc...
Maybe using this technique CQRS might even get some love from the DBA's as well ;-)
However, I can see the complexity grow to a new high as well using this approach, especially when using lots of negotiation logic... I guess I'll just try it on a real project one time and let you all know what the outcome is.

[Rinat Abdullin]

I've tried something like this in the production before. The approach looks simple to start with, but starts hitting a lot of edges really fast. It was not worth it in the long run, and maintenance was a mess. It gets worse, when you need to integrate together multiple systems being developed by different teams at different paces.

It's much easier (cheaper, dummer), to ditch storage specifics (SQL, SP, triggers and all the other relational quirks) and keep saga handling on the app server. I'm not even talking about scaling and cloud deployments.

But that's my personal experience. Your mileage might, as well, vary a lot.

Rinat

[Tom Janssens]

If you tried it and it failed, I'll just skip the attampt alltogether... Thanks !

Tom

[Greg Young]

Not replying in specific to this post was just the last one.

I think I will have to get livewriter setup and actually write a blog
post on the new codebetter system. Much of Udi's post and the ensuing
conversation to me has been very confused in many ways.

1) Sagas should not look like aggregates (in fact this is an
anti-pattern). Sagas should only contain routing logic (aggregates
often contain large amounts of business logic) and have very different
responsibilities than an aggregate. The only time this would be true
is in a very naive system. I guess if you are one of those people that
look at all code as being state machines you could say they look
similar but they have very different constraints/intents.

2) CQRS is being misused as a term. CQRS seems to be being defined as
an all-encompassing thing here when really it is a very simple
concept. Many get huge benefit just by separating reads from writes
and using a thin read layer instead of going through the domain model.

3) Event Sourcing if anything is generally brought in as a cost saving
measure, the assertion that it costs more doesn't make any sense to
me. Perhaps if you take the time to write your own event store as
opposed to using an existing one and use 50% of your project time
optimizing low level code this can make sense to me. EG: You already
have a log of events because you get value off it from a business
perspective and you have a 3nf/no-sql backing model. Keeping just the
events ends up being cheaper and removes code.

My guess is that much of the post is based on something I have seen as
well. People trying to use CQRS + ES on something like say a webpage
for their little sister's diary where it vastly over complicates
things (even doing analysis on such a system is likely vastly
overcomplicating things).

--

[@yreynhout]

I doubt he meant it literally (I could be wrong). I interpreted it as
aggregates (roots) that are the product of completed sagas. But that's
just me reading things into it, I guess.

On 25 apr, 18:12, Colin Jack <colin.j...@gmail.com> wrote:
> > I've been telling people that most Sagas are indeed Aggregates.
>
> What Udi actually said was:
>
> "Here’s the strongest indication I can give you to know that you’re doing
> CQRS correctly: Your aggregate roots are sagas."
>
> That confused me a bit, I can see sagas as aggregates but where does the all
> aggregate roots are sagas bit fit in?
>

[Nuno Lopes]

I believe that internally Sagas need to be consistent and change as a unit. For me this is the bottom line of an Aggregate. The question is, is it a business Entity? It depends, but if it can be referred it is an Entity IMHO.

Don't know much what message routing is in business terms. But here is IMHO what a Saga may feel in business terms.

It has been said that Sagas should be used in long running business transactions. I tend to think in terms of long running business activities that need to be consistent.

As such I tend to name them as *ing. Ex:

Shipping, Shopping, Selling, Approving, Reserving etc etc etc.

After

- The Purchase Order has been approved AND
- The Purchase Order has been paid for AND
- All Items in the Purchase Order have been reserved in stock AND
- Purchase Order has not been Cancelled

Then

- Items should be shipped.

This for me are business rules. It does not matter if they are then implemented using a finite state machine or whatever

Here are more examples:

Another:

After

- The Purchase Order has been Cancelled AND
- Items have been Reserved AND
- Purchase Order has been paid for a certain amount AND
- Items have been shipped

Then

- Purchase Order cancellation should not be approved.

Another:

After

- The Purchase Order has been Cancelled AND
- Items have been Reserved AND
- Purchase Order has been paid for a certain amount AND
- Items have not been shipped

Then

- Items reservation should be cancelled
- The amount should be refund

The good thing about conjunction is that it is commutative. Meaning that the Order of facts does not matter to the outcome.

How many Sagas are here? They should be in which Bounded Context? Shipping Management, Billing Management, Stock Management ...?

Things can get more complicated. For instance, suppose that the business ships orders partially based on some rules.

For instance:

After

- The Purchase Order has been approved AND
- Items over X amount in the Purchase Order have been reserved in stock AND
- Reserved Items has been paid for AND
- Rest of items fore cast is over a week AND
- Purchase Order has not been Cancelled

Then

- Reserved Items should be shipped.

Cheers,

Nuno
PS: And we are not counting business processes where payment happens within 60 days etc etc. Which basically means that we would need to deal with plafonds.

[Nuno Lopes]

That looks like an Event Processor.

Cheers,

Nuno

[Chris Nicola]

I don't agree that a Shopping Cart is necessarily a good example of when not to use CQRS or event ES.  What if the interaction and changes in the state of the shopping cart are important to the business.  A shopping cart is not always "simple".  Many businesses want to have a lot of rules around when customers get discounts, which things are counted as "bundles".  Furthermore the example has been used repeatedly as a reason *to* use ES, say you want to know how often an item is added and then removed from the shopping cart?  Is this data that you've recorded?  If you've used ES then it probably is, or at least it can be.

My problem with Udi's post is that he is trying to boil down when you should or should not apply CQRS/ES to a single concept, collaboration.  I can see why collaboration would be a reason to use CQRS/ES but I can't see where it is proven that the logical opposite is shown.  Just because collaboration supports using CQRS/ES does not imply that it is the only thing that supports it.  Likewise I'm sure a contrived example could be given where a system with collaboration doesn't need CQRS/ES.  I'm sorry the argument is just too simplistic to be taken at face value. 

Yes we all know there are no silver bullets, but this isn't news.

[Udi Dahan]

Let's see if I can clarify this a bit.

If you have separated your code into commands and queries but you
still have a single underlying data source, I'd call that CQS (or an
extension thereof) but not CQRS.
CQRS adds the separation at a data level as well, therefore requiring
some kind of synchronization mechanism.

One of the things that did not appear to get much reaction on this
thread was the statement about CQRS not being the top-level
architectural pattern (that being SOA). Some recalled my posts on SOA/
EDA combination (which is correct), others equated SOA with web
services (which isn't correct). I'm afraid we might be comparing
apples to oranges when comparing CQRS (as a full arch pattern) and SOA/
EDA with some CQRS inside services on an as needed basis.

One of the big differences is that when applying SOA/EDA at the top
level, entities tend to get vertically partitioned between services.
For example, you wouldn't see a customer's email address and their
shipping addresses on the same entity/class. Correspondingly, a screen
in the UI would also be a composition/layout of "widgets" belonging to
different services.

Very complex domains may thus be broken up into multiple small pieces
to the point where "naive" solutions (as Greg calls them) may be the
simplest thing that could work (for some of them). It is quite likely
that you don't even need a Fowler Domain Model for them. For the rest,
one characteristic that is similar between them (in my experience with
many clients/domains over the years) is that of collaboration. When
you start considering first-one-wins/last-one-wins concurrency is an
indication that you have a collaborative domain. In these, CQRS can
indeed make a lot of sense (as I wrote under the "where should we use
it" heading).

You can think of this as a rule of thumb - prescriptive, not
definitive; just like the saga/AR thing:

"Here’s the strongest indication I can give you to know that you’re
doing CQRS correctly: Your aggregate roots are sagas."

I did *not* say that all sagas are ARs or that all ARs are sagas. This
indicates that you're using long-running processes to govern the
collaboration in the collaborative domains and that you are avoiding
the use of the domain model pattern in the simple/naive places that
don't warrant CQRS.

One area where the above prescriptive guidance doesn't hold *as
strongly* is when developing *platforms* rather than solving a set of
defined business problems.

I might say (generalizing, of course) that often people use CQRS where
they shouldn't, but don't use it (properly) where they should.

Hope that clears it up.

With thanks,

-- Udi Dahan

[Greg Young]

I think the problems here are mostly in the constant redefinition of
what CQRS is.

> If you have separated your code into commands and queries but you
> still have a single underlying data source, I'd call that CQS (or an
> extension thereof) but not CQRS.
> CQRS adds the separation at a data level as well, therefore requiring
> some kind of synchronization mechanism.

You would need to redefine what CQS is in order for this statement to be true.

"It states that every method should either be a command that performs
an action, or a query that returns data to the caller, but not both.
In other words, asking a question should not change the answer. More
formally, methods should return a value only if they are referentially
transparent and hence possess no side effects."

This is not what people are doing when they still use a single
underlying data source.

You are putting forward the claim that people who use the same data
source "aren't doing CQRS" only those who push the specialization back
to the data sources and use events to synchronize the data sources.
This to me sounds very suspect as a definition and I believe is where
many people get the "CQRS is complex" views we have seen in the past.
If I am using the same data source, that means that I have chosen to
separate my interfaces but have decided for these interfaces that I
wish to integrate through the database in your version you integrate
through events. Why would changing the mode of integration of the two
interfaces make it a completely different pattern? Would it be yet
another pattern if I chose to integrate through non-event based batch
operations (not that I would recommend this).

You have also made statements like "If you don't use one-way commands
you aren't doing CQRS". I would disagree with this for the same
reasoning as with the dual data sources... There are lots of separate
things here.

The idea here is that you are taking a bunch of small patterns that
inter-operate with each other ... mashing them together and then
coming up with names for the resulting architectural pattern (which is
fine btw so long as they don't share the same names). I however prefer
to get these core building block patterns described (you can then
describe the resulting system through a series of building blocks). As
an example of these building blocks ... one way commands, CQRS
(described as the separation of reads from writes with the intent of
specializing them separately), event sourcing, all the messaging
patterns, thin read layer, etc.

I don't have any problem with the naming of these larger architectural
patterns (within a given service) but when we use the same pattern
name to describe many things it causes confusion amongst people. The
real value of a pattern is that we can communicate complex concepts
with very few words. We often have words that have multiple meanings
but the meanings tend to have completely different contexts (think car
... it could be an automobile or a car on a train, if we are in the
parking lot of the mall you are 99% sure you know which one I am
talking about. This relationship is not true for the multiple
definitions being used now).

"Very complex domains may thus be broken up into multiple small pieces
to the point where "naive" solutions (as Greg calls them) may be the
simplest thing that could work (for some of them). It is quite likely
that you don't even need a Fowler Domain Model for them. For the rest,
one characteristic that is similar between them (in my experience with
many clients/domains over the years) is that of collaboration. When
you start considering first-one-wins/last-one-wins concurrency is an
indication that you have a collaborative domain. In these, CQRS can
indeed make a lot of sense (as I wrote under the "where should we use
it" heading)."

This is plain DDD. Of course it tends to break things up slightly less
granular than SOA but this is strategic design. My guess would be that
no one is disagreeing because its already accepted and has been for
years.

Per collaborative domains, I can come up with non-collaborative
domains that would still benefit from applying said architectural
ideas (especially event sourcing or storing a historical event log)
although in general I think you will find collaborative domains are a
better match. I don't believe however that there is any direct
causation between collaboration and the use of these patterns.

--

[Nuno Lopes]

Hi, I mentioned Web Services in the Context of SOA. It could any other kind of protocol/method used to allow communication between modules of functionality deployed on a network.

If I gave the impression that I was equating SOA to Web Services that my intent neither the core of my observation.

Cheers,

Nuno

Sent from my iPhone

[michael-hamburg]

I think a Saga is simply a workflow that can be easily described with
a process model (See http://en.wikipedia.org/wiki/Business_Process_Modeling_Notation).
The actual work is done by the aggregates that react on commands send
by the Saga.

[Nuno Lopes]

Hi,

Udi,

> One of the big differences is that when applying SOA/EDA at the top
> level, entities tend to get vertically partitioned between services.

In my experience, if we apply modular programming principles, functionality tend to be partitioned vertically already. I mean vertically in the sense of vertical business functions (AssetManagement, UserManagement, SiteMapManagement, FileManagement, etc etc). The modules internally separate by more modules etc etc. This is a method to separate concerns at a hight level (modules are defined as namespaces in C#).

One relatively "new" concept that you introduce to SOA and that I've seen/heard being used in architectures such as the one used by Amazon are artifacts similar to the ones you as you define the concept of Autonomous Components. ACs seam to be runtime time functionality that in order to process a call it that don't depend on calls outside its runtime boundary.

I have yet to read a comprehensive methodology around partitioning ones system around fully Autonomous Components (Anyone has any links?) anywhere, and definitely not from SOA writings/books.

> For example, you wouldn't see a customer's email address and their
> shipping addresses on the same entity/class. Correspondingly, a screen
> in the UI would also be a composition/layout of "widgets" belonging to
> different services.

Hum. It depends on how values are used by the domain. For instance, in principle we would need to include the base price of a Product in a Purchase Order. It looks redundant since the price is established in some other Bounded Context that is easily accessible (say by calling a web service, or using some shared view). Yet business rules mandate that the base price should sated as a matter of FACT by the time the order is placed.

Within this we should decide what FACTS are relevant to the business and what facts aren't. To avoid such decision we could use the approach that all FACTs are relevant, that seams to be the case. This has some advantages but may increase complexity, especially when one is not used to the approach.

On another note I would love to read something about what particular aspect of a software architecture Autonomous Components solve.

My impression is that often people equate Autonomy with availability. I believe this is the wrong reading. If process an UI/Screen we require the services of more then one AC, if one is down, the page cannot be served, hence the system is unavailable.

The fundamental law about availability is IMHO redundancy. Within this, there is two types of redundancy:

1) State redundancy. This can be achieved by simple replication, or we can use for instance CQRS, to separate read state from write state. We can scale out the reads in multiple ways.

2) Runtime redundancy. We deploy multiple instances of the same functional module.

We can bring the two together to what we call Autonomous Component and deploy multiple instances of the same AC.

Since ACs don't seam to solve availability problems without redundant instances (separation of reads from writes does) I tend to think they tend to be used to solve performance problems! At runtime ACs seam to equate to processes or threads in gross terms. If I call two ACs I can for sure know that each call will be processed async.

Now IMHO should only be done if we have performance problems in the write side. This problems are usually dependent on bottlenecks around Aggregates. If we have lot's of transaction on the the same Aggregate probably our consistency boundaries are not up to the task. For instance, I can't imagine that there will be lot's of blocking transactions on a standard Order, Shipping Order etc etc. How many business users will make use of such Aggregate along the value chain?

A central point of a component autonomy is to be able process a single call it does not need to call any other AC (or external service). This may potentially make the component itself more available (not the system as a whole). So to this the state of a an external components needs to be replicated to the AC's internal state for reads why not use CQRS for that? Say the AC handles events from external systems feeding it's internal read model.

So there seams to be also a strong connection between ACs and CQRS from a source perspective (feeding the AC internals with external facts). In the end of the day if a system does not need CQRS or a variant of it it does not need ACs.

So should we also avoid AC's most of the time? I get the impression that is the case from your article. But again this is based on may somewhat limited knowledge on these matters.

This are my preliminary thoughts from what you have said and for sure they only show my own lack of understanding of your article.

Cheers,

Nuno

[Colin Jack]

> I did *not* say that all sagas are ARs or that all ARs are sagas. This
> indicates that you're using long-running processes to govern the
> collaboration in the collaborative domains and that you are avoiding
> the use of the domain model pattern in the simple/naive places that
> don't warrant CQRS.

Good clarification. Since more people will read the post than this thread I'm wondering if it might be worth rewording slightly. I just say this because "Your aggregate roots are sagas" definitely says to me that with this approach all ARs are sagas (hence my confusion).

[Sebastian]

I don't get why Udi said CQRS should be avoided in a collaborating
system. In fact, a shopping cart might be pretty complex with thousand
of discount rules for example and having stuff pre-calculated in the
reporting model according to the UI views makes sense to me.

[AlexY]

How does having rules makes a shopping cart a collaborating component? I believe Udi was referring to systems that are affected by multiple actors and a shopping cart is a single user component. No?

[Udi Dahan]

Greg,

> You would need to redefine what CQS is in order for this statement to be true.

If you recall our discussions with Martin (Fowler), he was against
naming a new pattern. My position on CQRS included the data separation
which, in my mind, made it different not just in degree but also in
kind from CQS. I thought (at the time) that you were in agreement.

> If I am using the same data source, that means that I have chosen to
> separate my interfaces but have decided for these interfaces that I
> wish to integrate through the database in your version you integrate
> through events. Why would changing the mode of integration of the two
> interfaces make it a completely different pattern? Would it be yet
> another pattern if I chose to integrate through non-event based batch
> operations (not that I would recommend this).

It is not a question of "mode of integration" (and I'm pretty sure you
know that). The eventual consistency arising from separate data
sources leads to a very different architectural and business context.
If one were to use say an ETL job for keeping those data sources in
sync, I would view that as a different implementation of the same
pattern (but a pretty poor and undesirable implementation all things
considered).

It is also this eventual consistency that is reflected by one-way
commands. Choosing synchronous commands indicates a non-eventually-
consistent architectural context.

I can understand your position of looking at all of the various pieces
as building blocks that can be used all together and/or in various
combinations.

> The real value of a pattern is that we can communicate complex concepts with very few words...

> when we use the same pattern name to describe many things it causes confusion amongst people

Agreed, but I'd say that a code-only definition of CQRS overlaps too
much with CQS potentially causing confusion that way. That was
Martin's position as well.
You could also say that the code-only definition is similarly an
extension of the single responsibility principle and the interface
segregation principle.
By applying the "prefer building block" state of mind you were
describing, we should be talking CQS, SRP, ISP rather than code-only
CQRS.

On the DDD bounded contexts (SOA/EDA breakdown), while you might say
that no one is disagreeing, the industry hasn't internalized that and
is (more often than not) trying to use CQRS at the top level.

> Per collaborative domains, I can come up with non-collaborative
> domains that would still benefit from applying said architectural
> ideas (especially event sourcing or storing a historical event log)
> although in general I think you will find collaborative domains are a
> better match. I don't believe however that there is any direct
> causation between collaboration and the use of these patterns.

I agree that the various building blocks can be used in all sorts of
combinations in many kinds of domains (including non-collaborative
ones). That being said, it is useful to have prescriptive guidance to
point people who may not understand all the nuances in (more or less)
the right direction. What I can say from a causation perspective is
the collaboration does mean system/user level eventual consistency, so
it is a pretty good heuristic for using CQRS as I describe it.

Cheers,

-- Udi Dahan

[Nuno Lopes]

Hi Udi and Greg,

I don't have your world wide experience on these matters, neither I have access to legends in the software industry to cross check my opinion and experiences. So take this with a pinch of salt and some leniency.

This will be a highly opinionated, yet hopefully not far from your understanding and the rest of the community.

Sometimes it helps to think about things algorithmically and put as far from the jargon of DDD, SOA, EDA etc.

So what is the problem that IMHO CQRS solves ...

Problem Statement (Detailed).

From experience, as long as there is one, unique representation of STATE and multiple agents that need to change or act based on it, contention/competition problem will exist.  The opposite of contention is collaboration. 

Some contention problems can be solves by transforming the rules of engagement in such way that agents no longer compete, instead collaborate. Yet there is a class of contention problems that can't be solved that way. Does that mean CQRS is not applicable?

Before we can even start to dive in to the design tools for collaboration we need to describe + the nature of the word "contention" in this context. It is my opinion that in one way or another this hasn't been done to its full extent. It is always mixed with other things such as SOA, ES cache etc etc. So here is one attempt to do so by looking without any preconceptions.

- Representation: Deciding for an ideal representation of STATE is more often then not constrained by ones algorithmic approach to automate a domain. 

A Tree structure may be better representation for one use case, on another a map, another a table with multiple indexes, yet on another a n-order graph representation is better, on another a matrix, on another a hash table and so on

. There no best representation for everything. It is highly an algorithmic matter.

- Transactions: A pure transaction semantics is one that a set of values within the state is changed as a unit in an instant (either changes all values or does change at all) or data is read as a unit (data is not changed while being read). There contention in access the state since multiple agents will compete for executing transactions so much that changes will be serialized.

- Access/Location: The location of state can be also a very real contention problem. For instance, say state is located in country A. Access from country B are extremely slow. So both locations compete for the proximity of State on the network to improve performance. This is can be a significant business problem especially in a highly competitive domain such as the Stock Exchange.

- Presentation: State can be formatted for presentation in many different ways. Presentation is structural in nature. This more in the realm of MVC, I have written an small article explaining the similarities of both patterns.

 

Unified Solutions for STATE representation (One ring rules them all):

Computer scientists (and mathematicians) love unified theories of the everything. So we came up with several unified approach's to solve the problems above. 

One very popular is the Relational Model and tech. 

In the relational relational model, there is only one way to represent state. Using record tables 

(non polymorphic), and table relationships. For a given domain there is only one model.

We have developed very sophisticated algorithms so that Relational representation fits all algorithmic needs. We embedded in the relational model very simple transaction semantics and constructs. Y

es today RDBMS provide multiple ways to configure the transaction semantics, but these configurations are only there to solve the native impediment of the paradigm (a compromise).  

In the relational model the data model was not thought to be highly distributed as it happens today (a lot of companies were using COBOL and central mainframes ... remember?) . Yes RDBMS systems provide use some configuration options that help us transparently distribute data across multiple database instances, but it does no natively provide us a way to deal with eventual consistency explicitly.

The relational paradigm is an amazing feet. With simple artifacts and rules we have a unified framework for information representation supporting state representation, transaction semantics and state location. They also solve a lot of the enumerated contention problems stated above to some point. Technically for most solutions its simplicity and benefits still outweigh the drawbacks.  

Funny enough COBOL did not provide an unified approach to neither state representation and transaction semantics, so we "abandoned" COBOL in favor of SQL  for some reason that we need not to forget.

The same thing happened with OODMS algebra. Yet in contrast the representation is more flexible with some similar costs. More often then not at some point multiple representations are needed :)

Guess which pattern came popular in the specific cases an unified state representation was of no good use? MVC. 

There is no way users would go on and read data from our tables :). We need alternative representations of state. So we developed a model for viewing (views), a model for transacting (domain model) and since are eventually consistent with the model, we have a controller that works has a mediator between the User, the Views and the Domain Model (considering the user is also an Object).

The problem of unified approaches such as the above is one of compromise. A compromise that for many solutions might be just affordable. But today we have sufficient forces making this type of approach not so affordable at least for everything.

The problems we have today aren't the same as the problems we had when SQL appeared as an alternative to COBOL ...

1) The problem today is that systems are more distributed yet more connected with each other then ever (fluent communication). This would not be much of a problem if not for the fact that each represent probably the same information in a different way especially when they overlap in duties. (Representation and Interpretation)

2) Information is more decentralized. Business want to deploy part of their business in the Cloud, another part on their Mainframes and another part in they regular IT infrastructure. (Location)

3) As services go to the web, for customer service and sales sake we need to serve the speed daemons of our society, our Customers. So specific algorithms need to be developed for particular use cases, hence specific state representations. (Transactions, faster transactions)

Eclectic Solutions

What if we don't go for an unified solution? How do we support multiple representation of the same state? How do we move data from a transactional state representation to the others?

To this, I see great value of CQRS inline with MVC. The solution seams to be fundamentally the same! One transactional model (one representation offering transaction guarantees  and multiple representations for observation by consumers (views). 

Yet this move rises several questions that need good answers. How do we maintain the multiple representations of the same state consistent with each other? Are these necessarily eventually consistent?

Sometimes yes, sometimes not. Two contention points pull solutions to opposite sides, one is location of state and another is transactions. Putting this to side a minute ...

To move state from one behavioral/business transactional representation (domain model) to the a business observation model (views) we two approaches:

1) Push State from A to Z (Publish / Subscribe / Populate)

2) Pull State from a to Z (ETL)

Each have advantages and disadvantages. In a green field solution I would probably go for 1), but on a brown field solution 2) might be the most cost effective.

CQRS description favors 1). This is so that state and its multiple presentation are being continuously integrated (continuous state integration), while 2) integration happens in some discrete moments.

In theory continuous state integration reduces the problem of eventually consistency since communication is fluent. Discrete state integration magnifies the problems. We can imagine that each of these multiple state representation are tightly coupled. 

One another note, fluent communication also requires more resources, both in bandwidth and computing power. The good news is that it scales better and costs tend to be predictable unlike discrete communications.

CQRS vs CQS

This, IMHO has been the source a lot of confusion.

IMHO considering the above, the driving forces around CQRS has nothing to do with CQS from Bertrand Mayer. At least as much to do with it as MVC.

Within many patterns we can apply this principle. For instance, SQL applies this principle having Commands (INSERT, DELETE, UPDATE) and queries (SELECT). Still CQS minimizes side effects to the point of facilitating  Assertions about the state (data). It would be hard to make assertion, since in imperative programming it implicitly required to use queries. If a query changed the state it could make the assertion false as a result while the assertion is true according to the previous state, making it difficult.

This does not seam to be the core concern of CQRS and in fact it may aggravate the issues CQS is intended solve further with eventual consistency. 

For instance we can in MVC also separate actions between commands and queries :) yet the pattern it's not argued anywhere close to CQS. In  controller action we can call the domain model for an operation, then call the domain model again to get a set of data to populate the view (Change and Pull State). Or we can call the domain model and let the view populate itself as per domain model notification (Subscribe and Populate).

CQRS and SOA

As far as I know SOA as a discipline concerns the challenge of designing and building distributed systems, We can imagine that CQRS is of great value to this considering the problem its intended to solve ... I will not say much about this.

SOA also follows a modular approach to system design and functionality. In lame terms each module/service is described as a set of input-ouput operations to which we call Interfaces

In CQRS inputs are Commands, outputs are Events. So an SOA module using CQRS is a set of Commands and a set of Events in response. The response is not necessarily synchronous. Based on responses views can be built. In fact view can be part of the module definition but not necessarily.

Of course SOA modules aren't defined arbitrary. We can use lessons learned from modular programming paradigms to do this or OOA. We make module out of related operations / functionality that is inline with business features.

There is also the concept of AC, which is a module of functionality that is not only autonomous in terms of code, but state and runtime. This is similar to Agents.

CQRS and DDD

I came to know a more formal description of the CQRS pattern through DDD. Concepts such as one Command is mapped to one Aggregate method. CQRS events become domain events and so on.

In a way CQRS applied to DDD you get an approach to a Distributed Domain Model which is interesting solution to some problems. 

Event Sourcing + DDD.

Inline with CQS vs CQRS missed conceptions, explaining CQRS in the light of Event Sourcing + DDD leads to some. Basically when teaching Event Sourcing along CQRS is a loose loose situation. If we first teach ES+DDD and CQRS separately and then together, only then it can be a win win situation :)

The reason for this for many this is all new. At least the formulation is new. Too many things to connect the dots at the same time you see. But once the dots are connected I think the realization is extraordinary. A brand new tool to solve problems :)

To understand ESS+DDD the following mindset helped a lot.

In OO we define an object has a series of input-ouput operations to which we call interface. Much in the same line has SOA does inline with modules. An object contains state. So go given an operation, the output is a product of its internal state and the input. This is in contrast with pure functions. We can turn all object operations into pure functions but that will defeat the purpose of OO. One of the ideas of OO is that the clients of an object need to have no specific knowledge about its internal representation of state in order to call an operation. 

Hence the term behavior. Behavior is a term that is defined over the notion that are some side effects that are necessary. CQRS principles tries to minimize the bad side effect problem to some degree.

ES in contrast requires rewire some OO notions. Instead of thinking of an Aggregate as an Entity described as a set of input-ouput operations we think about them as a set of FACTS whose the core subject is the Entity it represents and that it needs to maintain consistent by definition.

Methods of an Object are not longer operations. They produce not output. So what are these methods if not operations?

I lack the time to go deeper on this. But this methods seam to be Proofs. If is successful, it produces/proves a fact if not then is undetermined.

Say I do:

aOrder.Approve() -> OrderProved(...);

The Approve operation will try to prove that the fact OrderApproved is true. If the prove is successful, the result is true if not is false. If unsuccessful, the result is undetermined, it can either be true or false.

So how is able to "reason" over an object such as this? The set of facts of an object such as this are public. So if a client wants to know what has happened it can simply go directly to its set of facts and do what ever calculations to get the results it needs.

In a way, the set of facts of an object his its internal state yet it is public in its standard representation factName: values .... The client can safely peek on this list since a fact is immutable.

As you can imagine an ordered list of facts is not the best representation to perform all kinds of computations. So CQRS an be used for this problem.

There are a lot of benefits in thinking about objects on this way. I feel that a book explaining both the foundations and practice of this way of thinking is needed.

One of the good things about this approach as that everything we know about OO and DDD is still applicable, furthermore the best practices of OO are still valid in this realm.

Do we need ES for all problems?

ES as described above can be used for all problems. Still the burden is that we need CQRS in most situations.

ES + DDD + SOA

It turns out that SOA also needs CQRS. So we can leverage ES in SOA through CQRS. Furthermore, some problems that we have in SOA and CQRS regarding provisioning multiple eventually consistent representation may well be solved using the ES+DDD approach as the foundation for designing a domain model.

So How to Avoid CQRS?

CQRS can be fully avoided in situation where we have no need for multiple representations of the same State. So if you want to avoid it, design a unified state representation for an domain concept. You probably need to use Relational Model for that, and if using DDD and OO you will need a ORM.

The above is clearly impossible if not in the most simple ES workouts. 

Sorry for this long post and thank you for listening. If I'm completely off the mark please shoot.

Cheers,

Nuno

[Raoul Duke]

On Fri, May 6, 2011 at 4:21 AM, Nuno Lopes <nbpl...@gmail.com> wrote:
> Sorry for this long post and thank you for listening. If I'm completely off
> the mark please shoot.

great food for thought, still pondering... :-)

[Laurynas Pečiūra]

Hi Nuno,

I'll still go with Gregs definition of what CQRS is:

Many people have been getting confused over what CQRS is. They look at CQRS as being an architecture; it is not. CQRS is a very simple pattern that enables many opportunities for architecture that may otherwise not exist. CQRS is not eventual consistency, it is not eventing, it is not messaging, it is not having separated models for reading and writing, nor is it using event sourcing.
...
CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value).

For me personally, it did exactly the thing it describes - it very simply enabled many opportunities for solutions better suited to the problem at hand.

[Nuno Lopes]

"CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value)."

That definition for makes no sense. Of it the separation of Mayers why give it another name?

[Nuno Lopes]

Most problems in posted to this forum aren't about the having form queries and other for commands. Are about having a model for queries, separated from a model for transactions (commands).

So IMHO CQRS is about having two representations for what is basically the same set of facts, or put in another way, the same set of values aka State.

You really should have a valid reason for that.

Cheers,

Nuno

[Laurynas Pečiūra]

The link explains it in more detail.

[Laurynas Pečiūra]

Again, a quote from the post mentioned above:

In other words the interesting stuff is not really the CQRS pattern itself but in the architectural decisions that can be made around it. Don’t get me wrong there are a lot of interesting decisions that can be made around a system that has had CQRS applied … just don’t confuse all of those architectural decisions with CQRS itself.

[Nuno Lopes]

Hi Laurynas,

I know what Greg says about the pattern, I simply don't dig it. 

It is something like:

Problem: ?

Solution: "CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value)"

Quite often this means that the "?" can be equal to "Solution".

In that same article he writes:

"We could apply CQRS to a CRUD based interface (though things like creating separated data models would be much harder)."

Where is separated data models is anywhere in the "Solution". Is that part of the solution? If it is part of the solution why isn't it part of the solution description?

Again next:

"The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events." 

Again write / read models? Where is anywhere in the solutions description?

Again next:

"The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events."

The fact is, the two "models" seam to be so much into everything about CQRS that defining CQRS without integrating may be the source of a lot of confusion.

So in the end of the day, to use CQRS you need to have a valid reason to decide to have two or more representations of the same state. N for reading and one for transactions. Otherwise don't bother. Never mind the interesting properties that may or not be collateral.

In the end of the day I think me and Greg are probably saying the same thing. I'm trying to take an algorithmic view over this rather diving in questions of principle.

Cheers,

Nuno

[√iktor Ҡlang]

On Sat, May 7, 2011 at 10:17 AM, Nuno Lopes <nbpl...@gmail.com> wrote:

Hi Laurynas,

I know what Greg says about the pattern, I simply don't dig it. 

It is something like:

Problem: ?

Solution: "CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value)"

Does the definition of Query prohibit the use of side-effecting, which, by definition will update state?
 

Quite often this means that the "?" can be equal to "Solution".

In that same article he writes:

"We could apply CQRS to a CRUD based interface (though things like creating separated data models would be much harder)."

Where is separated data models is anywhere in the "Solution". Is that part of the solution? If it is part of the solution why isn't it part of the solution description?

Again next:

"The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events." 

Again write / read models? Where is anywhere in the solutions description?

Again next:

"The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events."

The fact is, the two "models" seam to be so much into everything about CQRS that defining CQRS without integrating may be the source of a lot of confusion.

So in the end of the day, to use CQRS you need to have a valid reason to decide to have two or more representations of the same state. N for reading and one for transactions. Otherwise don't bother. Never mind the interesting properties that may or not be collateral.

In the end of the day I think me and Greg are probably saying the same thing. I'm trying to take an algorithmic view over this rather diving in questions of principle.

Cheers,

Nuno

--
Viktor Klang,
Director of Research and Development
Scalable Solutions

Code:   github.com/viktorklang
Follow: twitter.com/viktorklang
Read:   klangism.tumblr.com

[Angel Java Lopez]

Hi people!

Command and Query separation are key concepts by Meyer. But, AFAIK, he never pushed for TWO separate objects. He struggled against having a method that returns a value AND update state.

Said that, questions:

- Why to have TWO separate objects? I could have a service with query methods AND command methods.

- Why to have TWO data models? I could implement command methods/objects using ORM, entities, DDD, etc... and I could implement query methods/objects using other simplified objects, DTOs, stored procedures, views, etc.. That is: TWO codes, ONE data model.

My guess: If you are new to CQRS, you could try:

- First, separate methods (not objects) in queries and commands
- Second, separate code (ORMs, repositories, entities for commands; DTOs, DAOs, and alikes for queries)
- Only then, if you need it, go for separate data models.

My own answers ;-):
- Separate objects for query and commands, is the way to reference to different underlying implementation (even with ONE data model). That is, ObjectWithQueryMethods could be injected with DAOs, and use DTOs, meanwhile, ObjectWithCommandMethods could be injected with ORM artifacts, repositories, etc..
- Separate data models? hmmm.. I don't have a compelling argument here. I should explore event sourcing benefits and implementation.

Angel "Java" Lopez
http://www.ajlopez.com
http://twitter.com/ajlopez

2011/5/7 √iktor Ҡlang <viktor...@gmail.com>

[Laurynas Pečiūra]

Hi Nuno,

I'll try to describe my take on the subject.

In that same article he writes:
"We could apply CQRS to a CRUD based interface (though things like creating separated data models would be much harder)."
Where is separated data models is anywhere in the "Solution". Is that part of the solution? If it is part of the solution why isn't it part of the solution description?

Again, separated data models are not the basis of CQRS. You could have them, or just simply have a single data store and separated objects for reads and writes. What would be the benefit of having a single data store and read write separated objects? I would say:

• Change of mindset. I look at data needed to populate the View as data. Not as read-only representations of your entities, just data formed whatever way the view needs it. Hence there are no smelly View related methods on you repositories/table gateways/data mappers.
  
• You can create new Views, or throw some old ones out without having to worry about your repositories/mappers/gateways. In most projects i encounter View lifetime is very short, hence this helps me out a lot.
  
• It is a very solid stepping stone for more interesting architectural decisions. For views that encounter performance issues or if you want different data structures for different purposes, you are still able to introduce events and a denormalizer without much hassle.
  

[Laurynas Pečiūra]

In addition:

• You can easily give read-only access to your datastore to another team and tell to knock themselves out building the frontend.
• You can easily use different technology stacks for your frontend and backend.

[Greg Young]

I think the biggest differences happen in the domain model when using a single data store.

1) the issues between strucyural and behavioral go away.
2) aggregate boundaries become clearer
3) a lot of non domain things end up in the read model (think repository methods that support paging, sorting, etc) these things often pollute domain models
4) if using an orm in domain model a lot of complexity can go away in terms of doing queries. Often this is the first level of 'specialization, both hit same store but in different ways/models)

[Laurynas Pečiūra]

Thank you very much Greg, you just said everything i wanted to say in 4 simple bullet points. Need to sharpen writing skills (lurking doesn't do any good).

[Udi Dahan]

Nuno,

 

To your assertion on services:

 

"In lame terms each module/service is described as a set of input-ouput operations to which we call Interfaces"

 

I strongly disagree with that statement. Services are not at all about input/output. What you have described is, at a logical level, a function.

 

Cheers,

 

-- Udi Dahan

 

 

 

[√iktor Ҡlang]

Seconded

>
>  
>
> Cheers,
>
>  
>
> -- Udi Dahan
>
>  
>
>  
>
>  

[Nuno Lopes]

Hi Laurynas and Greg,

I think you guys are in the end of the day agreeing with me. 

I remember back in 2009 I said to people that we could easily build Repositories on top of an ORM and directly query the relational model. 

The fact is that in this scenario we already have two models. The OO Model (Domain Model) and a Data Model (Relational Model) synched with a Mapper.

In this scenario we can easily apply CQRS by having Commands handled by the OO Model and Queries made directly to the Data Model.

Still this scenario albeit possible was considered not canonical (poor example).

"It very solid stepping stone for more interesting architectural decisions. For views that encounter performance issues or if you want different data structures for different purposes, you are still able to introduce events and a denormalizer without much hassle."

Exactly.

It is all about needing to representations of the same state. Even if it just a glitch in the mind set. Not needed in the above example has two representation are given for free.

"1) the issues between strucyural and behavioral go away."

2) aggregate boundaries become clearer"

I call it contention between state representation needs between a fluid relational representation for queries and discrete object representation for transactions.

"3) a lot of non domain things end up in the read model (think repository methods that support paging, sorting, etc) these things often pollute domain models"

Again, representation. State representation tailored for paging, sorting etc. Things that relational database internally handle transparently for us.

"4) if using an orm in domain model a lot of complexity can go away in terms of doing queries. Often this is the first level of 'specialization, both hit same store but in different ways/models)"

Absolutely.

Nuno

[Nuno Lopes]

Hi Udi

"In lame terms each module/service is described as a set of input-ouput operations to which we call Interfaces"

I strongly disagree with that statement. Services are not at all about input/output. 

You are totally correct in strongly disagreeing :) I should have added, deployed on a network described in a platform agnostic way. 

Still at a functional level, in terms of  separation of concerns the semantics of a Service is the same as of a Module IMHO.

Programmers (including me) have grown neglecting the importance of modules. We used modules for house keeping, arbitrarily distributing functions or class between modules as we see fit. If done badly this becomes a problem in large frameworks as much as it becomes a problem in SOA.

What you have described is, at a logical level, a function.

Hum, no. What I described is a module. I used the phrase "set of operations", so it is not just one operation.

"Modular programming is a software design technique that increases the extent to which software is composed of separate, interchangeable components, called modules by breaking down program functions into modules, each of which accomplishes one function and contains everything necessary to accomplish this.^[1] Conceptually, modules represent a separation of concerns, and improve maintainability by enforcing logical boundaries between components. "

...

"With modular programming, concerns are separated such that no (or few) modules depend upon other modules of the system. To have as few dependencies as possible is the goal." add reusable.

I think the above are Services premisses too with the difference (if any) that the context the network. That is, instead of a package deployed say in a dll, we have it deployed in the network with a implementation agnostic description. This in such way any other component on the network, developed in any other tech can bind to it and remotely call its functions.

Cheers,

Nuno

[Udi Dahan]

Greg,

The CQRS you're describing over a single data store sounds very much
like the Single Responsibility Principle along with the Interface
Segregation Principle.

-- Udi Dahan

On May 7, 5:51 pm, Greg Young <gregoryyou...@gmail.com> wrote:
> I think the biggest differences happen in the domain model when using a
> single data store.
>
> 1) the issues between strucyural and behavioral go away.
> 2) aggregate boundaries become clearer
> 3) a lot of non domain things end up in the read model (think repository
> methods that support paging, sorting, etc) these things often pollute domain
> models
> 4) if using an orm in domain model a lot of complexity can go away in terms
> of doing queries. Often this is the first level of 'specialization, both hit
> same store but in different ways/models)
>

> On 7 May 2011 15:45, "Laurynas Pečiūra" <laurynas.peci...@gmail.com> wrote:
> In addition:
>
>    - You can easily give read-only access to your datastore to another team

>    and tell to knock themselves out building the frontend.

>    - You can easily use different technology stacks for your frontend and
>    backend.

[Udi Dahan]

Nuno,

The deployment on the network and the platform agnosticism is, at
best, secondary to services.
A service is also not a module - services are not "interchangeable".

Anyway, that isn't the topic of this thread. I suppose the closest
thing in DDD would be a Bounded Context.

-- Udi Dahan

[Žilvinas Šaltys]

Yes you can say that CQRS *pattern* applied at the object level applies SRP and IS *principles*. However saying "I've applied CQRS at object level" says more than that you've applied SRP and IS principles. CQRS also describes how these principles were used as they can be used for many different things.

CQRS can also be applied at a higher level than just the object level.

That's why to me it makes sense to have a distinct pattern CQRS regardless if at some specific cases it has some similarities to SRP, IS or CQS.

[Nuno Lopes]

Hi,

There is no doubt in my mind that the Pattern can sustain on its own. Some patterns seam to come as a combination of others.

The question is why should this pattern be applied and when? Different context have different reason for it.

In abstract IMHO the bottom line is that we should apply it when we multiple distinct representations/structural of the same set of consistent values (state). The reasons for this need is what I try to describe in my previous post.

Of course, once we decide that we have those kinds of reasons, then when applied these problems go away. Other problems get in.

From CQRS to ES

Regarding ES the interesting tibit for me is that once we the choose to use domain events to push values from one transactional representation to the views (read models), we probably should question ourselves, why aren't this events stored for historical purposes? For sure this is useful when dialing with eventual consistency and related side effects.

Once we have an answer for that then we ask, why don't we use the history to compute state? (Greg' approach).

I would say answering the last question is a bigger jump then the former two from paradigm point of view. The first two don't require an change in paradigm (from modular / procedural or even OO), the later to use to its full extent it does IMHO.

The interesting part is that as it seams the people that made the jump in paradigm from traditional OO to this seam to find more intuitive then OO itself when modeling the business domain.

Personally, although I dig it, I still find that we need some logical foundations described in context as explained in my post.

From CQRS to EDA and SOA

It seams that SOA see events as a form of stimulus facilitating communication, data integration and service orchestration between services, opening the door for event-driven programming on top of SOA.

http://eventdrivenpgm.sourceforge.net/event_driven_programming.pdf

That IMHO does not seam to be the core point of ES in how Events are seen. At its core an event its a fact not stimulus in context of some DFD.

Cheers,

Nuno

PS: "There is a Monster in the end of this nook".

[Žilvinas Šaltys]

"In abstract IMHO the bottom line is that we should apply it when we have multiple distinct representations/structural of the same set of consistent values (state)."

I think I get where you're going as this *reasoning* can be applied to both full blown eventually consistent systems that apply DDD, CQRS, ES and also could be applied to any kind of other system that just uses CQRS at the object level to separate the domain model from the query data model.

I would agree that your reason *when* to use CQRS is somewhat valid. However it's quite broad and a little confusing when CQRS is applied at object level. Example reasons people use CQRS:

1. Uncluttering your domain.

2. Change in mindset, helps to better model OO domains.

3. Solid step towards other architecturally useful solutions for example ES, EDA.

One would have to be a bit more specific what was his exact reason to use CQRS. Even if someone applies CQRS where he didn't necessary have to I wouldn't find it that big of a deal. It's the same as if someone would apply Domain Model where Transaction Script would work well enough. The person might have just wasted some more time than he had to ;)

However I understand and I may be wrong this thread comes from the fact that Udi bundles a bunch of architectural decisions under one name of CQRS and says that people apply that bundle of patterns just because it's the cool thing to do. Then we start to question why would one use CQRS while the real question is: 

"When to avoid using CQRS + All other patterns CQRS tends to be mentioned with" and not "When to avoid CQRS as a single pattern"

[Greg Young]

Almost every pattern can have this said for it ...
MVC
Repository
Factory
Strategy
Proxy
Decorator

To name a few off the top of my head.

Greg

--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention

[Nuno Lopes]

Hi,

> 1. Uncluttering your domain.

A domain model is one possible interpretation of the world. Its as algorithmic as it is intuitive since it represents the UL in code.

Albeit it is a good representation from a transaction/linguistic point of view, for presentation for instance it fall short. Quite often people find it not good fit for queries too so much we develop sophisticate Object Mappers for that sole reason. Object navigation is somewhat secondary in the business world ... slicing sets of facts seams to be more useful.

> 2. Change in mindset, helps to better model OO domains.

That is a arguable IMHO. It depends on a lot of factors not directly related to CQRS. I've seen samples of domain models built using CQRS and ES that I don't that are good examples of domain modeling.

> 3. Solid step towards other architecturally useful solutions for example ES, EDA.

Also arguable IMHO. It depends on a lot of factors not directly related to CQRS.

ES for instance, is not so much an architecture decision, but pretty much domain driven. As for EDA, I agree, still the you can do without EDA in a lot of situations.

I can see what you are arguing, but in the end of the day a pattern definition should scoped on what it does and not on what it can potentially do once we add other stuff on top.

If we want to "sell" ideas on top of CQRS then focus the discussion on those ideas and explain why CQRS is necessary to them and not the other way around.

> I think I get where you're going as this *reasoning* can be applied to both full blown eventually consistent systems that apply DDD, CQRS, ES and also could be applied to any kind of other system that just uses CQRS at the object level to separate the domain model from the query data model.

Yes, every time we need more then one representation of state CQRS can be used and probably should be used. If we don't have such need, don't use it.

A very simple example. We had a workflow systems totally build using OO practices (closed vendor solution). The data model, was built to support Dynamic Property pattern and so on.

The result was that for a Task Based UI was fine and dandy (even generated them automatically), but to perform queries across activities and process was extremely slow due to dynamic properties. So we have built a query model and populated it using a ETL approach (every minute). The next step would to populate it using Push (events). This would allow "real time" population, but we haven't don't since it would require us to change the vendor code or data model and the purpose was really optimizing queries. The lag between the state in the domain model and the data in this query model wasn't a nuisance for the use cases at hand. But if we have done that then we could render all the UI using this new read model, in other words, it could be used for any use case, commands would go trough the vendor code and all queries would go through the new query model.

But you see, Commands and Events weren't central to this decision but CQRS helped to form the architecture of this solution.

I believe that in today's Enterprise's (medium to large) a lot of situations like this happen. This is basically a top down approach, from problem to solution in brown field environments.

In green field projects, then we have the liberty to make more profound architectural choices at right at the start including designing a solution towards multiple representation of state. Something that if our workflow vendor followed our lives would be much facilitated, not only in this context but also in other has you can imagine.

A part of the business and infrastructure is being moved to use Sales Force. Again we are facing very similar problems. The problem is not so much in terms of data integration but is actually is in terms of collaboration between bounded context. Without a cross system BUS where SalesForce is an endpoint (of sets of end points) and our systems are other end points ,the only solution is a mix of ETL and simple events store in databases to post process (everything is post processed). On the other side of the tail our internal systems aren't built to publish events for every single business fact.

CQRS is here is still a solution, but not so much a solutions as it is discussed for green field projects, SOA or non SOA. Most of us used to work in enterprises for long terms we have to deal mostly with other people mistakes with the understanding that people do things to the best of their knowledge, that includes vendors, external consultants and experts. The complexity here is different, we have to keep the business running long after short term consultants and vendors are gone.

The kind of reasoning around CQRS that I try to present is more realistic considering this landscape and does not close the doors at all to other things we can put on top towards:

> Solid step towards other architecturally useful solutions for example ES, EDA.

I miss green field projects.

Cheers,

Nuno

[Tom Janssens]

So CQRS is pattern !
LOL!

[Tom Janssens]

That previous post was supposed to be a sarcastic summary of this whole thread BTW ;-)