Maintaining Aggregate Root references in CQRS. Sagas vs command handlers

[Dominic Godin]

Posted this on Stack Overflow but just found this forum.  Think I might get a better quality response here.

I'm building a prototype to explore CQRS and Event sourcing.

Say I have 2 root Aggregates, Organisation and Position, a one to many relationship with a need for a two way reference. Positions can exist without Organisations (freelancers for example).

For this discussion lets assume both Organisation and Position objects have been created. I now have an AddPositionToOrganisation command.

Making sure the references are in sync I see I have two options.

1. The Organisation command handler can generate two events. One for the Organisation and one for the Position adding the corresponding references.
   
2. Create a Saga that listens for an PositionAddedOrganisation event and create a AddOrganisationToPositionCommand.
   

Is there a problem with using the Organisation command handler to generate events for the Position? On the plus side it groups related functionality together (maintaining references), but makes the Organisation responsible for some of the Position logic.

Some insight from more experienced developers would be most welcome.

[Dominic Godin]

I think this http://cqrs.nu/Faq/aggregates has just answered my question.  Option 2 it is.

[Herman Peeren]

Is there a difference in your domain between a position being added to an organisation or the other way around? Isn't this in fact one and the same action in your write-side? Resulting in two projections for the read-side: the current state of an organisation has a position added and the current state of a position has an organisation added.

[Herman Peeren]

Also: is there a word used in the domain instead of "add"? I've never seen organisations that "add" positions and even less the other way around.

On Friday, 9 September 2016 13:18:27 UTC+2, Dominic Godin wrote:

[Dominic Godin]

First thanks for the reply, most appreciated. 

You are right there is no difference. I'm still new to this so bare with me )

I am modelling a simple system, (it's a subsection of our existing CMS system) with three aggregate roots.  Person, Position and Organisation.  

• Both Organisations and People can exists in isolation, as in People and Organisation can exist with out Positions.  
  
• A Position however must have a Person but the Organisation is optional.

Originally I had Position as part of the Person aggregate and the Organisation holding ids for both the Person and Position. I have since read that you should only reference between aggregate roots, so I have refactored Position out to it's own root.

What I'm looking at now is maintaining the id references between aggregate roots.  What is best practice here?

Currently I only have a Saga that listens for a PositionAddedToOrganisation event and creates a corresponding AddOrganisationToPosition command.

As for terminology I believe here it is correct. The users of the system are researchers, in this case they are adding and remove people (via positions) to and from the organisations they are researching.  As this is just a feasibility study on the technology I'm not too concerned at this stage.

[Herman Peeren]

OK, that makes it a more understandable; sorry for my (false) assumptions.

However, to me a Saga sounds way off for a simple system with one command... Maybe I'm oversimplifying your case, but in my view the purpose of DDD-modeling is to simplify compexity, so I'd try to make things as simple as possible (but not simpler).

Could you give some examples of of the positions that are added to organisations? And why do you have a command to add positions while you say you are adding people to organisations?

If your main purpose is to explore CQRS, could you tell something about your read-side? What (denormalised) data do you want to display?

[Herman Peeren]

Maybe this helps: the read-side doesn't have "aggregates"; that is terminology for the write-side, the actions that change the system's state.

[Herman Peeren]

I'll just try a bit. Not one, but two commands:

• at time T assign person X (via position Y) to organisation Z. (one timestamp and 3 ids)
  
• at time T remove person X (via position Y) from  organisation Z.

What invariants are in play here? Must a position first have been assigned to a person? Can a person be assigned multiple times to an organisation? I still don't have a clear idea about your domain.

When invariants are met, this will result in events:

• at time T  person X was was assigned  (via position Y) to organisation Z.
  
• at time T person X was removed (via position Y) from  organisation Z.

Store those events, they are your single source of truth. You can reconstitute the current (and any past) state from them. Until now you still haven't updated or stored anything else.

You can build projections that handle those events and assemble/update your read-data as a result.

Does that make sense?

On Friday, 9 September 2016 19:20:54 UTC+2, Herman Peeren wrote:

[Peter Hageus]

Position sounds very much like lookup data to me. Surely the person assigment is the relevant behaviour?

/Peter

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Herman Peeren]

An aggregate in DDD is not just the whole object graph that you are used to from your ORM. It only contains the domain objects needed to guard the invariants while executing methods that change the system's state. If you have an aggregate for your starting and stopping scheduling of persons, organisations and positions, you don't need another aggregate (at least not for the behaviour you described). Ergo: no need to make Person, Organisation and Position aggregates (in the sense it is used in DDD as a consistency boundary); they are "just" entities.

[Dominic Godin]

Ok, I was keeping the domain simple in this example so I could focus on the mechanics of CQRS.

However I think it would help if I gave more detail of the domain and what I'm trying to achieve.

Currently we have a CMS system that feeds multiple websites, each with a different focus.  The CMS is old, it predates me at the company and I've been there over half a decade.  It's a desktop application with a normalised db as it's back end.  Triggers on this db copy data into partially denormalised databases that the websites use.  There are a lot more moving parts, solr indexes, Neo4j etc. but I'll be here all day :)  I have spent most of my time at this company working with these read only databases to make web applications.  The CMS is a nightmare to work with and is long overdue a rebuild.

One impotent feature it does badly is "Drafting and Proofing".  A users changes are held in a draft until they are submitted to an editor for proofing.  This is currently is done by locking records, storing the changes in xml and merging them back into the DB when proofed,  Anyone who has worked with file locking source control knows why this is bad.  When exploring alternative approaches a team member pointed me towards Event Sourcing.  Reading up on Event Sourcing introduced me to CQRS.

Lets say I had a light bulb moment.  We currently kind of have a read and write side, our normalised and web DBs.  CQRS looks like a much cleaner implementation.  Messaging and command handlers contain the domain logic, stopping it sprawling all over the application and its very testable.  So to test the concept and my ability to get to grips with it I have taken a small part of the legacy system, modelled a few commands and built an importer.  Basically the importer attempts to mimic what a user would have done to enter this data.  On the read side, I have a fully normalised DB representation of what is in the event store so I can inspect the contents easily.  More importantly there is a copy of the existing web database, this will only be updated by Publish events, other edits will be ignored.  This will allow us to update the CMS without updating all the websites all at once.  When we redevelop a website we will create a bespoke query model for it.  I see a lot of promise for this method but I need to be confident that I understand it properly before I commit to this path.

So the domain:

Organisations:  

• Can have 0..n Positions.  The order is important.  
  
• Can have 0..1 ParentOrganisations.
• Can have 0..n ChildOrganisations

People

• Can have 0..n Positions, the order is unimportant.

Positions

• Must have 1 Person
• Can have 0..1 Organisations

Organisations, People and Positions can be published separately.

In the full system there is a lot more complexity to these entities, they have categories, metadata, offices, contact details, aliases etc but I'm deliberately reducing the scope for the time being.

The importer creates a load of CreateOrganisation commands, followed by AddChildOrganisation commands.

Here I use a Saga to maintain parent child ids.  It listens for ChildOrganisationAdded events and sends SetParentOrganisation commands.

It then creates a load of CreatePerson commands, followed by CreatePosition commands.

I also use a Saga to listen for PositionCreated events and send AddPosition commands to the Person aggregate and Organisation aggregate when appropriate.

Finally Publish commands are created for each of the aggregates.

I have omitted the commands that fill in some of the aggregate details as I'm focusing on the relationships.  I know the naming is a bit clumsy and very CRUDy however this is the terminology the users use.

Using saga's to maintain the relationships seems to be working very well so far but the system is very small.

Hope that helps.

I am still new to DDD terminology, currently halfway through Eric Evans DDD book.  For example, I don't know the difference between aggregates and entities here.  To me they are both different words for class.  In the nTier world I use "entity" to mean a class that models a data record.

[Herman Peeren]

An essential part of Event Sourcing is modeling time: in what order (and possibly when) did an event happen. In your domain description until now I don't see any mention of temporal data. Is that because you use the data from a legacy CRUD-system in which exactly those temporal data is lost?

[Dominic Godin]

Yes you are correct.

The event store will track the time events were created.  My schema is currently.

AggragateId Guid

Verson int

CreatedDate DateTime

CorrilationId Guid

CreatedByUserId Guid

Data Json

The legacy CRUD system only records date and time when it's important to the domain.  Publish date for example.  The importer can not reconstruct the exact time updates were done on the legacy system so it doesn't try to. The CreatedDate will be the date the importer created the commands, and the user will be a Guid assigned to the importer.

The CreatedDate of the event store is use only by the infrastructure of the application.  If a date is relevant to the domain it is stored in the DataJson.  I have build an "Event Runner" tool which lets me rebuild Query models from the event store, this is dependent on the event CreatedDate to function correctly, here the order events happen is vital.  CreatedDate is not used in the domain at all.  As I see it the domain should not be anymore aware of the event store than a standard n-Tier domain should be aware of the database tables.

[Herman Peeren]

In real life giving birth to a child and becoming a parent are not two distinct actions. A Saga is a way to model distinct actions belonging to one transaction. You use a Saga as a technique to implement a sequence of programming commands, not of actions in the domain. Domain driven design is modeling using the language of the domain. AddChildOrganisation and SetParentOrganisation are not two distinct actions in the domain, they only exist as programming concepts.

On Saturday, 10 September 2016 13:56:10 UTC+2, Dominic Godin wrote:

[Herman Peeren]

Do you really create organisations in your domain? Legal stuff, hiring people, renting space, establish organisational structure, etc.? Or do you mean by CreateOrganisation that you are importing data of some organisation in your system?

Try to find words that describe the domain instead of using mere programming concepts.

On Saturday, 10 September 2016 13:56:10 UTC+2, Dominic Godin wrote:

[Herman Peeren]

With "You use..." I meant "what you, Dominic, are doing now...", definively not "you should use..."; language can be very ambiguous, hence this clarification.

[Herman Peeren]

OK, that is what I meant: the order in which the events happened is part of your model. You can for instance use it in invariants like "A person can can only be unscheduled from an organisation if it has earlier been scheduled (and since then not been unscheduled)".

[Dominic Godin]

I'll go through these one at a time.

Saga:  I get what you're saying about the commands should be about domain logic.  However without using saga's this way how is it possible to keep links between aggregate roots in sync. An Organisation must know what positions it has and a position must know if it has an Organisation.  A command issued to an Organisation shouldn't cross boundaries and add events to positions.  A saga on the other hand can bridge the gap.  I agree SetParentOrganisation is poorly named.  I'm on my first iteration and my main focus is on the mechanics of CQRS not the domain itself.  The domain here is not important, the domain could be anything.  What I'm investigating is how links between aggregate roots are maintained.

Ubiquitous Language:  I understand what your saying about the naming.  I have worked closely with the users of the system for 6+ years.  As it's a CMS for a directory, they see things like Organisations and People more like documents.  "I want to create an organisation and add positions to it" is something they would say.  I could swap "create" for "enter" or "input" but "create" is clearest.

Invariants:  The order is important as it provides the state of the system at that time.  However the domain itself can not look back in time.  In this case it does not need to.  The invariants depend on the current state not the past state.  So we have things like, "A Position can not be added to and Organisation if it already belongs to an other Organisation".  "An Organisation can not add the same position twice".  

It may be that I split Organisation positions from Freelance positions, move Organisation positions into the Organisation boundary and Freelance positions into the Person boundary, removing the Position aggregate root all together.  However I'm still left with a Organisation to Person link to keep in sync.

[Peter Hageus]

The drafting and proofing process sounds very much like branch, pull request, code review and merge.  GitHub essentially.

The domain looks very datacentric, I see no obvious behaviour here. I’ve personally not found event sourcing to add very much value in similar scenarios. 

Have you thought about modelling this as version managed documents?

/Peter

[Herman Peeren]

I cannot answer this without repeating the same things over and over again. Easier if you reread them. Maybe your example is not the best to learn more about Event Sourcing and CQRS, especially when you are stuck in the language and paradigms of an earlier solution. Sorry, I cannot help you further with this.

[Dominic Godin]

You are right, there isn't a huge amount of behaviour in the cms.  The domain is basically a big data store.  Bare in mind I'm only modelling a small subsection of it.  I'm trying to get to grips with the technology itself.

A few things have attracted me to CQRS:

• The containment of logic i.e. command handlers.
• The event stream.  This is a biggy for me.  The CMS feeds many websites, they are very varied in what they are interested in.  In turn the web databases feed other backend stores that support complex searches.  Solr and Neo4j.  Using event handlers to build bespoke query models and handle the Solr and Neo4j updates should be very powerful.  It has the potential to greatly reduce both our database and website code complexity.

Whilst I see you can use CQRS without event sourcing, I think rebuilding query models after schema changes could be difficult without it.  With the event store you can just update the event handlers and re-run the events though it.

As for version managed documents, no I haven't.  Thanks for the heads up I'll take a look. 

[Dominic Godin]

Thanks for the time you spent on this.

My mistake was providing an example.  The domain is not important for my question. Many domains will have links between aggregate roots.  If a command can only update the aggregate it targets how are links maintained?  Other sources have pointed me to Sagas.

Anyway thanks again for your time.

[Herman Peeren]

It is never a mistake to provide an example! I hope you keep exploring Event Sourcing and CQRS further and post your thoughts and questions on this mailing list. I'm sorry I was not able to make something clear I wanted to say, but that is my "mistake", not yours. Thank you for challenging us with your questions!

[Dominic Godin]

I will.  I clearly have a lot more to learn than I hoped.  I have a little time before I need to commit to CRQS or not so I will continue my research.

If I'm right my thinking is still too embedded in the old structural model.   I have taken a bit of time to go through the answers here again.

The query side I'm fine with.  It's the write side that is causing me issues.

So I'm working through the following "my aggregates are focused on guarding invariants and not object graphs".  Say I have Directory that is responsible for People and Organisations, and manages assigning positions.  The data store for my aggregates is an event store.  If Directory was the aggregate root the number events for it would be huge.  With more than a couple of users I'd hit versioning conflicts.

Maybe its because my core knowledge of CQRS comes from the examples given here http://cqrs.nu/.

To run a command I need to create an empty aggregate, rerun all it's events and than fire the command handler.  My approach has been to have an Organisation root that handles Organisation commands etc.

My understanding of aggregates is they must have a root entity.  You can only reference aggregates by this root entity.  People and Positions are used in other parts of the Domain.  News articles or events may link to either for example.  To me this makes both people and positions root entities.

Is it just I don't need to use a Saga to maintain the reference ids unless I'm explicitly enforcing an invariant.  Basically only one side needs the reference, in this case the Organisation -> Position.

Sorry didn't mean this to turn into a question, let alone a rambling one.  Just can't help myself :)

Thanks again.

[Johanna Belanger]

On Monday, September 12, 2016 at 2:41:18 AM UTC-7, Dominic Godin wrote:

So I'm working through the following "my aggregates are focused on guarding invariants and not object graphs".

 

Keep working on this, it will lead you in the right direction.

To run a command I need to create an empty aggregate, rerun all it's events and than fire the command handler.  My approach has been to have an Organisation root that handles Organisation commands etc.

That sounds right.

My understanding of aggregates is they must have a root entity.  You can only reference aggregates by this root entity.  People and Positions are used in other parts of the Domain.  News articles or events may link to either for example.  To me this makes both people and positions root entities.

An aggregate is a collection of entities which have invariant relationships to each other.  You should only make changes to the entities in an aggregate through the aggregate's root entity. If you were to make changes to the entities directly, they would not know to protect invariants that span multiple entities in the aggregate.

Reference in this case means an object reference to an actual entity object, which would allow the reference holder to make changes to the entity directly. It is ok for an aggregate to hold a reference ID for another aggregate or entity.

Is it just I don't need to use a Saga to maintain the reference ids unless I'm explicitly enforcing an invariant.  Basically only one side needs the reference, in this case the Organisation -> Position.

You don't need to maintain any data on the write side except what you need for processing commands/emitting events and maintaining invariants.

Hope that helps some. 

[Dominic Godin]

Thanks for this it helps a lot.

When taking about references I was indeed talking about reference IDs.

You've restored my confidence that I'm that I'm on the right path.  I happy that I have the pluming close to ready, so my focus has now switched to the modelling techniques of DDD.

Thanks again