Aggregates as consistency boundaries with system prevalence/memory image

[Michael Ainsworth]

After reading Martin Fowler's description of a Memory Image (http://martinfowler.com/bliki/MemoryImage.html), also known as System Prevalence or Object Prevalence (https://en.wikipedia.org/wiki/System_Prevalence), I've been wondering how this fits in with aggregates as transactional boundaries.

For example, the entire LMAX system operates in memory, and has been described as a "single aggregate" - the data within the entire (in memory) system is consistent by the time a command has finished executing. This has several benefits, including a global total ordering of events. A memory image is in contrast to a more distributed system, where each aggregate forms a transactional boundary. Such transactional boundaries are important because they enforce internal consistency at a micro-level while still allowing scalability.

How do you think a "global" aggregate impacts the modelling process, both positively and negatively? I can think of several changes in mindset.

Firstly, you do not need to store (sub-) aggregate identifiers, but can store direct references to objects. This can be a positive thing, in that there is no indirection (via a repository) to look up an object. On the negative side, this can cause issues in regards to memory management - cyclic references if you're using smart pointers, for example. On the positive side, this can be more expressive, as the domain is representative of the real world - I no longer store the customer "ID", which maps to a serialized blob/sequence of events on a disk or in a database or in the cloud - rather I just store a reference to the Customer directly.

Secondly, it introduces event handler complexity. I once worked on a piece of software that was written as a series of database stored procedures. After an INSERT/UPDATE/DELETE of data, this stored procedures would issues another SQL command (SQL triggers are effectively generic, non-semantic event handlers). The cascading affect was very hard to trace, but if one of the triggers had an issue, the whole system would be rolled back. In an in-memory situation, an error in an event handler would cause the entire system to crash, as there is no rollback facility in such an approach (unless you build one in of course).

Thirdly, it blurs the lines of invariants. E.g., an invariant is maintained by an aggregate/entity class. With a single-threaded, in-memory approach you can assume that one class will always be "in sync" with another. E.g., a Customer is always "preferential" if they have ordered more than $1,000 of product. This is a business "rule", but there may have been a time before this rule was implemented where preferential customers were those that subscribed to the mailing list. The invariants are what they are, but immediate consistency can fool people into mistaking relationships between entities as invariants.

What are other people's thoughts on the pros and cons of "system prevalence"?

[Harry McIntyre]

I built a system along these lines and it was the single most pleasurable software development experience of my career (except maybe the first ever 20 GOTO 10). Not having to faff around with stateful data stores, worry about aggregate boundaries was definitely very freeing, and being able to model everything without leaving the language was great. I did indexing and projections with in-memory Lucene indexes.

The biggest switch in mindset is in writing entirely deterministic code, and writing temporal rules (like the business rule you mentioned above) although I think that's bread and butter for anyone doing ES. 

The system I build was relatively low use, and didn't have a strict SLA, so there are some unanswered questions I still have, around high read/write contention and maintaining uptime (LMAX use several identical servers for redundancy). Not too worried about running out of RAM these days as you can get 6TB Dell servers (although the 96 cores seems like overkill for a single threaded architecture).

[Greg Young]

It is also a common pattern to still use aggregates in this type of
architecture. Basically you would model an aggregate as say Instrument
then you would shard to multiple threads based on the instrument id.
This keeps things single threaded per instrument but can have multiple
threads in multiple instruments. Depending on the amount of
calculation being done this may or may not be beneficial. For
something as simple as crossing orders its quite possible that this
will turn out to be worse than single threaded.

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test