Eventsourcing and multiple aggregates

[seagile]

I'd like to get to the bottom of this, because the advice I was given
in the "To UnitOfWork Or Not" thread seems to contradict Mark Nijhof's
popular example:

In his example he has an ActiveAccount (an aggregate root) that has a
Close behavior/method:

public ClosedAccount Close()
{
Guard();

IsAccountBalanceZero();

var closedAccount =
ClosedAccount.CreateNew(
Id, _clientId,
_ledgers, _accountName,
_accountNumber);

Apply(new AccountClosedEvent());
return closedAccount;
}

When we look at the ClosedAccount (also an aggregate root) we see
(this is an excerpt):

private ClosedAccount(
Guid accountId,
Guid clientId,
List<Ledger> ledgers,
string accountName,
string accountNumber) : this()
{
var Ledgers = new List<KeyValuePair<string, string>>();
ledgers.ForEach(x =>
Ledgers.Add(
new KeyValuePair<string, string>(
x.GetType().Name,
string.Format("{0}|{1}", ((decimal)x.Amount),
x.Account.Number))));

Apply(
new ClosedAccountCreatedEvent(
Guid.NewGuid(),
accountId,
clientId,
Ledgers,
accountName,
accountNumber));
}

public static ClosedAccount CreateNew(
Guid accountId,
Guid clientId,
List<Ledger> ledgers,
AccountName accountName,
AccountNumber accountNumber)
{
return new ClosedAccount(
accountId,
clientId,
ledgers,
accountName.Name,
accountNumber.Number);
}

Now tell me again how this is wrong? Because two aggregate roots are
producing events for one use case ("closing an account"). From reading
the code I gather they are persisted together (one transaction). If I
were to take the advice given in the previous thread, then would I
have to model the creation of the ClosedAccount as an event handler
issuing a command "CreateClosedAccount"?

Now for the critics out there:
I will follow you when you say that in a distributed high-perf
environment this may be a no go (Greg, I read Pat Helland's article),
but let's face it, not all systems are created equal.

I'm trying to establish if there is a middle ground (possibly with
pro's and con's).

Disclaimer: All credits go to Mark Nijhof - the above code was taken
from revision Fohjin-f85a251

[Szymon Pobiega]

I like the 'middle' solution: passing messages between aggregates. It maintains aggregate boundaries (in the sense of Pat's paper) and it is closed in the model (doesn't require using upper layers). 

Unfortuanately it is more complicated than simple top-down approach when command interacts with single aggregate which produces events. On the other hand, when all the interaction in on the model level, I would argue that this is more readable then processing events by external saga which issues commands.

Using this solution, it would look something like this:

public void Close()
       {
           Guard();

           IsAccountBalanceZero();

           var closeMessage = new CloseAccountMessage(Id, _clientId...);

           SentTo<ClosedAccount>(someId, closeMessage);
           Apply(new AccountClosedEvent());
       }

and ClosedAccount would be defined as

class ClosedAccount : IHandle<CloseAccountMessage>{

void Handle(CloseAccountMessage message)

{

...code goes here

}

}

2010/7/30 seagile <yves.r...@gmail.com>

[Mark Nijhof]

The example is indeed very simple ;)

Also After Greg went through this with me he suggested using an UoW
that gets registered in the AR where the AR will publish/register its
events to instead of keeping them locally. That way you can dispose
the AR before the transaction gets committed, and you are putting the
right responsibilities in the correct place. I just haven't found the
time to implement these changes.

-Mark

--
Mark Nijhof
m: 0047 95 00 99 37
e:  mark....@cre8ivethought.com
w: www.cre8ivethought.com

"Walking on water and developing software from a specification are
easy if both are frozen."

-- Edward V Berard

[seagile]

It's highly decoupled, but not as readable as Mark's example ... and
requires extra infrastructure inside the aggregate root (admitted, it
could be done via a base class or composition). Your SendTo method is
acting like a generic command publisher/handler (an in-memory bus?),
providing repository/factory (lookup/create) and unitofwork (change
tracking) like capabilities. I presume that you'd want ClosedAccount's
eventstream to be persisted after handling the message.

But it is an elegant solution to make sure that "aggregates" can be
treated like "documents".

Thanx Szymon!

[@NeilRobbins]

With the obvious caveat that everything depends on context ;)...

I wouldn't think I'd want an aggregate to explicitly have knowledge of
what handlers might be interested in its events (which is what I'm
assuming the line 'SentTo<ClosedAccount>(...)' is doing).

In my command handling pipeline I'm likely to have code which will

1. rehydrate the aggregate root
2. call the appropriate command method on the aggregate root (which as
a command method will be void)
3. call the only query method on the aggregate root (which will be get
changes)
4. persist the changes (the event objects) to the event store.

Now at this point either:
a. there will be a 5, which also publishes the events to a bus for pub-
sub style consumption (and which will probably be inside a transaction
with 4), or;
b. I will have the event store being used as a durable queue itself
(so like the Carrot meets CouchDB method I posted previously, or if
using SQL Server perhaps something like SQL Notification Services
subscriptions, or a host of other possibilities including just polling
for changes).

Either of the above (a or b) will lead to event handlers handling the
events, these will be responsible for updating the query side of the
system; other systems in the enterprise/wider system (so datamarts,
data warehouses, other applications like CRM, billing, etc...,
distributing to global replicas a la PNUTS, etc...). This may also
include handlers which lead to other commands being fired within the
source system, but on a different AR - just as if a user had done some
work and this had led to a command being fired.

As has been said, either in this thread or another one, when
rehydrating the AR then the events from the event store will be
reloaded, but they are not recorded in the GetChanges method. There is
an example of this in the (admittedly needing TLC) Java 1.4 example I
stuck up as a Gist, see the LoadFromHistory method on the
AggregateRoot class (the first one in the Gist) http://gist.github.com/481220

The advantages that I have found from working this way are:
loose coupling (through the pubsub style) - which leads to easier
extensibility, maintenance, etc...
clear boundaries to Aggregates (like Greg said, in terms of
consistency, as well as domain context)

On 30 July, 10:37, Szymon Pobiega <szymon.pobi...@gmail.com> wrote:
> I like the 'middle' solution: passing messages between aggregates. It
> maintains aggregate boundaries (in the sense of Pat's paper) and it is
> closed in the model (doesn't require using upper layers).
>
> Unfortuanately it is more complicated than simple top-down approach when
> command interacts with single aggregate which produces events. On the other
> hand, when all the interaction in on the model level, I would argue that
> this is more readable then processing events by external saga which issues
> commands.
>
> Using this solution, it would look something like this:
>
> public void Close()
>        {
>            Guard();
>
>            IsAccountBalanceZero();
>
>            var closeMessage = new CloseAccountMessage(Id, _clientId...);
>            SentTo<ClosedAccount>(someId, closeMessage);
>            Apply(new AccountClosedEvent());
>        }
>
> and ClosedAccount would be defined as
>
> class ClosedAccount : IHandle<CloseAccountMessage>{
> void Handle(CloseAccountMessage message)
> {
> ...code goes here
>
> }
> }
>

> 2010/7/30 seagile <yves.reynh...@gmail.com>

[seagile]

@Neil,

I don't think CloseAccountMessage is an event in this case. As Udi
would state: send is for commands, publish for events. And the intent
is made clear by sending it to a specific aggregate (existing or non-
existing that's up to the infrastructure) which is another thing that
differentiates an event from a message.

I think with all this decoupling you are loosing the overview of one
use case (getting that overview used to be pretty easy (and had its
flaws) in the pre-cqrs/eventsourcing era). It's readable, but not from
the point of "the command handler as a use case". That is I think the
biggest hurdle with regard to the clear separation of boundaries/
aggregates.

[seagile]

Mark, does your mentioned solution suggest to record events in a
(ambient or injected) unit of work for both aggregates (in which case
it's an implementation detail because you keep affecting two
aggregates)? Or just for one?

[@NeilRobbins]

Agree that it can make it a bit more difficult to track the flow
through a system. This is one of the 'cons' to event driven (pubsub)
architectures. I think it can be compensated for through things like
monitoring, auditing, etc... Publishing IHaveHandledAnEvent, events by
convention can help track who's handling what as well. You can see
what handles what (outside the originating AR clearly) by just looking
to see what HandledEvents were raised in response. But that clearly
comes at a cost.

Another approach, is that it's not hard to search in a codebase for
all the usages of say (IHandle<AccountClosed>) for example, to find
all the handlers. The key thing for keeping things trackable is only
having a single place where an event gets published. After that
locating subscribers is not really difficult I find (well, within a
particular codebase), though it does of course come with a couple more
clicks.

System Tests can help with this too (as per Nat Pryce's picture here -
http://www.natpryce.com/articles/000772.html), describing the
expectations when a particular event occurs. Clearly here, maintenance
can be an issue, but that's always the case with integration tests.
This probably best inside the originating system (eg including the
query side, but not the other systems/apps in the enterprise that
might subscribe, such as the CRM or the DataWarehouse).

It is a trade off, so context as always is king :)

[Mark Nijhof]

The mentioned solution was for individual AR's as well as when having
to do stuff with more then just one affected AR (this should really be
an exception to the rule).

@Neil I think one important part to this is also familiarity to the
concepts, meaning that once you understand the concepts being used
then navigating the code and understanding what happens when will be
so much easier. EDA is just very new to some people and for them it
will seem like magic :)

-Mark

--

[Szymon Pobiega]

Actually, I implemented SentTo simply as applying a generic MessageSentEvent. Likewise, when my aggregate receives a message, a generic MessageReceivedEvent is applied before message processing logic is invoked. Both events store the message object. I use it to establish communciation context. Any time (especially when replying) an aggregate can query its own state for messages it sent/received to/from other aggregates. I borrowed this concept from Pat's paper.

There is an external service which asynchronously processes event store, filters out MessageSentEvents and enqueues their messages to the entry queue of the system.

With aggregates able to send each other messages, this approach looks almost like an agent-driven system.

Szymon

2010/7/30 @NeilRobbins <neilar...@gmail.com>

[@NeilRobbins]

I like the term 'agent driven system' - not come across it before, but
I think it captures very nicely what's going on :)

It'd be very interesting to see some more of the code you're
describing. I don't suppose you fancy sticking it up on GitHub do
you?

On 30 July, 12:17, Szymon Pobiega <szymon.pobi...@gmail.com> wrote:
> Actually, I implemented SentTo simply as applying a generic
> MessageSentEvent. Likewise, when my aggregate receives a message, a generic
> MessageReceivedEvent is applied before message processing logic is invoked.
> Both events store the message object. I use it to establish communciation
> context. Any time (especially when replying) an aggregate can query its own
> state for messages it sent/received to/from other aggregates. I borrowed
> this concept from Pat's paper.
>
> There is an external service which asynchronously processes event store,
> filters out MessageSentEvents and enqueues their messages to the entry queue
> of the system.
>
> With aggregates able to send each other messages, this approach looks almost
> like an agent-driven system.
>
> Szymon
>

> 2010/7/30 @NeilRobbins <neilarobb...@gmail.com>

[Greg Young]

You can do this if you build an assumption into your system, you will
not partition your data storage. Building a unit of work in builds in
an assumption that you can have transactional consistency between two
partitions. Sometimes this is not a big deal, sometimes it is.

I think you will find though that very often you can change the
problem/boundaries so that you don't need to break aggregate
boundaries.

Another option that we have in this example is to have a saga that
handles the process of closing an account. What is the process of
closing an account? Are other things involved? The domain would
generate out the "Account Closed" event then other things would be
triggered asynchronously off of this.

Also in this particular example we can quite simply avoid the 2
aggregates. You will notice that a "closed account" is being created.
Yes the closed account is a different object but could the closed
account just be created off the event stream of the original account
(eg: it is just a different view of the same event stream?). It would
seem to me that this could be done relatively easily.

Cheers,

Greg

--
Les erreurs de grammaire et de syntaxe ont été incluses pour m'assurer
de votre attention

[CraigCav]

Looking at the particular example and problem (two aggregate roots
producing events for one use case - closing an account), my first
thought was "what are the responsibilities of each aggregate root"?.
By the looks of things, the "ClosedAccount" has no responsibilities/
behaviour, so I'd ask, what's the point?

For the moment, lets assume that a ClosedAccount had offered some
behaviour, and therefore is required in the domain.

The next clue that something seems out of place is provided by looking
at the event names:

AccountClosedEvent
ClosedAccountCreatedEvent

If I'm having a conversation with my domain experts and I said "when a
closed account is created", I'm sure I'd get a raised eyebrow or two.

I guess this would lead me to a similar place to greg's response;
closed accounts could in fact be a different view of the same event
stream. This way, no additional behaviour occurs on creation of a
ClosedAccount (we no longer create it, we just retrieve it).

If this approach was taken, we'd also no longer need to return a
"ClosedAccount" from Account.Close(), which this appears to be
currently required such that the new "ClosedAccount" can be saved in
the repository by the command handler.

Craig

[Greg Young]

I am pretty sure that the structure of that particular object comes
from a thing I talk about in my class occasionally similar to this
post http://codebetter.com/blogs/gregyoung/archive/2010/03/09/state-pattern-misuse.aspx

It is a very common pattern to model these things as different
objects. With events however these things are generally just different
views of the same event stream (its not a new aggregate for each one).

Cheers,

Greg

[seagile]

Could you elaborate ...

1. What would be in the event store? (from your explanation 1
eventstream for 1 aggregate) Is there any aggregate type information
in the eventstore? How would that affect the store if the aggregate
"changed" from one type to another (is it even logical to do that
because you'd be storing "current state" - what about replaying up to
a certain event)?
2. Does the store know anything about the other view on the aggregate?
If so, how would that affect the eventstore (e.g. how would I
represent that in a relational eventstore)?
3. How one would query for either the ClosedAccount or the
ActiveAccount? (I suppose one would do this explicitly)
4. How one would know in which one of these two states an account is?
(except for retrieving it from the read-side)

[CraigCav]

@seagile

Heres my current perspective:


1) In regards to the content of the event store, for this example, the
interesting event is the AccountClosedEvent. It contains the ID of the
Account. I don't think the event store necessarily needs the type
information of the aggregate (unless this is an implemenation detail
im not aware of?).

2) I dont think the event store _needs_ to know about the "views" on
the aggregate. AFAIK the event store will persist the raw events, and
the infrastuctural pieces pulls the _requested_ aggregate out with by
loading from the event stream/snapshots.

3) Assuming you mean retrieve by ID, I think Marks example looks like
this: GetById<TAggregate>(Guid id). If however you do in fact mean
query, then neither ClosedAccount or ActiveAccount are relevant, since
we'll be querying from the read model.

4) When retreiving an Account Aggregate by ID GetById<TAggregate>(Guid
id), TAggregate will either be ClosedAccount or ActiveAccount
depending on the events that have taken place. In either case, we need
to provide the context (the correct type of TAggregate) when calling
the repository. This context is provided by the read model.


So the way I see this playing out is something like this:

void Account.Close() called.
AccountClosedEvent raised.

At this point, the ActiveAccount aggregate is no longer of much use.
In Marks example, if further methods were invoked on this instance,
exceptions would be thrown. I actually dont think this instance needs
to be kept around. Again from marks example, an instance of
ClosedAccount is returned on calling Account.Close(). From what I can
tell though, this instance is only used by the
CloseAccountCommandHandler in saving the new object back to the
repository. Since this particular piece is no longer required (if we
have two "views" of the aggregate), we can instead return void.

I would assume then, that the next time the an action on this account
is performed, the action would take place in a specific context,
driven from a specific view of the system (for instance, after viewing
all accounts closed today, we might decide to re-activate one). In any
case, the context of whether the account is active/closed will now be
driven from the READ side, and we have this information when needing
to call GetById<ClosedAccount>(Guid id).

Does that make sense?

Craig

[seagile]

How would this affect snapshotting?
Would you presume both track the same state?

You also seem to presume the read side has to provide all the answers
regarding the current state of an account, which I find not always
feasable in light of eventual consistency. The state of the account
could be a domain only detail (admittedly probably not in this
example).
I integrate on a daily basis with systems that don't have the latest
view, so I have to make sure that at least my integration code has the
latest view. Either that or the domain. I guess the presumption is
indeed that the read model has the current state of the account. Also
as you stated rehydrating the ActiveAccount would probably set a flag
and cause it to become unusable once it was closed.

So in general I think I agree.

[seagile]

This also begs the question if an aggregate should provide state
handlers(ApplyXXX) for events that only make sense to other
"states" (e.g. Active/Closed) of the aggregate?

[CraigCav]

I'm not sure there would be a significant impact on snapshotting if we
took this approach.

Each Account is a different Aggregate in this model, and could be
retrieved as such (with its own snapshots). Continuing my assumption
that the read model can provide the context (whether we are working
with active/inactive), we can still retrieve the aggregate using its
specific snapshot.

Going back to the example (I like working with concrete scenarios), I
would imagine that since ActiveAccount offers very specific behaviour
to ClosedAccount (which can't, for example, make withdrawels/
deposits), it would require very different internal state to maintain
its invarients. I dont see the ClosedAccount requiring ledgers for its
internal state for instance, if its only behaviour is "allowing re-
activation".

I guess if we take this approach for this example, the only handlers
common to both Aggregates are likely to be the entry and exit points
for each state - "onAccountClosed" and potentially
"onAccountReActivated".

How does that sound?

I dont think the read side should be left to provide all the answers
regarding state - the aggregates in the domain will retain enough
state to maintain their invarients to ensure consistency within the
domain. As you pointed out though, it is particularly appropriate, in
this example, that the state of the account (active/closed) is exposed
to the users through the read model. That said, I don't for a minute
think that every state transition should or could be handled in the
same way.

Craig

[seagile]

@CraigCav

I'll rephrase:

What if ClosedAccount and ActiveAccount are the same aggregate (just a
different view - as stated by Greg) and thus have the same
eventstream? What would be the effect of replaying the events (image
without snapshotting for one second) on ClosedAccount or ActiveAccount
depending on the "current" state in the read model?

1. The account is active (assuming it has prior events)
2. Issue a close command on the active account
3. The command handler would query for
GetById<ActiveAccount>(command.AccountId)
4. account.Close() would be called and the ClosedAccount event would
be recorded and stored
5. Issue a re-open command on the closed account
6. The command handler would query for
GetById<ClosedAccount>(command.AccountId). I'm making the assumption
we lost state of the aggregate on the server and need to rebuild from
the eventstore.
The closed account object will have to replay events that have no
meaning to it (still no snapshotting). One could just ignore the
events of no meaning and only apply those that do have meaning. Is
this the way to go?

Now imagine with snapshotting ...
Unless the snapshot is readable by both (and probably ignorable by
one), I don't see how I can deal with this.

Rhetorical question: What are the dangers of not having a state
handler for an event and just skip it? How do you differentiate
between a forgotten one and a skipped one? I would answer: the tests.

[CraigCav]

I see ClosedAccount and ActiveAccount providing different roles in the
system, which is why I was inferring that they were distinct
Aggregates. You're right in pointing out this doesn't necessarily tie
up with what Greg said (just different views in the same aggregate).

Agree also that I can't see how snapshotting could work if
ClosedAccount needed to be loaded of a ActiveAccount Snapshot.

Craig

[Greg Young]

They are different aggregates that provide different roles but are
different views of the same Event Stream ... It can be a bit tricky in
that they have to share some invariants etc but can be a quite
efficient way to model things.

Greg

--

[seagile]

Yet aggregate's have keys and either they share the same key or they
don't. Calling GetById<TAggregate>() with different identifiers, how
is the event store to know what stream you are interested in? The
identifier identifies the stream, TAggregate the projection/extent you
want, AFAIK. Trying to get to the bottom of the mechanics here ...

[Greg Young]

Yes it stores the stream ... that's it.

With something like an account open/closed even the snapshots will
likely be the same. The objects are only behaviorally different (eg
they offer different roles).

Greg

--