Event Properties vs. Event Metadata

[Jeff Harris]

I started off doing something like this:

interface IDomainEvent(Id,AggregateId,AggregateVersion)

Similar to: https://github.com/MarkNijhof/Fohjin/blob/master/Fohjin.DDD.Example/Fohjin.DDD.Events/DomainEvent.cs

But I'm now seeing this is a bit silly as "AggregateId" and "AggregateVersion" are not really a business-level concern -- looking at my Aggregates, the Aggregate Id will *naturally* be one of the properties of the Event (e.g. PersonId, AccountId).  I guess another way of saying it is that, by looking at any Domain event, it's "source stream" can be determined by looking at the values of its properties.

So it feels like these properties might fit better in metadata.

Also, it's entirely possible that some events do correspond to an aggregate and don't change state -- so version isn't as helpful, also some events don't necessarily correspond to an Aggregate directly, so why force these weird properties into domain events.

Using something like GES, the "stream" is set by looking at the Aggregate/Event-sourced-thing we've applied these events to, so they will get saved to the correct stream, and we'll always load them back from the correct stream also. 

Or if they don't correspond to an Aggregate, events can be written explicitly into whatever stream is appropriate.

All good so far -- I'll just include that in metadata. Maybe my only property on a domain event base-class/interface is "Id".

Now for tests.

However, after watching Greg's Assert.That(We.Understand) talk -- I feel like it makes sense that a helpful level of test-abstraction is to say: 

"With this handler, given these events, when this command/event occurs, expect this series of events (or exception)"

I feel like Aggregates shouldn't be the concern of these kinds of tests -- it's irrelevant which Aggregate(s) these events apply to. I should be able to "refactor" my Aggregates as long as my events and commands stay consistent.

The issue becomes how to provide the "Given()" enumerable of events that have already occurred in such a way that the "FakeEventStore" can identify which stream they belong to.

I can think of three possible solutions:

1) I'm (as usual) over-complicating things, just include the Aggregate explicitly in the tests: e.g. https://github.com/MarkNijhof/Fohjin/tree/master/Fohjin.DDD.Example/Test.Fohjin.DDD

2) Give the domain events some (non-serialized) way to indicate which stream is their "source" based on their properties (e.g. IDomainEvent { string GetSourceStream(); }, or *groan* an Attribute (C#))

3) Add logic into the "EventBuilder" classes so that they can handle dealing with appropriate metadata -- now the tests are aware of event metadata.

How are other people handling this? 

[João Bragança]

In your OnHandle of your test you can route the messages to a Dictionary. then you have a fake repository that wraps the dictionary

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sent from my regular computer

http://twitter.com/thefringeninja

http://www.thefringeninja.com/

[Jeff Harris]

That as my original approach, when I realized that I need some way for my "Given()" method to specify which "stream" these events are associated with, so that when my handlers call "Load<>" the fake repository knows which events go with the requested aggregate.

That would work great if I had "AggregateType" and "AggregateId" as a property of my Event, but now that I'm keeping that sort of thing in metadata, I have to return something else from "Given()" other than just events, or I have to add some kind of attribute or something to a base "Event" class to know which aggregate this is from.

*or* I have to assume that these tests will *only-ever* test a single Aggregate at a time.

My current approach is to have my "Given()" method return a sort of "EventWrapper { Event, Metadata }" -- and I've modified my Aggregate base class to emit these.  This way the metadata can keep information like this event's "source stream" (using a GES-style model where we thing of a stream as being something like "aggregateType-aggregateId").

This feels a little bit heavy, but carrying this metadata around is helpful for some other infrastructure things too, I guess.

Plus keeping metadata out of events directly means that my "Expect()" logic in my tests can focus on the business logic that's expected.

Still feels a little bit wrong :-/

[João Bragança]

I misspoke earlier. You just need a list, not a dictionary if you are testing an aggregate - you should not be changing multiple aggregates in the same transaction :)

--

You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeff Harris]

Joao,

First, yes, I only modify one Aggregate (or stream for events that are not generated by an aggregate) during a *transaction*.

So I had thought of just keeping a list and requiring each test to *only* test a single stream.

But, my *hope* was to write tests at a higher level than just a single Aggregate -- actually I'm now seeing the discussion here: https://groups.google.com/forum/#!searchin/dddcqrs/testing/dddcqrs/06eLaMrZQVA/HPvHoFwuLUsJ and this is actually what I'm trying to do, I think. 

I guess I'm trying to blend the Assert.That(We.Understand) practical approach with the idea of testing at the "command handler" level, not the "aggregate" level.

If anybody has figured this out, feel free to chime in, otherwise I'll let you know where I end up.

[Greg Young]

I normally write tests at the command handler level not the aggregate
level ... in class this is exactly what we go through :)

I just deleted my 25 line framework for doing this about an hour ago
(from class) otherwise I would just put it up for you. I might get
some time over the weekend to write it up quickly for you.

Cheers,

Greg

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

Studying for the Turing test

[Philip Jander]

Hi Jeff,

as Greg, I do not test aggregates. I test a unit of functionality, which commonly means to test the result of handling a single command given a history.
As a sidenote, I usually don't limit myself to one transaction-one aggregate, see below.

Towards your original question, I tried multiple approaches in the past, two of which I still use:

a) streams are prior to events, i.e. each event has metadata of stream id and stream version. Events outside of streams do not exist. Thus for testing, I prepare the streams with the correct IDs and versions (and: each aggregate is a stream). That's likely the "mainstream" approach. There is no conceptual problem if you set up history for tests. However, a bad feeling remained for me in that I always felt that events should have a potentially longer lifespan than any software implementation, therefore longer than aggregate boundaries and therefore longer than the concept of a specific stream.

b) events are prior to streams, i.e. each event exists independent of streams. Streams are a projection of the total history given a specific filter, which operates on the event's payload. That means that in order for an event to become eglible for projection into a stream, it must contain a property with the ID of the stream-to-be.
Example:
event Customer_escalated_service_request { RequestId Request, CustomerId Customer, ServiceAgentId Agent, ...further payload... }
could (potentially) be projected using three filters (_=>_.Request==stream) (_=>_.Customer==stream) and (_=>_.Agent==stream).

Hence, an event contains only business-related information. Here, streams are a derived concept. For testing that is quite nice as you simple prepare a set of business-related events.
Upside of this approach:
 - feels 'right' in that one can defined events without having defined aggregates.
 - Two aggregates can share an event (either physically or cloned), which probably stems from a point-like interaction between those events. Note that this violates your one-TX/one-aggregate constraint.
 - But: artificial duplications of events for interactions vanish, i.e.: Student_joined_class & Class_accepted_student. Note that you are still free to create both events if modelling leads you to that. But the situation I can avoid is: problem domain modelling gives you a single event, but solution domain modelling leaves you with two aggregates and hence the requirement to have two events if the states of both aggregates change. Which in turn requires a process to coordinate the events since their emission should be coupled, only that it's not ok to emit them in a single transaction ;) This is a lot simple if I just say that two aggregates meet in a point in time to emit a single business-oriented event recording their interaction.
- This leaves room for business events that do not (yet) clearly belong to any aggregate in the sense that there is (at this time) an invariant depending on their properties, but seem too important to not capture.
- Redesigning aggregates is simplified.

Downside: stream versions are a difficult concept as a stream version will change if the stream definition changes. Vector clocks are a solution, as is semi-global ordering but both come with conceptual problems. OTOH, changing stream versions is a problem with redesigning the aggregates in a), too. Finally, I don't think that this approach is broadly used, so there are probably some problems with it yet to be discovered...

Maybe that gives you some ideas if you want to think out of the aggregate=stream=list_of_events box.

Cheers
Phil

[Jeff Harris]

Phillip, thanks for your detailed response -- this is really helpful for my thought process, and I'm still digesting it.

I'm leaning towards "a" for reasons that mostly stem out of my "one-aggregate-per-transaction" rule.

- I like the Jonathan-Oliver-Saga type of entity (and/or Projections) where there is a specific meaning and change of functionality for having an event exist in a different streams.

- Logically, events that have already occurred *must* have a source stream -- to abstract out that concept means that my tests are looking too different from the "real world" with implementations like GES.  It's too likely that my tests would miss important bugs.

- I think I need "student joined class" and "class accepted student" to be different events. 

I try to avoid this exact scenario by having only one aggregate care about this event and using the Read Model to denormalize things like lists of students that are in a class.

If the "class" really needs to protect an invariant, then I try to do something like "student requested to join class" and "class accepted student" if this kind of language makes sense in the domain.  Then it's easier to add compensating actions (e.g. "class cannot accept student")

I still need to think this through, but I think "a" makes the most sense for me.

[Jeff Harris]

Greg -- thanks for your help -- sounds like I should try to get to one of your classes :-)

[Jeff Harris]

@Phillip,

I was thinking through your thoughts and a little more about a Jonathan-Oliver-style Saga with my system: http://blog.jonathanoliver.com/cqrs-sagas-with-event-sourcing-part-ii-of-ii/ 

I'm not doing JO's approach exactly, but to me the interesting thing about his "Sagas" is that their state is determined by replaying events, but they can also change state by consuming events that they do not originate.

Thinking about it in a GES-way -- they can consume events from other streams and put those events into their own stream.

The logical effect of this is not that the event "occurred" again, but that the event was now "recorded" by another stream -- so we now know that this "JO-Saga" entity has now taken the appropriate action after witnessing this event.

I'm curious to dive into more details about Persistent Subscriptions in GES and how that enables message-bus-like functionality via the EventStore directly.

I think I can now get a lot of usefulness by capturing the "Source Stream" of my Events.  Essentially, the only events that need to be considered as "dispatchable" are events written to their own source streams.  These same events may exist in other streams, but when they are written there, they do not need to be dispatched on the bus.

I think, if I'm okay with at-least-once Command-invocation, an IOC container, and a tiny bit of external data to get GES working as a message bus using Catch-Up subscriptions.

All I'd need would be a Projection that takes all events that have been written to streams for which they are the source.

Granted -- a Bus would need to process Commands as well, but any Command can be written as an event (e.g. append "Requested" to the end).

Or -- I could just write a wrapper event "CommandDispatchRequested(command)".

Still pondering the side-effects of this way of thinking -- but it feels intruiging :-)

Thanks again for your sharing some of your previous approaches.

[@yreynhout]

Shivers down my spine ... you may wanna look at AggregateSource.Testing and steal the bits that fit your needs.

[Greg Young]

"That as my original approach, when I realized that I need some way
for my "Given()" method to specify which "stream" these events are
associated with, so that when my handlers call "Load<>" the fake
repository knows which events go with the requested aggregate."

Why not have Given() return the events?

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--

[Jeff Harris]

@yves: shivers down my spine as well -- i felt like i was missing something here.

I hadn't seen your project previously, it looks like you may have already solved the issue I'm trying to figure out here in a much simpler way.

Usually I think of the over-complicated solutions first, and it takes me a while to figure out something simpler.

[Jeff Harris]

@yves -- Okay, so if I'm understanding you're approach to tests that are not directly referencing aggregates, you are:

- requiring that the AggregateId be supplied with each "Given()"  to compose a "Fact"

- taking advantage of the each Aggregate's Id being unique (at least within a single test) so that the system knows which events need to be replayed to which Aggregates

Also, I don't see anything in your framework that is both (a) an event-sourced thing and (b) is something other than an Aggregate. I was wondering if you have found that that you haven't needed event-sourced process-managers/sagas?

@greg -- I was trying to just return *only* events from my "Given()" -- the issue was getting my "FakeEventStore" to replay events back to the correct Aggregate for scenarios that involve more than one Aggregate.

My current solution is to return Event + Metadata from my "Given()" -- which I think is essentially what @yves is doing with the "Fact" class (although his is a single field, not a whole set of metadata).

[@yreynhout]

The stream name/aggregate identifier (depends on the perspective) composed together with the event itself is indeed what I call a Fact in AS. The how is less important: an interface, a base class, an attribute, explicit style (what AS.Testing embraces) are all options, each with different trade offs and personal levels of "what feels right". As a library, I didn't want more coupling than the current IAggregateRootEntity, hence why I tend to prefer POCO events, instead of the alternatives. Now, the fluent test specification authoring syntax in AS.Testing is more about keeping the narrative of a scenario intact, i.e. allowing you to mix multiple events across the same or different aggregates, even repeating the same aggregate identifier if that fits the story better (this can be achieved in both the givens and thens). Allow me to sidestep the debate of what style of aggregate collaboration one is into here. At the end of it all, you're left with a specification of the scenario/test you need to execute. How you go about that is again a matter of preference, convenience, technical constraints even, etc ... Your analysis of bringing the right set of events to a specific aggregate using the Fact (AggregateId, Event tuple) is spot on.
AS is not about process managers, it never will be, that's a deliberate choice. I'm building something else to fill that gap, but it's too early.
Do I not use process managers? Not a lot, no. Most of the time I get by with simple reactive behavior but that's probably because the systems I work on have no inherent use for coordination (well, not a lot). Yet, at times I do feel the need ... when and if, custom code has been the answer most of the time (as said I'm working on getting rid of the word custom).