Question on FAQ : When should I enrich my events?

[Derick Schoonbee]

I've been adding some questions on the FAQ (with some help from others
- thanks) and would like to involve the community to resolve following
question:

-----
When should I enrich my events?

Q: I have an event with only a reference to some data. For example
ProductId in OrderLineItemAddedEvent. How and when should I resolve/
get the ProductDetailedDescription when I need to populate the read
model?
----
(See http://cqrs-faq.wikispaces.com/)

If you want to say "depends on the domain" or "this is not a real
world example"... please add some details to the scenario to explain
and make the question/scenario fit your requirements ;)

We can either discuss it here to be put on the wiki later, or you can
update the wiki... it's public.. no need to register. And oh, while
you're at it.. why not try to answer some other questions.

[Derick Schoonbee]

I do not have CQRS experience to give some expert advice but would
like to discuss it. Here is the question:

When should I enrich my events

Q: I have an event with only a reference to some data. For
example ProductId in OrderLineItemAddedEvent. How and when should I

resolve/get the ProductDetailedDescription when I need to populate the
read model? 

Approaches:
1) Enrich when publishing / persisting to the EventStore. Thus, add
the description property to the event and fill it when publishing.
2) When handling the event use the read model / denormalizer also.

In any case here is my braindump:

For 1 you need to find where to enrich. Keeping in mind that your
domain originates the events first, enriching before is not an option
since the event does not exist yet. I think the domain code is not
such a good option in THIS case (not a domain concern).

Also, you would not want to persist the enriched event since your
handler does not make use when applying / replaying. Except if your
domain code enriches the event internally. (Good?)

I was thinking of putting some enrich code in the event itself but we
need to keep the event class “dumb” or “DTO like” so maybe an
EventEnricher role but where to we call it? Domain or when
dispatching?

Also, if you enrich too early the message size might get to big for
your infrastructure to handle (I think I saw some cloud message size
limitations somewhere). Thus, we might have to go finer grained... is
that good?

For 2 I thinks it's easy to implement since you can access the read
store(s) to get what you want.

Any comments / experiences?

On Mar 7, 11:33 am, Derick Schoonbee <derick.schoon...@gmail.com>
wrote:

> I've been adding some questions on the FAQ (with some help from others
> - thanks) and would like to involve the community to resolve following
> question:
>
> -----
> When should I enrich my events?
>
> Q: I have an event with only a reference to some data. For example
> ProductId in OrderLineItemAddedEvent. How and when should I resolve/
> get the ProductDetailedDescription when I need to populate the read
> model?
> ----

>  (Seehttp://cqrs-faq.wikispaces.com/)

[@seagile]

I'd say try to stick as much as possible with option 1. It's very much
a "domain" concern in the sense that *meaningful* events should come
out of your domain. But you do touch upon a valid point: events
contain data to bring an aggregate back to its current state as well
as data that is meaningfull to event consumers. Don't try to separate
those concerns though. Option 2 is only valid if the data from the
read model is "good enough". Even then I'd question why the domain is
not emitting that "enriched" form of the events. How can you say
something happened and then add more data after the fact - to me
events are immutable. The only reason I can think of is that you want
to provide more context, but maybe that's a sign to start looking at
language.

[Simon Harris]

On Mar 9, 8:03 pm, "@seagile" <yves.reynh...@gmail.com> wrote:

> I'd say try to stick as much as possible with option 1. It's very much
> a "domain" concern in the sense that *meaningful* events should come
> out of your domain. But you do touch upon a valid point: events
> contain data to bring an aggregate back to its current state as well
> as data that is meaningfull to event consumers. Don't try to separate
> those concerns though. Option 2 is only valid if the data from the
> read model is "good enough". Even then I'd question why the domain is
> not emitting that "enriched" form of the events. How can you say
> something happened and then add more data after the fact - to me
> events are immutable. The only reason I can think of is that you want
> to provide more context, but maybe that's a sign to start looking at
> language.

My issues with option 1 (and I freely admit it comes out of a position
of ignorance more than anything) are:

1. I won't always know in advance what I might need in the read model,
and
2. Even if I did know, now my write model is coupled in some ways to
what my read model wants which almost defeats part of the beauty of
the approach.

Thoughts?

[Nuno Lopes]

What happens to the Order if the Description of the Product Changes? Even after the Order is payed and shipped?

Has for detailed description I'm not sure what it means. Is it the full description in the Product Catalogue?

If that is the case and maybe a simple link to another view is all that is necessary to be presented in the UI. If that is not the case you have two options:

1) Shared a product view within the Order BC with the Product Catalogue BC
2) Have the UI handling the Mashup between a view in the Product Catalogue BC and the Order BC

> Also, you would not want to persist the enriched event since your
> handler does not make use when applying / replaying. Except if your
> domain code enriches the event internally. (Good?)

Right.

Cheers,

Nuno

[Derick Schoonbee]

Thanks for your inputs, both beginners and experts are welcome to jump
in :) Great stuff!

In terms of the question “When should I enrich my events” it seems
that @seagile (and Nuno) are pointing to publishing “rich events” from
the domain itself... (right?) I think Rinat (and Greg?) also has this
opinion:
http://stackoverflow.com/questions/4064067/event-sourcing-and-read-model-generation

@seagile:

> It's very much a "domain" concern in the sense that *meaningful* events should come out of your domain.

Meaningful to whom? Is it a requirement for the domain to publish
events that are *meaningful*:
- Meaningful within the domain? “skinny” events (see below)
- To the read model? This would require “fatter” events.
- What about other bounded contexts? “Way fatter events”
- The business? I guess the do not care at this stage since they'll be
looking at the read model(s)

Consider this: On the write side we have business rules keeping the
domain in a valid state. Domain Events are mainly used to track state
changes so that we can rebuild the *Aggregate Root's state*. So my
question is: Why do we add data/state concerns in our domain when it's
not used in validating state. Is this not a “read side” or
“integration concern”?

In this example the OrderLine does not care about the
ProductDetailDescription since it does not need it to validate the
state. If I would to go for “rich/fat events” I can then have
ProductId, ProductDetailDescription, ProductUnitOfMeasure,
ProductBatchNumber, ProductExpiryDate … etc. The list can go on and
on, imagine when I have multiple views needing different pieces of
product or order line information. Or should one add this type of
“fluff” and not concern ourselves with our aggregates getting “too
fat” or becoming “Manager/God objects”?

(I'm just wondering about practical approaches / experiences , no
rambling intended ;)

@Nuno

> What happens to the Order if the Description of the Product Changes? Even after the Order is payed and shipped?

Ok, I want to keep the discussion as general as possible to cover the
concepts / best practices with CQRS without delving *too deep* into
the domain. I think we should stick to “technical concerns” such as
Event Store, Read Model, Write Model etc. Thus let's see how far we
can go with this to be able to derive some answers/discussions to the
question. I know that there's some dangers with simple examples as
opposed to real world scenarios but we need small chunks to digest...

So I ran to the business and asked them ;) They said: If Shipped,
order stays unchanged. If In Progress, update the order line with the
new description (which comes from the Procuct Catalog).

I'm sorry Nuno, but the question is about *when* to enrich the Event
(objects). In this case when populating the Read Model. Where do I
code this feature. From what I can see you are referring the views
that are already populated with “enriched data”. How / where do you
enrich that? In the example how/when/where did you get
ProductDetailDescription?

@Simon:

> I won't always know in advance what I might need in the read model

I guess the fact the views / read model can be rebuilt from scratch
addresses that concern. The question still remains when: On rebuild/
handling the event or publishing or somewhere else.

> Now my write model is coupled in some ways to what my read model
Ok, here we are touching on another FAQ that I would like to start
soon ;) “Can my domain make use of data from my read model?” I guess
one workaround / approach to remove read model coupling is here:
http://seabites.wordpress.com/2010/10/31/guards-and-queries-in-the-domain-model/

PS: I'll try to compile some conclusions / summary at one stage in the
FAQ...

[Nuno Lopes]

Hi,

> In terms of the question “When should I enrich my events” it seems
> that @seagile (and Nuno) are pointing to publishing “rich events” from
> the domain itself... (right?)

No, that is not my opinion :) At least that is not what I wanted to write.

Cheers,

Nuno

[Nuno Lopes]

Hi,

In terms of the question “When should I enrich my events” it seems
that @seagile (and Nuno) are pointing to publishing “rich events” from
the domain itself... (right?) I think Rinat (and Greg?) also has this
opinion:
http://stackoverflow.com/questions/4064067/event-sourcing-and-read-model-generation

On the page you linked to ....

If the data you need is already part of the Aggregate publishing the event it is trivial to put that data in the event.

What Aggregate published the QuestionAsked event? If one exists and it contains the username, then it is a simple matter of including the username in the event as advised by Rinat.

If the Aggregate publishing the event does not contain the data then ...

On the premiss that you already technically how to have any piece of software construct denormalizers subscribe to events including.

Suppose that the Product Catalogue BC publishes events around product information (say ProductDetailAdded, ProductDetailChanged etc). You can have your views subscribing to any events, there is no technical limitation on that.

Suppose your Product Catalogue is in some legacy system that can't publish events:

- Does it expose a data view with the product detail(say a database view)? If so why not query it?

- Can it expose a web service? Why not have the UI mashup information on the view with information service by by the WebService?

- If everything fails why not have some DTS to import that information periodically into your view?

There are several solutions for enriching views with data outside a Bounded Context, without enriching events.

You see, the UI can send commands to multiple BCs and an query multiple BC to make the screen.

Does this help?

Nuno

[@seagile]

> 1. I won't always know in advance what I might need in the read model,
> and

You have bigger problems if you're designing a feature and don't know
what the corresponding commands and events are.
That is not say that you should be clairvoyant :)

> 2. Even if I did know, now my write model is coupled in some ways to
> what my read model wants which almost defeats part of the beauty of
> the approach.

I don't design systems out of beauty or love for SOC. I strive to not
muddle the write model with concerns that don't belong there.
But I don't consider providing context to my event subscribers to be
muddling the write model. It's highly debatable how far you should
take this.

The core problem here is that we are trying to discuss this from an
abstract POV (without a concrete context). It will fail miserably.

[@seagile]

I get the feeling you want an answer to "How 'big' should my events
be?". Honestly, I can't give an absolute answer to that. I doubt
anyone can beyond the standard "It depends".
I'll give you some examples where I made deliberate choices about
enriching events
- An appointment is rescheduled from one schedule to another. The
resulting event AppointmentRescheduledEvent contains both the schedule
I rescheduled from and the schedule I rescheduled to. It makes it
easier on my consumers (they don't have to figure out where it was
rescheduled from by querying their own state (if they keep any state
at all)). Because the appointment aggregate knew in which schedule it
was before the reschedule command, it was very easy to enrich. Do I
use the AppointmentRescheduledEvent.FromSchedule property when
restoring (replaying) the state of my appointment aggregate? No. So
what?
- An end user fixes a typo in the name of a patient. The resulting
event PatientNameTypoFixedEvent contains the fixed name. Do I use the
PatientNameTypoFixedEvent or any of its properties to restore the
patient aggregate's state? No. So what? I can tell you that some of my
event consumers are very much interested in this event (UI, HL7,
reporting, services etc...) while others couldn't care less (slot
search). Event conflict resolution does not allow two users to do this
concurrently though (in the sense of optimistic concurrency - Gregory
Young's second video).
- Each time a command is executed on an appointment aggregate the
resulting event contains a "soft" reference (an identifier of some
sort) to the schedule, specialty, location, site, patient of the
appointment. Do I reuse these when restoring appointment aggregate
state? Depends on the type of event really. If an event "changed" one
or more or the aforementioned attributes then yes, those are used to
restore state. Do I find it confusing at times that there's no clear
difference between what is innate to the event and what is enriched?
Sometimes.

I try to solve as much as possible of the event enrichment using the
domain. Any other way makes it too complex for my poor head. All these
questions start popping up in my brain: will the data I want to enrich
the event with be there? will I issue a new event? does it have the
same identity? how do I know what caused that new event from a
traceability perspective? what am I going to call that new enriched
event (feels awkward wanting to say the same thing but have two
different "models" for it)? ... and I don't feel like answering them.
Seriously, try option 1 before option 2. That's my advice.

The other thing you are touching upon is 'Data Inheritance' and how to
handle it in DDD+CQRS. Anything to make that aggregate self-contained
and be able to get it's job done during it's life-cycle. That could
mean copying some state from another aggregate when it's constructed
or altered (bad choice of words - insert name of your command here).
It could also mean that we should send a command to the aggregate to
update some information inside of it. View(model)s on the other hand
should just consume all necessary events and be able to figure out
what to update. So product id and price at the time of order would
probably be enough for the order aggregate (write side). If the order
viewmodel (read side) wants to expand that product into many more
attributes, then these could be copied over from another table in the
read model (assuming relational here). If an aspect of a product
changes, it's highly contextual which orders in the order viewmodel
get affected by that (e.g. only future outstanding orders). Mind you
the product aspect change (represented by an event) could also cause a
command to be sent to the order aggregate if decisions need to be
based on it further on the life-cycle of the order aggregate. Again,
there's no one size fits all. Talk to your domain experts and business
process analysts, and pray they know what they're talking about. Stop
focusing on the abstract.
Anyways, I don't feel very comfortable talking about Products and
Orders because quiet frankly it's not really a domain I know a whole
lot about. Merely speculation on my behalf of how things should
work :-)

On 9 mrt, 15:17, Derick Schoonbee <derick.schoon...@gmail.com> wrote:
> Thanks for your inputs, both beginners and experts are welcome to jump
> in :) Great stuff!
>
> In terms of the question “When should I enrich my events” it seems
> that @seagile (and Nuno) are pointing to publishing “rich events” from
> the domain itself... (right?) I think Rinat (and Greg?) also has this
> opinion:

>  http://stackoverflow.com/questions/4064067/event-sourcing-and-read-mo...

> one workaround / approach to remove read model coupling is here:http://seabites.wordpress.com/2010/10/31/guards-and-queries-in-the-do...

[Ramin]

You forgot Option 3) Have the command handler query the read model for
the ProductDetailedDescription and send that information inside the
method call to the domain along with the other information such as the
ProductId.

Option 4) would be to include the ProductDetailedDescription in the
command sent by the UI and have the command handler simply pass that
information through to the domain, but I think that would only apply
if the UI can be trusted.

In Option 1) your domain would have to query the read model, in Option
2) you would modify your event after it got published by the domain. I
must admit that I am used to doing it "the Option 1 way" myself, but
from things I have read over the time, it seems that many apply Option
3), and some even apply Option 4).

Oops, I forgot Option 5). Use temporal objects. Have the command
handler query the read model for the current temporal object for the
ProductId and send that temporal object into the domain. The temporal
object is immutable, meaning that if the ProductDetailedDescription
changes, a new temporal object will be added for that ProductId. If
your event includes the temporal object ID, you basically included an
immutable ProductDetailedDescription.

enjoy,
Ramin

On 8 Mrz., 13:20, Derick Schoonbee <derick.schoon...@gmail.com> wrote:
> I do not have CQRS experience to give some expert advice but would
> like to discuss it.  Here is the question:
>
> When should I enrich my events
>
> Q: I have an event with only a reference to some data. For
> example ProductId in OrderLineItemAddedEvent. How and when should I
> resolve/get the ProductDetailedDescription when I need to populate the
> read model? 
>
> Approaches:
> 1) Enrich when publishing / persisting to the EventStore. Thus, add
> the description property to the event and fill it when publishing.
> 2) When handling the event use the read model / denormalizer also.
>
> In any case here is my braindump:
>
> For 1 you need to find where to enrich. Keeping in mind that your
> domain originates the events first, enriching before is not an option
> since the event does not exist yet. I think the domain code is not
> such a good option in THIS case (not a domain concern).
>

> Also, you would not want to persist the enriched event since yourhandlerdoes not make use when applying / replaying. Except if your

[Ramin]

So you have many options on "how" to get hold of the information with
which to enrich the events (I think I was wrong about temporal objects
being immutable, so ignore Option 5).

The second part of the question now is "what" to include in the
events. As Yves said, that is impossible to discuss with an abstract
example. This can only be answered by the domain experts. If the
domain expert sais that the ProductDetailedDescription is an immutable
part of the order line, then he is telling you that that information
has to be part of the event.

So there's your rule: include information in your event if the
ubiquitous language tells you so. You forgot to incude something?
Include it in a new version of your event (using domain event
versioning). If the read model requires information the domain doesn't
need? Well, the read model is part of the domain and the ubiquitous
language, so if the read model needs it, include it.

Just my 2 cents,
enjoy,
Ramin

[Scott Reynolds]

We do a mix of mass as they are called on the UI for writing information like who changed what (the username etc) and option 5 adding data at the command handler leveell. For us the command is a kind of coordinator validation additional rules and enriching the command for the domain.

We regularly publish events from the domains we don't really use in the domain at this time.

[Tom Janssens]

You should not enrich your events; you should enrich your commands in the command handler, and pass the stuff on to the events.

[Ramin]

So we are in agreement on Option 3 I threw in (post #11)? If not, how
not? If yes, good, I finally got something right ;-)

[@yreynhout]

That's a matter of security/trust.

[Tom Janssens]

Yes, afaik there is no other way in CQRS; here is some (pseudo) code that clarifies how CQRS works IMO... 

(For the ones who have suggestions: forking a gist is easy ;-) )

https://gist.github.com/956750

//---><8--------------------------- UI loads viewmodels and generates commands ----

UI.Viewmodel = ViewModelStore.Query(UI.SomeParameters);
UI.HandleInput = GeneratedCommand => CommandStore.Add(GeneratedCommand);

//---><8--------------------------- Commands to events -----------------------------------------

// This uses an iterator because a command should only be handled one time

while(CommandStore.HasCommands) 
{
  var cmd = CommandStore.Next();
  var cmdhandler = CommandHandlers.GetHandlerForCommand(cmd);
  cmdhandler.DecorateCommand(cmd); 
  var AR = ARStore.CreateOrLoad(cmdhandler.GetARInfo(cmd));
  cmdhandler.HandleCommand(AR,cmd);
  EventStore.AddRange(AR.NewEvents);
}
//---><8--------------------------- Events to viewmodel/AR updates or commands----------

// this uses a pointer instead of an iterator since events can be handled multiple times

while(EventStore.HasNewEvents(ref someeventpointer))
{
  var evt = EventStore.Next(ref someeventpointer);
  foreach (var eventhandler in ARHandlers.GetHandlersForEvent(evt))
  {
    var AR = ARStore.CreateOrLoad(eventhandler.GetARInfo(evt));
    eventhandler.ProcessEvent(AR,evt);
    if (AR.HasChanges)
      ARStore.SaveChanges(AR);
  }
  foreach (var eventhandler in SagaHandlers.GetHandlersForEvent(evt))
  {
    var saga = SagaStore.CreateOrLoad(eventhandler.GetSagaInfo(evt)));
    var cmd = eventhandler.ProcessEvent(saga,evt);
    if (cmd!=null) 
      CommandStore.Add(cmd);
    if (saga.IsEOL)
      SagaStore.Purge(saga);
    else if (saga.HasChanges)
      SagaStore.SaveChanges(saga);
  }
  foreach (var eventhandler in ViewModelHandlers.GetHandlersForEvent(evt))
  {
    var viewmodel = ViewModelStore.CreateOrLoad(eventhandler.GetViewModelInfo(evt));
    var ischanged = eventhandler.ProcessEvent(viewmodel,evt);
    if (viewmodel.HasChanges)
      ViewModelStore.SaveChanges(viewmodel);
  }
}

//---><8---------------------------

[Tom Janssens]

I forgot timeouts on saga's and modified the gist; this is the extra code:

//---><8--------------------------- Time triggers for sagas ----------
while (SagaStore.HasOverDueSagas())
{
  var saga = SagaStore.NextOverdueSaga();
  var overduehandler = SagaHandlers.GetHandlerForOverdue(saga);
  var cmd = overduehandler.ProcessOverdue(saga);
  if (cmd!=null) 
    CommandStore.Add(cmd);
  if (saga.IsEOL)
    SagaStore.Purge(saga);
  else if (saga.HasChanges)
    SagaStore.SaveChanges(saga);

[Nuno Lopes]

What does the word "enrich" means in this context?

Cheers,

Nuno

[Nuno Lopes]

> Q: I have an event with only a reference to some data. For
> example ProductId in OrderLineItemAddedEvent. How and when should I
> resolve/get the ProductDetailedDescription when I need to populate the
> read model?

You need to know what defines something to be an Order in business terms. If an Purchase Order line is defined as (ProducRef, Name, base price, ....) that is what should be. So all this information is passed when the user declares the products he/she wants to order.

Notice, your are not enriching anything. You just stating what it is, not more, not less. If someone changes the description of the product in the order for Ferrari G40 to Potatoes someone would find incomprehensible that that have order a $300.000 bag of potatoes.

> 1) Enrich when publishing / persisting to the EventStore. Thus, add
> the description property to the event and fill it when publishing.

No

> 2) When handling the event use the read model / denormalizer also.

No

Cheers,

Nuno

[Nuno Lopes]

Hi,

> I have an event with only a reference to some data. For
> example ProductId in OrderLineItemAddedEvent. How and when should I
> resolve/get the ProductDetailedDescription when I need to populate the
> read model?

Has for detailed description, it can be a simple link to the UI to some other web page with the full product description.

Cheers,

Nuno

[Nuno Lopes]

Sorry for four posts in a roll.

If you where to archive an Order what elements should be there? These are the elements that make up Order as such be in part of its definition.

On an another note. This make sound strange ... an Order is not an Order until placed :) So probably you should not have methods such AddOrderLine and so on.

Instead, use a separate concept that Builds the Order on behalf of the User. Once building process is concluded, the result is an Order.

Check the Builder Pattern (http://en.wikipedia.org/wiki/Builder_pattern).

You can have the Shopping Cart acting as the builder or some other concept.

Hope it helps,

Nuno
PS: A Plane is not a missile up until built. For instance, before being built a Plane cannot fly so how can it be a Plane before? Hope it helps.