About correlation ids

[Danil Suits]

Quick review, looking for affirmation that I'm on the right track, or the swift stick of correction.

There seems to be a conflict in the lessons I've taken from managing correlation ids.  One is that the correlation id of the cause is copied to the correlation id of the effect (ie, we take the correlation id from the command, and copy it as is into the event.

At the other end, when we start a process, we copy the id of the process manager into the correlation id of the commands that we send, so that the events that appear on the other side get delivered to the right instance.

Where I get tangled: I had picked up the idea, somewhere, that the correlation id is always copied from one message to the next.  Which would introduce a conflict if there already was one present.  For instance, if a command from outside the domain arrives with a correlation id, which is copied by the persistence layer into the metadata of the events generated by the aggregate.  Now, if one of those events is supposed to start a process, I need to wipe out the old correlation id to replace it with the saga id.

My conclusion is: yup, that's fine -- every time we start a process, the correlation id changes.   The justification being that each of the processes is a distinct conversation that the remote client is not a part of. Plus the convenience that I don't have to worry when the events in one process start launching additional processes of their own.  Each can row merrily along without interfering with the others.

[Greg Young]

Correlation id and causation id are simple. They work as follows.

Every message has 3 ids, MessageId, CorrelationId, CausationId. When
you are responding to a message (either a command or and event) you
copy the correlationId of the message you are responding to to your
message. The CausationId of your message is the MessageId of the
message you are responding to.

Greg

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

[Danil Suits]

A workflow

Command A arrives(this could be from the client, or from an internal processmanager)
A.messageId = 1
A.aggregateId = 2
A.causationId = 3
A.correlationId = 4

aggregate 2 emits event B in response
B.messageId = 5
B.aggregateId = 2
B.causationId = 1
B.correlationId = 4

this is observed by the event handler, who in this example creates a new process instance, which records its own history with an event C
C.messageId = 6
C.sagaId = 7
C.causationId = 5
C.correlationId = 4

And issues a command D (to a different aggregate, referenced in the payload of event B).

D.messageId =8
D.aggregateId = 9
D.causationId = 5? 6?
D.correlationId = 4

Which in turn emits another event E

E.messageId = 10
E.aggegateId = 9
E.causationId = 8
E.correlationId = 4

How does the event handler arrange that event E(10,9,8,4) gets passed to the saga that was created when we processed B(5,2,1,4)?  How do we keep the history of the two together?  Or more carefully, how do we keep this history together, without also introducing the constraint that there's only one process involved in any given message?

The only way to do it that I've been able to suss out is changing D.correlationId -- the asynchronous command D is not "responding to" either the event B or event C, or by leaving correlationId as is, and injecting a new piece of metadata to keep the wheels turning.

[Greg Young]

First off get rid of "saga id" and "aggregate id" they are meaningless
here. I have done that below.

"this is observed by the event handler, who in this example creates a
new process instance, which records its own history with an event C
C.messageId = 6

C.causationId = 5
C.correlationId = 4

And issues a command D (to a different aggregate, referenced in the
payload of event B).

D.messageId =8

D.causationId = 5? 6?
D.correlationId = 4

Which in turn emits another event E

E.messageId = 10

E.causationId = 8
E.correlationId = 4:



D.causationId = 6 (the message its responding to)

"How does the event handler arrange that event E(10,9,8,4) gets passed
to the saga that was created when we processed B(5,2,1,4)? How do we
keep the history of the two together? Or more carefully, how do we
keep this history together, without also introducing the constraint
that there's only one process involved in any given message?"

The process manager simply subscribes to correlation id. Note every
event here will have the same correlation id. As for only one process
per correlationid can you come up with any sane reason why you would
want multiple processes concurrently operating on a given
correlationid?

[Danil Suits]

"As for only one process per correlationid can you come up with any sane reason why you would want multiple processes concurrently operating on a given correlationid?"

Because I want to decompose a monolithic process into a number of smaller, focused processes, which can possibly be composed in other ways?

Because I don't want to load the entire pile of fallen dominoes to handle an event whose effects are local?

Isn't this analogous to why we factor a bounded context into multiple aggregates? What don't I get that makes it different?

[Kijana Woodard]

"...multiple processes concurrently..."

I think your description is still one [overall] process or multiple processes serially...depending on the definition of the overloaded word "process".

[Greg Young]

Multiple concurrent processes would be both fun to rationalize about
and maintain!

[Greg Young]

See sub-process.

You don't have n processes concurrently but pass ownership

[Newion]

On Tuesday, January 12, 2016 at 3:35:07 PM UTC+1, Greg Young wrote:

The process manager simply subscribes to correlation id. Note every
event here will have the same correlation id. As for only one process
per correlationid can you come up with any sane reason why you would
want multiple processes concurrently operating on a given
correlationid?

I think the Danil's question is different. 

**Given:** 

- event E1 is raised in domain D1

- E1 triggers process P2 in domain D2

- P2 issues a command C2 that results in E2 being raised (in D2)

that results in 

- E1 message is assigned P2Id correlationId (in domain D2),

- C2 message is assigned P2Id correlationId (in domain D2),

**When:**

- E2 triggers process P3 in domain D3

**Then:**

- E2 message must be assigned P3Id correlationId 

- Since E2 message is already assigned P2Id correlationId (assigned in domain D2), should P2Id collerationId be preserved somehow (to be able to track events participating in process chain) ?

[Greg Young]

If you just follow the rules above you will have correlation id working here.



E1 msgid 5 corrid 17 causeid -1
C2 msgid 6 corrid 17 causeid 5
E2 msgid 7 corrid 17 causeid 6

- E2 message must be assigned P3Id correlationId

Think about correlation id/subscription as being a chat room about a
topic. You can at any point open up a new chat room about a different
topic if you so want.

[Fredrik Wiström]

Would an incoming command from the UI have a CausationId?

[Greg Young]

what caused it? in the example above some have causation ids some don't

On Wed, Jan 13, 2016 at 12:27 PM, Fredrik Wiström
<wistrom...@gmail.com> wrote:
> Would an incoming command from the UI have a CausationId?
>

[Simon Harris]

Nice. Let's say some part of a business process requires input from a user or another system, do you typically then copy the correlation ID to/from the UI? Or would you accept this as effectively the start of another "topic" in your analogy?