"Out-of-band" information

[ahjohannessen]

Hi All :)

I am very curious how eventsourced users deal with "out-of-band" information that might be relevant to an application, 

but not necessarily the domain (processors). This can be metadata like:

• UserId 
  
• IP-Address
  
• CommandSource
  
• CorrelationId
  
• CausationId
  
• You name it...

that is helpful when determining things like:

• What caused a particular event? 
  
• What originally caused this event? 
  
• What other commands have been triggered in response to this event? 
  
• Who triggered this command? 
  
• When did they trigger this command?

which is very useful for all sorts of analytics and keeping operational sanity. Would it be useful to introduce

• headers : Map[String, String] // or similar
  

to eventsourced Message type?

[Duncan DeVore]

Hi 

In our system we generally just keep this as part of the message itself.

Cheers,

The preceding email message may contain confidential information of Viridity Energy, Inc. It is not intended for transmission to, or receipt by, any unauthorized persons. If you have received this message in error, please (i) do not read it, (ii) reply to the sender that you received the message in error, and (iii) erase or destroy the message.

[Martin Krasser]

We also keep this as part of the Message payload. I'm not sure if a general header field should be introduced as the eventsourced Message type represents a journaling/persitence protocol rather than one for general-purpose messaging.

--
You received this message because you are subscribed to the Google Groups "Eventsourced User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eventsourced...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ahjohannessen]

I see. 

I can think of some ways to introduce this on the payload:

 - Explicit fields on the payload itself.

   => This is easy until you get 100s of messages and explosion of right to left copying in order to ensure metadata is passed on. 

 - Explicit fields on some base trait.

   => Hard to maintain and change, pollutes payload with one context to rule 'em all.

 - Headers field on a base trait that payload extends.

   => Flexible, not type safe and needs some sort of extractors. This is similar to having it on Message, but is brittle when doing stuff like msg.copy(evt = differentPayload).

        You have to remember doing the right sort of gymnastics all over the place.  

and then there is the option of:

 - Envelope that contains headers field and payload

   => This means that we need to maintain our own receiver of sorts that wraps / unwraps and can drift from how eventsourced itself works.

        One additional level of indirection is a bit awkward in my book. 

We have 100s of commands and events in just *one* of our bounded contexts and this is a pretty common requirement. 

That is why I was thinking about how eventsourced itself can make life easier for us developers.

When you guys say that you keep that information on the payload itself, what kind of data is this and is it in a header field?

[ahjohannessen]

I am probably over thinking this and a combo of correlationId and causationId would probably provide a way to build a trace graph and connect the dots in most scenarios.

However, I think it is an interesting problem in the context of actors and eventsourced. I believe Finagle is more capable than Akka in this regard.

[Duncan DeVore]

+1 on Martin's comment.

Why not just use your Map[String, String] as one of the attributes on your commands/events? This way you can add new out-of-bound attributes and you won't have to deal w/ command/event versioning. 

[ahjohannessen]

Mostly because this troubles me:

1) Brittleness: msg.copy(evt = newPayload) => oops, forgot to copy over headers.

2) Repetition: emit(SomeEvent(headers = cmd.headers, args*))

Being required to remember repeating this sort of gymnastics all over the place seems a bit error prone.

I understand the viewpoint of Martin, but I wish the toolkit could help me in some way.

[Martin Krasser]

On 22.08.13 12:00, ahjohannessen wrote:
> I see.
>
> I can think of some ways to introduce this on the payload:
>
> - Explicit fields on the payload itself.
>
> => This is easy until you get 100s of messages and explosion of
> right to left copying in order to ensure metadata is passed on.
>
> - Explicit fields on some base trait.
>
> => Hard to maintain and change, pollutes payload with one context
> to rule 'em all.
>
> - Headers field on a base trait that payload extends.
>
> => Flexible, not type safe and needs some sort of extractors. This
> is similar to having it on Message, but is brittle when doing stuff
> like msg.copy(evt = differentPayload).
> You have to remember doing the right sort of gymnastics all
> over the place.
>
> and then there is the option of:
>
> - Envelope that contains headers field and payload

That's what I'd recommend. Use your own envelope as payload of Message.
That's a clean way of extending a protocol stack and it even allows you
to use your own application-specific protocol indepent of Message.

>
> => This means that we need to maintain our own receiver of sorts
> that wraps / unwraps

That's one additional (application-specific) stackable trait in addition
to those provided by eventsourced. This allows you to keep header-copy
code in a single place and compose this functionality with eventsourced.
You could even use this additional stackable trait independent of
eventsourced (again), as a result of proper separating concerns.

Summary:

- define your own application protocol and layer it on existing protocol
stacks where needed
- deal with your own application protocol by using stackable traits on
the endpoints (actors)

> and can drift from how eventsourced itself works.
> One additional level of indirection is a bit awkward in my book.
>
>
> We have 100s of commands and events in just *one* of our bounded
> contexts and this is a pretty common requirement.
>
> That is why I was thinking about how eventsourced itself can make life
> easier for us developers.
>
> When you guys say that you keep that information on the payload
> itself, what kind of data is this and is it in a header field?
>

--
Martin Krasser

blog: http://krasserm.blogspot.com
code: http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

[ahjohannessen]

Martin, after considering various approaches I come to the same conclusion as you.

Thanks for your input guys :)