Domain Events versioning (and preventing accidental changes)

[Damian Hickey]

This is really a development process in .net question... As those of who do event sourcing know, you can't alter a domain event's shape once it has been written. For deserialization and subsequent up-transform purposes it type is kept around forever. Over time you may have accumulate hundreds of versions. How are you managing these and what is your unit of version? Are you happy with it?

1. By type name: MyDomainEvent_v1, ..., MyDomainEvent_vN
2. By namespace: v1.MyDomainEvent,... vN.MyDomainEvent
3. By assembly: DomainEvent_v1.dll, DomainEvents_v2.dll
4. Something else?
   

If you are doing 1 and/or 2, how are you preventing accidental (resharper, developer etc) changes to 'earlier' versions in your code base? The only thing I can come up with is some sort of file signing. Is there a better way?

When adding a new version, do you make effort to hide the previous 'legacy' domain event types from your projections / event listeners?

I've search the group archives and haven't found really anything relevant. If there is, please point me :) Any insights (problems / solutions) anyone has in this area is appreciated.

- Damian

[Nils Kilden-Pedersen]

I do manual mapping. Sure it's slightly tedious, but it really doesn't take that much time and if there's any change, I get compiler errors. This also allows me to keep the name of the current event, and only have weird names (Event_v2) for deprecated events.

[@yreynhout]

Depends on your release cycle and how good you know the business. If you release every 3 or 6 months, I doubt you'll run into many versioning issues (as opposed to continuous deployment where things MIGHT be different, but again that would be like breaking your API every day/hours/minute). Saying you'll have an abundant amount of versions is like saying you don't know how to model things that happen inside that business, or that you're really bad at it. So going with 1 is not all that bad. Also, there's ways around keeping the classes around.

[@yreynhout]

Come to think of it, you could keep schema versions (xml schema as an example) around and generate code off of that. Won't prevent devs from fucking up, though. Instead of doing file signing, you could do a types checksum and add a unit test or build condition that verifies the checksum for each released version, such that the build fails if someone touches an old schema ... I'm not convinced this is all worth the effort, though. Will have to think about it some more.

[Damian Hickey]

Still doesn't quite prevent the accidental change...  Resharper's rename refactor / namspace fix / would ensure that you don't get a compiler error for instance.

I do like the idea of mapping to domain event that is shipped and can be depended on by external code (projections), but is not used as in the event stream itself.

[Greg Young]

I have put schemas into source control before and locked them. Then as
part of build process the events were generated (developers could do
it locally as well but they were .gitignored). This prevented
accidental change while allowing them to work locally. If they made an
accidental change CI would give a failure.

Does this make sense and help with your situation?

Cheers,

Greg

--
Le doute n'est pas une condition agréable, mais la certitude est absurde.

[Damian Hickey]

I am finding that domain event versioning is resistive to continuous deployment. It one tries to be agile and do incremental and iterative development, and release often, it tends to create Domain event churn and that has a cost.

As you said, it could be bad modelling, but it can also be that the model may be emergent or the business changes. I am taking a long term view of this of at least several several years (financial regulatory requirements) and trying to devise a process that will stand the test of time. Including the fact there will be other people, possibly less knowledgeable in ES, maintaining it.

I am just trying to reason a process of casting in stone a domain event shape that is hopefully easy to do with regular releases.

[Greg Young]

Have you looked at using JSON and a weaker serialization structure for
your events?

[Damian Hickey]

Chewed on it last night, I am thinking similar. If I can create a tool to create these schema 'locks' + tests easily, maybe it could get somewhere. Worth a day's spike I think.

>Won't prevent devs from fucking up...

Dev's are lazy, if the bar is raised a bit ;)

> I'm not convinced this is all worth the effort, though

Not 100% either here. It's just the business area is paranoid.

Thanks for your response.

[Damian Hickey]

I am aware of that (and am using JSON). I am not deeply knowledgeable in JSON serialization though, and would be concerned if there is any chance of ambiguity. Serializers change over time (*ahem* Json.Net & DateTime). Think I'd like to be more explicit so your original suggestion appeals to me more.

Thx for the responses :)

[Greg Young]

While this is true serializers certainly version less than events :)

[Henrik Feldt]

All my messages are .Net interfaces.

interface MyKittenJustTimeJumped { Coordinates WhereTo {get;} }
interface MyKittenJustTimeJumpedFromEarth : MyKittenJustTimeJumped { string Reporter {get;} }

that way you can let the routing mechanism for your consuming service handle the versioning, and if you have something in production that you can't change, then you simply introduce a new interface that inherits the old. Until you find that you are introducing a new concept completely, then you create a new event instead. You can then let infrastructure handle the instantiation etc.

(WRT. JSON, MT has been adding its own converter instances to the Newtonsoft.NET deserialization pipeline for some time now, so that should be fairly stable)

[@nbarnwell]

One other point is that if you're worried about the long-term, you could devise database migrations to "upgrade" old events (providing the changes were incremental changes to the actual same domain event and it's definitely appropriate) and re-store them? If there are then event types for which you have no actual events persisted for, you can clean up your codebase.

[Damian Hickey]

Yes agreed. Fowler mention's same in one of his bliki's. A migration is definitely a strategy when the event shapes diverge and/or version management becomes too much of a burden. For my case, I would see that as a maximum once-a-year operation.

[Damian Hickey]

Just to follow up on the direction I've decided to go in. Firstly, thanks to all for your insightful help and suggestions, and secondly, I think this will work for me for and my particular requirements - I'm certainly not prescribing this as 'best practice' or anything.

Anyhow, I've decided to two representations of a Domain Event, which I have called "Application Domain Event" and the other "Storage Domain Event"

Application Domain Event:

1. Is published on bus
2. Can be referenced and depended on by any part of the application / system.
   
3. Can follow OO patterns, such as DRY. i.e. sharing a type / enum with a command in a common location is permitted. Can perhaps have interfaces as Herik mentioned, but I'll probably not be doing that.
   
4. Is versioned is the standard .net application / assembly manner. Old versions are not kept around.

Storage Domain Event:

1. Is the schema format for serializing to, and deserializing from, the event store.
2. Is in a project that has no external dependencies, except for 'System'. (And maybe System.Runtime.Serialization if they need to be marked [DataContract] etc).
3. Is not referenced or consumed by any part of the system / application except for the event store.
4. Versioning is done for the entire set of domain events, rather than individual. Currently this is organised by namespace, but may do seperate assemblies later.
   
5. Will be kept around 'forever'.
   
6. Will, on occasion, utilize wholesale migration if it makes sense.

I use joliver's eventstore library and am utilizing pipeline hooks to convert between Storage and Application events when read (up conversion), when committed (commit conversion) and published (an IDispatchCommits wrapper). These converters are in a separate library and are all, at this point, AutoMapper based with minimal configuration. There is now an additional developer cost in creating / managing the storage representation as well as the application one, but it doesn't seem to be completely wild (yet). A test using AutoMapper's AssertConfigurationIsValid catches most event shape mismatch issues very quickly. There is also a perf hit in the mapping but my gut tells me it's small compared to serialization and event store I/O.

For 'locking' a version of the storage events I have a glorified copy -> fine/replace script+tool (me-ware) that duplicates a storage version, changes the namespace (i.e. ".V1." -> ".V2."), generates a type signature, creates a test to make sure that types don't change and updates converters. I'm not convinced that this is ultimately necessary or worth it. I have some time to chew on that and am in no particular rush to settle on an approach.

So, will this stand the test of time? Time will tell I suppose :)

Thx for the help,

- Damian

[Chris Condron]

How did this work out?

-Chris

[Greg Young]

trollolol

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test