Event Sourcing and Data Migration

[Setya]

Hi,

From my naive understanding, it seems that using event sourcing requires us to maintain consistency between 2 sets of data, one is the series of events and the other one is data used for reporting, etc. Hence updating report data without going through events invocation is strictly discouraged since it will harm the consistency between the two. What if you need to migrate existing legacy data from old system into your new CQRS (event sourced) system ?

Thanks & Regards,

Setya

[Allard Buijze]

Hi Setya,

I've dealt with this problem once in the past. There, we decided to translate the state of the old application into commands for the new version. We then ran a little tool to run queries and fire commands. But this only given you an "illusion" of some history. The advantage is that -besides a little tool- you don't need any extra logic in your application.

An alternative would be to create special events "dataimportedEvent" and store these events in the Event Store. Event handlers could then use these events to build up their query tables. This gives you a proper view on history, but involves more coding.

Since this is more a general question about CQRS and Event Sourcing, I'd suggest to post this question on the dddcqrs mailinglist as well: ddd...@googlegroups.com. I'm sure there are a few people there that have tried different approaches.

Cheers,

Allard

[Setya Djajadinata]

Hi Allard,

Thank you for sharing your experience.

I've dealt with this problem once in the past. There, we decided to translate the state of the old application into commands for the new version. We then ran a little tool to run queries and fire commands. But this only given you an "illusion" of some history. The advantage is that -besides a little tool- you don't need any extra logic in your application.

An alternative would be to create special events "dataimportedEvent" and store these events in the Event Store. Event handlers could then use these events to build up their query tables. This gives you a proper view on history, but involves more coding.

Does this means that dataImportedEvent serves similliar to snapshot of events since it summarize 'events' of the old system ? 

 

Since this is more a general question about CQRS and Event Sourcing, I'd suggest to post this question on the dddcqrs mailinglist as well: ddd...@googlegroups.com. I'm sure there are a few people there that have tried different approaches.

Sure I will.

Although I like the CQRS (esp. event sourcing) idea a lot, I still can not imagine the cost of implementing it in an highly dynamic application in term of data and business rules especially having to maintain the consistency as I mentioned before.

I'd like to hear someone with such experience.

Thanks & Regards,

Setya

[Allard Buijze]

Hi Setya,

Does this means that dataImportedEvent serves similliar to snapshot of events since it summarize 'events' of the old system ?  

They're almost the same, but then stored as regular events, instead of snapshots. Snapshots can be seen as a temporary replacement for a number of events, while this is a "real" event.  

Although I like the CQRS (esp. event sourcing) idea a lot, I still can not imagine the cost of implementing it in an highly dynamic application in term of data and business rules especially having to maintain the consistency as I mentioned before. 

Event Sourcing doesn't come cheap if your business logic changes constantly. You'll just need to spend a lot of effort into building upcasters and tools to migrate data. The big challenge is to weigh the cost against the benefits of having the entire history of things at your disposal. Make sure there is a business case for it.

I am currently working on 2 projects where we use event sourcing. And for both, there is great value in having the events, making the extra work for maintaining those events worthwile.

Cheers,

Allard