We currently need a cached eventstore so the aggregates rebuilding goes faster. We are currently importing some data which takes +3 days to persist everything to the database. We loose the most time when querying the aggregates from the eventstore, so we would like to cache them because they will be used in multiple iterations during our dataconversion. We really need to improve this performance, because we need to do this dataconversion within a weekend. We spent the whole day measuring today and the most performance improvement can be gathered from the aggregate rebuilding. This takes about 40 milliseconds. Everything else is most of the time below 15 milliseconds.
Is there someone who knows enough of the ncqrs eventstore to provide us a cached version of the MsSqlEventStore?
On Friday, March 30, 2012 6:36:09 PM UTC+3, @marcofranssen wrote:
> We currently need a cached eventstore so the aggregates rebuilding goes > faster. We are currently importing some data which takes +3 days to persist > everything to the database. We loose the most time when querying the > aggregates from the eventstore, so we would like to cache them because they > will be used in multiple iterations during our dataconversion. We really > need to improve this performance, because we need to do this dataconversion > within a weekend. We spent the whole day measuring today and the most > performance improvement can be gathered from the aggregate rebuilding. This > takes about 40 milliseconds. Everything else is most of the time below 15 > milliseconds.
> Is there someone who knows enough of the ncqrs eventstore to provide us a > cached version of the MsSqlEventStore?
> If it hurts stop doing it. Implement snapshots. A decorated MsSqlEventStore with a caching layer is not going to achieve anything. Your problem isn't the read speed of Sql Server - you've reached a point where you are replaying too many events on the aggregate.
> On Fri, Mar 30, 2012 at 1:06 PM, mynkow <myn...@gmail.com> wrote: > How this cache should work? Cache is never simple.
> On Friday, March 30, 2012 6:36:09 PM UTC+3, @marcofranssen wrote: > We currently need a cached eventstore so the aggregates rebuilding goes faster. We are currently importing some data which takes +3 days to persist everything to the database. We loose the most time when querying the aggregates from the eventstore, so we would like to cache them because they will be used in multiple iterations during our dataconversion. We really need to improve this performance, because we need to do this dataconversion within a weekend. We spent the whole day measuring today and the most performance improvement can be gathered from the aggregate rebuilding. This takes about 40 milliseconds. Everything else is most of the time below 15 milliseconds.
> Is there someone who knows enough of the ncqrs eventstore to provide us a cached version of the MsSqlEventStore?
What do you mean by "importing data"? Are processes kicked off in reaction to events in this scenario, or is it just persisting ARs? Why not use some form of scale out and consolidate afterwards (bulk inserting into your sql server)? Ensuring sequential processing on an aggregate basis will allow you to keep the aggregate materialized in memory and just apply one event after another. You could collect batches of those changes and flush them as you see fit. Events are immutable so there should be no problem caching them (you could keep reading entirely out of the equation).
On Friday, March 30, 2012 5:36:09 PM UTC+2, @marcofranssen wrote:
> We currently need a cached eventstore so the aggregates rebuilding goes > faster. We are currently importing some data which takes +3 days to persist > everything to the database. We loose the most time when querying the > aggregates from the eventstore, so we would like to cache them because they > will be used in multiple iterations during our dataconversion. We really > need to improve this performance, because we need to do this dataconversion > within a weekend. We spent the whole day measuring today and the most > performance improvement can be gathered from the aggregate rebuilding. This > takes about 40 milliseconds. Everything else is most of the time below 15 > milliseconds.
> Is there someone who knows enough of the ncqrs eventstore to provide us a > cached version of the MsSqlEventStore?
