replaying eventstore

[@marcofranssen]

Currently I build a tool to fix lots of readmodel problems. The tool just replays the eventstore and the denormalizers do the rest. Currently it takes about 24 hours to replay ca 3 million events. 

I optimized the tool to fill a queue with event parallel and let another thread play the events from the queue so I will never have a delay when querying events.

I also process each events in parallel on the different denormalizers.

I put some indexes on the databases so the readmodel update queries are quicker.

I'm using the MsSqlEventStorage....

Are there any ideas to spead this up? 

[Greg Young]

batching? Just collect the sql that the projections want to execute
and send 100 statements at a time (will be a bunch faster). Also are
your projections current state based or are they writing other stuff
as well? If current state based just bring forward an object until you
hit the end of the log.

--
Le doute n'est pas une condition agréable, mais la certitude est absurde.

[@marcofranssen]

We use entityframework for the readmodels.

For example like this in the denormalizers:

public void Handle(IPublishedEvent<AliquotUnpositionedEvent> evnt)
        {
            var payload = evnt.Payload;
            Handle(context =>
            {
                var aliquot = context.ClaimedAliquotPositionsReadModels.SingleOrDefault(a => a.AliquotId == payload.AliquotId);
                if (aliquot == null) return;
 
                aliquot.BoxId = Guid.Empty;
                aliquot.Position = -1;
                aliquot.Positioned = false;
                aliquot.BoxName = string.Empty;
            });
        }

protected virtual void Handle(Action<WebsiteReadModelContext> handler)
        {
            try
            {
                using (var context = new WebsiteReadModelContext())
                {
                    handler(context);
                    context.SaveChanges();
                }
            }
            catch (DbUpdateException)
            {
                Handle(handler);
            }
        }


What you are trying to tell me is I should do the context.SaveChanges() not per event, but per batch of events. Am I right?  However that needs me to do a structural refactor. I need to provide a EF Context to each denormalizer and call the savechanges somewhere else... Where would be the best place to do the savechanges? At the moment each event gets saved in the eventhandler in the denormalizer. However when I am going to use this context at higher level I will also need it to query so I need to provide it also to my repositories, because not everything is persisted immediately. What would be the best strategy to manage this context?

Hope you can help me out with this Greg. This will probably improve our overall performance... However the impact would be high I think in order to get this working and test it.

On Monday, July 30, 2012 3:36:12 PM UTC+2, Greg Young wrote:

batching? Just collect the sql that the projections want to execute
and send 100 statements at a time (will be a bunch faster). Also are
your projections current state based or are they writing other stuff
as well? If current state based just bring forward an object until you
hit the end of the log.

[Greg Young]

Generally when dealing with this kind of thing you use a n writes or t
time mechanism (eg you batch but flush on intervals if not). I am not
sure how well EF supports batching and I am the wrong guy to ask on
that part unfortunately. Have you considered Julie Lerman? She seems
to know EF pretty well. I have to imagine they support it.

[@marcofranssen]

Entity framework does support it. It just keeps the changes in memory until I call savechanges on the context. But where should I make my calls to context.savechanges() and disposing my context. In short where should I manage my context? So I can easily manage / test it? So my question to you is for the architecture part. I will contact Julie on twitter for the entity framework part...

On Monday, July 30, 2012 3:55:56 PM UTC+2, Greg Young wrote:

Generally when dealing with this kind of thing you use a n writes or t
time mechanism (eg you batch but flush on intervals if not). I am not
sure how well EF supports batching and I am the wrong guy to ask on
that part unfortunately. Have you considered Julie Lerman? She seems
to know EF pretty well. I have to imagine they support it.

[Greg Young]

I dont deal much with EF sorry. I would find using it to be overkill
in most cases (eg it would create more problems than it solved for
me). Maybe someone else here has more specific EF knowledge.

[@marcofranssen]

Ok thanks... Jus t got the message from Julie it doesn't support batching. It will send one at a time even if there are multiple changes.... Hmmm that would mean I have to decide if we change our persisting layer or just do the migration in a weekend.

Thanks for your help...

On Monday, July 30, 2012 4:07:44 PM UTC+2, Greg Young wrote:

I dont deal much with EF sorry. I would find using it to be overkill
in most cases (eg it would create more problems than it solved for
me). Maybe someone else here has more specific EF knowledge.

[@yreynhout]

How many projections are we talking about? i.e. to make an informed trade-off between rewrite of the handlers with a batchable approach or a wrapper around the context (smaller refactor).

[Marco Franssen]

I have 24 readmodels/projections for my website and 20 for my domain. Both the domain as the website denormalizers have the protected handle method I showed to Greg in a baseclass.

2012/7/30 @yreynhout <yves.r...@gmail.com>

[@yreynhout]

Might be easier if you catch me on gtalk.

On Monday, July 30, 2012 5:10:44 PM UTC+2, @marcofranssen wrote:

I have 24 readmodels/projections for my website and 20 for my domain. Both the domain as the website denormalizers have the protected handle method I showed to Greg in a baseclass.

2012/7/30 @yreynhout

How many projections are we talking about? i.e. to make an informed trade-off between rewrite of the handlers with a batchable approach or a wrapper around the context (smaller refactor).

2012/7/30 @yreynhout

How many projections are we talking about? i.e. to make an informed trade-off between rewrite of the handlers with a batchable approach or a wrapper around the context (smaller refactor).

[Galen]

I'm very keen to see any progress or results you have made with this concern.

[@marcofranssen]

I did refactored the context into a wrapper object which I bootstrap in my dependency container. So I was able to make 2 implementations. One for my database rebuild tool. One for my production environment. I also encapsulated the complete DbCOntext object, which enables me to implement a wrapper object that for example uses NHibernate or plain sql. However I choose the easy way and just did some entity framework configuration options and some semi batching. So I did put all the entityframework configuration options to false (autodetect changes, proxy etc.) And I collect 10000 changes before calling context.ChangeTrakcker.DetectChanges() and context.SaveChanges(). With this small tweaks my database rebuild gone from 24+ hours to +- 12 hours. Because of the context wrapping my production implementation didn't changed, so minimal risk for the production environment.