Building an event chaser for read models

[Bennie Copeland]

What are the recommendations for generating read models for an ASP.Net Core web application (or in general)?

My current thoughts based on research are:

1. One projection builder per read model that reads from the eventstream and keeps track of its location in the stream. This will allow new projections/read models to be introduced without rerunning existing projections.

2. This should be done outside of ASP.Net due to issues with background tasks, ideally a Windows service.

3. One thread per projection builder

My questions revolve around how to trigger a projection to update. I can see two methods, but I'm unsure how to implement either one.

1. Pull method
1. I can have a thread for each projection builder, and a timer per thread to kick off the update periodically. What I'm not sure of is what the frequency of the timer should be. I would like the read models updated not too far behind the event store, but not kill the server with an endless loop.
2. Push method
1. The ASP.Net application can notify the service that the projections need updating after an event has been stored. No idea how to implement this. Maybe a message queue with an UpdateProjections message or something.

I lean towards the Pull method due to its simplicity and decoupling.

[Greg Young]

Ideally you support both. See CatchUpSubscription in ES. It
automatically switches between push and pull

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> Visit this group at https://groups.google.com/group/dddcqrs.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

[Bennie Copeland]

Greg,

I've been looking into it the CatchUpSubscription, and that is quite a web of code there. I'm learning a lot looking through it. I'll need to find a good resource on Task programming to understand some of what I'm looking at.

On a separate, but semi-related topic, how do you generate models that track things late in the stream? Getting a list of current things is easy, add a model on the Create event, and remove it on the Deleted event. But how do you catch the opposite? For example, a list of disabled Users, or retired products? The three ways I've come up with is keep a read model of all Users and just filter on disabled, or listen for the disabled event and then read that streams event to build the model, or listen for the disabled event, and source the models data from an existing read model.

[Greg Young]

You dont need to look at the code, just use subscribetostreamfrom and
it handles this for you.

"On a separate, but semi-related topic, how do you generate models
that track things late in the stream? Getting a list of current things
is easy, add a model on the Create event, and remove it on the Deleted
event. But how do you catch the opposite? For example, a list of
disabled Users, or retired products? The three ways I've come up with
is keep a read model of all Users and just filter on disabled, or
listen for the disabled event and then read that streams event to
build the model, or listen for the disabled event, and source the
models data from an existing read model."

Or keep intermediary state along the way for the second option as
opposed to re-reading later.

On Sun, Feb 26, 2017 at 7:43 PM, Bennie Copeland

[Bennie Copeland]

Oh, I'm not using EventStore. I'm using my own implementation until SqlStreamStore has been ported to .Net Core.

[Greg Young]

Seeing polling client in neventstore as well (basically the same thing
as catchupsubscription without the concept of a push)

On Mon, Feb 27, 2017 at 5:55 AM, Bennie Copeland
<bennie.r...@gmail.com> wrote:
> Oh, I'm not using EventStore. I'm using my own implementation until SqlStreamStore has been ported to .Net Core.
>

[Ben Kloosterman]

A major requirement of an event DB/store  is insert speed which is one of the things SQL DBs are poorest at unless you use bcp which kind of defeats the purpose.. I have worked with SQL stores thinking it was easy and can share arch with readmodel and keep the dbas happy ,but the performance led to significant issues and i now think its  more trouble than its worth.  Then again your system could be small to medium in which case ignore my warning :-)

Ben

On Mon, Feb 27, 2017 at 4:55 PM, Bennie Copeland <bennie.r...@gmail.com> wrote:

Oh, I'm not using EventStore. I'm using my own implementation until SqlStreamStore has been ported to .Net Core.

--

[Bennie Copeland]

It is definitely small to medium.