No persitance read models

[Ben Kloosterman]

I noticed in a lot of cases you can build the whole read model in < 10 seconds ( till you get to over 1M events )  , so I was thinking why not do this at boot for most small services and have no persistence.  The extra memory overhead would be cheap compared to the cost of writing and managing the persistence and once the time / memory becomes an issue introduce persistence for the service in question. Heck in most clusters it can take a minute or 2 to spin up a machine and you just dont make it visible to the load balancer till its finished. 

Has anyone else done this ? Are there any issues with small to medium services  ?  Is persistence on read models for small to medium micro services premature optimisation ?

I know it will make the SQL DBAs / Stored proc bunnies squeal but jarring them out of their low productivity comfort zone is half the fun,.

Ben 

[Denny Crane]

I guess it would really depend on a few things, no? For example, how is the service being hosted? Docker, for example? How many other services are running on the same resources? Enough to justify worrying about hitting your scaling threshold (resulting number of nodes) quicker? I guess if the amount of data in the read model is gonna be super small, then it would be ok, but then I'd probably ask you why you chose to use CQRS in such a small service if not just for the auditing and ability to replay.

[Rickard Öberg]

Hey,

This could work well if you have a database that supports in-mem as an
option, like Neo4j. Once you switch to on-disk storage, the changes
should be minimal.

/Rickard

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> Visit this group at https://groups.google.com/group/dddcqrs.
> For more options, visit https://groups.google.com/d/optout.

[Alexander Langer]

We've been doing this for a long time in one project. It simplifies
projections a lot and customers are impressed by the latency of
in-memory read model lookups.

Our in-memory read model is currently about ~6 GB in memory and is built
at application startup. It's, however, using much more memory (5x) than
we originally estimated, but we were too lazy to figure why (blaming
inefficient Java collections) and ignored it since RAM is so much
cheaper than developer time :-)

I can definitely recommend this approach for small to medium services.

We are currently pondering moving parts of the read model to an
in-memory-grid like Hazelcast.

> --
> You received this message because you are subscribed to the Google
> Groups "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dddcqrs+u...@googlegroups.com

> <mailto:dddcqrs+u...@googlegroups.com>.

[Leif Battermann]

Hi,

in fact I just recently tried exactly that for a simple/"toy" reference implementation of an ES/CQRS version of a tile matching game. (See source code)

Akka is a great tool for that. Anyway I don't know how it scales, but it adheres to KISS, YAGNI and maybe premature optimisation principles, imo.

Leif

[Ben Kloosterman]

Thanks for all that . I have used in memory but persistence backed for a while and was thinking why bother with persistence  at all, a LOT of development time is waited on persistence and we already have the event store to worry about it. 

Re costs Agree you can get high memory machines for peanuts compared to dev time - the main limitation being the read time of the event store,

Why do we have small - medium CQRS services ?  Because we have a distributed system with shared mutable state running on vehicles etc  so we have to event source and CQRS makes that model easier. ( We also use CRUD services where there is little logic ).  Also we archive our state in the cloud ( with blobs) once it is finalised so we only ever look at a 30-60 day history for live data which limits the number of aggregates and hence event count.

Regards,

Ben 

--

You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+unsubscribe@googlegroups.com.

[Peter Hageus]

I started with in memory projections on a current project, but recently added the ability to snapshot projection state at periodic intervals. (just saves one big json file per projection). Going live with this in a couple of weeks and was a little concerned about startup times, especially on a brand new service that might see a lot of updates initially.

This also adds a couple of other advantages: Easy to debug state (just open the file and have a look), and a feel for relative memory consumption (just compare file sizes).

/Peter

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.

[Leif Battermann]

Just bumped into this: https://martinfowler.com/bliki/MemoryImage.html

Isn't that exactly your approach?

Am Dienstag, 21. März 2017 01:10:49 UTC+1 schrieb Ben Kloosterman: