Event Store: integration layer between microservices or private data store for one microservice

[Zach Blocker]

At the risk of being slightly off-topic for this group (apologies - I am not terribly familiar yet with DDD but am trying to build a microservices application with event sourcing as a foundational idea), I am wondering about the role of an event store in the larger microservices architecture.  

Specifically we are evaluating Greg Young's Event Store, but conceptually we could be talking about any event storage database that also has messaging component so services can subscribe to get event notifications. 

Listening to speakers like Chris Richardson (see latest talk on microservices + event sourcing), I get the idea that event sourcing solves the problem of publishing events atomically and letting other microservices have access to them.  What that seems to imply is that an event store needs to not belong to a single microservice; it is really an integration layer between microservices.  Therefore domain events stored there would not be specific to one bounded context or one microservice; the domain events would instead model domain objects ("aggregates"?) in a context-neutral way. 

However, in the Event Store group, someone raised the idea that the event store should belong to a single microservice, as part of its encapsulated data storage.  This was in response to my question about how to control access by various microservices to certain event streams, so that a given microservice could publish a certain event type "E", let's say, but no other microservices could publish an "E" event.  

I was curious what peoples' thoughts were on the idea of this very specific type of shared database between services, acting as more of a persistent, replayable message queue than an actual materialized view... 

And if you feel that event stores should be private to a single microservice, how would you solve the problem of publishing domain events so that other services could react to them?  And do so in an atomic way, without a two-phase commit of some kind?

[Greg Young]

"However, in the Event Store group, someone raised the idea that the
event store should belong to a single microservice, as part of its
encapsulated data storage. This was in response to my question about
how to control access by various microservices to certain event
streams, so that a given microservice could publish a certain event
type "E", let's say, but no other microservices could publish an "E"
event. "

If you look further there are other discussions as well where its a
tradeoff. Its the same with any db/microservice. Do you prefer ease of
join between two sets or do you prefer encapsulation? There is not a
right or wrong answer.

Greg

> --
> You received this message because you are subscribed to the Google Groups
> "DDD/CQRS" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to dddcqrs+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Studying for the Turing test

[Zach Blocker]

Thank you.  That's basically the high level answer I was looking for.  There are times to use one or the other, etc...

The way I'm modeling it right now is that the Event Store is one of the inter-service integration layers in our application, and each microservice manages its own materialized views of the event-driven data.  I am happy with this approach, but I wanted to sanity check my design against the growing body of best-practice knowledge. 

[Kirill Chilingarashvili]

I think domain events within one service /context are internal to this context,

Also you can use them in other contexts, you need to think about situation when event schema by some reason changes within one context,

So there are two things:

1) event structure

2) event transport between contexts

One known headache  with microservices is their interface cannot change at same pace as code within service,

each service can be pushed independently of others,

but for interface between the services you need to make sure both contexts understand the changes

And just to make this process simpler, I think it is better to differentiate events for interfacing from events within the context,

Just declare them in some shared package and share unit tests for interfaces

So another - second point is transportation, is it RPS, push message bus?

I would just define another abstraction name IBus for message bus and keep it that way,

push there messages from level upper to domain - gateway, and process them in anticorruption layer

Higher in infrastructure just implement IBus using Event Store - it works just fine

but if you have it abstracted you can switch to different implementation later if needed

[Thomas Schanko]

If there is an interaction between two contexts (Microsrvices in this case), the interaction defines a third context of it's own right. If you care about always doing things right (whatever that means) a context should have no knowledge about things beyond it's own boundary. So, it would be the responsibility of the interaction context to retrieve the events from one Microservice and translate to something the other Micro service understands.
As a side effect, the interaction becomes explicit removing the need for those unit tests. But as mentioned before, sometimes a simpler solution is just good enough.

[Ben Kloosterman]

my 2c looking at similar things.

1) Dont make all micro services Event Sourced only where appropriate . For CRUD services fire just events needed for inter service communication. Simple CRUD services are a god sent for a lot of requirements.

2) Yes it acts as a good integration layer between services don't make this heavy or into a framework

3)  re one event store or shared .To me the main decision relies on being able to restore a single service or replace the DB with say a graph db without affecting others ( Do you use a single SQL DB for your services ?)  .You may have a separate Kafka / ESB  like Event Store acting as your bus for inter service messages ( to have a single point and not couple all the services) but these are just for inter service communication messages not Event Sourcing and will normally trigger something in other services.  Though as Greg said this encapsulation comes at a cost .

"And if you feel that event stores should be private to a single microservice, how would you solve the problem of publishing domain events so that other services could react to them?  And do so in an atomic way, without a two-phase commit of some kind?"

If you use a  Event Store for inter store messages and another for the service you would just get the inter service store to subscribe for relevant messages. If the service is a different data source it gets a bit harder ( this may be relevant  https://www.youtube.com/watch?v=WG3xKyldSK0) , there are many ways here depending on  need . 

Note events in the inter service data store only need to be kept for a short time ( 1- 30 days) since the services are still responsible for the data. 

Ben