NodeJS + MongoDb with CQRS/ES/DDD

[Aaron Adams]

I hope this is the right group for these questions:

If using MongoDb and Entity Sourcing how could we do snapshots?

Should we use something like RabbitMQ or ZMQ or something else for messaging if using MongoDb as persistence? What do messages look like? Should we even use messages?

If using Eventual Consistency on the read side, how do you projecting or publish from your Event Source to your Read Model? What if the client needs to display the data on a results screen after a submit it just sent over the API to the Event Store along with say, a mash up of some kind based on business logic the server is responsible for? Should a message to some Read Model repository be sent asking for a query once it's consistent and push this to the client, as the client waits?

What does the Domain Models look like for an event? I have a lot of relationships some with behavior and some without, is that a problem?

What does a Event look like in terms of a Model or Schema, in other words how do you build up a schema from events, what does it look like? If that is in fact the way it is done...

It's been a struggle to find up to date info on this in NodeJS for some reason.

[Greg Young]

As I mentioned in the other thread much of this is very platform/language agnostic and a ton of information can be found on dddcqrs.com

As a warning building your own event store is non-trvial, there are many subtleties when working with one

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Studying for the Turing test

[Aaron Adams]

Thanks for your reply Greg.  I'll bet you've heard these questions a thousand times over, and know the answers to each one -- but the answers of course vary based on use cases/context/situation I suppose...

I'm trying to start a saas-b2b here and would like some help evaluating different persistence options, this is my goal.  While doing research, I've found others that are doing NodeJS => MongoDb with ES, CQRS, and DDDD, so it's not out of the question to use MongoDb from what I gather.  It's just like pulling teeth to get comprehensive, detailed, current, and correct answers as you can I'm sure well imagine.   

One issue I'm having right now your product is the fact that it doesn't have NodeJS drivers, it's AtomPub -- which I don't know how to work with.  

Look, if your product can help me save time and become productive faster -- that's great.  I would like to have someone from your company talk to me about how exactly your product can help me get up to speed and what it can do over other solutions.  I'd be willing to enter into a service agreement under favorable terms.   

Thanks again

[Greg Young]

Atom Feeds are just a restful protocol used all over the web http://en.wikipedia.org/wiki/Atom_(standard)

There are existing libraries for atom feeds in node.js that can be leveraged. 

https://github.com/eventstore/eventstore/wiki/Getting-Started-HTTP is a good place to get started (there are further docs on everything in the atom api such as "Writing to a stream" https://github.com/eventstore/eventstore/wiki/Writing-to-a-Stream-%28HTTP%29 here https://github.com/eventstore/eventstore/wiki under http api. All docs use curl to show the exchanges so if you understand curl you should be able to replicate a client in any environment relatively easily. For node its actually even easier as there is an atom variant application/vnd.eventstore.atom+json which gives the same structure as atom standard but in json as opposed to xml (much easier to parse)

Questions like:

If using MongoDb and Entity Sourcing how could we do snapshots?

Should we use something like RabbitMQ or ZMQ or something else for messaging if using MongoDb as persistence? What do messages look like? Should we even use messages?

If using Eventual Consistency on the read side, how do you projecting or publish from your Event Source to your Read Model? What if the client needs to display the data on a results screen after a submit it just sent over the API to the Event Store along with say, a mash up of some kind based on business logic the server is responsible for? Should a message to some Read Model repository be sent asking for a query once it's consistent and push this to the client, as the client waits?

What does the Domain Models look like for an event? I have a lot of relationships some with behavior and some without, is that a problem?

What does a Event look like in terms of a Model or Schema, in other words how do you build up a schema from events, what does it look like? If that is in fact the way it is done...

Are covered in the video I mentioned as they are not node specific.

Cheers,

Greg

[Ron Liu]

Hi Aaron,

Actually, I am trying the similar thing on MEAN architecture. Most of your questions I struggled with before, and right now, I finished most of my solution, and it does work for me, but I really don't know if I did the right thing.  Let me explain as clear as I can below, and please correct me. 

1. Event store with mongo. Mongo doesn't support transaction, or I should say it doesn't support transaction across aggregates or statements. At beginning, I tried to put all events inside aggregate, but mongo has a 16M document size limitation, so it doesn't work in long term. I also thought about Grey's event store, but for node.js it only support ATOM/REST interface, which means it need  many roundtrips to get all events for one aggregate. Finally, I decided to use SQL to get better consistence, actually POSTGRES as my write store. Based on Grey's cqrs document , I wrote a very simple basic node-eventstore-postgre . It works very well for me now. though it doesn't snapshot, but very easily to make it happen, I will do it when my project need it, I think it is pretty soon. Also because postgre support query json column, so later it even can query events if needed.

2. Read side persistence. I choose mongoDb of course because I like the aggregate query language it provides. At beginning, I code each event listener for each domain event, but it end up with lots of duplicated code and tends to buggy. Later I used a shortcut, I publish an event called aggregateChanged once eventStore save the aggregate, and the unified aggregateChanged event listener will write the aggregate to the read persistence. The drawback of this solution is that the read model is exactly same as the domain model. But it didn't bother me a lot, because in most cases that is fine, and I still can listen to the domain events to create special read model which is different from domain model. 

3. Queue selection. At beginning, I create my own memory bus to start with, and later I switch to rabbit mq successfully with tiny code changes. As you mentioned, there are some cases, we need to read immediately after we write, and due to cqrs it decoupled write and read sides, so the read-after-write inconsistence exists. I solve this problem by introduce the memory bus I created at beginning back into my project. In memory bus, when send a command, it return a promise which will be resolved after all events are handled including the read model updates. So finally, I have two queues in my project, and I can set up for different commands. And, only few cases need memory bus. To be honesty, I am really not sure if it is right to use memory bus to handler read-after-write consistence issue, looks like it's better to avoid such cases in cqrs way.

4. event schema. node.js is schema-less, which is the beauty part of js but sometimes pain. So we have to validate each command once when receive it. I created my own one validate-obj , but I am sure there exists many better ones. 

All in all, my tech stack is: node.js + postgres(write side) + express + socket.io(publish done to clients) + mongodb(read side) + angular

Cheers,

Ron