CQRS on a frontend javascript Single-Page-App

[Sébastien Lorber]

Hi,

First I'm still learning CQRS so let me know if I say anything weird to you.

CQRS and EventSourcing are traditionnally used on the server side...

Just wanted to know if some of you have already thought of using CQRS principles in a frontend single page application running inside the browser.

Because I do and it works pretty well until now.

I think it can make it easier to scale a frontend application on a large team, as we can design some kind of "UI widgets" which are autonomous components backed by their own internal data structure, and that all listen to a common event stream.

No need for event sourcing on the client as we don't really need to replay the events on the client and can trash all of them on browser refresh...

I am using ReactJS which is a pretty cool JS framework to build UI in a pure functional way.

Facebook proposed an architecture called Flux that goes with ReactJS (but usable with other frameworks as well). 

It has similarities with CQRS as they have the concept "Actions" which are somehow similar to events.

They also have the concept of Store which is an in-memory JSON data structure (or browser based storage system like localstorage?) that receive the "actions" and update itself. The only way to modify this store is to publish actions to a "dispatcher" which somehow acts like an event bus. You have multiple stores per application. The stores data is then used to build the DOM that is displayed to the user.

The thing I don't like in their Flux architecture is that they have introduced dependencies between the stores. One store can get data from another store after receiving an action, so they have introduced a way to tell their dispatcher in which order the stores have to receive an action. In a CQRS architecture for me this somehow means that the order in which event handlers treat an event matters.

I think these dependencies do not help to scale the JS app on a large code base so we tried using CQRS principles on top of Fux, by using "autonomous component widgets". 

I've written a little bit here: http://stackoverflow.com/questions/25791034/om-but-in-javascript

Each widget has its own store/db/datastructure and all widgets receive the same events. To compare it to a backend architecture, it is like each widget is running in its own server and does not share anything with others except the event stream. In reality, everything runs inside the same browser, so this means that sometimes widgets are interested in the same pieces of data from the event stream, so this means that this leads to a little bit more data duplication inside the browser (I don't think it's a big problem, as most heavy data comes from an API, like a large string, this just means that X widgets will reference that same string instance and it won't take much more browser memory...)

So I would like to have your feedback on the work done here. What do you think of these ideas?

Also as I'm not a CQRS expert I will probably have a lot of questions on how to translate some CQRS concepts from the server to the client side.

For exemple, I did not introduce Sagas in our internal framework, but already see there are some cross-cutting concerns in our JS app where we may have done things in a better way (I'm thinking about application onboarding, application backward/forward navigation, generic error handling of ajax request failures, detecting user disconnection...)

Bye

[Rinat Abdullin]

> I think these dependencies do not help to scale the JS app on a large code base 

I think, Flux works ok at Facebook scale. How large is your codebase going to be?

> Each widget has its own store/db/datastructure and all widgets receive the same events

This might lead to a very fragmented and complex design. In my experience, there is rarely 1:1 mapping between front-end components and back-end modules. However, your mileage may vary.

Rinat Abdullin | Writer at Abdullin.com

--
You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brian Mavity]

I have done something similar (based off of my understanding of Greg Young's talk) using nodejs and it has worked out ok.

I have some commands handlers that live on both client and server.

I have a projection library that lives on both client and server. (https://github.com/bmavity/bjorling)

1. On page load, the browser storage queries the server for the relevant projection and stores it as a baseline.
   
2. Once loaded, the DOM is databound (using Knockout)
3. When a user performs an action (submitting a form, for instance), the command handler is invoked in the browser and sent to the server.
4. Browser command handler execution results in events being raised, which gets applied to the projection.
5. DOM updates itself due to projection change
6. Server processes command.
7. (This is only partially done) Resulting events are received by the client and conflict resolution performed.

Each projection holds the state for a given UI component and communication between components is done with client only events. As Rinat said, it can get complex, and server/ui data needs are different, but my projections were completely tailored to the UI and I did not have to implement a domain (although theoretically I could share the domain code with the client). You just need to make sure keep an eye out for pain and don't get so married to your structure that you force things when another way would be more appropriate.

[Raif Harik]

I see this as having the advantage of making the client submit data in the exact, validated, command format.

In my stuff I go from client to web.api.  I clean up and sanity check the data sent in the web.api.  I then hand that to a command builder which puts it in a command ( further validates it ) and sends it on it's way.  

What it sounds like to me is that you are doing that same sort of stuff, but on the client.  This is great, if you own the client, and the client is the only guy who will ever be interacting with your domain.  But I can't help but feel that you will find you still need all that command creating validation and work on the server side.

With my stuff at home, I do own all the inputs and can make sure I'm submitting data just the way my command want it, but at work we accept input from all manner of degenerates and we need the anti-corruption layer stuff.

r

[Brian Mavity]

It's nodejs, so the same code runs in both places. There are also additional validation/security rules on the server.

[Per Wiklander]

On Thursday, November 20, 2014 12:46:09 PM UTC+1, Rinat Abdullin wrote:

> Each widget has its own store/db/datastructure and all widgets receive the same events

This might lead to a very fragmented and complex design. In my experience, there is rarely 1:1 mapping between front-end components and back-end modules. However, your mileage may vary.

I think he was only talking about frontend here. He was just trying to come up with a good name for the thing that Flux calls  a "store". So the idea was to let each frontend widget handle all the data it needs and gather that data from the incoming events. Just like aggregates in different BCs keep the data they need to do their work. 

[Sébastien Lorber]

Hi

Sorry was very busy these days to check the answers.

Yes Per Wiklander exactly, this is only on the frontend, there is no "mapping" with a backend server.

Our backend is an API which does not even use CQRS.

I wanted to discuss how to scale large javascript applications based on CQRS techniques (it does not need to involve any backend at all btw).

I don't know so well DDD terminology but I guess what I'm trying to do is that each widget is somehow a bounded context with an aggregate (called store) which receive events. And there's no dependencies between the stores.

I think it can help to scale the app because now every frontend developer can have the responsibility of a set of widgets without having to know how other widgets are working. Each widget is autonomous and has no data dependency on the others.

--
You received this message because you are subscribed to a topic in the Google Groups "DDD/CQRS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dddcqrs/WABQ8sgcUyw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dddcqrs+u...@googlegroups.com.

[Ben Kloosterman]

Submitting / Validating commands and accessing the read model is fine but for the rest ?  

Its not a question of scale its just a question that its massive over engineering and doesn't pay the benefits to justify those costs till you get to large scales eg the performance separation comes form DB transaction issues writing to the same table , you dont get that at the front end .. Also UIs dont have much real business logic just behaviours they want to see but not something you really want to model in detail . So you don't get much out of the DDD style modelling which removes the infrastructure crap either. 

Id be looking at type safety ( typescript ?) and MVVM models angular/knockout long before cqrs in a single page app. 

Rinat's point is very valid .. You rarely want the UI component to be closely mapped to the back end  and logic . You normally want to build many different views / apps to the same back end logic.  Even worse it often leads to Crud style commands and events. 

Ben

[Tom Janssens]

TBH I think the overhead can be minimal; I'm using events for inter-component UI communication as well in the browser, with minimal overhead, not using a framework:

I have one class which is just a wrapper around "domelement.dispatchEvent", "new CustomEvent" and contains a "registerComponent" which registers the component's event handlers ("addEventlistener") and views (instances that have a "viewActivated" event handler).

I raise commands by invoking the component's functions, and these raise events. I don't respect CQRS, but I do respect CQS.

The JavaScript code for this is fairly minimal, and it works perfectly for me. I don't send commands over the bus (or whatever you might name this). One important aspect for me is that I can trust the client who uses this, as I have almost no server-side code. (i.e. The server only performs persistence.)

One example that takes this approach really to the next level is the ELM language & debugging framework (http://elm-lang.org/), but for now it is way to complex for my needs.

Op zondag 30 november 2014 07:45:11 UTC+1 schreef Bennie Kloosteman:

[Sébastien Lorber]

Its not a question of scale its just a question that its massive over engineering and doesn't pay the benefits to justify those costs till you get to large scales eg the performance separation comes form DB transaction issues writing to the same table , you dont get that at the front end .. Also UIs dont have much real business logic just behaviours they want to see but not something you really want to model in detail . So you don't get much out of the DDD style modelling which removes the infrastructure crap either. 

Actually I think somehow UIs do have a lot of business logic, it is just it is not exactly the same domain as what you are used on the server side.

The domain of an UI will have events like "AjaxRequestToGetDataFired" and "MenuHovered". 

The point of using CQRS in the frontend is not for performances, just to scale on large code bases.

 

Id be looking at type safety ( typescript ?) and MVVM models angular/knockout long before cqrs in a single page app. 

Actually I'm a functional developer so I like type safety a lot. I have experience with AngularJS, Backbone and Knockout but I prefer a lot ReactJS as it enables developers to write UI's in a pure functional way, by pushing side effects (dom mutations) to the edges of the system.

This usage of ReactJS has been popularized last year with the Om's Clojurescript framework. Check this nice talk: http://www.youtube.com/watch?v=DMtwq3QtddY

 I don't think MVVM models will actually help you reduce the complexity or your app. Many frontend apps using these frameworks actually are a big mess of spagheti code very hard to reason about.

Using Angular and CQRS does not seem incompatible for me. You can update your Angular scopes according to an event stream.

Same for backbone instead of models/collections.

 

Rinat's point is very valid .. You rarely want the UI component to be closely mapped to the back end  and logic . You normally want to build many different views / apps to the same back end logic.  Even worse it often leads to Crud style commands and events. 

 

Yes, as what I alredy said this is not at all what I'm trying to do. There is no mapping at all between the backend and the frontend. Actually I don't care who and how the backend is developer, it could also be a public API. I'm talking about the frontend only here

[Sébastien Lorber]

TBH I think the overhead can be minimal; I'm using events for inter-component UI communication as well in the browser, with minimal overhead, not using a framework:

Yes using events does not provide any overhead. It is pretty simple and by the way it has been used for years on UI's.

Backbone has an event bus. GWT too... Angular has some kind of event system.. But these event buses are often used as a way to make different elements of the UI to communicate. I've often seen these event buses used in an "RPC way".

So I don't think using events is a problem or overengeneering: it is just already used by any large sized frontend application.

What I just want is using this bus not in the "traditionnal RPC way" but rather with CQRS principles. All the dat shown by the UI is actually a projection of the event stream.

 

I have one class which is just a wrapper around "domelement.dispatchEvent", "new CustomEvent" and contains a "registerComponent" which registers the component's event handlers ("addEventlistener") and views (instances that have a "viewActivated" event handler).

I raise commands by invoking the component's functions, and these raise events. I don't respect CQRS, but I do respect CQS.

The JavaScript code for this is fairly minimal, and it works perfectly for me. I don't send commands over the bus (or whatever you might name this). One important aspect for me is that I can trust the client who uses this, as I have almost no server-side code. (i.e. The server only performs persistence.)

One example that takes this approach really to the next level is the ELM language & debugging framework (http://elm-lang.org/), but for now it is way to complex for my needs.

 Haven't had time to check ELM but I know it has quiite inspired David Nolen for its Om framework. Check the video above.

With such frameworks the UI is purely functional. You have an efficient way to transform a JSON into a DOM tree. So the DOM is not a problem anymore. 

The problem that remains is: how do you manage the local state of a large frontend JS application? 

When ajax request completes, where do you put the data? 

if 2 widgets on the screen do show the same kind of data but presented in different ways, does this mean that we have one json object shared between the 2? Does this mean that if one of the widgets evolve and need a data structure change, you are forced to update the 2nd too because of the shared dependency? 

A frontend application evolves a lot other time and in my experience, with current frameworks and practices, it is way too easy to get very messy.

Most UI I see are monolithic applications with very tight coupling. 

On these apps it is very commun to modify a little detail of the app and introduce regressions on many other places. It also becomes very hard over time to refactor the code, as you can't easily see all the impacts of your refactoring attempt.

I'm trying to solve these problems so that you can:

- Easily introduce new UI widgets without having to understand how others are working

- Easily refactor an existing widget without fear of impacting others

[Tom Janssens]

The problem that remains is: how do you manage the local state of a large frontend JS application? 

When ajax request completes, where do you put the data? 

if 2 widgets on the screen do show the same kind of data but presented in different ways, does this mean that we have one json object shared between the 2? Does this mean that if one of the widgets evolve and need a data structure change, you are forced to update the 2nd too because of the shared dependency? 

I can't tell from your situation, but I have similar issues. The way I solved it, is having components that don't have views. These components emit events that get picked up by the views. 

 

A frontend application evolves a lot other time and in my experience, with current frameworks and practices, it is way too easy to get very messy.

Most UI I see are monolithic applications with very tight coupling. 

On these apps it is very commun to modify a little detail of the app and introduce regressions on many other places. It also becomes very hard over time to refactor the code, as you can't easily see all the impacts of your refactoring attempt.

I'm trying to solve these problems so that you can:

- Easily introduce new UI widgets without having to understand how others are working

- Easily refactor an existing widget without fear of impacting others

I have a very simple contract for my components:

- Commands are the public functions on my classes

- The names of the event handlers are a property on my class: i.e. this.eventHandlers = ["objectUpdated","renderRequested"]

- I currently don't have a formal way to define which events are emitted, but just grepping for "publish(" should be good enough.

I don't expose any state directly at all. I use events to propagate all state changes. 

This implies you need to have some discipline when handling the events: you are not allowed to mutate an event simply because it might be handled by other handlers too, or even expose some of the internal state.

On some short lists I just use a "renderRequested" event together with the something that mimics the full state of the list. If your state is not too large this might work for you too.

I do believe the way I build components is heavily influenced by DDD practices, and for now this approach works perfectlyfor me (which implies I'll probably be able to tell you in about 5 years why it wasn't good enough, but that is a different story).

 

[Sébastien Lorber]

Hey,

Rinat I saw you will give a talk at DDD belgium next week about DDD / CQRS in the frontend

http://www.meetup.com/dddbelgium/events/218600059/

This seems to go in the same direction as what I'm trying to achieve (except maybe you agree better with Flux?)

I discussed with a facebook React dev and he told me that they developed Flux internally even before using a system like React, and that when they "opensourced" it they were quite surprised to see so many implementations, as they know themselves it is not so perfect even if it works quite well for them.

So I guess you understand that I'm not trying to create a "mapping" between a CQRS backend and a CQRS frontend or something like that, but rather having a CQRS / DDD inspired frontend running against a black box REST API.

And I guess this is what you will present in this talk too right?

I hope it will be recorded!

If you can I would like to continue this discussion to understand better how to apply CQRS / DDD concepts to the browser.

What is bothering me with Flux is that stores as originally described can depend one on another, and the application order of events to these stores do matter.

They are not really supposed to contain "denormalized" data but would rather look like collection of items.

They are also supposed to contain the data that we render to the user.

So for me it's not really clear if a store should be compared to a repository, an aggregate root, a reporting vue...

Some of my thought:

- A browser runs on a single computer, so adopting a distributable architecture like CQRS/DDD also means that we can make all the formerly eventually consistent components consistent. I mean we can easily update 2 components state in the same transaction in an atomic way (using something like STM / immutable data structures) without any major performance bottleneck as there's no IO involved.

- On the backend, we construct reporting views (eventually consistent). Flux stores are used to "draw" the UI. I think drawing the UI is the equivalent of "reporting" on the browser. I tend to think a Flux store is then a "reporting database". This is why I denormalize data into flux stores and make them independant like in the backend. So I created "widget stores" that will contain everything for a widget to render. Each widget has its own reporting database (there's also a widget to drive the layout). Then I just have to create an UI which is an aggregation of all my widgets. A bit like creating a consolidated report.

Flux for me seems to create a single huge "reporting database" (composed of all the stores) and then rendering it. This means that all the developers must understand that huge reporting database, and the rendering function so not sure it scales better...

- The domain of an UI app is not necessarily the same as a backend application domain. In your UI domain, commands and events such as "contactBarHovered" or "submitForm" are also part of the domain. Reports can only be updated by listening to events, and drawing the UI is the main reporting need of the browser. Thus is hovering the contact bar makes a new item to be shawn, this is a valid application event. 

- Anothing interesting thing to notice is that it seems to me some events are "self-validating" in a sens of if the browser triggers a "hover" dom event, the event already happened, so it does not make any sens to issue a "hoverContactBarCommand". Also, it does not make sense to allow the user to click on a button if that action is not allowed so for now I didn't really find any usecase for command validation. If an user command is supposed to fail, if is preferable to disable the button generating that command

- It is not so often that the client/browser is considered a "part of the distributed system" and is (eventually) consistent with the backend by an event log system. Most often we use APIs, and it's common that not all the data is stored locally because of storage capacity issues. Then many times, the client has probably stale data locally. Business commands can't be validated safely locally because the validator state can be stale and does not give any guarantee so it seems better to not validate locally and send the command to the server. As far as I understand the safest place to do validation in CQRS/DDD is at the aggregate root level. So basically I didn't find any need for local command validation until now.

- Sagas can be helpful too. I used them for things like app onboarding, infinite scrolling... 

A basic exemple would be: on scroll event if selected view is of type timeline, issue command to load more items.

So in my current app I don't really have anything for aggregate root. Don't know how to apply this to the frontend yet...

Does all this make any sense to you?

--

[Rinat Abdullin]

Hi Sebastien,

Thank you for continuing this interesting discussion! I guess you already seen the video recording of the talk at DDDBE (http://abdullin.com/talks/2015-01-06-dddbe-facebook-flux-reactjs/). I'll continue from there.

I would agree with you, that in some domains there could be a strong mapping between DDD process and decomposition of the User Interfaces into the reusable components (especially within something that works as well as React.JS). We could reuse DDD methodology to break UI into components and refine their boundaries.

There could also be a very strong relation between principles of the event-driven design and user interactions in the UIs. Although there are differences. In my opinion "actions" and "action creators" from Flux do not map to command handlers and commands in CQRS (I cover that in the talk). There are more like UI events and very thin event creators (responsible mostly for interacting with API). That is similar to what Facebook is doing with "action creators" (except for the name).

Stores don't map very well to anything in the classical CQRS/ES domain as well. However I think about Stores as a simple event-driven modules (in micro-services sense, like we did at HappyPancake), where each module has an API with getters and subscribes to the events do fill in some state. GET requests on API might perform some computations on the stored view state in order to return the response.

In other words, there are no pure Aggregates, AggregateRoots and view projections in Flux, these are just optional implementation patterns for an event-driven system. However, some elements of Flux/ReactJS design (as defined by Facebook and Yahoo teams) share some traits with these patterns.

Regarding widgets and alignment of the UI with the Stores. I tend to see UIs as hierarchies of ReactJS components: state-full and pure. State-full components are located and root and key nodes, they are bound to the stores and get state changes from them (some folks from DDDBE compared them to Aggregate Roots in that sense). Once state-full ReactJS component has got the state, it propagates it down to the children via the props, most of which are pure and simple. From this perspective we can say that there is some correlation between these state-full ReactJS components and Stores.

I agree with you, there is a strong distinction between different kinds of events in the Flux/React design: events specific to the UI domain (tasksLoaded, postingCommentFailedOnServer etc) and browser events (represented as synthetic ReactJS events like buttonClicked or valueChanged). These events are different from the backend domain-specific events (taskAdded, taskRenamed etc).

Re infinite scrolling. Afaik, we solved that problem at HPC without the need of any component external to Flux design. However, there was a need in a state-full process manager for dealing with event streams and event polling.

Best regards,

Rinat

Rinat Abdullin | Writer at Abdullin.com

--

You received this message because you are subscribed to the Google Groups "DDD/CQRS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dddcqrs+u...@googlegroups.com.