Architecture Question / Handling app-state transactions outside of the component hierarchy?

[Nikolas Göbel]

Hey everyone,

I've started experimenting with ClojureScript + OM for a rewrite of a medium-sized Web-App. I started in PlainJS + React/Flux but quickly found myself hacking together things already available in ClJS, so now I'm here.

May I ask for some feedback on the architecture im planning? Especially if I'm missing/violating any established practices:

As in Flux, I plan to establish a single core.async channel on which every action the user triggers is published as some kind of tagged event. My first idea was to have components in the hierarchy subscribe to different tags, depending on their level of knowledge. A top-level component would then carry out the actual app-state transaction. This is a variation of the channel based communication shown in the Basic Tutorial[1] except that my approach would rely on a single shared channel and tagged events, instead of different channels for each purpose.

Thinking further about the problem I came to the conclusion that separating transaction handling from the component hierarchy would be much more desirable.
My rationale for this is that having components update the app-state kind of breaks the unidirectional flow paradigm. Also, I would prefer to have all logical interactions of the UI abstracted away, such that it becomes reusable (maybe for mobile applications).

Flux kind of solves this by intertwining Stores with update logic (which is ok I guess, but not possible with an atom). I figured that in CLJS I would simply subscribe functions to the dispatcher channel and have them transact the app-state atom.

Since I can't om/transact! on the atom directly, I would like to know if there is a way to pass cursors / create new cursors outside of the component hierarchy? Or can I use swap! instead? Would I be losing anything more than the tx-chan observability (which I really don't want to lose...)?

Thank you and please feel free to completely tear apart any of this if it doesn't make sense in CLJS world (which I'm rather new to).

[1] - https://github.com/swannodette/om/wiki/Basic-Tutorial#intercomponent-communication

[Daniel Kersten]

You can use swap! directly, but doing so will lose some of what Om gives you (like :tx-listen) and therefore is somewhat discouraged.

You can pass cursors around and transact on them even outside of an Om component (but remember that if you are not in the render phase you must use @cursor to read its value).

What I personally do is have an invisible root component which is responsible for managing various things such as which "page" component is visible and transacting on the root app-state. This design would likely work for you.

How I'm structuring my app is that all non-local user actions are put on a channel. These sound exactly like what you use: [id data] and I use (async/pub ch first) and then I can subscribe to different action ids. All of my event handlers simply put to this channel and I have helpers that subscribe (using om-tools mixins so that they auto-unsubscribe on unmount) to event ids putting the event data into component local state. Something like this (not my actual code):

(reify

  om/IInitState

  (init-state [_]

    {:x "Not Clicked"})

  om/IWillMount

  (will-mount [_]

    (.subscribe owner :foo [:x]))

  om/IRenderState

  (render-state [_ {:eys [x]}]

    (dom/div #js {:onClick (async/put! my-dispatch-ch [:foo "clicked"])}

      x)))

This way I can send non-domain-data update messages to components in a clean and consistent way. Any domain data updates are handled in my invisible root component (which then transacts on the app-state). 

However, I do use cursors to update app-state in a component if the change is entirely encapsulated in the component. For example, if I have a component that displays a list and clicking on a list item deletes the entry, then that components can transact on the list directly to make the change.

My logic for this is that if a component can be made completely self contained, then it should be. Aids reusability and makes the code clearer. But if components need non-local knowledge then the logic is pulled out and handled externally (by the invisible root component).

I've found this to work quite well so far. I will blog about it sometime when I have some more time.

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojurescript.

[Mike Haney]

I've taken a different approach which I think has really worked well, so I thought I would share it.

The inspiration for my approach came from this article http://tonsky.me/blog/datascript-chat, in which he uses Datascript, core.async, and uses React directly for rendering (using Sablano for templating). I've adapted this basic approach with a few changes:

1) Using React directly is fine, although you are relying solely on React's DOM differencing for performance, with no IShouldUpdate optimizations like you get with Om/Reagent. To be honest, that's probably fine for most apps, but I really like Reagent and wanted to simplify my templates a bit, so I combined the Datascript approach with Dave Dixon's binding code (here: https://gist.github.com/allgress/11348685). This has worked fantastically!

So instead of monitoring the transaction stream and re-rendering the UI on every change (as in the original article), individual Reagent components can bind a datalog query to a reagent/atom and it will be updated whenever that data changes in Datascript. This works for arbitrarily complex queries and I have been very pleased with the results in practice.

For example, in our app you can select an employee from a dropdown and then it will list work orders for the selected employee - a fairly common use case. This was trivial to implement with 3 queries - a query for the selected employee, a query for all employees (to show in the dropdown menu) and a query for all workorders assigned to the selected employee. Each Reagent component was built with its respective query bound to a r/atom, and that's literally all there is to it. When the selected employee changes, the workorder query is automatically re-run and its atom updated, which triggers Reagent to re-render the workorder list.

2) So what about transient state - such as whether the dropdown is collapsed or not? This could be represented as component state easily in Reagent (or Om, for that matter), but that brings up other issues. For example, when a dropdown menu is expanded, it's usually expected that clicking outside of the menu will collapse it (we've all seen those annoying sites where you can only close the menu by clicking on it again - UX Fail!).

This is tricky to do with local state and it's easy to couple components that really have no business knowing about each other. Well thought out event pipelines with core.async can solve this, but I try to be lazy whenever possible. So this transient state is kept in the DB as well.

There are a couple of ways to do this. The easiest is to have a single "UI state" entity and just lump all the transient state in there, and any component needing access to it can just bind a r/atom to it. I created a bind-entity variant of the binding code to do just that - bind a r/atom to a single entity and update whenever it changes. What I ended up using is a more granular approach, where each component can create its own entity to track transient state and get a bound r/atom representing that state. That was easy to setup with a few helper functions.

3) I'm using plain functions instead of core.async for event handling. Just a personal preference. IMO, since the DB is the single source of authority and any component can easily bind to the DB and react to changes, I found that pub/sub was just unnecessary complexity (it could easily be added back if I find I need it, though).

4) I wasn't comfortable with having the Datascript DB (and core.async channels, which I'm no longer using) as global state, so I created a quasi component system for my app. (NOTE to Stuart Sierra - PLEASE port component to Clojurescript! If you don't, I may just have to tackle it myself soon). This adds some nice structure to app and is generally OK, but to be honest I'm still not 100% convinced the additional complexity is worth it, at least for smaller apps. The main reason I stuck with it is because although this current app is small-medium in size, and I could get away with not structuring the code, then next phase of this project will entail building a significantly larger app (hopefully reusing many of the components from this app).

I still have mixed feelings about that last one. Just having a global DB and event bus that everything accesses certainly makes things MUCH easier, and global state seems to be at least overlooked if not accepted in the Javascript world. But it just feels icky to me - 20+ years of professional experience tells me that shortcuts like that come back to bite you in the a** more often than not. I would be interested in hearing others' opinions - maybe I'm just being too "old school"?

Well hopefully someone finds this approach interesting. Whether or not you use core.async/functions or components/global-state, the basic approach of Datascript + Reagent + bound queries is a BIG win in my book, and I've been very happy with the results. If people are interested, I've thought about pulling out the key pieces into a demo app (minus the proprietary stuff for my project), but it will probably be at least a couple weeks before I could get to that. In the meantime, if anyone has specific questions I can try to answer and maybe pull out a few snippets as gists.

[Matt Dean]

Mike, thanks for sharing! This helped clarify some of my thoughts and introduced me to a couple of possibilities I hadn't considered. I'd enjoy seeing a demo app if you do decide to publish one.

[David Pidcock]

At first I read DB and Transient state and was "huh?". Then I skimmed over the linked article.
Nice. So if I were to hazard an analogy, its like data script is a root Om atom with querys in the place of cursors, right?

[Justin Reidy]

This approach is fantastic. +1 on sharing, +1000 if you put together a generic example. :)

[Mike Haney]

Om and Reagent have different approaches to managing state, but that analogy is pretty close. If you are partial to Om, look into Ian Eslick's derive library, which he discusses in another recent thread on this group. It takes a similar approach to what I've described using Om and NativeStore (although he says it could be adapted to Datascript easily). Without something like derive, this approach would be difficult with Om, so it's great to see someone working in that area. That's one of the main reasons I went with reagent initially, but I really like it and will stick with it. I do think derive could be useful with reagent as well, I just haven't had time to dig into it.

Also, I realized after I posted I didn't answer your original question, how to manage State changes that happen outside your ui components. With the datascript/bound-query approach this is trivial. I have components listening for server changes that simply transact against the Datascript db whenever changes occur, then those changes are reflected in the bound queries and the ui is updated if needed. I'm using Firebase for this particular mobile app, but it would work just as well with sente, raw web sockets, or even Ajax calls.

[Nikolas Göbel]

Just wanted to thank you guys for sharing your approaches, its much appreciated.

I will think through those and try to find the best approach for me.

Thanks again,
Niko

[Scott Nelson]

I've been using Om with cursors but have very quickly encountered the limitations of purely hierarchical data and have been playing with DataScript as an alternative. The application I'm working on will perform fine without the IShouldUpdate optimizations so I'm trying out a simple approach like this: https://gist.github.com/swannodette/11308901

When using Om's cursors I had a number of top level entries in the app state (i.e. things like current route, error messages, current user, etc.). Mike, how are you modeling these kind of singleton objects as DataScript entities? Or are you using separate atoms for this kind of thing?

Thanks!

-Scott

[Andrew Stoeckley]

Regarding app-state manipulation outside of the render phase, can anyone clarify for me how we know with certainty if a function is likely to run during this phase? For Om my naive interpretation is that only code inside an IRender or IRenderState is executing in the render phase. And I notice that all :onClick handler functions appear to always run outside the render phase. Is this a fair assumption or would there ever be a case where an :onClick might actually be considered a part of rendering?

[David Nolen]

Note that these complications are slated for removal. Automatic cursor consistency is a goal for 0.8.0, the only big question is what to do if a cursor has become invalid due to deletion. Likely we'll add a predicate for those people who want to be safe.

On Thursday, October 30, 2014, Andrew Stoeckley <andr...@gmail.com> wrote:

Regarding app-state manipulation outside of the render phase, can anyone clarify for me how we know with certainty if a function is likely to run during this phase? For Om my naive interpretation is that only code inside an IRender or IRenderState is executing in the render phase. And I notice that all :onClick handler functions appear to always run outside the render phase. Is this a fair assumption or would there ever be a case where an :onClick might actually be considered a part of rendering?

[Mike Haney]

On Wednesday, October 29, 2014 3:48:35 PM UTC-5, Scott Nelson wrote:
> When using Om's cursors I had a number of top level entries in the app state (i.e. things like current route, error messages, current user, etc.). Mike, how are you modeling these kind of singleton objects as DataScript entities? Or are you using separate atoms for this kind of thing?

There are a couple of ways to do this. You can create separate entities in Datascript to store these things. This is generally the approach I used and I created some helper functions to more easily setup and track these "singleton" entities. But I was evolving towards the second approach, as my app grew.

The second approach is to decorate existing entities with additional attributes. For something like current-user, assuming you already have a list of users in your DB, you could add a :user/selected boolean attribute. For single selection you have a DB fn that resets all users to 'false' except the selected one, but you could change that to multi-selection simply by not resetting the flag on other users. It's more work on a case by case basis, but if you start thinking in abstractions like this you can create generic DB functions that can be applied across a wide range of entities. Abstractions like "selectable", "hideable", "toggleable" can be reused in many parts of most UIs.

I'm speaking in past tense, because I have temporarily had to stop using this approach in my app. As my app grew I began to notice substantial lag at times running on my target mobile devices. It was never a problem testing on a desktop browser, but of course mobile browsers have much less horsepower.

The problem stemmed from a few things:
1) the binding code was very inefficient
2) each reagent component simply binds whatever state it needs, so for example if 5 different components need the user list, they each bind their own r/atom.
3) the combination of 1 and 2 means several dozen inefficient binding listeners running after every transaction, which became noticeable on mobile.

I had a couple of choices - remove Reagent and use Quiescient or React directly for rendering, as tonsky did in his demo app, or remove Datascript and continue to use Reagent. I chose the latter, with the intention to be able to go back to using Datascript once I work through the problems.

I think the key to fixing this is this feature: https://github.com/tonsky/datascript/issues/32 which fortunately has now been implemented (just noticed that when I popped over to grab the link). Having transaction metadata should allow me to optimize the binding code significantly.

Since I can't risk these experiments in my production app at this point, I'll probably pull out code and create a demo app I can experiment on and share, then when I am confident that I have the binding code optimized I can re-introduce Datascript to my production app. I'm in the process of setting up my blog and my goal is to have the first post or two on this finished before the Conj, so look for something over the next 2-3 weeks.

[Dylan Butman]

There was some more work done on supporting query change binding in datascript https://github.com/tonsky/datascript/pull/12

and there's also a larger project https://github.com/vitalreactor/derive that aims to provide similar functionality on top of either their own in memory db NativeStore or on top of datascript. There was a larger thread about it https://groups.google.com/forum/#!searchin/clojurescript/derive/clojurescript/WCz57-k8leY/cdLHHpBJ6qEJ before ref cursors.

Next rainy day I want to sit down and write parallel implementations of something like a chat system using

Om w/ ref cursors
Datascript w/
Derive
Unmerged query watching
Reagent w/ multiple atoms

Maybe a port of tonky's chatting cats example? Any thoughts on other combinations that might be useful in a side-by-side comparison?