re-frame: updating data in app-db tree

[Jamie Orchard-Hays]

In Om, I didn't have to think about how to find the data in app-db to update as Om has cursors. In the cursorless world of re-frame, I am wondering what are favorite strategies for updating data deep inside a decently rich app-db tree.

ie, say I want to update a field that is in a map that is in a list in app-db. I want to update the :name value in one of the maps in :some-list:

(def app-db (atom
{:some-list
[{:id "asdfadfa" :name "Foo Bar" ...}
{:id "gfdsalkjc" :name "Boo Baz" ...}
....]}))

This isn't so difficult to approach, but what about a field in a map in a list in a map in a list... You get the idea.

Cheers,

Jamie

[Taylor Sando]

There was handler middleware for helping with deeply nested structure

middleware/path
A middleware factory which supplies a sub-tree of `db` to the handler.
Works a bit like update-in. Supplies a narrowed data structure for the handler.
Afterwards, grafts the result of the handler back into db.
Usage

https://github.com/Day8/re-frame/blob/master/src/re_frame/middleware.cljs

So if your data is at :path :to :value, you could have a middleware handle definition like:

(register-handler :some/key (path [:path :to :value]) hander-fn)

Then handler-fn would be passed the value, and all you need to return is the updated value from the handler-fn, it will be placed into the app at the path location.

[Jamie Orchard-Hays]

Thanks, Taylor. I'd seen path. It doesn't do what I need, but after reviewing and thinking about it, it may be a good part of a solution. Mainly what is the hard part is dealing with a tree of nested maps/vectors. You won't necessarily know the path until run-time, and certainly don't want to rely on the index location in a vec in case there have been data transformations along the way to the view code. So, when updating a value in the view and sending that value back to app-db, I need to be able to reliable find the correct item in the tree and update it.

I've been playing with my own naive code, clojure.walk and zippers. I find zippers conceptually understandable, but practically daunting. However, I came across this link, which does what I'm looking for:

https://gist.github.com/renegr/9493967

I'm curious how other users of Reagent and Re-Frame (and Quiescent) and handling the updating of app-db trees more complex than a simple one-level map.

Cheers,

Jamie

> --
> Note that posts from new members are moderated - please be patient with your first post.
> ---
> You received this message because you are subscribed to the Google Groups "ClojureScript" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
> To post to this group, send email to clojur...@googlegroups.com.
> Visit this group at http://groups.google.com/group/clojurescript.

[Daniel Kersten]

I haven't run into this yet as I've only been playing with re-frame for about a week, but you could write your own middleware to help you (middleware is easy to write). Maybe something like:

(dynamic-path [:path :to :* :value :*])

And it replaces the :* fields with data from the event. Eg:

(dispatch [:foo [0 1] :other-data])

Which would then have the handler receive and update data at path [:path :to 0 :value 1]

> To unsubscribe from this group and stop receiving emails from it, send an email to clojurescript+unsubscribe@googlegroups.com.

> To post to this group, send email to clojur...@googlegroups.com.
> Visit this group at http://groups.google.com/group/clojurescript.

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojurescript+unsubscribe@googlegroups.com.

[Mike Thompson]

Tim Peters guides us in this area by recommending: "Flat is better than nested".
https://www.python.org/dev/peps/pep-0020/

But that isn't always possible, I know, so to answer your question instead of providing gratuitous quotations ...

As much as possible, I try to conceptualise "app-db" to be a database. That's the reason for that "db" in the name.

And, if you wanted to "update" a database, you need:
- the unique key of the thing to be updated
- the name of the table (collection).

Generally "path middleware" nicely handles the "name of the table" part of this pair. It gets your handler to the "root" of the data structure (table? collection?).

In the situation you are describing, the "unique key" bit is some sort of "path" into a data structure. There are layers of vectors and maps to traverse, even if middleware gets you to the base of this data structure you want to update.

In such a case, you'll have to "build the unique key" as you descend. A read-only cursor-like process.

So, let's imagine you represent your path as a vector (of keys or indexes), then in each level you would pass down this path vector, conj-ed with a new part of the id, incrementally building the unique key as you descend, and passing it down.

Then, at the leaves, when you "dispatch", you can include the path vector in the event payload. The handler then has the information it needs to perform the update.

This deep-tree situation is not a common usecase for me, but if you ran into this situation a lot, then I'd think about adapting the reagent cursors for the job.

In summary: when you dispatch, pass through the "unique key" (path), of the thing you want updated. Your handler may additionally have "path middleware" on it, which resolves the “name of the table” part of any update. So you get the job done via that combination.

A super simple version of this, can be seen here:
https://github.com/Day8/re-frame/blob/46b8848f3bab432832a4517f0cccf36cb4197f65/examples/todomvc/src/todomvc/handlers.cljs#L54-L58

"todo-ware" contains path middleware which gets the handler directly to the "todos table" (see the parameter "todos"), and the "id" in the event is the unique key required for the update. In your case, you'd just have a more complicated "id" (it's a vector created in a read-only cursor-like fashion).

I worry that I've simply told you what you already know here.


--
Mike

[Jamie Orchard-Hays]

Thanks for that, Mike. Like I've written earlier in this thread, I've been trying a few approaches and have been wondering what others have come up with. (I don't need to reinvent the wheel.)

I hadn't quite gotten as far as you've described, though I was definitely on that path with one approach. Another I've tinkered with is using clojure.walk, but I dislike that it touches every node. Expensive on a large data set. Zippers don't make me happy either.

Definitely agree with keeping data as flat as possible, but even so you can end up with a decent tree shape, which the data structures I'll be consuming and generating actually have.

In my app, a Product is a Component (not a React/Reagent/Om component) that has many Components. Each Component can have many Components or many Substances. Even though the components are stored flat, they form a logical tree of n-levels of Components. Components and Substances can have other lists as well.

So, you can see that my needs to traverse into a rich data structure are immediate, even with flattened storage of the Components in a single list on the root Product.

Interestingly, last Summer I started on this with Reagent and later Om using the unflattened data (why?--it was flat in the db!). It got turned into an app for different data that was not recursive in any way, but I'd still solved that problem. I turned to Om then b/c it has cursors and they solved this problem so well that I felt it was worth the other pains involved in using Om, so I gave up on Reagent. In a nutshell: Reagent made developing React-style components really simple and easy, but dealing with a rich app-data atom was such a PITA, that I switched to Om whose cursors made that easy while making component development much more cumbersome. Since seeing and playing with Re-Frame, I've decided to do this next project with re-frame.

Anyway, thanks again for the info, as it definitely helps me along.

Jamie

> --
> Note that posts from new members are moderated - please be patient with your first post.
> ---
> You received this message because you are subscribed to the Google Groups "ClojureScript" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.

[Mike Thompson]

On Thursday, March 26, 2015 at 2:03:21 PM UTC+11, Jamie Orchard-Hays wrote:
> Thanks for that, Mike. Like I've written earlier in this thread, I've been trying a few approaches and have been wondering what others have come up with. (I don't need to reinvent the wheel.)
>
> I hadn't quite gotten as far as you've described, though I was definitely on that path with one approach. Another I've tinkered with is using clojure.walk, but I dislike that it touches every node. Expensive on a large data set. Zippers don't make me happy either.
>
> Definitely agree with keeping data as flat as possible, but even so you can end up with a decent tree shape, which the data structures I'll be consuming and generating actually have.
>
> In my app, a Product is a Component (not a React/Reagent/Om component) that has many Components. Each Component can have many Components or many Substances. Even though the components are stored flat, they form a logical tree of n-levels of Components. Components and Substances can have other lists as well.
>
> So, you can see that my needs to traverse into a rich data structure are immediate, even with flattened storage of the Components in a single list on the root Product.
>
> Interestingly, last Summer I started on this with Reagent and later Om using the unflattened data (why?--it was flat in the db!). It got turned into an app for different data that was not recursive in any way, but I'd still solved that problem. I turned to Om then b/c it has cursors and they solved this problem so well that I felt it was worth the other pains involved in using Om, so I gave up on Reagent. In a nutshell: Reagent made developing React-style components really simple and easy, but dealing with a rich app-data atom was such a PITA, that I switched to Om whose cursors made that easy while making component development much more cumbersome. Since seeing and playing with Re-Frame, I've decided to do this next project with re-frame.
>
> Anyway, thanks again for the info, as it definitely helps me along.
>

Yep, you've got a recursive data structure there which just naturally leads to nesting. No getting away from it.

One thought: you talk about storing all the "Components" flat (even though they nest). Wouldn't that mean each Component has an "id" (unique key)? If so, do you really need to know the "path" used to traverse to the Component, or could any dispatch just reference the "id"?

--
Mike

[keegan myers]

Sorry for posting to an older discussion, but I recently came across this issue as well. After evaluating zippers and code walking I decided instead to implement my own protocol to solve the issue.

I'm using re-frame along with sente. I've structured my application so data only enters the front-end app-db initially from a single function. As such I am pre-processing data as it enters app-db to make it easier to index. So far I have been appending information to the meta data of my data structures so that I don't have to strip unnecessary identifiers when I push changes to the server. This allows me to call

(update-by-elem collection element-to-be-changed function-to-change-it) from a handler.

The code is very rough and still has a ton of issues, but it may be of use to anyone still struggling with this issue:

https://gist.github.com/KeeganMyers/311fc4fa80222a693d18

example use:

(def test1 (add-meta-id [1 2 3 [4 5 6]]))
(update-by-elem test1 (last test1) (fn [coll2] (map dec coll2)))

(def test2 (add-meta-id {:test1 "value" :test2 {:test3 "value3"}}))
(update-by-elem test2 (:test2 test2) (fn [coll2] (assoc coll2 :test2 "val4")))

(def test3 (add-meta-id #{1 2 3 #{1 2 3}}))
(update-by-elem test3 (last test3) (fn [coll2] (map inc coll2)))