How can find something inside heavily nested data structure ?

[Hussein B.]

Hi,

I have transformed JSON response into the equivalent data structure using Cheshire library.

The result is a huge nested data structure , mostly vectors and maps. It is actually a tree.

How to find a property that is nested deep inside the tree ? For example I'm search for the node that has the value zyx for property "uuid".

What I'm supposed to use? Something like zipper or walk? or something simpler is available?

Thanks for help.

[Marc O'Morain]

Hi Hussein,

A combination of filter and tree-seq might do what you need.

Marc

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Andy-]

I have yet to evaluate it myself but this might do help you:

https://github.com/nathanmarz/specter

[Gary Verhaegen]

If you want more specific answers, you'll need to describe the
structure of your tree. In particular, what is the relationship
between your conceptual nodes and your data structures (vectors and
maps)?

[Hussein B.]

Each node is represented as map:

{prop1 prop2 prop3 children}

children is a vector of nested nodes

[
prop1
prop2
children
 [
 {prop1 prop2 children []}
 {prop1 prop2 children [{new-node} {new-node} ... {another-node}]
 ]
 [
 {prop1 prop2 children []}
 {prop1 prop2 children [{new-node} {new-node}]
 ]
]

I need to traverse the structure to find the node (the map data structure) that has a specific value, say for prop2.

I tried:

(defn is-it-node? [coll]

  (not (nil? (get coll "children"))))

(filter #(= "12345" (get % "prop2")) (tree-seq #(is-it-node? %) identity navigation-tree))

But I'm getting nothing actually.

[Hussein B.]

Here is more concrete example

(def s [{"n" {"id" "a"} "d" 2 "children" [{"n" {"id" "c"} "d" 4 "children" nil}]} {"n" {"id" "b"} "d" 3 "children" nil}])

I want to find the map that has value "c" for "id". If found, I need to return the map {"n" {"id" "c"} "d" 4 "children" nil}

On Wednesday, August 19, 2015 at 5:36:42 PM UTC+2, Gary Verhaegen wrote:

[Alan Forrester]

On 19 Aug 2015, at 18:08, Hussein B. <hubag...@gmail.com> wrote:

> Here is more concrete example
>
> (def s [{"n" {"id" "a"} "d" 2 "children" [{"n" {"id" "c"} "d" 4 "children" nil}]} {"n" {"id" "b"} "d" 3 "children" nil}])
>
>
> I want to find the map that has value "c" for "id". If found, I need to return the map {"n" {"id" "c"} "d" 4 "children" nil}

One way to do this follows. First get all of the sub-maps, which you can do with the following two functions:

(defn get-lower [x] (tree-seq coll? identity x))

(defn get-maps-from-lower [x] (filter map? (get-lower x))).

The first function gets all of the lower level colls, the second picks the maps out of those colls.

You then need a function that will rummage round in a coll x looking to see if it has the relevant element r:

(defn rummager [x r] (some #(= r %) x))

You then use rummager to get maps with the appropriate val

(defn get-map-with-val [v x]
(first (filter #(rummager (vals %) v) (get-maps-from-lower x)))).

Trying this out in the repl I get

=> (get-map-with-val {"id" "c"} s)

{"d" 4, "n" {"id" "c"}, "children" nil}

which I believe is the result you wanted.

Alan

[Dave Tenny]

I have found the core 'get-in' function to be useful for extracting data from big trees of clojure data structures, perhaps

that will help.

I'm still in search of tools that let me get a good sense of *what* to navigate when looking at such trees

of data structures from API's  and/or data sources I'm unfamiliar with.  I find that to be pretty painful, not least

because emacs (and maybe the REPL) get seriously compute wedged printing large data structures.

An example of this is Amazonica, which will flatten ALL the data in a hierarchy of java data structures from

the AWS SDK into one gigantic tree.  It's a nifty tool, but the resulting flood of data with one careless query or printed tree fragment has been known to knock out my emacs session for minutes at a time.

I wrote a tool to analyze unique key paths in large trees so that I might know, for a given tree, what would be useful to provide as the access path to 'get-in'.

However I'm not happy enough with my tool to post it here, it needs some kind of syntax for describing sequences in the tree (since you don't want to see every unique key

used to access a sequence of 10,000 things).

On Wednesday, August 19, 2015 at 10:18:06 AM UTC-4, Hussein B. wrote:

[Hussein B.]

Yes, that does the job. Thanks for your help and time.

[henrik42]

I like using tree-seq and core.match when working on instaparse ASTs - like this:

(require ['clojure.core.match :as 'm])

(def s
  [{"n" {"id" "a"} "d" 2 "children" [{"n" {"id" "c"} "d" 4 "children" nil}]} {"n" {"id" "b"} "d" 3 "children" nil}])

;; top-down-traversal
(defn nodes [x]
  (tree-seq coll? seq x))

;; target pred
(defn sel-pred [n]
  (and (map? n)
       (some
        #{{"id" "c"}}
        (vals n))))
(some
 (fn [x]
   (m/match x
            (_ :guard #(sel-pred %)) x
            :else nil))
 (tree-seq coll? seq s))

The guard is a little clumbsy but as far as I can see match does not support map-matches with binding-keys like {k {"id" "c"}}.

[henrik42]

OK,  so in this special case you one would just use

(def s
  [{"n" {"id" "a"} "d" 2 "children" [{"n" {"id" "c"} "d" 4 "children" nil}]} {"n" {"id" "b"} "d" 3 "children" nil}])

(some
 (fn [x]
   (and (map? x)
        (some
         #{{"id" "c"}}
         (vals x))))
 (tree-seq coll? seq s))

[Brian Marick]

Andy- wrote:
> I have yet to evaluate it myself but this might do help you:
>
> https://github.com/nathanmarz/specter
>

Specter is great.

[Dave Tenny]

Specter looks nice.  I didn't see any examples in the readme or tests for working with more deeply nested data structures such as those discussed in this thread, any pointers?

[Brian Marick]

Dave Tenny wrote:
> Specter looks nice. I didn't see any examples in the readme or tests
> for working with more deeply nested data structures such as those
> discussed in this thread, any pointers?

Here's an example that might be relevant to the original question.
Suppose you have this structure:

(def deep {:a [{:b [{:key 33 :uuid "zyx"}]}
{:b [{:key 4, :uuid "zzz"}]}]})

You want to print out the terminal maps where the `:uuid` is "zyx". You
can do that like this:

user=> (select [:a ALL :b ALL #(= "zyx" (:uuid %))] deep)
[{:key 33, :uuid "zyx"}]


My `structural-typing` library builds a DSL for sort-of type
declarations on top of specter.
https://github.com/marick/structural-typing/ The code is kind of tricksy
in the parts that use Specter, so I don't know if it'll be of much help.

I plan to propose a short (50 pp.) booklet on specter, to be published
on Leanpub. I'll announce it on this mailing list shortly, and use
Leanpub to gauge whether there's enough interest.

[Sean Duckett]

If you're using emacs with cider, you might try `cider-repl-clear-buffer`. This
has saved my emacs session in situations where I accidentally print a large XML
structure.

--smd.