Reduce + merge-with error on lazy sequence of maps

[Nathan Smutz]

Hi all,

I've had a case where reduce with merging operations throws errors on lazy sequences.  I thought I should check in for insight before logging a bug.  Maybe using maps as keys has something to do with it. too  (I've since refactored)

(reduce merge-student-demand everybody)

Throws: 

IllegalArgumentException contains? not supported on type:clojure.lang.LazySeq  clojure.lang.RT.contains (RT.java:814)

However: 

(reduce merge-student-demand (take 5 everybody)) 

works just fine.  

As does:

 

(merge-student-demand {} (first everybody))

None of my code uses contains? directly, neither does reduce, so I figure it must be one of the invocations of merge-with in my code.

Here are the relevant chunks of code:

;; I can see how this would produce a LazySeq of hash-maps

(defn prioritized-courses [needed-courses]

  (-> 

    (for [course-set needed-courses

          :let [demand (course-demand course-set)]]  

      (map #(assoc % :demand demand) (:courses course-set))) 

    flatten))

;; There are two merge-with instances here.  This one is used inside the next one.

(defn merge-demand [acc m]

  ;; I was admittedly doing something corny here, using a map as a key.

  (merge-with + acc {{:prefix (:prefix m) :number (:number m)} (:demand m)})) ;; Sum demand by course

;; Merging again here:

(defn merge-student-demand [acc student]

  (merge-with merge-demand acc {(:id student) (prioritized-courses (:needed-courses student))})) 

(reduce merge-student-demand everybody)

=> IllegalArgumentException contains? not supported on type:clojure.lang.LazySeq  clojure.lang.RT.contains (RT.java:814)

I figure it's likely the Clojure core (1.8.0) merge-with function, being the first instance of contains? I could find:

text-mining.core=> (source merge-with)

(defn merge-with

  "Returns a map that consists of the rest of the maps conj-ed onto

  the first.  If a key occurs in more than one map, the mapping(s)

  from the latter (left-to-right) will be combined with the mapping in

  the result by calling (f val-in-result val-in-latter)."

  {:added "1.0"

   :static true}

  [f & maps]

  (when (some identity maps)

    (let [merge-entry (fn [m e]

(let [k (key e) v (val e)]

  (if (contains? m k)                ;; contains? invoked here

    (assoc m k (f (get m k) v))

    (assoc m k v))))

          merge2 (fn [m1 m2]

   (reduce1 merge-entry (or m1 {}) (seq m2)))]

      (reduce1 merge2 maps))))

Well that's my mystery.  I should get around to fiddling and seeing if I can reproduce this with a more straighforward program.

Best,

Nathan

[James Reeves]

There are a couple of errors in your code:

1. You have no initial value for the reduce, yet your output seems different to your input

2. prioritized-courses returns a seq, but merge-demand expects a map

My guess is that you only get the error when you hit data in your input sequence with the same student ID. So if you shorten your input with (take 5 ...) you don't hit the conflict, and you never get the error.

- James

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nathan Smutz]

Thanks James,

There was definitely a missing aggregation step in there. I appreciate your eyeballing it.

Best,
Nathan