Using map on multiple collections.

[Nicolas Buduroi]

Hi, today I needed to use the map function on multiple collections
which didn't had all the same length. In this case, it returns a
sequence of the size of smallest one. But the problem I was facing was
required to map until the end of the longest, padding the smaller ones
with a default value. I came up with this:

(defn mappad [f val & colls]
(let [longest (apply max (map count colls))
colls (map #(if (< (count %) longest)
(concat % (repeat val))
%) colls)]
(apply (partial map f) colls)))

user> (mappad #(+ %1 %2 %3) 0 [1] [2 3] [4 5 6])
(7 8 6)

Is there a better way?

- budu

[kyle smith]

It's a little shorter if you unconditionally concat & repeat.

(defn append-val [val & colls]
(let [maxlen (apply max (map count colls))]
(map #(concat % (repeat (- maxlen (count %)) val)) colls)))

user> (apply map + (append-val 0 [1] [2 3] [4 5 6]))
(7 8 6)

[Nicolas Buduroi]

Thanks for the suggestion.

[Tom Hicks]

A slight modification, which I think avoids counting each collection
twice:

(defn append-val [val & colls]

(let [lengths (map count colls)
maxlen (apply max lengths)]
(map #(concat %1 (repeat (- maxlen %2) val)) colls lengths)
)
)

[Heinz N. Gies]

On Dec 23, 2009, at 6:04 , Nicolas Buduroi wrote:

> Hi, today I needed to use the map function on multiple collections
> which didn't had all the same length. In this case, it returns a
> sequence of the size of smallest one. But the problem I was facing was
> required to map until the end of the longest, padding the smaller ones
> with a default value. I came up with this:

No code but an idea, calling length on the collection might force them to be itterated entirely why not define something like a lazy seq that takes:

1) the sequence to extend
2) the default value
3) a number of other sets

As long as any of the lists is not epty return either first or the default value, this saves iterating through all lists, and is lazy :)

A quick not tested and proably horribl wrong example:

(defn extend-list [main-list default & other-lists]
(if (some #(not (empty? %)) other-lists)
(if
(cons
(empty? main-list) default (first main-list)) (apply extend-list (next main-list) default (map next other-lists)))))

(this is not tested and just an idea, sorry short in time!

[Nicolas Buduroi]

On Dec 25, 1:52 am, Tom Hicks <hickstoh...@gmail.com> wrote:
> A slight modification, which I think avoids counting each collection
> twice:
>
> (defn append-val [val & colls]
>   (let [lengths (map count colls)
>         maxlen (apply max lengths)]
>     (map #(concat %1 (repeat (- maxlen %2) val)) colls lengths)
>   )
> )

Good one!

[Nicolas Buduroi]

I'll have a look at this, it would certainly make this function more
idiomatic to Clojurians.

Thanks for the suggestion.

[Heinz N. Gies]

On Dec 26, 2009, at 2:37 , Nicolas Buduroi wrote:

I'll have a look at this, it would certainly make this function more
idiomatic to Clojurians.

Thanks for the suggestion.

I've another one I like even better, it get's a bit away from mapping over n list but it could achiev the same:

(defn extend-tupel [default & lists]

  (if (some (complement empty?) lists)

    (cons

      (map (fn [l] (if (empty? l) default (first l))) lists)

      (lazy-seq (apply extend-tupel default (map next lists))))

    nil))

This takes a default value and N lists, then creates s tupels form this lists untill all are empty, filling the ones that are empty before the rest with the default value. You could use destruction in your map to make it look like mapping over multiple lists. I like this better since each list will only walked once, this is especially nicer in my eyes when it are lazy lists that need to be evaluated each time the list is accessed.

Best Regards,

Heinz

[Daniel Werner]

Another, rather radical approach is to re-implement map with intrinsic
padding support (obviously based on clojure.core/map):

(defn map-longest
"Returns a lazy sequence consisting of the result of applying f to
the
set of first items of each coll, followed by applying f to the set
of second items in each coll, until *all* of the colls are
exhausted.
Missing items in exhausted colls are replaced with pad. Function
f should accept number-of-colls arguments."
([f pad coll]
(map f coll))
([f pad c1 c2]
(lazy-seq
(let [s1 (seq c1) s2 (seq c2)]
(when (or s1 s2)
(let [s1 (or s1 [pad])
s2 (or s2 [pad])]
(cons (f (first s1) (first s2))
(map-longest f pad (rest s1) (rest s2))))))))
([f pad c1 c2 c3]
(lazy-seq
(let [s1 (seq c1) s2 (seq c2) s3 (seq c3)]
(when (or s1 s2 s3)
(let [s1 (or s1 [pad])
s2 (or s2 [pad])
s3 (or s3 [pad])]
(cons (f (first s1) (first s2) (first s3))
(map-longest f pad (rest s1) (rest s2) (rest
s3))))))))
([f pad c1 c2 c3 & colls]
(let [step (fn step [cs]
(lazy-seq
(let [ss (map seq cs)]
(when (some identity ss)
(let [ss (reduce #(conj %1 (or %2 [pad])) []
ss)]
(cons (map first ss) (step (map rest
ss))))))))]
(map #(apply f %) (step (conj colls c3 c2 c1))))))

Of course, this solution results in lots of code duplication, looks
rather unwieldy and is less general than Heinz' solution. I'm curious:
Does it have any merits that could make its use worthwhile?

[Meikel Brandmeyer]

Hi,

Here is a more idiomatic (and fully lazy) version using the fnil function proposed by Rich here: http://groups.google.com/group/clojure/msg/f251cfd9baab440a

(defn extend-tupel
[default & lists]

(lazy-seq
(let [seqs (map seq lists)]
(when (some identity seqs)
(cons (map (fnil first default) seqs)
(apply extend-tupel default (map rest seqs)))))))

Sincerely
Meikel

[Heinz N. Gies]

On Dec 26, 2009, at 21:09 , Meikel Brandmeyer wrote:
> Here is a more idiomatic (and fully lazy) version using the fnil function proposed by Rich here: http://groups.google.com/group/clojure/msg/f251cfd9baab440a
>
> (defn extend-tupel
> [default & lists]
> (lazy-seq
> (let [seqs (map seq lists)]
> (when (some identity seqs)
> (cons (map (fnil first default) seqs)
> (apply extend-tupel default (map rest seqs)))))))
>
> Sincerely
> Meikel
>

I didn't checked for nil but for empty? on purpose, nil might also be a value in a list - I know it's will not be common but a list that has '(nil 1 2 3) would become '(<default> 1 2 3) if checking for nil and not empty. So your approach to wrap the lazy seq around it all is a good idea and makes the whole thing much nicer :).

Best Regards,
Heinz

[Meikel Brandmeyer]

Hi,

Am 27.12.2009 um 12:22 schrieb Heinz N. Gies:

>> (defn extend-tupel
>> [default & lists]
>> (lazy-seq
>> (let [seqs (map seq lists)]
>> (when (some identity seqs)

>> (cons (map (fnil first [default]) seqs) ; <---- Note previously missing brackets.

>> (apply extend-tupel default (map rest seqs)))))))
>

> I didn't checked for nil but for empty? on purpose, nil might also be a value in a list - I know it's will not be common but a list that has '(nil 1 2 3) would become '(<default> 1 2 3) if checking for nil and not empty.

Checking for nil here is independent on the contents of the lists. The `(map seq lists)` first turns everything in lists into a seq. If a collection is empty it will be turned into nil. This nil is independent of any contained nil of any other collection at that point in the iteration. The second point where nil shows up is fnil. first returns nil when it gets passed nil. We have to modify it to return the default in that case. However the passed nil is again from the seq itself not the contents. So contained nils are not a problem at all.

user=> (extend-tupel :x [1 nil 3] [4 5 6 7])
((1 4) (nil 5) (3 6) (:x 7))

Note, there were some missing brackets in my code. I corrected the above quote.

Sincerely
Meikel

[Heinz N. Gies]

On Dec 27, 2009, at 22:33 , Meikel Brandmeyer wrote:

> Checking for nil here is independent on the contents of the lists. The `(map seq lists)` first turns everything in lists into a seq. If a collection is empty it will be turned into nil. This nil is independent of any contained nil of any other collection at that point in the iteration. The second point where nil shows up is fnil. first returns nil when it gets passed nil. We have to modify it to return the default in that case. However the passed nil is again from the seq itself not the contents. So contained nils are not a problem at all.
>
> user=> (extend-tupel :x [1 nil 3] [4 5 6 7])
> ((1 4) (nil 5) (3 6) (:x 7))

Ah sorry, you're right, I misinterpreted fnil wrong :) so yes, that is really way nicer :) thanks for the hint to fnil too, so it seems not to be in the stable clojure yet :( too sad, but copy & paste from your link does :D.

Best regards,
Heinz

[Nicolas Buduroi]

Big thanks to everyone, the suggestions given are all very welcome,
even if I didn't really needed a better version as my use of mappad is
really simple for now. It was just curiosity on my part.

The lazy version by Heinz could be quite useful in other situations,
I've added it to my toolbox. Using lazy sequence with recursion is a
really elegant way to resolve most problems, it make up for the lack
of TCO in lots of cases. Also, the use of fnil is interesting. It
seems like a quite useful function, but it's a little bit confusing,
need more studying on that one. So to wrap up this post here's the
code for both lazy and non-lazy version of mappad:

(defn extend-tuple
"Lazily creates tuples from given colls, filling the ones that are

empty before the rest with the default value."

[default & colls]
(if (some (complement empty?) colls)
(cons
(map (fn [l] (if (empty? l) default (first l))) colls)
(lazy-seq (apply extend-tuple default (map next colls))))
nil))

(defn append-default
"Returns the given colls padded with the given default value."
[default & colls]

(let [lengths (map count colls)
maxlen (apply max lengths)]

(map #(concat %1 (repeat (- maxlen %2) default)) colls lengths)))

(defn mappad
"Like map but if collections aren't all of the same size, the
smaller
ones are padded with the given default value."
[f default & colls]
(apply map f (apply append-default default colls)))

(defn lazy-mappad
"Like mappad but lazy."
[f default & colls]
(map #(apply f %) (apply extend-tuple default colls)))

As you see, I renamed it extend-tuple as tupel was probably a typo
unless I'm mistaken. I'd like to find a better name, but these kind of
functions are always hard to name. I'll eventually release these and
other simple helpers in a small library where I keep these things. But
maybe one of these version (the lazy one at least) warrant being
included in clojure.contrib.seq-utils. I'll open a ticket for it in
the Assembla page later this week.

Thanks again and happy new year!

- budu