Hello everyone,
I work with lists of hash-maps constantly. As such, I have a
series of proposals I would like to add to contrib, based on my
efforts. They are mostly related to data wrangling, and I have found
that they work together nicely. The high level of proposals is
* add the proj function, a utility to apply functions in parallel.
* some additions to map-utils, in order to make manipulating hash-
maps easier
* a new library, pred-utils, for creating complex predicates
* a new library, table-utils, for manipulating lists of hash-maps.
This includes outer (left, right, full) and inner (equi, natural,
cross) join operations, as well as pivoting operations (inspired by
Excel).
I would ask the group for patience as I present the proposals over the
next week. With that in mind, I'd like to present my first proposed
addition to clojure.contrib.core, the proj function.
* proj & comp *
In order to understand proj(ection), I'd like to first talk about comp
(osition). Comp can be defined in terms of reduce like so:
(defn my-comp [& fns]
(fn [args]
(reduce
(fn[accum f](f accum))
(conj (reverse (seq fns)) args))))
Granted, this isn't 100% equivalent to the Clojure comp function, but
it is very, very close. What it demonstrates is that comp applies a
list of functions in series using reduce. After writing Clojure for a
while, I found frequent need to apply a list of functions in parallel
using map. proj can be defined as follows
(defn proj [& fns]
(fn[arg] (map #(% args) fns)))
Notice that proj creates a closure. Initially, I used this as a way
to access values from a map.
user=>(def test-map {:a "1" :b "2" :c "3" :d "4"})
user=>((proj :a :c) test-map)
("1" "3")
However, as I used proj more and more, I found it to be a useful way
perform many operations on a map at once. For example
;assume parse-int turns a string to an int appropriately
user=>((proj :a (comp parse-int :c)) test-map)
("1" 3)
Since proj returns a closure, it is very useful in any place I would
use a map operation as well. For example, this made turning a list of
maps into a list of lists very easy. Also, this made it very easy to
determine if a sub-selection of a hash-map is equal to another hash-
map
user=>(def test-proj (proj :a :c))
user=>(= (test-proj {:a 1 :b 2 :c 3}) (test-proj {:a 1 :b 34 :c 3}))
true
One thing that is very interesting is that this function allows me to
simulate the behavior of let in a point-free style. This is something
I am still experimenting with.
;This is deliberate overkill for a small example
;Generate a list of squares
;Notice that the proj fn uses the range twice
user=>((partial map (proj identity #(* % %)))
(range 1 6))
((1 1) (2 4) (3 9) (4 16) (5 25))
I suspect that there are other uses for this type of function out
there that others can see more clearly than me. If this is something
others would be interested in, I'd gladly submit a patch.
For those of you that would like to further evaluate the proposal, you
can find the code in my library here:
http://github.com/francoisdevlin/devlinsf-clojure-utils/
Look in the lib.devlinsf.core namespace
In a few days I'll post some proposed additions to the map-utils
library. For now, I thank you for your time.
Sean Devlin