post-query and transform-by

[CGAT]

I have a small feature request/idea.

It's fairly common (at least for me) to extract particular
information from all of the records returned by a select
query, with the information that is extracted varying
at different points in the code, even if the same query
is used.

This leads to constructs like

   (mapv :id (select ...))  and  (mapv :key (select ...))

popping all over which is both more ugly and less composable than it
could be. We could do (->> query select (mapv :id)) and
(->> query select (mapv :key)), allowing query reuse, which is better.
But that makes the transformation visible at use time, and in many cases, we
might prefer to keep the extraction logic at the query definition.

The post-query construct could work for this purpose, but, the
post-query operations act on the entire result set. They also happen
before the registered transforms, if any. We could thread
(post-query (partial mapv :key)) into the queries, but that is
both a bit wordy and is fragile because it must account for the
structure of any already-registered transforms.

Instead, I'd propose a new query operation  transform-by,
which adds a per-item results transform to the current query
only. (I'm not tied to the name.:)

  (select entity
     ...
     (transform-by f)
     (transform-by g)
     (transform-by h)
     ...)

This causes transforms f, g, and h (in that order) to act
on each item in the results set, after any pre-registered
transforms.  For instance,
(transform-by :key) or (transform-by (juxt :id :key))
or (transform-by (comp edn/read-string :info)).

The function could be defined as follows:

(defn transform-by
  "Add a per-item result transform for this query only"
  [query f]
  (let [entity (:ent query)
        vec?   (vector? entity)
        ks     (if vec? [:ent 0 :transforms] [:ent :transforms])]
    (if (and vec? (not (map? (get entity 0))))
      query
      (update-in query ks conj f))))

I handled the vector case here becaue I use aliasing at times, as I
described in a recent post ('aliasing with vector entities'). As the
Korma code is right now, the transform will not be applied in the
aliased case because the :ent field of the query is a vector of maps
rather than a map. But if a small change to korma.core/apply-transforms,
as I described, then this will work in both cases. To punt on the vector
case, just have the function return query as is. But the vector part does
no harm.

Either way, this lets us do

(let [widget-q  (-> (select* widgets) ....)
      widget-ids  (-> widget-q (transform-by :id))
      widget-keys  (-> widget-q (transform-by :key))]
 ...
 (select widget-keys)
 ...
 (select widget-ids)
 ...
 (-> widget-q (transform-by (juxt :id :key)) select (partial apply hash-map))
 ...)
 

or whatever. It's a small thing I know, but I've been trying it and do like the flexibility.

Thanks.