side-effecting re-/transducers

335 views
Skip to first unread message

Sam Raker

unread,
Jun 25, 2015, 2:30:21 PM6/25/15
to clo...@googlegroups.com
This seems bad, is this bad:

(defn to-db
 
[conn]
 
(fn
     
([val] (upload-to-my-db conn val))
     
([_ val] (upload-to-my-db conn val)))

(defn -main []
 
(transduce my-preprocessing-xf (to-db (get-db-conn)) seq-of-things-to-preprocess-and-upload))

I ask only because
1) Plugging the side-effecting part into the transducer pipeline seems cleaner and potentially more efficient than e.g. 
(doseq [v (sequence my-preprocessing-xf sea-of-things)] (upload-to-my-db conn v))
which is what I was doing
2) The addition of `run!`[1] in 1.7 seems to perhaps implicitly condone this kind of thing. There's very little out there about it, though, so I could very well be wrong.




Stuart Sierra

unread,
Jun 25, 2015, 4:01:12 PM6/25/15
to clo...@googlegroups.com, sam....@gmail.com
Hi Sam,

Transducers are a new feature, and best practices are still emerging.

Transducing/reducing is always non-lazy, so it's *less* risky to have side effects in a reduce compared with side effects in a lazy sequence.

Still, I would offer the same advice I give for lazy sequences. Keep your side-effects as separated as possible from the purely-functional computation that feeds them.

Don't put a side-effect in the middle of a chain of map/filter-like operations. Do all your map/filter/whatever with pure functions, then process the entire result non-lazily to do the side effects.

The "driver" for that process could be `doseq`, `run!`, `transduce`, or even `reduce`. `doseq` has overhead to allocate the sequence, which the others avoid.

–S
Reply all
Reply to author
Forward
0 new messages