Understanding init (the zero arity function) for transducers.

Patrick Curran

unread,

Feb 29, 2016, 4:27:15 PM2/29/16

to Clojure

Hi,

I was trying to write a transducer and the 0-arity part of it never got called, which was unexpected. I did some searching and found this post: https://groups.google.com/forum/#!msg/clojure/uVKP4_0KMwQ/-oUJahvUarIJ. What Dan is proposing in that post would essentially solve my problem, but it doesn't look like his proposal has gotten much traction...

Specifically I was trying to implement scan.

(defn scan
([f] (scan f (f)))
([f init]
   (fn [xf]
     (let [state (volatile! init)]
       (fn
         ([] (xf (xf) init))
         ([result] (xf result))
         ([result input]
          (let [next-state (f @state input)]
            (vreset! state next-state)
            (xf result next-state))))))))

Which results in the following:

(require '[clojure.core.reducers :as r])
(r/reduce ((scan + 3) conj) [1 2 3])
=> [3 4 6 9]
(transduce (scan + 3) conj [1 2 3])
=> [4 6 9]
(transduce (scan + 3) conj (((scan + 3) conj)) [1 2 3])
=> [3 4 6 9]

My expectation would be that we'd always get the 3 at the front of the vector.

I'm actually using core.async and I'm expecting that the initial value be available to be taken from the channel.

(require '[clojure.core.async :as a :include-macros true])
(def c (a/chan 1 (scan + 3)))
(a/go (println (a/<! c)))
; expecting 3 to immediately be printed.
(a/>!! c 1)
=> 4

So this is more of a conceptual thing rather than just how transduce is implemented.

I'd love to hear other people's thoughts on this. I'm quite new, but Dan's proposal definitely feels "correct" and the current implementation definitely feels "wrong".

--Patrick

Alex Miller

unread,

Feb 29, 2016, 5:10:53 PM2/29/16

to Clojure

I think that Rich had an objection to this, however in the haziness of time I don't recall specifically what it was. If I get a chance, I will ask him this week.

Patrick Curran

unread,

Mar 8, 2016, 9:43:27 PM3/8/16

to Clojure

Thanks Alex,

If you ever do get a chance, I'd be curious to know what it was. The more I think about it the more I think Dan is correct. Also "scan" seems like a natural thing that one should be able to do without having to jump through hoops.

Stephen Nelson

unread,

Mar 9, 2016, 3:24:12 PM3/9/16

to Clojure

This was discussed further in http://dev.clojure.org/jira/browse/CLJ-1569

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sean Corfield

unread,

Mar 9, 2016, 8:15:43 PM3/9/16

to Clojure Mailing List

Can we at least get an example of situation where the zero-arity version would be called?

Right now it seems that all the transducer literature out there says there must be three arities – and that’s how map etc are defined – but it doesn’t seem, based on various people’s simple tests, that the zero-arity version is ever called… so why would we define it?

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

Philos Kim

unread,

Mar 9, 2016, 8:42:29 PM3/9/16

to Clojure

I wrote the next example to trace the inner workings of transducer. I hope that this will help.

The next filter-t(transducer), map-t(transducer) and conj-t(reducer) functions are excerpted from the filter, map and conj from clojure.core and then simplified and modified to focus on the understanding of the inner workings.

(defn filter-t

[pred]

;; The first fn is a transducer. It receives the reducer rf and returns

;; the reducer(the second fn part of this code).

(fn [rf]

(fn

([]

(let [r (rf)]

(println "filter-t [] post: result =" r)

r))

([result]

(println "filter-t [result] pre: result =" result)

(let [r (rf result)]

(println "filter-t [result] post: result =" r)

r))

([result input]

(println "filter-t [result input] pre: result =" result ", input =" input)

(let [r (if (pred input)

(rf result input)

result)]

(println "filter-t [result input] post: result =" r)

r)))))

(defn map-t

[f]

(fn [rf]

(fn

([]

(let [r (rf)]

(println "map-t [] post: result =" r)

r))

([result]

(println "map-t [result] pre: result =" result)

(let [r (rf result)]

(println "map-t [result] post: result =" r)

r))

([result input]

(println "map-t [result input] pre: result =" result ", input =" input)

(let [r (rf result (f input))]

(println "map-t [result input] post: result =" r)

r)))))

(defn ^:static conj-t

[]

;; This is a reducer itself, not a transducer, because it doesn't receive the reducer

;; and return a reducer as a transducer.

(fn

([]

(println "conj-t []: result =" [])

[])

([result]

(println "conj-t [result]: result =" result)

result)

([result input]

(println "conj-t [result input] pre: result =" result ", input =" input)

(let [r (. clojure.lang.RT (conj result input))]

(println "conj-t [result input] post: retrun =" r)

r) )))

The oupput is edited to facilitate the understandings.

(def xform (comp (filter-t odd?) (map-t #(* % 10))))

(transduce xform (conj-t) [1 2 3 4 5])

;>> conj-t []: result = []

;

; filter-t [result input] pre: result = [] , input = 1

; map-t [result input] pre: result = [] , input = 1

; conj-t [result input] pre: result = [] , input = 10

; conj-t [result input] post: retrun = [10]

; map-t [result input] post: result = [10]

; filter-t [result input] post: result = [10]

;

; filter-t [result input] pre: result = [10] , input = 2

; filter-t [result input] post: result = [10]

;

; filter-t [result input] pre: result = [10] , input = 3

; map-t [result input] pre: result = [10] , input = 3

; conj-t [result input] pre: result = [10] , input = 30

; conj-t [result input] post: retrun = [10 30]

; map-t [result input] post: result = [10 30]

; filter-t [result input] post: result = [10 30]

;

; filter-t [result input] pre: result = [10 30] , input = 4

; filter-t [result input] post: result = [10 30]

;

; filter-t [result input] pre: result = [10 30] , input = 5

; map-t [result input] pre: result = [10 30] , input = 5

; conj-t [result input] pre: result = [10 30] , input = 50

; conj-t [result input] post: retrun = [10 30 50]

; map-t [result input] post: result = [10 30 50]

; filter-t [result input] post: result = [10 30 50]

;

; filter-t [result] pre: result = [10 30 50]

; map-t [result] pre: result = [10 30 50]

; conj-t [result]: result = [10 30 50]

; map-t [result] post: result = [10 30 50]

; filter-t [result] post: result = [10 30 50]

;=> [10 30 50]

From the above output, my conclusion is that the init part(with no argument) of reducer is called only in the last reducer(conj-t in this case) and never called in the reducers within the transducers(filter-t and map-t).

If you give the init value to the transduce function as follows,

(transduce xform (conj-t) [] [1 2 3 4 5])

;>> filter-t [result input] pre: result = [] , input = 1

; map-t [result input] pre: result = [] , input = 1

; conj-t [result input] pre: result = [] , input = 10

; conj-t [result input] post: retrun = [10]

; map-t [result input] post: result = [10]

; filter-t [result input] post: result = [10]

;

; filter-t [result input] pre: result = [10] , input = 2

; filter-t [result input] post: result = [10]

;

; filter-t [result input] pre: result = [10] , input = 3

; map-t [result input] pre: result = [10] , input = 3

; conj-t [result input] pre: result = [10] , input = 30

; conj-t [result input] post: retrun = [10 30]

; map-t [result input] post: result = [10 30]

; filter-t [result input] post: result = [10 30]

;

; filter-t [result input] pre: result = [10 30] , input = 4

; filter-t [result input] post: result = [10 30]

;

; filter-t [result input] pre: result = [10 30] , input = 5

; map-t [result input] pre: result = [10 30] , input = 5

; conj-t [result input] pre: result = [10 30] , input = 50

; conj-t [result input] post: retrun = [10 30 50]

; map-t [result input] post: result = [10 30 50]

; filter-t [result input] post: result = [10 30 50]

;

; filter-t [result] pre: result = [10 30 50]

; map-t [result] pre: result = [10 30 50]

; conj-t [result]: result = [10 30 50]

; map-t [result] post: result = [10 30 50]

; filter-t [result] post: result = [10 30 50]

;=> [10 30 50]

even the init part(with no argument) of reducer(conj-t in this case) is not called as above.

Wiithin into and sequence functions, the init part(with no argument) of reducer are never called as follows.

(into () xform [1 2 3 4 5])

;>> filter-t [result input] pre: result = () , input = 1

; map-t [result input] pre: result = () , input = 1

; map-t [result input] post: result = (10)

; filter-t [result input] post: result = (10)

;

; filter-t [result input] pre: result = (10) , input = 2

; filter-t [result input] post: result = (10)

;

; filter-t [result input] pre: result = (10) , input = 3

; map-t [result input] pre: result = (10) , input = 3

; map-t [result input] post: result = (30 10)

; filter-t [result input] post: result = (30 10)

;

; filter-t [result input] pre: result = (30 10) , input = 4

; filter-t [result input] post: result = (30 10)

;

; filter-t [result input] pre: result = (30 10) , input = 5

; map-t [result input] pre: result = (30 10) , input = 5

; map-t [result input] post: result = (50 30 10)

; filter-t [result input] post: result = (50 30 10)

;

; filter-t [result] pre: result = (50 30 10)

; map-t [result] pre: result = (50 30 10)

; map-t [result] post: result = (50 30 10)

; filter-t [result] post: result = (50 30 10)

;=> (50 30 10)

(sequence xform [1 2 3 4 5])

;>> filter-t [result input] pre: result = nil , input = 1

; map-t [result input] pre: result = nil , input = 1

; map-t [result input] post: result = nil