Use of volatile! in stateful transducers

[Jörg Winter]

Hi,

As seen in this example of a stateful transducer...

http://crossclj.info/ns/org.clojure/clojure/latest/clojure.core.html#_partition-by

... I am wondering what is the concrete motivation behind using 'volatile!' instead of say a simple (mutable) Java-Object wrapper ?

In the partition-all example, an ArrayList is used for aggregating the 'temporary' results for the step-function, so this mutable state is not concerned with threading at all.

Why then is there a threading-concern with pv (the volatile!) ?

As far as I understand, the step-function of a transducer is never(?) accessed concurrently by more than 1 thread.

Is volatile! necessary because transducers should be usable with core.async ?

Or is it just an easy way to get a mutable object in Clojure ?

Best,

Joerg

[Timothy Baldridge]

"As far as I understand, the step-function of a transducer is never(?) accessed concurrently by more than 1 thread." 

It's actually "one thread at a time". And you're right, stuff like Core.async may bounce a transducer between several different threads, but only 1 thread "owns" it at a given time. 

Timothy

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Jörg Winter]

So if I'd need that extra performance, say in a private library of transducers not intended to be shared with other Clojure developers, it is perfectly Ok to use a java.lang.Object instead of a volatile, right ?

J 

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/CjxK7xEsOKQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Alexander Gunnarson]

I was wondering the same thing, Jörg. This thread talks about it as well. I posted a note there which I will reproduce here for your convenience:

I think it's safe to assume that since `ArrayList` uses unsynchronized mutability internally (a quick review of the GrepCode entry for `ArrayList` confirms this), then we can rest assured that a `volatile` box as opposed to a totally unsynchronized mutable variable is unnecessary, even in the context of `fold`. After all, `reduce` (and by extension, `transduce`) is only ever going to be single-threaded unless the data structure in question unexpectedly implements a multithreaded reduce, which should never happen (and if it does, you likely have bigger problems). To be honest, I'm not sure why `volatile` is used in transducers instead of e.g. an `unsynchronized-mutable` box. There may be a good reason, but I'm not seeing it immediately. I'd love to learn.

[Alexander Gunnarson]

EDIT: Transducers are not safe in `fold` contexts either:

(let [f (fn [i x] (println (str "i " i " " (Thread/currentThread))) (flush) x)

      r-map-indexed #(r/folder %2 (map-indexed %1))]

  (->> [6 7 8 9 10]

       (r-map-indexed f)

       (r/fold 1 (fn ([] (vector)) ([x] x) ([a b] (into a b))) conj)))

Produces:

i 0 Thread[ForkJoinPool-1-worker-2,5,main]
i 2 Thread[ForkJoinPool-1-worker-1,5,main]
i 3 Thread[ForkJoinPool-1-worker-1,5,main]
i 4 Thread[ForkJoinPool-1-worker-1,5,main]
i 1 Thread[ForkJoinPool-1-worker-3,5,main]

So you would have to be careful to create different `map-indexed` transducers for single-threaded and multi-threaded contexts.