Why not mutable locals?

[Rich Hickey]

This question came in private email (please use the group!), but I
thought it might be of general interest:

-------------
Anyway, I've been studying language design quite a bit over the
past several weeks, and obviously something that sets Clojure apart
from many languages is the fact that variables are not really
variables in that they can be rebound (at least not locals anyway). Is
there a particular reason for this other than it was something you
liked? What I mean is, could you have mutable variables and immutable
data structures and still get much the same result as Clojure, or do
the immutable variables make certain things possible?
-------------

One of the primary objectives of Clojure is that it makes it
straightforward to build programs that behave well in the context of
multiple threads. That means that any mutation facilities must be
thread aware, as are Clojure's Vars, Refs and Agents. If you use these
constructs (and Clojure's persistent data structures) and avoid Java
side-effects, you can write robust multi-threaded programs easily.

If locals were variables, i.e. mutable, then closures could close over
mutable state, and, given that closures can escape (without some extra
prohibition on same), the result would be thread-unsafe. And people
would certainly do so, e.g. closure-based pseudo-objects. The result
would be a huge hole in Clojure's approach.

Without mutable locals, people are forced to use recur, a functional
looping construct. While this may seem strange at first, it is just as
succinct as loops with mutation, and the resulting patterns can be
reused elsewhere in Clojure, i.e. recur, reduce, alter, commute etc
are all (logically) very similar. Even though I could detect and
prevent mutating closures from escaping, I decided to keep it this way
for consistency. Even in the smallest context, non-mutating loops are
easier to understand and debug than mutating ones. In any case, Vars
are available for use when appropriate.

I would hope that programmers without prior functional programming
experience would grow accustomed to Clojure's functional style, and,
even if they never do multithreaded programming, would benefit from
the more robust and easier-to-understand programs it yields.

Rich

[John Cowan]

On Jan 23, 2008 9:30 AM, Rich Hickey <richh...@gmail.com> wrote:

> If locals were variables, i.e. mutable, then closures could close over
> mutable state, and, given that closures can escape (without some extra
> prohibition on same), the result would be thread-unsafe. And people
> would certainly do so, e.g. closure-based pseudo-objects. The result
> would be a huge hole in Clojure's approach.

FWIW, although Scheme supports mutable locals, it's quite common for
compilers to treat that as syntax sugar, and to replace references to
mutable variables with immutable references to mutable boxes on the
heap. This is called "assignment conversion". AFAICT, though, a
Clojure Var can't be the value of a local variable -- it can only be
referred to by its name.

--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures

[Rich Hickey]

On Jan 23, 11:22 am, "John Cowan" <johnwco...@gmail.com> wrote:

> On Jan 23, 2008 9:30 AM, Rich Hickey <richhic...@gmail.com> wrote:
>
> > If locals were variables, i.e. mutable, then closures could close over
> > mutable state, and, given that closures can escape (without some extra
> > prohibition on same), the result would be thread-unsafe. And people
> > would certainly do so, e.g. closure-based pseudo-objects. The result
> > would be a huge hole in Clojure's approach.
>
> FWIW, although Scheme supports mutable locals, it's quite common for
> compilers to treat that as syntax sugar, and to replace references to
> mutable variables with immutable references to mutable boxes on the
> heap. This is called "assignment conversion".

I'm fully aware of that conversion - prior to its move to primarily-
functional Clojure had mutable locals and did exactly that (using
clojure.lang.Box :) . But whether locals live on the heap or stack
isn't relevant here (they have to live on the heap in the JVM in order
to escape), what matters is the mutability. As long as it can be
shared, which it must be in order to support closure semantics, a
mutable 'box' has the same threading problems.

Note that by problems I don't mean that meaningful multithreaded
semantics couldn't be assigned - they could just be last-one-in-wins
volatiles. What I mean is that they would be awful as primitives for
multithreaded programs, with no read-modify-write or coordination
capabilities.

> AFAICT, though, a
> Clojure Var can't be the value of a local variable -- it can only be
> referred to by its name.
>

Vars are first class, although there aren't any helper functions yet.
Here's how they work:

(import '(clojure.lang Var))
(def x (. Var (create)))

;x is a var that holds a var, could just as easily be a local

(. x (get))
-> java.lang.IllegalStateException: Var null is unbound.

(. x (bindRoot "fred"))
(. x (get))
-> "fred"

(try
(. Var (pushThreadBindings {x "ethel"}))
(. x (get))
(finally (. Var (popThreadBindings))))
-> "ethel"

(. x (get))
-> "fred"

There's also a set method, which throws if the var is not thread-
locally bound.

Note that if one were to use a Var as the value of a local variable it
still wouldn't be the same as a true mutable local, as it would have
different behavior were any capturing closure to be run outside the
dynamic scope of the binding.

Rich

[Rich Hickey]

On Jan 23, 1:45 pm, Rich Hickey <richhic...@gmail.com> wrote:

> Vars are first class, although there aren't any helper functions yet.
> Here's how they work:
>

Hmmm... now that I've explained that, I expect everyone is writing
their own with-local-vars macro right now... :(

[Rich Hickey]

Here it is, for those interested:

(defn take-nth [n coll]
(when (seq coll)
(lazy-cons (first coll) (take-nth n (drop n coll)))))

(defn interleave [& colls]
(apply concat (apply map list colls)))

(defn get-var [#^clojure.lang.Var x]
(. x (get)))

(defn set-var [#^clojure.lang.Var x val]
(. x (set val)))

(defmacro with-local-vars [name-vals-vec & body]
`(let [~@(interleave (take-nth 2 name-vals-vec)
(repeat '(. clojure.lang.Var (create))))]
(try
(. clojure.lang.Var (pushThreadBindings (hash-map ~@name-vals-
vec)))
~@body
(finally (. clojure.lang.Var (popThreadBindings))))))


(with-local-vars [a 1 b 2 c 3]
(set-var c 10)
(+ (get-var a) (get-var b) (get-var c)))

-> 13

Rich