strange result from the STM example

[DiG]

Hi,

I've tried executing the STM example and the first time I am getting a
different sum. I am putting the transcript of the session bellow.

[16:08:31]$ java -jar clojure.jar src/boot.clj
Clojure
user=> (import '(java.util.concurrent Executors))
(defn test-stm [nitems nthreads niters]
(let [refs (apply vector (map ref (replicate nitems 0)))
exec (. Executors (newFixedThreadPool nthreads))
tasks (map (fn [t]
(fn []
(dotimes n niters
(sync nil
(dolist r refs
(set r (+ @r t)))))))
(take nthreads (iterate inc 1)))]
(. exec (invokeAll tasks))
(. exec (shutdown))
(map deref! refs)))
(time (test-stm 10 10 10000))nil
user=> #<Var: user/test-stm>
user=>
()
"Elapsed time: 2377.848376 msecs"
(461489 461489 461489 461489 461489 461489 461489 461489 461489
461489)
user=> ()
user=> (time (test-stm 10 10 10000))
"Elapsed time: 786.606839 msecs"
(550000 550000 550000 550000 550000 550000 550000 550000 550000
550000)
user=>

I can repeat the execution and numbers look quite different every time
I execute it. I am running it on the quad core machine in 64 bit
Linux. It could be a bug in 1.6 jvm. As I understand it shutdown is
suppose to wait for already running tasks to complete, but it seems it
does not.

If I add (. exec (awaitTermination 10 (. TimeUnit SECONDS))) after
the invokeAll then it waits for 10 seconds and then exits with again
sometimes incorrect results.

I am just playing around with the code, but if somebody can shed any
light on what is going on, I would appreciate it.

Regards,
DiG

[Rich Hickey]

You may be getting an exception in one of the threads, which can
happen since this example is maximally contentious and there are retry
limits

You won't see the exception unless you try to get the result of the
thread. Try this:

(defn test-stm [nitems nthreads niters]
(let [refs (apply vector (map ref (replicate nitems 0)))
exec (. Executors (newFixedThreadPool nthreads))
tasks (map (fn [t]
(fn []
(dotimes n niters
(sync nil
(dolist r refs
(set r (+ @r t)))))

t))

(take nthreads (iterate inc 1)))]

(let [ret (. exec (invokeAll tasks))]
(. exec (shutdown))
[ (map (fn [f] (. f (get))) ret) (map deref! refs)])))

[Rich Hickey]

> Yep, I've got the exception:

Well, that's good news in that the STM is working correctly. In
overly contentious situations that exception will always be possible.
I'm a bit puzzled that you see it with only 10 threads and 10 items -
I run this on a dual and quad (OS X) boxes and never see the retry
exception.

The default retry limit is currently set very high IMO, at 1000. So
it might be a difference in the Linux JVM wait/notify fairness.

In any case, 'normal' use of the STM, unlike this pounding, will have
less continuous contention.

Also, I will be exposing the retry limit and timeout times in the
first arg to sync, soon. This will let each transaction control those
limits.

Let me know if you have any questions and thanks for your feedback.

Rich

On Oct 21, 2007, at 7:40 PM, DiG wrote:

> Yep, I've got the exception:

>
> user=> (time (test-stm 10 10 10000))

> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:
> 39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:
> 25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:37)
> at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:25)
> at user.test_stm$fn__209.invoke(Unknown Source)
> at clojure.map.invoke(boot.clj:405)
> at user.test_stm.invoke(Unknown Source)
> at user.fn__211.invoke(Unknown Source)
> at clojure.lang.AFn.applyToHelper(AFn.java:156)
> at clojure.lang.AFn.applyTo(AFn.java:149)
> at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:1711)
> at clojure.lang.Compiler.eval(Compiler.java:2551)
> at clojure.lang.Compiler.main(Compiler.java:2724)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.Exception: Transaction failed after reaching retry limit
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> ... 15 more
> Caused by: java.lang.Exception: Transaction failed after reaching
> retry limit
> at clojure.lang.LockingTransaction.run(LockingTransaction.java:273)
> at
> clojure.lang.LockingTransaction.runInTransaction
> (LockingTransaction.java:
> 179)
> at user.test_stm$fn__206$fn__207.invoke(Unknown Source)
> at clojure.lang.AFn.call(AFn.java:29)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:619)