"Oh, yeah, transients are fast!"

[Raoul Duke]

hm.

i copied and pasted the examples from the transient page into a local
test.clj, and i don't really see a significant performance difference
(it should be an order of magnitude according to the web page).
macbook pro. just downloaded sources and built
clojure-1.1.0-alpha-SNAPSHOT.jar for this. is there something about
having to have the jit warm up or something over a zillion runs?

help? :-)

user=> (load-file "/tmp/test.clj")
#'user/vrange2
user=> (time (def v2 (vrange2 1000000)))
"Elapsed time: 502.842 msecs"
#'user/v2
user=> (time (def v2 (vrange2 1000000)))
"Elapsed time: 407.568 msecs"
#'user/v2
user=> (time (def v2 (vrange2 1000000)))
"Elapsed time: 364.29 msecs"
#'user/v2

user=> (time (def v (vrange 1000000)))
"Elapsed time: 558.235 msecs"

user=> (. System getProperty "java.version")
"1.5.0_20"

[Stuart Halloway]

I have seen perf consistent with the documentation when using Java 6
and server vm.

Stu

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient
> with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Richard Newman]

> clojure-1.1.0-alpha-SNAPSHOT.jar for this. is there something about
> having to have the jit warm up or something over a zillion runs?

I see an immediate speed improvement from transients with a recent
Clojure, and after a few runs the difference is obvious.

user=> (time (def v (vrange 1000000)))

"Elapsed time: 342.455 msecs"

user=> (time (def v2 (vrange2 1000000)))

"Elapsed time: 39.054 msecs"

Often vrange takes 800+msec, even after a number of runs.

I'm using 1.6.0_15, -server.

Try with a 1.6 JVM...

[Raoul Duke]

> Try with a 1.6 JVM...

wow. it actually got worse than when i was using 1.5. ... so much for
hallowed write-once-run-the-same-anywhere-ish of the jvm, d'oh.

Clojure 1.1.0-alpha-SNAPSHOT

user=> (load-file "/tmp/test.clj")
#'user/vrange2

user=> (. System getProperty "java.version")

"1.6.0_15"

user=> (time (def v2 (vrange2 1000000)))

"Elapsed time: 329.906 msecs"
#'user/v2

user=> (time (def v2 (vrange2 1000000)))

"Elapsed time: 888.734 msecs"
#'user/v2

user=> (time (def v2 (vrange2 1000000)))

"Elapsed time: 546.982 msecs"
#'user/v2

user=> (time (def v2 (vrange2 1000000)))

"Elapsed time: 517.969 msecs"
#'user/v2

user=> (time (def v (vrange 1000000)))

"Elapsed time: 1649.449 msecs"
#'user/v

user=> (time (def v (vrange 1000000)))

--- never came back, i had to ^C^C (in emacs buffer)!!

[Raoul Duke]

> user=> (time (def v (vrange 1000000)))
> --- never came back, i had to ^C^C (in emacs buffer)!!

p.s. so at least the transient version never did that ;-)

[John Harrop]

You've got some kind of system problem confounding your results, I'll bet. It got slower? One test actually hung?

My suspicion, of course, lies with the emacs environment you've just confessed to using. Half the traffic on this list at times is from people having problems with configuring either emacs or vi to work well with Clojure.

Try it in a modern IDE, or from the command line, and see if the problem goes away. If it does, then you know the likely culprit and it isn't Clojure's transients. :)

[Michael Wood]

2009/11/21 John Harrop <jharr...@gmail.com>:

To be fair, John, you've had an issue or two with the REPL provided by
your modern IDE too :)

Raoul, are you using the server VM? Not sure how much of a difference
it makes, and I haven't benchmarked this myself.

--
Michael Wood <esio...@gmail.com>

[John Harrop]

On Sun, Nov 22, 2009 at 6:05 AM, Michael Wood <esio...@gmail.com> wrote:

> You've got some kind of system problem confounding your results, I'll bet.
> It got slower? One test actually hung?
> My suspicion, of course, lies with the emacs environment you've just
> confessed to using. Half the traffic on this list at times is from people
> having problems with configuring either emacs or vi to work well with
> Clojure.
> Try it in a modern IDE, or from the command line, and see if the problem
> goes away. If it does, then you know the likely culprit and it isn't
> Clojure's transients. :)

To be fair, John, you've had an issue or two with the REPL provided by
your modern IDE too :)

Nothing so serious as a hang, though, and at least I can do basic things in my IDE without reaching for the frelling manual every two minutes to look up some key-combination :)

[David Brown]

On Sun, Nov 22, 2009 at 11:45:46AM -0500, John Harrop wrote:

>Nothing so serious as a hang, though, and at least I can do basic things in
>my IDE without reaching for the frelling manual every two minutes to look up
>some key-combination :)

I suspect both models are going to be important. I feel the same way
you do when I'm running an IDE rather than vim.

Oddly enough, I've found myself most productive with a repl in it's
own window, and a short script (below) in user.clj to let me specify a
namespace to test.

I put this in a 'repl' directory that I only add to my classpath for
the repl script.

David

(println "Running user startup")
(use 'clojure.contrib.repl-utils)
(use 'clojure.contrib.str-utils)
(use 'clojure.contrib.java-utils)
(use 'clojure.contrib.def)
(use 'clojure.contrib.ns-utils)
(use 'clojure.contrib.test-is)
(use 'clojure.contrib.sql)
(use 'clojure.stacktrace)

(set! *warn-on-reflection* true)

;;; The NS that we are testing/working on.
;;; This can be given through the java property work-ns, e.g.
;;; ./run.sh -Dwork-ns=org.davidb.chunk-file
(def *work-ns*
(symbol
(or (System/getProperty "work-ns")
'org.davidb.foo)))

;;; A testing NS.
(def *test-ns*
(symbol
(or (System/getProperty "test-ns")
(symbol
(re-sub #"[^\.]+$" #(str "test-" %)
(name *work-ns*))))))

; (use *work-ns*)
(defn l [] (binding [*warn-on-reflection* true]
(use *work-ns* :reload)))
(defn t []
(binding [*warn-on-reflection* true]
(require *test-ns* :reload)
(run-tests *test-ns*)))
(defn i [] (in-ns *work-ns*))

; Load the User's test-user.clj if present.
(try (load "/test-user")
(catch java.io.FileNotFoundException e
(println "No test-user.clj found in classpath.")))

[Raoul Duke]

i have tried:

1.5
1.6
1.6 -server
the last i did both in repl-in-emacs, and in a repl-in-straightup-shell.

the numbers i get are weird. it does seem like v2 is faster than v,
but never gets stupendously fast (never faster than 500 msec on a dual
core macbook pro 2.2ghz core 2 duo 4gb ram), and v does a strange
thing where i run v2 a bunch then the first time i run v it is like 2x
v2, and then subsequent runs of timing v are like 8x v2. and here i
thought hotspot was supposed to get faster, not slower, over time.

oy veh.

[sross]

I believe that this is most likely a symptom of the Apple JVM and not
that of transients
as the change from persistents to transients is far more substantial
on one of our linux
servers than it is on my macbook pro (6x vs 2x speedup)

I'm not entirely sure as to why this is the case but I suspect that GC
has something to
do with it as the following example, which simply bumps a counter in a
map, exhibits
the more impressive performance improvement

(defn make-n [x n]
(loop [i 0 v []]
(if (< i n)
(recur (inc i) (conj v x))
v)))

(defn make-n2 [x n]
(loop [i (int 0) v (transient [])]
(if (< i n)
(recur (inc i) (conj! v x))
(persistent! v))))

(dotimes [x 10]
(do (System/gc) (time (make-n 1 100000)) 1))

(dotimes [x 10]
(do (System/gc) (time (make-n2 1 100000)) 1))

[Raoul Duke]

> I believe that this is most likely a symptom of the Apple JVM and not

yeah, given Apple's wonderful treatment of Java over the years, i
could believe your theory.

[Richard Newman]

> I believe that this is most likely a symptom of the Apple JVM and not
> that of transients as the change from persistents to transients is
> far more substantial
> on one of our linux servers than it is on my macbook pro (6x vs 2x
> speedup)

Not necessarily so. My times earlier in this thread (4GB Core 2 Duo
MacBook Pro, Apple's JDK 6, Regular Leopard) showed a 10x speedup.

Most likely it's related to the JVM version (1.5 by default), the
amount of memory allocated (I use -Xmx512m), the amount of L2 cache, a
HotSpot tuning parameter, or something else along those lines.

There are far too many possibilities to consider, and too little
evidence, to support a conclusion of "it's just Apple's JVM".

[Raoul Duke]

> Most likely it's related to the JVM version (1.5 by default), the
> amount of memory allocated (I use -Xmx512m), the amount of L2 cache, a
> HotSpot tuning parameter, or something else along those lines.
>
> There are far too many possibilities to consider, and too little
> evidence, to support a conclusion of "it's just Apple's JVM".

(i love how supposedly we've come so far with our systems, only to
have them become overly complex. it is to sigh. on the other hand i
guess it is 'job security'.)

[Richard Newman]

> (i love how supposedly we've come so far with our systems, only to
> have them become overly complex. it is to sigh. on the other hand i
> guess it is 'job security'.)

Heh, true. Reading articles about JVM tuning reminds me what the "M"
stands for -- it's as complicated a topic as optimizing the interface
between your code and a real machine.

E.g.,

http://blogs.sun.com/bharath/entry/taming_gc_for_sip_applications
http://blog.headius.com/2009/01/my-favorite-hotspot-jvm-flags.html

Your point about job security is spot on, though -- every time the
topics of VHLLs, autonomic systems, and so on arise, I gently point
out that somebody needs to figure out what's going on when they stop
working, and the smarter they get, the more fiendish the problems!

[Sergey Didenko]

Here are my results.

The transient version performs slightly better on windows 32bit client
JVM, and considerably better on linux 64bit server JVM (they are both
1.06.0_17)