Mysterious performance anomalies

[Jason Wolfe]

I was doing some microbenchmarking earlier, and I noticed some very
very weird anomalies. If anyone could shed some light on what's
going on that would be awesome. (I'm using the latest SVN, and have
verified this on a totally clean repl).

Simplified as much as possible, the heart of what I observed is:

user> (prn (time (loop [i (int 0)] (if (< i (int 30000000)) (recur
(inc i)) i))))
"Elapsed time: 4247.477 msecs"
30000000
nil

user> (time (loop [i (int 0)] (if (< i (int 30000000)) (recur (inc i))
i)))
"Elapsed time: 128.37 msecs"
30000000

Weird, right? The prn is *outside* the loop, and yet it still affects
the timing somehow. Maybe this is something specific to printing?
Nope:

user> (first (time (loop [i (int 0)] (if (< i (int 30000000)) (recur
(inc i)) [i]))))
"Elapsed time: 4264.847 msecs"
30000000

user> (time (loop [i (int 0)] (if (< i (int 30000000)) (recur (inc i))
[i])))
"Elapsed time: 130.099 msecs"
[30000000]

But, some other expressions around the "time" don't affect it in the
same way:

user> (when (time (loop [i (int 0)] (if (< i (int 30000000)) (recur
(inc i)) [i]))) 12)
"Elapsed time: 130.236 msecs"
12

In case you were wondering, this has nothing to do with the "time"
macro.

user> (first (loop [i (int 0)] (if (< i (int 30000000)) (recur (inc
i)) [i])))
; ... 4 seconds pass on my stopwatch ...
30000000

And the slowness is by a multiplicative, not additive factor:

user> (first (time (loop [i (int 0)] (if (< i (int 60000000)) (recur
(inc i)) [i]))))
"Elapsed time: 8576.649 msecs"
60000000

user> (time (loop [i (int 0)] (if (< i (int 60000000)) (recur (inc i))
[i])))
"Elapsed time: 250.407 msecs"
[60000000]

I'm at a total loss for what's going on. Anyway, I'll stop here for
now in case I'm missing something stupid or obvious.

Thanks for your help!
Jason

[Christian Vest Hansen]

Here's my theory.

In your when example, "when" is a macro that is expanded to an "if"
special form. Your other examples, however, wrap your code in a
function call.

Now, functions in Clojure can't really take primitive arguments, so in
spite of your coercion efforts Clojure introduces boxing in your loop
and this is what slows you down.

A work-around is to either use a (def variable ...) or something like this:

(let [x (time (loop [i (int 0)] (if (< i (int 30000000)) (recur (inc
i)) i)))] (prn x))

How does that sound?

--
Venlig hilsen / Kind regards,
Christian Vest Hansen.

[Rich Hickey]

On Jan 16, 5:15 am, Christian Vest Hansen <karmazi...@gmail.com>
wrote:

The bytecode produced for the loop is exactly the same in both cases.
It seems the presence of first (or really anything) on the stack
during looping causes HotSpot to bail on optimizing. This is a case
where the bytecode is unlike any produced by javac (in Java, loops
cannot be expressions).

For now, I've made it so that when a loop occurs as an expression it
is lifted out into a fn, and the performance is identical.

SVN 1216 - thanks for the report.

Rich

[e]

This is probably a silly question, well outside the scope of this conversation (but maybe not, and I didn't want to start a whole thread on it).

Is it much much easier to make byte code than assembly code?  I mean, I understand why running on a VM makes sense as far as instantly inheriting all the massive amounts of things out here for Java .... but would it be possible to shoot for something down the road that, behind the scenes, uses jvm whenever you say, "import" . . .and writes and compiles assembler whenever you are doing purely algorithmic things?  In a way, I guess what I'm saying is that this was a fix that helped the hotspot compiler along.  But (and I'm TOTALLY ignorant about all of this so maybe I should do more reading before asking the question) that technology is a moving target and varies from one  VM to another.  It's sort of analogous to the discussion on why the Jython approach is problematic because it's got to stay in sync with Python.

Is this a silly question?

Thanks.

[Konrad Hinsen]

On Jan 16, 2009, at 15:47, e wrote:

> Is it much much easier to make byte code than assembly code? I
> mean, I understand why running on a VM makes sense as far as
> instantly inheriting all the massive amounts of things out here for
> Java .... but would it be possible to shoot for something down the
> road that, behind the scenes, uses jvm whenever you say,
> "import" . . .and writes and compiles assembler whenever you are
> doing purely algorithmic things?

There are (at least) two reasons not to go that way:

1) The JVM provides a lot of infrastructure for programming languages
that otherwise each compiler/interpreter would have to reinvent.
Memory handling with garbage collection, for example. And of course a
lot of platform-specific OS interfacing. Reinventing them is not only
a lot of work, but also creates compatibility barriers in multi-
language programming. It is very difficult to combine languages A and
B if each of them has its own memory management.

2) Compiling to a mix of native code and JVM bytecode brings up a lot
of ugly platform dependencies, the kind that makes the Java native
interface so unpleasant to use.

There is a reason why virtual machines like the JVM and CLI (.Net/
Mono) are gaining popularity. They provide security (by supervising
programs' acces to the machine resources), portability,
infrastructure, and the possibility to mix languages easily. There is
a performance price to pay for this, but apparently more and more
people are willing to do so, and at the same time that price is going
down.

Konrad.

[Stuart Sierra]

On Jan 16, 9:47 am, e <evier...@gmail.com> wrote:
> Is it much much easier to make byte code than assembly code?  

To some extent, yes. JVM bytecode is certainly simpler than x86
assembly. The big win is that bytecode is platform-neutral, so you
don't have to rewrite your compiler for each architecture. Java's
just-in-time compiler generates optimized assembly for every
architecture. It's not perfect, but steadily improving.

-Stuart Sierra

[Jason Wolfe]

> SVN 1216 - thanks for the report.

Thanks for the quick fix!

-Jason

[Mark H.]

On Jan 16, 6:47 am, e <evier...@gmail.com> wrote:
> Is it much much easier to make byte code than assembly code?

I'll chime in too to say that x86 is only king of the desktop / laptop
world -- many portable devices are ARM-based (and a lot of Windows
apps run on ARM), and there are other architectures used for
enterprise and HPC servers. Plus it's not clear to me that x86 will
win, esp. in power-constrained arenas. (All those legacy instructions
and the translation from x86 ops into reasonable microops eat power
and area.) I've dealt with at least six different instruction sets in
my HPC work and the JVM runs on at least five of them: instant
portability!

mfh

[e]

That's a great argument.  I need arguments like these.  I work with people who dismiss JVM.  Even though there are many non-Sun JVM's, folks say, "Sun is dead -> java is dead -> jvm is dead." ..... even though Java is the most popular language right now.
http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

I wonder if there will ever be a JM ... that is a chip that natively executes byte code.  I wonder what they'd have to say, then.  I think I'll do a Google search.  I also wonder if it was a tough decision for Rich to cut the CLI support.  I know he feels fine looking back.

[e]

I see.  From wikipedia: "A Jazelle-aware Java Virtual Machine (JVM) will attempt to run Java bytecodes in hardware, while returning to the software for more complicated, or lesser-used bytecode operations. ARM claim that approximately 95% of bytecode in typical program usage ends up being directly processed in the hardware."

That's awesome.

Also, I've only read the title and skimmed here, so far, but it seems relevant: http://blogs.vmware.com/performance/2008/08/esx-runs-java-v.html

[Mark H.]

On Jan 18, 8:48 am, e <evier...@gmail.com> wrote:
> That's a great argument.  I need arguments like these.  I work with people
> who dismiss JVM.  Even though there are many non-Sun JVM's, folks say, "Sun
> is dead -> java is dead -> jvm is dead." ..... even though Java is the most
> popular language right now.http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

A lot of people agree that having some kind of VM is very helpful. It
may not necessarily be the JVM in the future, but right now the JVM is
popular enough that it's worth using it rather than relying on some
other VM that might also go away. VMs provide portability, can do a
lot of low-level optimizations (that would otherwise have to go into
the compiler), and have useful services (like garbage collection) that
save folks like Rich a lot of trouble when implementing a new
language.

Saying we shouldn't use the JVM because Sun might go away is like
saying we shouldn't bother improving gas mileage in cars because
gasoline might go away. (Of course it pays to think about the long
term, but shorter-term gains are worth the work.)

mfh

>
> I wonder if there will ever be a JM ... that is a chip that natively
> executes byte code.  I wonder what they'd have to say, then.  I think I'll
> do a Google search.  I also wonder if it was a tough decision for Rich to
> cut the CLI support.  I know he feels fine looking back.
>

[e]

That's a solid arg, too . . . but it would be stronger if we weren't importing things from java all the time.  If we said like, "(gui-frame "hello"), which happened to be implemented as a JFrame . . . then that'd be even stronger.  Drop in a different REPL and you'd still get a JFrame-like thing even if it weren't from a java lib via a JVM.

[ivant]

On Jan 20, 3:14 am, e <evier...@gmail.com> wrote:
> That's a solid arg, too . . . but it would be stronger if we weren't
> importing things from java all the time.  If we said like, "(gui-frame
> "hello"), which happened to be implemented as a JFrame . . . then that'd be
> even stronger.  Drop in a different REPL and you'd still get a JFrame-like
> thing even if it weren't from a java lib via a JVM.
>

For me it doesn't make sense to invest in writing a GUI wrapper for
Swing (and for SWT, Qt-whatever-was-it's-name) and make it work on a
VVM (Virtual Virtual Machine). People who do and know GUI programming
now, will find it easier to work with their GUI lib directly, instead
of learning one more wrapper. Also there are quite a few differences
with GUI libraries, and you'll have to think of a way to wrap these.
And in the end you'll end up with imperfect wrapper which will miss
specific features from each library it supports. The GUIs done with
this wrapper will be most probably uglier or less functional than the
ones done with the underlying library directly, so who would want to
port them to VVM then?

Besides, GUIs aren't the only thing. You can say the same for
database access, persistence, sockets, and everything which is not (re)
implemented in Clojure.

For me, JVM is a good choice. JVM is multiplatform, with very good
garbage collector and with tons of libraries. And it's getting
better. IIRC, the only major complain Rich has about JVM is the lack
of tail-call optimization.

--
Ivan

[cliffc]

fyi... non of the bytecode machines made much headway; once you get
above a certain (actually fairly small) size device you can afford a
tiny JIT - and even a trivial JIT instantly blows away pure bytecode
interpreters; to the point that the energy consumed is actually less
even counting the JIT cost against you because the resulting code is
so much better.

Other things a JVM gives you:
- Code gen & Register allocation
- A reliable multi-threaded Memory Model, which otherwise varies from
platform to platform, and even within just X86 systems
- *fairly* reliable performance; obviously Clojure is walking the edge
here
- Already mentioned: OS interface, GC/memory allocation, Security,
Vast pre-existing libs
- Steady engineering providing continuous incremental gains

Cliff

On Jan 18, 9:31 am, e <evier...@gmail.com> wrote:
> I see.  From wikipedia: "A Jazelle-aware Java Virtual

> Machine<http://en.wikipedia.org/wiki/Java_Virtual_Machine>(JVM) will

> attempt to run Java bytecodes in hardware, while returning to the
> software for more complicated, or lesser-used bytecode operations. ARM claim
> that approximately 95% of bytecode in typical program usage ends up being
> directly processed in the hardware."
>
> That's awesome.
>
> Also, I've only read the title and skimmed here, so far, but it seems
> relevant:http://blogs.vmware.com/performance/2008/08/esx-runs-java-v.html
>

> On Sun, Jan 18, 2009 at 11:48 AM, e <evier...@gmail.com> wrote:
> > That's a great argument.  I need arguments like these.  I work with people
> > who dismiss JVM.  Even though there are many non-Sun JVM's, folks say, "Sun
> > is dead -> java is dead -> jvm is dead." ..... even though Java is the most
> > popular language right now.
> >http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
>
> > I wonder if there will ever be a JM ... that is a chip that natively
> > executes byte code.  I wonder what they'd have to say, then.  I think I'll
> > do a Google search.  I also wonder if it was a tough decision for Rich to
> > cut the CLI support.  I know he feels fine looking back.
>

[Mark H.]

On Jan 20, 12:01 am, ivant <itosh...@gmail.com> wrote:
> On Jan 20, 3:14 am, e <evier...@gmail.com> wrote:
>
> > That's a solid arg, too . . . but it would be stronger if we weren't
> > importing things from java all the time.  If we said like, "(gui-frame
> > "hello"), which happened to be implemented as a JFrame . . . then that'd be
> > even stronger.  Drop in a different REPL and you'd still get a JFrame-like
> > thing even if it weren't from a java lib via a JVM.
>
> For me it doesn't make sense to invest in writing a GUI wrapper for

> Swing...

Plus other folks have invested the time already in writing cross-
platform GUI libs -- the only thing to do is establish cross-language
bindings.

mfh

[Jon Harrop]

On Tuesday 20 January 2009 08:01:19 ivant wrote:
> IIRC, the only major complain Rich has about JVM is the lack
> of tail-call optimization.

That's a pretty major problem. :-)

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

[Cosmin Stejerean]

On Tue, Jan 20, 2009 at 11:31 AM, Jon Harrop <j...@ffconsultancy.com> wrote:

On Tuesday 20 January 2009 08:01:19 ivant wrote:
> IIRC, the only major complain Rich has about JVM is the lack
> of tail-call optimization.

That's a pretty major problem. :-)

 

It's a problem, and I hope future versions of the JVM will address it but I don't think it's a huge problem. There are plenty of languages that also don't support TCO and there are usually ways to work around them. Clojure already provides two such options, recur and trampolines.

--
Cosmin Stejerean
http://offbytwo.com

[Mark H.]

On Jan 20, 9:31 am, Jon Harrop <j...@ffconsultancy.com> wrote:
> On Tuesday 20 January 2009 08:01:19 ivant wrote:
>
> > IIRC, the only major complain Rich has about JVM is the lack
> > of tail-call optimization.
>
> That's a pretty major problem. :-)

btw, did you ever stop trolling comp.lang.lisp? You were the main
reason I figured out how to filter individual names out of a Google
Groups feed ;-P

mfh

[Rich Hickey]

Let's keep from making the same mistakes as c.l.l, including calling
people trolls.

This issue (TCO) is resolved - it's a limitation of the JVM that
Clojure accepts. If that is a significant problem for anyone they
should either not use Clojure or work on adding TCO to the JVM via the
MLVM effort:

http://openjdk.java.net/projects/mlvm/

Rich

[Mark H.]

On Jan 20, 1:41 pm, Rich Hickey <richhic...@gmail.com> wrote:
> Let's keep from making the same mistakes as c.l.l, including calling
> people trolls.

Sorry, yes Rich, I will refrain from using such titles in the future.

mfh

[Jon Harrop]

On Tuesday 20 January 2009 21:41:29 Rich Hickey wrote:
> This issue (TCO) is resolved - it's a limitation of the JVM that
> Clojure accepts. If that is a significant problem for anyone they
> should either not use Clojure or work on adding TCO to the JVM via the
> MLVM effort:
>
> http://openjdk.java.net/projects/mlvm/

Too late:

http://mail.openjdk.java.net/pipermail/mlvm-dev/2009-January/000331.html

[Rich Hickey]

On Jan 21, 5:02 pm, Jon Harrop <j...@ffconsultancy.com> wrote:
> On Tuesday 20 January 2009 21:41:29 Rich Hickey wrote:
>
> > This issue (TCO) is resolved - it's a limitation of the JVM that
> > Clojure accepts. If that is a significant problem for anyone they
> > should either not use Clojure or work on adding TCO to the JVM via the
> > MLVM effort:
>
> >http://openjdk.java.net/projects/mlvm/
>
> Too late:
>
> http://mail.openjdk.java.net/pipermail/mlvm-dev/2009-January/000331.html
>

I saw that too, very promising. Someone's working on continuations as
well.

Rich

[e]

amazing.  Some day there will be a book written about this perfect storm: "The Language Age".

Ok, I admit.  I started watching this presentation last night: http://www.parleys.com/display/PARLEYS/Home#talk=2556139;slide=1;title=The%20future%20will%20be%20about%20programming%20languages

did I see that link here?