Java vs Python Benchmarks: Java is faster
Bryan
For what it is worth to people, I have taken the liberty of benchmarking
the new Java SDK ( 1.4.0 ) against Python 2.1 and 2.2. The tests originate
from:
http://www.twistedmatrix.com/users/glyph/rant/python-vs-java.html
Point form Summary:
- Java has improved its Standard Output
- Java has improved its Hash table implementation
- Java has improved its IO routines
- Java has improved its Object Allocation
- Python has improved its Native methods
- Python has improved its interpreter speed
- Python allocates objects 57 times slower than java
- Both Python and Java have increased their interpreter initialization
times, but Python has done so by a much smaller margin.
Interesting differences between Python 2.1 and 2.2:
- I/O is slower in 2.2
- List and Hash are slightly improved in 2.2
- Object allocation shows slight improvement
- Interpreter intialization time has doubled in 2.2
The Results (fixed-font, space-formatted):
JDK 1.4.0 vs Python on an 800Mhz Pentium 3, 512 MB RAM,
Unloaded linux 2.4.8 i686 system, no X-Windows, few daemons.
Internet: 400 KB/s Down, 15 KB/s up. Results are an average of 10 tests.
Java2 Python 2.1 Summary
----- ---------- -------
ConsoleTest 12.34 8.3 Python is 1.4 times faster
HashTest 0.98 2.14 Java is 2.18 times faster
IOTest 2.63 10.25 Java is 3.8 times faster
ListTest 0.65 2.95 Java is 4.5 times faster
NativeTest 1.40 2.29 Java is 1.6 times faster
NoTest 0.34 0.02 Python is 17 times faster
ObjectTest 1.10 62.79 Java is 57 times faster
SpeedTest 0.34 0.64 Java is 1.8 times faster
URLTest 14.53 13.69 Python and Java are equivalent*
Java2 Python 2.2 Summary
----- ---------- -------
ConsoleTest 12.34 8.72 Python is 1.4 times faster
HashTest 0.98 2.1 Java is 2.14 times faster
IOTest 2.63 11.63 Java is 4.4 times faster
ListTest 0.65 2.71 Java is 4.16 times faster
NativeTest 1.40 2.26 Java is 1.6 times faster
NoTest 0.34 0.04 Python is 8.5 times faster
ObjectTest 1.10 61.71 Java is 56.1 times faster
SpeedTest 0.34 0.63 Java is 1.8 times faster
URLTest 14.53 14.71 Python and Java are equivalent*
* The slight discrepancies are most likely due to network fluctuations.
As a side note, I am very much a python proponent. Speed is not everything,
otherwise we would still be coding in straight machine language :) If this
serves to debunk one pythoneer's baseless comments on the slow speed of
Java ( as it did to me ) then it has served its purpose.
Happy coding,
--
Bryan Mongeau
http://www.eevolved.com/bryan
--
"Imagination is more important than knowledge."-- Einstein
bru...@tbye.com
> For what it is worth to people, I have taken the liberty of
> benchmarking the new Java SDK ( 1.4.0 ) against Python 2.1 and 2.2.
> The tests originate from:
Thanks for running those tests, Bryan. Pretty interesting stuff. I wasn't
sure from reading your post, but I'm assuming that the Java stuff had the
JIT enabled? I'd be curious to know what Java numbers would be without
it - I was really expecting Java + JIT to be a lot faster than those
tests showed... all that extra technology and meager (except ObjectTest!)
speed improvements.
-Dave
Bill Tate
> For what it is worth to people, I have taken the liberty of benchmarking
> the new Java SDK ( 1.4.0 ) against Python 2.1 and 2.2. The tests originate
> from:
>
> http://www.twistedmatrix.com/users/glyph/rant/python-vs-java.html
>
Very interesting stuff. Do you have numbers for Java 1.3.x - just
curious to see if there were major gains between 1.3 and 1.4?
Bryan
> Bryan,
> Very interesting stuff. Do you have numbers for Java 1.3.x - just
> curious to see if there were major gains between 1.3 and 1.4?
Unfortunately I don't have the time to do so, but I encourage you to. I
hope Glyph will post my results on his site, his seem very skewed in light
of new python and java developments.
--
Bryan Mongeau
http://www.eevolved.com/bryan
--
"The true function of life, that which is being maximized in the natural
world, is DNA survival. But DNA is not floating free; it is locked up in
living bodies and it has make the most of the levers of power at its
disposal." -- Richard Dawkins
dman
I've heard that Reflection in Java is many times faster in 1.4 than
any earlier version. On the Jython list a while back someone posted
PyStone results showing jython on jdk 1.4 (beta something or other)
beat out cpython by a clear margin.
-D
--
The crucible for silver and the furnace for gold,
but the Lord tests the heart.
Proverbs 17:3
dman
|
| For what it is worth to people, I have taken the liberty of benchmarking
| the new Java SDK ( 1.4.0 ) against Python 2.1 and 2.2.
| As a side note, I am very much a python proponent. Speed is not everything,
| otherwise we would still be coding in straight machine language :) If this
| serves to debunk one pythoneer's baseless comments on the slow speed of
| Java ( as it did to me ) then it has served its purpose.
I'm still not willing to give java the benefit of the doubt yet.
Did you try any GUI stuff in the tests? Did you try any largish
programs? In all my experiences running java apps (ie JBuilder,
'deploytool' from J2EE, IBM VisualAge for Java, Wilmascope (a J3D OSS
app), etc) they tend to be slow and memory hogs. Interestingly
enough, too, there doesn't seem to be a way to increase the max heap
size of javac and I've seen it bomb with java.lang.OutOfMemoryError
trying to compile a large source tree while the system still had lots
of RAM/swap to play with.
A little while back I wrote a script in python to obfuscate package
and class names in a java source tree. (for in-house use at my
current employer) I used CPython 2.2 and a bunch of regexes for
identifying the occurences of the class/package names in the source
files. It can process the whole 15MB source tree in about 30 seconds
and using no more than 7MB RAM at peak with 2-3MB most of the time.
I'll likely have to recode it in java (no one else there speaks
python), but I'll bet it won't be nearly as good (both in
terms of read/write-ability of the code and performance).
Any comments?
-D
(admittedly I didn't yet follow the URL you posted, but I will check
it out later)
--
Your beauty should not come from outward adornment, such as braided hair
and the wearing of gold jewelry and fine clothes. Instead, it should be
that of your inner self, the unfading beauty of a gentle and quiet
spirit, which is of GREAT WORTH in God's sight. For this is the way the
holy women of the past used to make themselves beautiful.
I Peter 3:3-5
Bryan
> - Python allocates objects 57 times slower than java
As a side note, Michael Robin I suggested I attempt a "new-style" object
test using the new __slots__ mechanism.
It does make a difference. New Style classes take 92 seconds to pass
the Object Test, whereas old style takes 61.
--
Bryan Mongeau
http://www.eevolved.com/bryan
--
"I think there is a world market for maybe five computers." -- Thomas J.
Watson
Bryan
> I'm still not willing to give java the benefit of the doubt yet.
Agreed. The test suites are rather limited in nature. I would welcome the
results of a more in-depth analysis. However, the trollish nature of my
post and resulting low number of replies leads me to believe no one can or
is willing to really back up any claims to the contrary.
> Did you try any GUI stuff in the tests? Did you try any largish
> programs? In all my experiences running java apps (ie JBuilder,
> 'deploytool' from J2EE, IBM VisualAge for Java, Wilmascope (a J3D OSS
> app), etc) they tend to be slow and memory hogs. Interestingly
> enough, too, there doesn't seem to be a way to increase the max heap
> size of javac and I've seen it bomb with java.lang.OutOfMemoryError
> trying to compile a large source tree while the system still had lots
> of RAM/swap to play with.
>
No GUI tests were involved. I concurr that Java is a memory hog, perhaps I
should have also profiled memory usage for each test?
> A little while back I wrote a script in python to obfuscate package
> and class names in a java source tree. (for in-house use at my
> current employer) I used CPython 2.2 and a bunch of regexes for
> identifying the occurences of the class/package names in the source
> files. It can process the whole 15MB source tree in about 30 seconds
> and using no more than 7MB RAM at peak with 2-3MB most of the time.
> I'll likely have to recode it in java (no one else there speaks
> python), but I'll bet it won't be nearly as good (both in
> terms of read/write-ability of the code and performance).
Let me know how it turns out... The tests show file IO is faster in Java,
yet from personal experience, regex in python is superior.
> (admittedly I didn't yet follow the URL you posted, but I will check
> it out later)
Check it out if you want to run the tests yourself. I find the page rather
biased and hope they will include my updated results.
Happy coding,
--
Bryan Mongeau
http://www.eevolved.com/bryan
--
"Let us understand what our own selfish genes are up to, because we may
then at least have a chance to upset their designs, something that no other
species has ever aspired to do." -- Richard Dawkins
Donn Cave
|
| For what it is worth to people, I have taken the liberty of benchmarking
| the new Java SDK ( 1.4.0 ) against Python 2.1 and 2.2. The tests originate
| from:
|
| http://www.twistedmatrix.com/users/glyph/rant/python-vs-java.html
|
| Point form Summary:
| Interesting differences between Python 2.1 and 2.2:
|
| - I/O is slower in 2.2
| - List and Hash are slightly improved in 2.2
| - Object allocation shows slight improvement
| - Interpreter intialization time has doubled in 2.2
Doubled!!? Could that be some kind of site glitch?
Donn Cave, do...@drizzle.com
Jason Orendorff
> the max heap size of javac and I've seen it bomb with
java -cp $JAVA_HOME/lib/tools.jar -Xmx... sun.tools.javac.Main ...
> A little while back I wrote a script in python to obfuscate package
> and class names in a java source tree. (for in-house use at my
> current employer) I used CPython 2.2 and a bunch of regexes for
> identifying the occurences of the class/package names in the source
> files. It can process the whole 15MB source tree in about 30 seconds
> and using no more than 7MB RAM at peak with 2-3MB most of the time.
> I'll likely have to recode it in java (no one else there speaks
> python), but I'll bet it won't be nearly as good (both in
> terms of read/write-ability of the code and performance).
>
> Any comments?
1. You can buy a real obfuscator for a few hundred bucks.
2. Whatever you buy will be written in Java. (And it will
probably work at the class-file level rather than the
source level, which is a good thing.)
3. If you must do this yourself: ANTLR (for example) comes
with everything you need to tokenize Java source code (or
fully parse it, if you prefer). It should be quite speedy,
and more foolproof than slinging regular expressions.
## Jason Orendorff http://www.jorendorff.com/
Courageous
>1. You can buy a real obfuscator for a few hundred bucks.
Give the man a dollar. :)
>3. If you must do this yourself: ANTLR (for example) comes
> with everything you need to tokenize Java source code ...
I have to say, I heartily recommend ANTLR all the way around.
It's _far and above_ the best lexer/compiler/ast/walker generator
I've ever encountered. That it combines all of these duties into
one integrated whole and does so very well is simply amazing.
C//
Bryan
> | - Interpreter intialization time has doubled in 2.2
>
> Doubled!!? Could that be some kind of site glitch?
It is possible, but I doubt it. I ran the tests ten times at various points
during my day and consistently saw results that indicated that 2.2 takes
twice as long to initialize the interpreter. Perform your own tests:
perhaps I am overlooking something on my end.
--
Bryan Mongeau
http://www.eevolved.com/bryan
--
"A person starts to live when he can live outside himself."-- Einstein
Bill Tate
> Bill Tate wrote:
>
> > Bryan,
> > Very interesting stuff. Do you have numbers for Java 1.3.x - just
> > curious to see if there were major gains between 1.3 and 1.4?
>
> Unfortunately I don't have the time to do so, but I encourage you to. I
> hope Glyph will post my results on his site, his seem very skewed in light
> of new python and java developments.
I'll give it shot - I'll post when I get it done. Some of the Jython
comments are starting to peek my interest as well.
Thanks again for the info.
bru...@tbye.com
> > I'm still not willing to give java the benefit of the doubt yet.
>
> Agreed. The test suites are rather limited in nature. I would welcome the
> results of a more in-depth analysis. However, the trollish nature of my
> post and resulting low number of replies leads me to believe no one can or
> is willing to really back up any claims to the contrary.
That's why I asked whether or not the JIT was enabled in the tests and, if
so, if you could re-run them without it. Maybe the low number of replies
is because it's not suprising that a semi-compiled-to-native program is
faster?
-Dave
Emile van Sebille
<bru...@tbye.com> wrote in message
news:mailman.101257878...@python.org...
> That's why I asked whether or not the JIT was enabled in the tests
and, if
> so, if you could re-run them without it. Maybe the low number of
replies
> is because it's not suprising that a semi-compiled-to-native program
is
> faster?
>
Or that a lot of us already know that there are faster languages, but
that python is fast enough anyway?
--
Emile van Sebille
em...@fenx.com
---------
Anders Dahlberg
<bru...@tbye.com> skrev i meddelandet
news:mailman.101257878...@python.org...
well, but why would you want to compare interpreted java with python? As
java is not supposed to be *only* interpreteded anymore (it once was in the
cold distant past - like 4-5 years ago) but compiled ("jitted") to native
code during run-time? If you were to compare interpreted java with python
should you then compare interpreted c++ with java (same ballpark)?
just some observations ;)
Would be fun to see comparisons on jython (running on for example jdk1.4)
versus python2.2!
>
> -Dave
>
/Anders
bru...@tbye.com
> could re-run them without it. Maybe the low number of replies > is
> because it's not suprising that a semi-compiled-to-native program is >
> faster?
>
> well, but why would you want to compare interpreted java with python?
Well for one, it'd be nice to be a little closer to comparing apples to
apples (how fast a VM can interpret and execute the bytecodes). Secondly,
I was very suprised that, with a JIT, Java was only meagerly faster than
Python. I was expecting an order of magnitude faster for most stuff, so I
was curious to know how slow Java was without the JIT. Finally, even
though Java and Python are wildly different in many ways, knowing the
interpreted vs. JIT'ed speeds might give a little insight into the
potential benefits of a Python JIT.
If I was really really interested in the Python-Java debate I'd go do the
tests myself, but since I'm just sorta curious I was asking Bryan in the
hopes that he still had the tests setup and it'd be easy to re-run them.
> code during run-time? If you were to compare interpreted java with python
> should you then compare interpreted c++ with java (same ballpark)?
Sure, why not?
-Dave
Anders Dahlberg
<bru...@tbye.com> skrev i meddelandet
news:mailman.1012587303...@python.org...
> On Fri, 1 Feb 2002, Anders Dahlberg wrote:
>
> > could re-run them without it. Maybe the low number of replies > is
> > because it's not suprising that a semi-compiled-to-native program is >
> > faster?
> >
> > well, but why would you want to compare interpreted java with python?
>
> Well for one, it'd be nice to be a little closer to comparing apples to
> apples (how fast a VM can interpret and execute the bytecodes).
Somewhat valid point (IMO that is)
>Secondly,
> I was very suprised that, with a JIT, Java was only meagerly faster than
> Python.
Well, python is written in c or something equivalent right?
>I was expecting an order of magnitude faster for most stuff,
object allocation (java need it alot)...
>so I
> was curious to know how slow Java was without the JIT. Finally, even
> though Java and Python are wildly different in many ways, knowing the
> interpreted vs. JIT'ed speeds might give a little insight into the
> potential benefits of a Python JIT.
I don't really know if python needs a jit, it would probably not harm
python, but is it really worth the effort?
>
> If I was really really interested in the Python-Java debate I'd go do the
> tests myself, but since I'm just sorta curious I was asking Bryan in the
> hopes that he still had the tests setup and it'd be easy to re-run them.
>
> > code during run-time? If you were to compare interpreted java with
python
> > should you then compare interpreted c++ with java (same ballpark)?
>
> Sure, why not?
well probably because you IMO should compare tools "as they are intended to
be used" (hope you understand what I'm getting at here).
> -Dave
/Anders
bru...@tbye.com
>
> <bru...@tbye.com> skrev i meddelandet
> news:mailman.1012587303...@python.org...
> > On Fri, 1 Feb 2002, Anders Dahlberg wrote:
> >
> > > could re-run them without it. Maybe the low number of replies > is
> > > because it's not suprising that a semi-compiled-to-native program is >
> > > faster?
> > >
> > > well, but why would you want to compare interpreted java with python?
> >
> > Well for one, it'd be nice to be a little closer to comparing apples to
> > apples (how fast a VM can interpret and execute the bytecodes).
>
> Somewhat valid point (IMO that is)
>
> >Secondly,
> > I was very suprised that, with a JIT, Java was only meagerly faster than
> > Python.
>
> Well, python is written in c or something equivalent right?
Yes, the Python VM is written and C. And what do you think the Java VM
is written in?
> > was curious to know how slow Java was without the JIT. Finally, even
> > though Java and Python are wildly different in many ways, knowing the
> > interpreted vs. JIT'ed speeds might give a little insight into the
> > potential benefits of a Python JIT.
>
> I don't really know if python needs a jit, it would probably not harm
> python, but is it really worth the effort?
Heehee... that's the whole reason I asked my question in the first place.
-Dave
Steve Holden
> > > was curious to know how slow Java was without the JIT. Finally, even
> > > though Java and Python are wildly different in many ways, knowing the
> > > interpreted vs. JIT'ed speeds might give a little insight into the
> > > potential benefits of a Python JIT.
> >
> > I don't really know if python needs a jit, it would probably not harm
> > python, but is it really worth the effort?
>
> Heehee... that's the whole reason I asked my question in the first place.
>
unlikely that a JIT would yield the same benefits as it does for Java,
although recent experiments with compiler techniques have shown interesting
speedups on restricted cases.
The major problem is that with dynamic (latent) typing, the JIT code has to
examine the type of its operands in order to determine which of several
type-based operations to perform. Java can avoid this due to its strong
typing mechanism. But let's not forget: "Java is the COBOL of the
object-oriented world".
regards
Steve
--
Consulting, training, speaking: http://www.holdenweb.com/
Python Web Programming: http://pydish.holdenweb.com/pwp/
Goh S H
For those not familiar with psyco, point to
Kragen Sitaker
> Without any introduction of static typing into the language, it's extremely
> unlikely that a JIT would yield the same benefits as it does for Java,
> although recent experiments with compiler techniques have shown interesting
> speedups on restricted cases.
This is completely unfounded and false. The JIT techniques used for
Java were developed for Self, a language just as dynamic as Python,
and compiled Self code to within a factor of 2 of C --- the same kind
of speed Java JITs get today. The Self experiments were carried out
during the late 1980s and early 1990s, and they were not "restricted
cases".
> The major problem is that with dynamic (latent) typing, the JIT code has to
> examine the type of its operands in order to determine which of several
> type-based operations to perform. Java can avoid this due to its strong
> typing mechanism. [inflammatory remark snipped]
Java can avoid this in some cases, because the types of some variables
(those of primitive, non-Object types) are known at compile-time, but
just like Python, the types of most objects are not known at
compile-time; most classes and most methods are not final. It's
optimizing these dynamic calls that is hard.
It's true that Java objects all have declared types, but that doesn't
save you as much as you'd think; it just allows more efficient dynamic
method lookup in cases where the JIT can't do anything smarter --- but
it's still too slow.
Courageous
>It's true that Java objects all have declared types, but that doesn't
>save you as much as you'd think;
While you obviously know what you're talking about and don't need my
help, I'd just like to add that those folks talking about an optional
type declaration system for speeding up Python don't seem to have a
good idea of where the time in Python goes. There are more important
fish to fry, as I'm sure you realize.
C//
Jason Orendorff
On behalf of all of us that don't know what we're talking about and
do need your help...
I guess I imagine that Python's speed issues are:
1. Ultra-late binding of, and consequent repetitive lookups of,
global names, attributes, methods, module members, etc.
(Of course, unless names like 'int' and 'float' can be bound
at compile time, optional type declarations are unhelpful.)
Since many people like this about Python, and many have used it
in existing code, effort is focused on making lookups cheaper
(or caching the results) rather than changing the binding
behavior to require fewer lookups overall.
2. Nothing ever gets compiled to anything even close to
machine code. This is largely because the Python development
team doesn't have the resources to invest in it. Clearly
there's something to gain by moving the huge ceval.c loop/switch
from runtime to compile time. No one knows how much. Clearly
there's something to lose from the resulting explosion in the
complexity of the compiler.
3. Even if you did compile Python to machine code, simple things
like "x = y + z" would still be packed with object-oriented
wackiness and much chasing-about of heap pointers. It doesn't
compile to simple register code like you want.
4. Function calls are slow.
5. Python still has SET_LINENO, which is basically a waste
of time. (This is low-hanging fruit, but apparently the
team is too busy to grab it.)
I believe static typing would help with #1 and #3.
More later.
Emile van Sebille
"Jason Orendorff" <ja...@jorendorff.com> wrote in message
news:mailman.1012761186...@python.org...
> 5. Python still has SET_LINENO, which is basically a waste
> of time. (This is low-hanging fruit, but apparently the
> team is too busy to grab it.)
>
Note the -O (and -OO) command line options to python already eliminate
this.
Courageous
>On behalf of all of us that don't know what we're talking about and
>do need your help...
Ah, but you're one of those who _do_ know what you're talking about.
For example, the following comment illustrates knowledge of one of the
very first things one notices in profiling Python itself:
> 1. Ultra-late binding of, and consequent repetitive lookups of,
> global names, attributes, methods, module members, etc.
And likewise this:
> Clearly there's something to gain by moving the huge ceval.c
> loop/switch from runtime to compile time.
Type declarations target higher hanging fruit. For example, there
is quite a lot more which can be achieved in the pursuit of #1 than
has been so far, _without_ the gains which type declarations would
offer. It would therefor follow to my sorry logical mind that
_first_ attempting a declaration system is a premature optimization.
C//
Ype Kingma
you wrote:
>
<snip>
>
> I guess I imagine that Python's speed issues are:
>
> 1. Ultra-late binding of, and consequent repetitive lookups of,
> global names, attributes, methods, module members, etc.
> (Of course, unless names like 'int' and 'float' can be bound
> at compile time, optional type declarations are unhelpful.)
>
> Since many people like this about Python, and many have used it
> in existing code, effort is focused on making lookups cheaper
> (or caching the results) rather than changing the binding
> behavior to require fewer lookups overall.
>
> 2. Nothing ever gets compiled to anything even close to
> machine code. This is largely because the Python development
> team doesn't have the resources to invest in it. Clearly
> there's something to gain by moving the huge ceval.c loop/switch
> from runtime to compile time. No one knows how much. Clearly
> there's something to lose from the resulting explosion in the
> complexity of the compiler.
>
Please note that jython is compiled directly to Java opcodes.
Anything that makes interpreting these faster (JIT, native compilation)
will make jython faster.
Regards,
Ype