Parrot vs NekoVM
Nicolas Cannasse
Yesterday I did a quick fib(30) benchmark comparing Parrot Win32 daily
build (using jit core) and NekoVM (http://nekovm.org). The results are
showing that Parrot is 5 times slower than Neko (see my blog post on
this point there : http://ncannasse.free.fr/?p=66).
I would like to understand why there is a so much big speed difference
while NekoVM does not have JIT yet. Is there a semantical difference for
integer calculus or is it just a VM implementation issue ?
Best,
Nicolas
Leopold Toetsch
On Feb 28, 2006, at 12:09, Nicolas Cannasse wrote:
> Yesterday I did a quick fib(30) benchmark comparing Parrot Win32 daily
> build (using jit core) and NekoVM (http://nekovm.org). The results are
> showing that Parrot is 5 times slower than Neko (see my blog post on
> this point there : http://ncannasse.free.fr/?p=66).
Benchmarks are just damn lies ;)
$ time ./parrot -j fib.pir 30
Fib(30): 1346269
real 0m4.774s
Ok that's slow (AMD X2@2000, unoptimized parrot build), you are right.
But:
$ time ./parrot -Cj fib.pir 30
Fib(30): 1346269
real 0m0.036s
$ time ./parrot -Cj fib.pir 38
Fib(38): 63245986
real 0m0.675s
$ time ./parrot -Cj fibn.pir 38
Fib(38.0): 63245986.0
real 0m1.475s
The rather slow function call performance is coming from a complex call
frame creation and flexible argument passing. The whole code involved
with calls/returns isn't optimized either.
The '-Cj' runtime options tries to compile simple subs to native
assembler code (and obviously succeeds here ;)
leo
Jonathan Worthington
>
> Yesterday I did a quick fib(30) benchmark comparing Parrot Win32 daily
> build (using jit core)
pointing out that it ain't an optimized build. But I think leo supplied the
real reason for the slowness - calling isn't that fast yet (and isn't
optimized, and where it can be optimized that's not enabled by default -
yet).
Jonathan
Nicolas Cannasse
> On Feb 28, 2006, at 12:09, Nicolas Cannasse wrote:
>
>> Yesterday I did a quick fib(30) benchmark comparing Parrot Win32 daily
>> build (using jit core) and NekoVM (http://nekovm.org). The results are
>> showing that Parrot is 5 times slower than Neko (see my blog post on
>> this point there : http://ncannasse.free.fr/?p=66).
>
>
> Benchmarks are just damn lies ;)
Not exactly lies, but they are quite different from reality ;)
> The '-Cj' runtime options tries to compile simple subs to native
> assembler code (and obviously succeeds here ;)
Yes I understand that there is different cores for Parrot, but what are
the flags appropriate for doing some comparisons ? Some flags might do
very good in some cases and quite bad in some others. They might also
take tremendous time to JIT the code, hence not being usable on larger
applications.
Is there one single config that the Parrot team is focusing on bringing
to 1.0 ?
Nicolas
Joshua Isom
adding -Oc as well. On ppc, -C and -Cj are often the fastest. On x86,
-j is most often the fastest. But here's the cavaet, to use JIT, you
of course need someone to port it to that arch. With -C, your compiler
has to support some of the source code features of the CGP core, which
gcc does. In general, as far as I know anyway, -C will be your generic
fast runcore. If your application's actually time-able, as in it won't
continuously run, you can just test various runcores with your
platform. But the way your program is coded has an impact. You can
write a program/subroutine to be very optimized for easy jitting, or
write it in a simpler and quicker to develop manner. And some of the
opcodes used just aren't jitted for one reason or another, which kind
of stops the jitability of that sub. There is no one fastest runcore
for all situations(although -t7 is probably the slowest). It depends
on the PIR and the architecture.
On Feb 28, 2006, at 12:59 PM, Nicolas Cannasse wrote:
> [...]
Joshua Juran
>> On Feb 28, 2006, at 12:09, Nicolas Cannasse wrote:
>>
>>> Yesterday I did a quick fib(30) benchmark comparing Parrot Win32
>>> daily
>>> build (using jit core) and NekoVM (http://nekovm.org). The
>>> results are
>>> showing that Parrot is 5 times slower than Neko (see my blog post on
>>> this point there : http://ncannasse.free.fr/?p=66).
>>
>> Benchmarks are just damn lies ;)
Worse than that...
> Not exactly lies, but they are quite different from reality ;)
...they're statistics. :-)
(With apologies to Mark Twain.)
Josh
Leopold Toetsch
On Feb 28, 2006, at 19:59, Nicolas Cannasse wrote:
> Yes I understand that there is different cores for Parrot, but what are
> the flags appropriate for doing some comparisons ? Some flags might do
> very good in some cases and quite bad in some others. They might also
> take tremendous time to JIT the code, hence not being usable on larger
> applications.
The time needed to JIT some code isn't the big deal, the more that the
plan is to recompile heavily used (or small) subroutines only, and not
whole applications. The currently exisiting JIT code (-j) is compiling
a whole file. The new (and still experimental) -{S,C}j options will
recompile code per subroutine or even per basic block for the direct
threaded run core.
> Is there one single config that the Parrot team is focusing on bringing
> to 1.0 ?
A single option can't cover all possible usage of Parrot. But it
basically boils down to two options:
-Os ... optimize for size (multiple processes of Parrot with shared
code)
-Ot ... use more memory to run faster
> Nicolas
leo
Nicolas Cannasse
> Fib(30): 1346269
>
> real 0m4.774s
>
> Ok that's slow (AMD X2@2000, unoptimized parrot build), you are right. But:
>
> $ time ./parrot -Cj fib.pir 30
> Fib(30): 1346269
-Cj does not produce different results than -j on the Win32 build of
Parrot. Is -Cj supported on this architecture ?
Nicolas
Leopold Toetsch
On Mar 4, 2006, at 18:05, Nicolas Cannasse wrote:
> -Cj does not produce different results than -j on the Win32 build of
> Parrot. Is -Cj supported on this architecture ?
Yes, it should work. It might depend on, how fib is actually written in
PIR. As said this option is in a rather early state. Compiling a
subroutine to machine code is currently only done, if all Parrot
registers are fitting into CPU registers. The fib function below is
working here on x86/linux.
Mmm - actually -C needs computed goto, which isn't supported by all C
compilers. You can try:
$ ./parrot -Sj fib.pir 38
Fib(38): 63245986
> Nicolas
leo
.sub main :main
.param pmc argv
.local int argc, n
argc = argv
n = 1
unless argc == 2 goto argsok
$S0 = argv[1]
n = $S0
argsok:
$P0 = getinterp
$P0.'recursion_limit'(100000)
.local pmc array
array = new .FixedFloatArray
array = 2
$I1 = FibInt(n)
array[0] = n
array[1] = $I1
$S0 = sprintf <<"END", array
Fib(%d): %d
END
print $S0
.end
.sub FibInt
.param int n
unless n < 2 goto endif
.return(1)
endif:
.local int tmp
tmp = n - 2
$I0 = FibInt(tmp)
tmp = n - 1
tmp = FibInt(tmp)
$I0 += tmp
.return($I0)
.end
Jonathan Worthington
> Mmm - actually -C needs computed goto, which isn't supported by all C
> compilers.
tested with (MS Visual C++). Shouldn't it give a "we don't have a computed
goto runcore" error though when you supply -C on platforms without it?
Jonathan
Leopold Toetsch
Definitely yes. What does happen currently, when running 'parrot -C' or
'parrot -Cj' with msvc?
> Jonathan
leo