Parrot Benchmark
Nicolas Cannasse
I tried to have a deeper look at Parrot so I installed latest PXPerl with
Parrot 0.2.3 and I tried to run some benchmarks on my computer in order to
compare with Neko.
I was thinking that Parrot JIT would outperform Neko VM easily but I got
theses results :
fib(35) Neko : 8 seconds
fib(35) Parrot (benchmarks/fib.imc) : 34 seconds
So I would like to ask if the JIT is enabled in my Parrot install and if not
how can I enable it ?
Or am I missing something about the way Parrot is handling integer values ?
Thanks,
Nicolas
Leopold Toetsch
fib is one of the few benchmarks that currently performs very badly due
to huge function call overhead (mainly L2 cache misses). I'm currently
working on a different call scheme, which eventually gets rid of these
problems.
For reasonable performance comparisons you would also compile an
optimized executable:
$ make realclean
$ perl Configure.pl --optimize
$ make
To see, if JIT really works (and how fast it is)
$ ./parrot -j examples/assembly/mops.pasm
The MOPS test shows the theoretical maximum of opcode dispatch, which
is one hardware CPU instruction / parrot instruction for JIT on i386
and ppc. You can evaluate different run loops:
$ ./parrot -S ... # switched core
$ ./parrot -C ... # direct threaded CGP core
...
In examples/mops you find more mops tests for different HLLs.
A more realistic benchmarks is:
$ ./parrot -j examples/assembly/md5sum.imc parrot
which runs on x86 about at half the speed of Perl/XS, which is written
in C.
> Thanks,
> Nicolas
leo
Nicolas Cannasse
> > values ?
>
> fib is one of the few benchmarks that currently performs very badly due
> to huge function call overhead (mainly L2 cache misses). I'm currently
> working on a different call scheme, which eventually gets rid of these
> problems.
Even with such problems, I was thinking that Parrot JIT will be faster than
NekoVM which does not have a JIT yet.
> For reasonable performance comparisons you would also compile an
> optimized executable:
> $ make realclean
> $ perl Configure.pl --optimize
> $ make
I don't run sources from Parrot , I have PxPerl installed.
> To see, if JIT really works (and how fast it is)
> $ ./parrot -j examples/assembly/mops.pasm
I get the following results :
Elapsed time: 3.281000
M op/s: 60.957023
Looks like it means that JIT is not active. How can I activate it ?
> The MOPS test shows the theoretical maximum of opcode dispatch, which
> is one hardware CPU instruction / parrot instruction for JIT on i386
> and ppc. You can evaluate different run loops:
>
> $ ./parrot -S ... # switched core
I get :
M op/s: 101.574410
> $ ./parrot -C ... # direct threaded CGP core
This one give the following error :
Couldn't find init_func for core 6
> ...
> In examples/mops you find more mops tests for different HLLs.
So parrot have several version of the interpreter/jit embedded ? What are
the current choices ?
Thanks for your answers,
Best,
Nicolas
Leopold Toetsch
> Even with such problems, I was thinking that Parrot JIT will be faster than
> NekoVM which does not have a JIT yet.
The fib benchmark is mainly function calls only. JIT doesn't really help
here. We will optimize function call performance later.
> I don't run sources from Parrot , I have PxPerl installed.
Yes I know. You should install Parrot from SVN and compile it optimized
for benchmarking.
>>$ ./parrot -j examples/assembly/mops.pasm
>
>
> I get the following results :
> Elapsed time: 3.281000
> M op/s: 60.957023
> Looks like it means that JIT is not active. How can I activate it ?
Seems that it isn't compiled with JIT support but don't ask me, how it
was disabled in the first place (and why you don't get an error message).
>>$ ./parrot -C ... # direct threaded CGP core
>
> This one give the following error :
> Couldn't find init_func for core 6
And it was obviously configure with --cgoto=0, or compiled with a
microsoft? compiler that doesn't support computed goto.
> So parrot have several version of the interpreter/jit embedded ? What are
> the current choices ?
There are several "run loops". 'parrot --help' shows various 'Run core
options'. CGoto & CGP depend on compiler support. JIT should always be
available on x86.
> Thanks for your answers,
> Best,
> Nicolas
leo