Thanks for bringing this up and for the benchmarks!
I had heard of pypy/RPython before, but I was scared about having so much fanciness (black magic? :)) going on under my runtime. Hearing of this and other success stories I decided to give it a try anyway.
So I started a Kernel implementation in RPython. I started from scratch, and not forking plang or pumice, the two previous Kernels-in-RPython that I knew about, because AFAICT these didn't support tail call optimization (pumice didn't do continuations at all), and that looked like something that might be hard to retrofit later on. For the basic structure of the interpreter I borrowed heavily from pycket, a RPython implementation of Racket:
https://github.com/samth/pyckethttp://samth.github.io/pycket-dyla.pdfMy results so far is in
https://github.com/euccastro/icbinkIt is currently in a proof-of-concept stage, so many primitives are missing, but most of the fundamental mechanisms are there: $vau, tail call optimization, guarded continuations, with an error signaling mechanism based on these, first-class environments, encapsulations, promises, keyed dynamic and static variables, integers of arbitrary precision, and a rudimentary (and buggy -- the irony!) debugger.
Most glaringly missing so far are ports, chars, and all optional modules except environment mutation; most notably, mutable pairs and any kind of special handling of cyclic lists. I figured that without the optional mutable pairs module it's impossible to build a cyclic list anyway(?), so I have little motivation to go the extra mile for these. Anyway, I haven't given this much thought and I may change my mind on this.
All in all, I think it's an useful data point as for what performance to expect from RPython.
WRT benchmarks, in my machine (a core duo i5... something), tak.k takes the following in these implementations:
Klisp: ~1.7s
Compiled plang: ~0.3s
Compiled icbink (no jit): ~0.47s
Compiled icbink (jit): ~0.25s
So I guess there is significant overhead in the (naive) way I do continuations, but even a jit with no hints whatsoever (takes two lines to enable it; search for 'jit' in primitive.py) makes up for that. There are more gains to be had in other obvious optimizations, as explained in this series of articles:
http://morepypy.blogspot.com/2011/03/controlling-tracing-of-interpreter-with.htmland other ideas for improvements may be gotten by studying the pycket code.
That said, the stat I like the most so far is that the whole thing takes 1511 (non blank or comment) lines of rather straightforward (R)Python*, although brevity was never much of a goal. Also, it took ~3 weeks of part-time effort by one programmer with barely any experience implementing languages (I was already fluent in Python, though).
In the long run a compiler/interpreter that is meant for Kernel from the ground up (perhaps Bronze Age or what comes from Andrés's work with abstract interpretation) will probably beat anything that we can get from RPython. But in the meanwhile, and especially while Kernel doesn't get more popularity and programmer*years, taking advantage of the amazing work of the pypy folks makes perfect economic sense.
My current plan is to go on and fill in the blanks to make this a conformant and robust (but not necessarily complete) Kernel implementation. Then I'll work on some practicalities like better error messages and debug facilities, and ffi. Only after that I'll look at adding optimizations (this may involve branching out 'development' and 'release' interpreters, the latter being stripped of debug functionality for performance, although I hope that's not necessary).
If anyone has interest in working on/forking this, I'm happy to reply any questions regarding the code. I'll try and add some docs, but probably only as I reply actual questions.
One area in which I think kernel implementations can help each other is in building a corpus of conformance tests in pure kernel. I'll take a look at what's there and perhaps spin this off as an independent project.
Another thing that would be nice to standardize is the tree of error continuations. I have the following tentative hierarchy so far:
root
error
system-error
user-error
type-error
encapsulation-type-error
operand-mismatch
arity-mismatch
symbol-not-found
unbound-dynamic-key
unbound-static-key
add-positive-to-negative-infinity
but it's all ad hoc. Just looking at it now, maybe symbol-not-found and the unbound-* continuations could use a common ancestor like lookup-error... or maybe, on the contrary, the branch on type-error should be flattened. Or the names changed, etc.
For the rest of the stuff that is missing in the R-1RK, I like Klisp's policy of using R7RS as a fallback reference.
I hope you find this stuff interesting.
Happy Kerneling!
- Estevo
* plus some 80 lines of kernel library which I just copied from the reference implementations in the R-1RK.