"Research group for an experimental, fast Ruby compiler."
which I think is a bit misleading (or I don't share the goal ;)).
My problem is "fast compiler" which is definetely not a high prio imho. A compiler able to produce "fast code" would be another thing... ;)
A really high prio imho:
1. Written in as pure-Ruby as possible (because we want max involvement from a community which will never happen if written in other lang, also because a compiler should be self-hosting/bootstrapped since if we like a lang we want to use it (ok, langs can be specific to certain purposes or better/worse for some situations but I think Ruby will be just fine for writing compiler...)
> "Research group for an experimental, fast Ruby compiler." > which I think is a bit misleading (or I don't share the goal ;)).
you're right, the title is misleading.
> My problem is "fast compiler" which is definetely not a high prio imho. > A compiler able to produce "fast code" would be another thing... ;)
able to produce fast code. you're right. so the title should be "Research group for an experimental Ruby compiler"
> 1. Written in as pure-Ruby as possible (because we want max involvement > from a community which will never happen if written in other lang, also > because a compiler should be self-hosting/bootstrapped since if we like > a lang we want to use it (ok, langs can be specific to certain purposes > or better/worse for some situations but I think Ruby will be just fine > for writing compiler...)
I think this aligns well with Alex's Rubydium and with Florian Groß's approach too
Very modular architecture is also a key to wider adoption, to enable parallel participation. (as long as this doesn't trump with runtime performance)
> BTW, I've resurrected some old files of mine that is relevant for this > discussion. They are available here: > http://www.pronovomundo.com/rubyrubyvm
404 - Not Found The requested URL /rubyrubyvm was not found on this server. Apache/2.0.51 (Gentoo/Linux) DAV/2 SVN/1.1.1 Server at www.pronovomundo.com Port 80
Alex's project's name is Rubydium, and the latest public release was in Feb 9, 2005, Rubydium 0.3, accompanied by Nanovm, a simple VM written in C.
in Alex's words: Rubydium aims to become an optimising reimplementation of the Ruby 1.8 interpreter.
In Alex's announcement on ruby-talk he said: "Rubydium works by generating optimized platform specific code on the fly, as needed, using instance type information gathered during the execution itself. Optimisations are possible as the generated code can be specialized for the types in question, therefore allowing heavy inlining of speed critical sections."
Initially "Robert Feldt's awesome Ruth library" was used on matz's C-Ruby to get bootstrapped quickly, he later produced Rblex partially based on Pyggy (a GLR parser/lexer generator written in Python) with provision for a theoretical all-ruby parser/lexer to replace Pyggy entirely later.
In the beggining Alex started using libjit, which was largely unmantained for more than a year around the time Alex was working on Rubydium, so Alex roughly reimplemented the best things he saw in libjit and integrated them into Rubydium and Nanovm.
The latest libjit release was the August 8, 2004, but the libjit guys recently (January 2006) released a new version, so we may want to check again whether we can trust libjit again to offload some work. http://www.southern-storm.com.au/libjit.html
This is very roughly my understanding for what I can remember right now. (please correct me, Alex, wherever I'm wrong)
As for Florian's and Mauricio's work, I'm not familiar enough with their compiler work yet.
Florian's stated main objective is to have a Ruby compiler targeting the .NET CLR. His current work environment is Win32 but he plans to move to OS-X soon, so he will likely be moving to Mono.
Mauricio has lately spent some time analysing Alex's Rubydium, he has also analysed YARV and matz's C-Ruby in depth in the past, his experience (particularly about Ruby and YARV innards) plus some applied information science can be tracked at http://www.eigenclass.org
Please note that I have not actually done anything that will be useful yet.
I just thought about possibly doing a Ruby in Ruby on .NET compiler for the next Summer of Code for Mono.
My goals for this are getting something that is in a hackable (and fun to do so) state as soon as possible. That means cheating a lot. Not parsing Ruby at all is one way to cheat. Not optimizing is another one. The compiler being self-hosted is the primary goal for me. I think it would be a lot less useful if it wasn't possible for everyone with Ruby knowledge to modify and contribute to it.
I picked .NET because it seems quite easy to go self-hosting there. You can write a .NET bytecode compiler in Ruby by using a Ruby<>.NET bridge to use the code generation APIs, after that you would just need to get a minimal subset of Ruby working so that you can use the (very rich and comfortable to use) .NET standard library and base a full Ruby implementation around that. I think it is a very simple approach, but haven't actually tested it just yet so it's probably best taken with a huge grain of salt.
My basic goal was to keep things as minimal as possible so that I would actually have a chance of producing something useful. Otherwise the risk of cancelling the project before it becomes useful is way too high.
Right now I'm not sure if it still makes sense to do that project. There's already people working on a Ruby.NET after all.
> > BTW, I've resurrected some old files of mine that is relevant for this > > discussion. They are available here: > > http://www.pronovomundo.com/rubyrubyvm
> 404 - Not Found > The requested URL /rubyrubyvm was not found on this server. > Apache/2.0.51 (Gentoo/Linux) DAV/2 SVN/1.1.1 Server at > www.pronovomundo.com Port 80
On 4/1/06, flor...@gmail.com <flor...@gmail.com> wrote:
> Please note that I have not actually done anything that will be useful > yet.
> I just thought about possibly doing a Ruby in Ruby on .NET compiler for > the next Summer of Code for Mono.
> My goals for this are getting something that is in a hackable (and fun > to do so) state as soon as possible. That means cheating a lot. Not > parsing Ruby at all is one way to cheat. Not optimizing is another one. > The compiler being self-hosted is the primary goal for me. I think it > would be a lot less useful if it wasn't possible for everyone with Ruby > knowledge to modify and contribute to it.
Yes, this is key imho.
> I picked .NET because it seems quite easy to go self-hosting there. You > can write a .NET bytecode compiler in Ruby by using a Ruby<>.NET bridge > to use the code generation APIs, after that you would just need to get > a minimal subset of Ruby working so that you can use the (very rich and > comfortable to use) .NET standard library and base a full Ruby > implementation around that. I think it is a very simple approach, but > haven't actually tested it just yet so it's probably best taken with a > huge grain of salt.
Our group has also been working on targetting DotNet. There is a paper from one of my students at
and we later worked with Hal Fulton who had some funding for looking into this. However, our ambition was probably too high by writing a full CLR assembler in pure-Ruby; we came pretty close though so it might still be an interesting approach. But we also spiked using one of the Ruby<->DotNet bridges and calling the Reflection.Emit API. Problem with the latter is that there are still things that the API does not allow you to do but which you can do if you control it all from your own code.
Anyway, the main problems with DotNet will be continuations but there are other problems also.
Personally, I think neko (nekovm.org) is a much easier first target. It is well adapted to dynlangs and the author listens and extends it as one proposes things that is missing. Performance is supposed to be good.
> My basic goal was to keep things as minimal as possible so that I would > actually have a chance of producing something useful. Otherwise the > risk of cancelling the project before it becomes useful is way too > high.
vruz <horacio.lo...@gmail.com> writes: >> "Research group for an experimental, fast Ruby compiler." >> which I think is a bit misleading (or I don't share the goal ;)).
> you're right, the title is misleading.
>> My problem is "fast compiler" which is definetely not a high prio imho. >> A compiler able to produce "fast code" would be another thing... ;)
> able to produce fast code. you're right. > so the title should be "Research group for an experimental Ruby compiler"
How about "Research group for an experimental, efficient Ruby compiler"?
>> 1. Written in as pure-Ruby as possible (because we want max involvement >> from a community which will never happen if written in other lang, also >> because a compiler should be self-hosting/bootstrapped since if we like >> a lang we want to use it (ok, langs can be specific to certain purposes >> or better/worse for some situations but I think Ruby will be just fine >> for writing compiler...)
> I think this aligns well with Alex's Rubydium and with Florian Groß's > approach too
> Very modular architecture is also a key to wider adoption, to enable > parallel participation. (as long as this doesn't trump with runtime > performance)
Writing the compiler in Ruby is a requirement IMO too, ideally, the whole thing should be able to bootstrap from a Matz-Ruby 1.8+ and a C compiler...
Read that one back when I was working on the JScript.NET run time. It was a very interesting read.
> Anyway, the main problems with DotNet will be continuations but there > are other problems also.
True, it's not easy to do 100% of Ruby. I've spent some time thinking about continuations and think it would be possible to generally transform all methods through which continuations could pass as in http://flgr.dyndns.org/callcc-cil.rb (simple sample).
It wouldn't be trivial and I'm pretty sure there would be border cases to take care off, but I think it might be possible to make continuations work correctly with quite some effort. It's something for later, however.
I'm not sure about Neko. Haven't had a in-depth look at it yet. But you think it would be easy to map Ruby to it?
> Right now I'm not sure if it still makes sense to do that project.
I think it does makes sense, see the other mail about the QUT licencing scheme.
Since targetting the CLR was one of the goals of the team leaded by Robert Feldt, I think it's really worth having a look whether something can be done to help each other.
Acknowledgely the legal stuff is not a scientific nor technological matter, yet it's a very important one if you consider the impact it can have in the real world.
> How about "Research group for an experimental, efficient Ruby compiler"?
Agreed, it's set to this title you suggested now, unless somebody comes up with something better.
>Writing the compiler in Ruby is a requirement IMO too, ideally, the >whole thing should be able to bootstrap from a Matz-Ruby 1.8+ and a C >compiler...
I think we all can agree with this. (if not, please raise your hand)
Using Matz's C-Ruby 1.8.x is the key to wider adoption.
Also, writing it in Ruby, or a subset of Ruby (like Robert Feldt's SRuby) should allow for better chances of self-hosting-ness in the future.
(we still have to discuss this other, self-hosting matters, and how much priority weight it has specifically later)
side note: you are all gonna have to put up with me being a bitter git right now :P
On Apr 1, 2006, at 11:01 PM, Horacio Lopez wrote:
> Using Matz's C-Ruby 1.8.x is the key to wider adoption.
i don't understand what you mean by using...
in any case, compiling a self hosting version down to a memory dump that can be bootstrapped with a small c/llvm program is the way to go.
> Also, writing it in Ruby, or a subset of Ruby (like Robert Feldt's > SRuby) should allow for better chances of self-hosting-ness in the > future.
use of a subset is nice only for getting a first version up and running quickly, namely, not requiring full dynamic type-feedback based persisted type annotations to allow bootstrapping of a fast enough version. but instead going for a simple type-inference-able implementation which a subset language would support.
> (we still have to discuss this other, self-hosting matters, and how > much priority weight it has specifically later)
you mean wrt "i'd suggest you take a look at pypy. now that you've started talking about both self hosting and targetting multiple languages i realize this project really ain't gonna get very far..." ?
pypy is a 12-18 man year project thats all i wanted to point out multiple backends + self hosting in their aim and its going to be 2-3 years before they finish (in my eyes finish == 10x faster than python) even though i think they have many more developers than we could ever hope to amass...
self hosting remains the highest priority for me. but both? thats a no go.
just don't want this to be vapourware, so cutting down the list of requirements and being insanely pragmatic is the only real option.
nope, not bitter at all, I agree with the total pragmatism in this case if we want to see some actual working code one of these days.
and no, I wasn't talking about Pypy exactly, I'm already aware of the huge investment they made (and continue to make) I have a huge respect for the Pypy guys, and I'm aware of the brilliance of Armin Rigo and others. besides, you already know all the things I'm aware of, it's unnecessary to keep stating them here :-)
just pointing out that as of now, nothing is definitive, and this is a discussion where other people are sharing their personal experiences as well.
we don't want anything to be left out. even the bad.
we already know that *none* of these personal experiences really worked out, so that's precisely why we are here, to learn what was right, and to polish whatever went wrong.
> now that you've started talking about both self hosting and targetting > multiple languages i realize this project really ain't gonna get > very far..."
for further precision, in reply to: "this project really ain't gonna get very far"
there is no project, it's research, it's a discussion too soon to say it will or it won't do anything.
your comments about "BOTH self-hosting and targetting multiple languages" are more clear now, thanks for the clarification.
> > now that you've started talking about both self hosting and targetting > > multiple languages i realize this project really ain't gonna get > > very far..."
> for further precision, in reply to: > "this project really ain't gonna get very far"
> there is no project, it's research, it's a discussion > too soon to say it will or it won't do anything.
Yes, this is my take on it also; we have many and varied goals. This is a place for discussing options imho.
Neko: Is a very interesting target. He actively supports compilation of dynlangs like Python and Ruby and will extend neko if needed. He already added self-style chaining of prototypes to support Ruby's object model on my request. He will add for a __resolve__ method to be called if no matching method is found in the chain (for simper method_missing handling) etc.
|
