Seven new VMs, all in a row
Peter Suk
I thought I'd talk about my new project here, since there is a good
chance that someone might be interested in it. I'm planning to put
Ruby on top of Smalltalk VMs. Ruby and Smalltalk are very similar
under the covers, so Smalltalk VMs are a very good match for the
language. This will give Ruby a much faster execution environment
(perhaps 30X), VMs which are capable of incremental garbage collection,
generational garbage collection that is so fast your progra still works
even with an infinite loop allocating new objects (I do this as a lark
sometimes), a wonderful debugger which will let programmers modify
methods on the fly & continue execution, a "workspace" window where you
can execute arbitrary code, a visual "inspect", a powerful "Refactoring
Browser," an industrial strength OODB (Gemstone) with objects and
methods you can define in Ruby, and a readily accessible meta-level
which will allow Rubyists to readily modify their own language. (For
example, you could then use Method wrappers to very quickly implement
an Aspect-Oriented Ruby.)
My strategy for doing this involves writing a Ruby parser (or, rather,
translating the existing one in JRuby to Ruby) then writing a Smalltalk
Parser object to request parsing of Ruby code into an AST from a Ruby
program outside of Smalltalk. We then reify the AST inside the image
and use it to compile Ruby methods into bytecodes. (Multiple syntaxes
can coexist in one Smalltalk image.) Once this is done, we can then
compile the external Ruby parser and bring it into Smalltalk.
Afterwards, we can use the Refactoring Browser Smalltalk parser plus a
little runtime type inferencing to incrementally transform the image
into pure Ruby.
We can do all of this without changing the structure of Ruby files &
Modules or requiring Rubyists to do Smalltalk style image oriented
development. And for those of you sharp enough to wonder: yes, we can
handle Modules, Mixins, and fully qualified Method names without
changing the Smalltalk VMs. (At least those that have Namespaces.)
If anyone is interested, please drop me a line.
--Peter
--
There's neither heaven nor hell, save what we grant ourselves.
There's neither fairness nor justice, save what we grant each other.
R. Mark Volkmann
VM.
Quoting Peter Suk <peter.kwa...@mac.com>:
--
R. Mark Volkmann
Partner, Object Computing, Inc.
Peter Suk
> I think it would be really cool if you could do this on top of the
> free Squeak
> VM.
Yes, I was thinking along these lines. However, no real Namespaces
(yet) in Squeak. We could finagle this by referencing classes with
Module-name prefixes. (This has already been proposed.) When real
Namespaces appear in Squeak, we could then properly place the classes
in modules/namespaces and rename them programatically.
The VMs I am targeting first are commercial ones, but they have free
versions/licenses. (VisualWorks and Gemstone) In addition, they are
very attractive for implementing servers. (VisualWorks is damn fast,
and Gemstone is a mature, rock-solid OODB.)
Lothar Scholz
PS> Hello everyone,
PS> I thought I'd talk about my new project here, since there is a good
PS> chance that someone might be interested in it. I'm planning to put
Is this just brainstorming or have you already done something ?
--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's
Peter Suk
> Hello Peter,
>
> PS> Hello everyone,
>
> PS> I thought I'd talk about my new project here, since there is a good
> PS> chance that someone might be interested in it. I'm planning to put
>
> Is this just brainstorming or have you already done something ?
Lothar,
I have my hands on a working copy of Racc, and the JRuby source. I've
begun the port of the RubyLexer into Smalltalk, but I've decided to
abandon that and use the bootstrap from outside the image technique
instead. I've also started the port of DefaultRubyParser.y from the
JRuby source. I have a simple Ruby script that uses Regexps to
transform the DefaultRubyParser.ry into Ruby. (Regexps are not
powerful enough to tackle the job generically -- that would take a
Context Free Grammar -- but I am including enough specific context in
the regexps that it will work in the specific cases needed.)
I have also worked out and demonstrated the manipulation of method
dictionaries so that we can compile fully qualified Ruby method names
for the VisualWorks VM. I have sketched out a design for handling Ruby
Mixins and assignment-implicit accessors.
I have also recruited a helper for translating the AST node classes.
(There is something to be said for little languages. It's a lot easier
to build syntax driven tools for them!)
Next, I will be taking a very simplified Ruby subset (the calculator
example from the "Dragon Book" which comes with Racc, and the ability
to define methods in Ruby classes) and working out the
Parser-Outside-The-Image scheme with that. Hopefully I'll have that by
this weekend.
jc
- Would this support programming both within or without an image,
choice up to the user?
- From what I understand Smalltalk has a small VM written in C, with
the rest written in Smalltalk itself. Would your version be the Ruby
equivalent of this (VM with everything else in Ruby), or would there
still be some Smalltalk behind the Ruby?
-jc
pat eyler
See the MetaRuby project:
http://blog.zenspider.com/archives/metaruby/index.html (among other places)
(Although this is about Ruby in Ruby, not Ruby in smalltalk)
>
> -jc
>
>
--
thanks,
-pate
-------------------------
We are often unable to tell people what they need to know, because
they want to know something else, and would therefore only
misunderstand what we said
- the Raven (George MacDonald, Lilith)
Florian Groß
> I thought I'd talk about my new project here, since there is a good
> chance that someone might be interested in it. I'm planning to put
> Ruby on top of Smalltalk VMs.
This does indeed sound promising. Having a self-hosted Ruby would have
certain benefits as well.
> a wonderful debugger which will let programmers modify methods on the
> fly & continue execution
That is already possible with ruby-breakpoint which spawns an IRB
session at a specific place of your code. See
http://ruby-breakpoint.rubyforge.org/ if you are interested in this sort
of things.
It sure can't hurt to have more options, though.
Peter Suk
On Apr 7, 2005, at 2:19 PM, jc wrote:
> Hi Peter,
>
> - Would this support programming both within or without an image,
> choice up to the user?
Basically, you could treat the image as just a very powerful
debugger/IDE. The image would spit out new versions of your module
files, which you could then edit/diff to your liking. I have devised a
very powerful algorithm to respect comments, based on diffs of token
sequences.
> - From what I understand Smalltalk has a small VM written in C, with
> the rest written in Smalltalk itself. Would your version be the Ruby
> equivalent of this (VM with everything else in Ruby), or would there
> still be some Smalltalk behind the Ruby?
Initially, there would be some Smalltalk behind the Ruby, but we'd fix
it so you'd never see it. Eventually, we can develop a completely Ruby
meta-layer.
Peter Suk
On Apr 7, 2005, at 2:36 PM, pat eyler wrote:
> On Apr 7, 2005 12:19 PM, jc <james.c...@gmail.com> wrote:
>> Hi Peter,
>>
>> - Would this support programming both within or without an image,
>> choice up to the user?
>>
>> - From what I understand Smalltalk has a small VM written in C, with
>> the rest written in Smalltalk itself. Would your version be the Ruby
>> equivalent of this (VM with everything else in Ruby), or would there
>> still be some Smalltalk behind the Ruby?
>
> See the MetaRuby project:
>
> http://blog.zenspider.com/archives/metaruby/index.html (among other
> places)
>
> (Although this is about Ruby in Ruby, not Ruby in smalltalk)
I am aware of MetaRuby. This project is also about Ruby in Ruby.
(Smalltalk is in Smalltalk. But the VM doesn't care which
language/meta-language it is running!)
However, this meta-Ruby will run two orders of magnitude faster!
Peter Suk
In the Smalltalk debuggers, you can spawn off as many debugger sessions
as you want by highlighting arbitrary code in the debugger as well as
any browser window, as well as visually "inspect" any object (with each
one appearing in a window), with the context of every window sensitive
to the class being displayed. The effect is basically an
"irb-anywhere" which you can arbitrarily interleave with any number of
visual inspects. That, and a self-hosted language, faster VM, more
garbage collection options, etc...
Lothar Scholz
PS> to the class being displayed. The effect is basically an
PS> "irb-anywhere" which you can arbitrarily interleave with any number of
So it can crash anywhere :-) SCNR.
Yes maybe its possible to transfer part of the language (all?) into an
smalltalk VM. But you can never get all the binary extensions to work,
an emulation layer would kill all your speed benefits if possible
at all. So maybe you should simply announce a ruby branch called SmallRuby,
since this would not have so much todo with the current ruby anymore.
But i'm courious whats your result in the next years.
Is this your Ph.D thesis ?
Glenn Parker
--
Glenn Parker | glenn.parker-AT-comcast.net | <http://www.tetrafoil.com/>
Thomas E Enebo
>
> Yes maybe its possible to transfer part of the language (all?) into an
> smalltalk VM. But you can never get all the binary extensions to work,
> an emulation layer would kill all your speed benefits if possible
> at all. So maybe you should simply announce a ruby branch called SmallRuby,
> since this would not have so much todo with the current ruby anymore.
OTOH, if it is 30x faster and makes an alternate implementation of socket
(which likely would not have any speed deficiency) and a few other extensions
it may become an attractive branch to use for many people.
Seems like an idea worth pursuing to me...Proofs in the pudding though.
It will be interesting to see what gains materialize...
-Tom
--
+ http://www.tc.umn.edu/~enebo +---- mailto:en...@acm.org ----+
| Thomas E Enebo, Protagonist | "A word is worth a thousand |
| | pictures" -Bruce Tognazzini |
Eric Hodel
> Yes maybe its possible to transfer part of the language (all?) into an
> smalltalk VM. But you can never get all the binary extensions to work,
> an emulation layer would kill all your speed benefits if possible
> at all. So maybe you should simply announce a ruby branch called
> SmallRuby,
Wow, could you possibly be more negative?
This kind of dismissive overlording behavior does not reflect the usual
spirit of acceptance and togetherness found on this list, especially
since so many projects of the same spirit are in progress at the same
time (YARV, Rubidium, JRuby, MetaRuby and a few less-publicized ones).
This is a time for offering constructive criticism and helpful insight
into the difficult problems ahead, not a time to say things are
impossible!
--
Eric Hodel - drb...@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04
Peter Suk
> Hello Peter,
>
> PS> to the class being displayed. The effect is basically an
> PS> "irb-anywhere" which you can arbitrarily interleave with any
> number of
>
> So it can crash anywhere :-) SCNR.
In general this sort of thing doesn't crash under Smalltalk. In fact,
the debugger is just a normal application doing a few funky things with
Processes. It's just a browser for the reified context stack. In
fact, you can write your own feature-poor, but working debugger in
Smalltalk and have it running in a matter or minutes.
> Yes maybe its possible to transfer part of the language (all?) into an
> smalltalk VM.
Not what I'm trying to do. The Smalltalk VM has basically *none* of
the *language* in it.
> But you can never get all the binary extensions to work,
Why not? There are plenty of binary extensions to the various
Smalltalks. They are designed to make it easy to build in binary
extensions.
> an emulation layer would kill all your speed benefits if possible
> at all. So maybe you should simply announce a ruby branch called
> SmallRuby,
> since this would not have so much todo with the current ruby anymore.
You missed something in my previous posts, or do not have some pieces
of background concerning how Smalltalk works. There is no emulation
going on, except stepping during debugging, and that happens at the
level of the bytecodes so all VM/Object semantics are preserved. Ruby
will be running on top of a Just-In-Time compiler, translating the
bytecodes into native machine instructions. The VisualWorks JIT has
been in development since 1989, so is one of the most mature JIT
interpreters around. Its speed rivals Hotspot in real-world situations
where GC is significant. Object allocation will be native -- Objects
will just be Objects to the VM/garbage collector, with the
Ruby/Smalltalk implementation language being moot.
> But i'm courious whats your result in the next years.
> Is this your Ph.D thesis ?
No. I want to make the world safe for Pure-OO. A way I can help
Pure-OO development is to vastly increase the power of the Ruby
community by giving it access to the great technology developed for
Smalltalk.
Other things that putting Ruby on top of Smalltalk VMs could garner for
the Ruby community:
The Smalltalk MT environment can produce native Windows DLLs that are
indistinguishable from C++ DLLs. We would theoretically be able to do
the same for Ruby when we can host Ruby on that VM. There could be a
Gemstone Ruby -- with a name like that there *should* be -- giving the
community access to a great OODB. There is also a fast Smalltalk VM
for .NET which could be used for Ruby. If we can overcome the lack of
real Namespaces in Squeak, then we'd have a Ruby VM that runs
completely bit-identical on 32 platforms.
Also, I'd like to do automated syntax + type-inference Ruby <-->
Smalltalk translations so that Ruby can benefit from the wealth of
great stuff done in Smalltalk, and Smalltalkers can be invigorated by
contact with a larger, very active community.
--Peter
Peter Suk
On Apr 7, 2005, at 4:45 PM, Glenn Parker wrote:
> So, what are the potential platforms for Ruby on a Smalltalk VM?
Squeak alone has been ported to 32 platforms, running in all of them
bit-identically, including Acorn Risc, WinCE, Mac OSX, Mac System 9,
Windows, Linux, various Unixes. There is a Smalltalk VM for PalmOS.
(slow, though) The first target, VisualWorks, runs on 8 platforms,
including Various Windows, WinCE, Mac 8, Mac 9, Mac OSX, HPUX, AIX,
IRIX, Solaris-SPARC, Linux x86, Linux SPARC, Linux PPC.
--Peter
Avi Bryant
Peter Suk wrote:
> Hello everyone,
>
> I thought I'd talk about my new project here, since there is a good
> chance that someone might be interested in it. I'm planning to put
> Ruby on top of Smalltalk VMs.
Hi Peter,
I'm definitely interested in this; please keep me posted. Actually, a
couple of people are aware that I was looking into the possibility of
doing this as a commercial product - since, from my benchmarking, we
can get about a 20x to 30x speed increase by hosting Ruby on a
commercial Smalltalk VM like VisualWorks, it seemed like it might be
worthwhile. However, the interest wasn't great enough for me to pursue
it, so I'm very glad that someone is.
A small note: you say "at least those that have namespaces", but
there's no need to be concerned about that. Namespaces are an
image-level thing, not a VM-level thing; any Smalltalk VM worth its
salt can support namespaces just fine (and in fact constructs like pool
variables are effectively namespaces already, they just have somewhat
different conventions than Ruby's).
Avi
Phil Tomson
Thomas E Enebo <en...@acm.org> wrote:
>On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:
>>
>> Yes maybe its possible to transfer part of the language (all?) into an
>> smalltalk VM. But you can never get all the binary extensions to work,
>> an emulation layer would kill all your speed benefits if possible
>> at all. So maybe you should simply announce a ruby branch called SmallRuby,
>> since this would not have so much todo with the current ruby anymore.
>
> OTOH, if it is 30x faster and makes an alternate implementation of socket
I'm very skeptical about the 30x faster claim.
Phil
Phil Tomson
Avi Bryant <avi.b...@gmail.com> wrote:
>
>Peter Suk wrote:
>> Hello everyone,
>>
>> I thought I'd talk about my new project here, since there is a good
>> chance that someone might be interested in it. I'm planning to put
>> Ruby on top of Smalltalk VMs.
>
>Hi Peter,
>
>I'm definitely interested in this; please keep me posted. Actually, a
>couple of people are aware that I was looking into the possibility of
>doing this as a commercial product - since, from my benchmarking, we
>can get about a 20x to 30x speed increase by hosting Ruby on a
>commercial Smalltalk VM like VisualWorks, it seemed like it might be
This "20x to 30x speed increase" stuff is interesting. I would have
thought it would be closer to something like 5x. Can anyone elaborate on
how those increases are being acheived in the SmallTalk VMs? I'm wondering
if it might also be worthwhile to incorporate some of these ideas into YARV -
parhaps that would get us to high-speed Ruby even faster than trying to
put Ruby on top of a SmallTalk VM?
Phil
Bill Guindon
I might be skeptical, but if I voice my skepticism loud enough, I may
discourage those who might try (and may succeed).
So yeah... I think that 30x faster is quite possible ;-)
--
Bill Guindon (aka aGorilla)
vruz
>
> I'm very skeptical about the 30x faster claim.
>
> Phil
For what I've seen, reusing something like the Mono native backend
(available for major processor architectures) can yield even higher
performance gains.
A custom tailored VM in assembler can provide 100x and possibly more
depending on the architecture.
30x actually looks very conservative to me, but may well be worth the
effort to reuse an already existing codebase.
Isaac Gouy
> This "20x to 30x speed increase" stuff is interesting. I would have
> thought it would be closer to something like 5x.
More than 5x
http://www.lissett.com/ben/
Phil Tomson
Please, prove my skepticism to be wrong. A Ruby that was 30x faster (on
the same hardware) would be great. As I said in the other post
responding to Avi, if it's possible to speed things up that much with a
SmallTalk VM, maybe we should be studying said VM to figure out how to
make it happen in YARV (since matz has recently said that YARV is the
future VM for Ruby).
Phil
Bill Guindon
> In article <67a2229205040...@mail.gmail.com>,
> Bill Guindon <agor...@gmail.com> wrote:
> >On Apr 7, 2005 10:04 PM, Phil Tomson <pt...@aracnet.com> wrote:
> >> In article <20050407220...@garnet.tc.umn.edu>,
> >> Thomas E Enebo <en...@acm.org> wrote:
> >> >On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:
> >> >>
> >> >> Yes maybe its possible to transfer part of the language (all?) into an
> >> >> smalltalk VM. But you can never get all the binary extensions to work,
> >> >> an emulation layer would kill all your speed benefits if possible
> >> >> at all. So maybe you should simply announce a ruby branch called
> >SmallRuby,
> >> >> since this would not have so much todo with the current ruby anymore.
> >> >
> >> > OTOH, if it is 30x faster and makes an alternate implementation of socket
> >>
> >> I'm very skeptical about the 30x faster claim.
> >>
> >
> >I might be skeptical, but if I voice my skepticism loud enough, I may
> >discourage those who might try (and may succeed).
> >
> >So yeah... I think that 30x faster is quite possible ;-)
> >
>
> Please, prove my skepticism to be wrong.
I leave that job to Peter, my job is to encourage him to try.
> A Ruby that was 30x faster (on
> the same hardware) would be great. As I said in the other post
> responding to Avi, if it's possible to speed things up that much with a
> SmallTalk VM, maybe we should be studying said VM to figure out how to
> make it happen in YARV (since matz has recently said that YARV is the
> future VM for Ruby).
Agreed, and I hope Peter considers that, but if he's hell bent on
building one on top of a SmallTalk VM, then I say go right ahead, and
if it works, we can dissect that later.
Yukihiro Matsumoto
In message "Re: Seven new VMs, all in a row"
on Fri, 8 Apr 2005 13:19:41 +0900, pt...@aracnet.com (Phil Tomson) writes:
|Please, prove my skepticism to be wrong. A Ruby that was 30x faster (on
|the same hardware) would be great. As I said in the other post
|responding to Avi, if it's possible to speed things up that much with a
|SmallTalk VM, maybe we should be studying said VM to figure out how to
|make it happen in YARV (since matz has recently said that YARV is the
|future VM for Ruby).
It's not impossible to make Ruby run 30x faster for some tasks, for
example, YARV runs more than 100x faster in some small benchmarks.
But I'm not sure if it's possible to make it 30x faster in average.
matz.
Bill Guindon
Yes, and my point is that there's a fine line between "I'm not sure if
it's possible" and "I doubt it's possible". One challenges, the other
discourages. Btw, I admire that you're not sure, and if you aren't
sure, how can anyone else be sure until it's tried?
Not knowing if it's possible, if Peter wants to try, we should
encourage it. If he's right, and we're wrong, and we convince him not
to try, we all lose. If he's wrong, what has he wasted but his own
time? He's not on my payroll, so try he should! If he were on my
payroll, I might think differently ;-)
Here's the way I see it, in Ruby...
class Person
def initialize
srand Time.now.to_i
@wants = (rand(24) + 64).chr
end
# what a person wants is (somewhat) read only, and fairly random.
def wants
"Wants: " + @wants
end
# what a person works on is usually the same as what they want.
def works_on
"Works on: " + @wants
end
end
p = Person.new
puts p.wants
puts p.works_on
p.wants = "Z"
Yukihiro Matsumoto
In message "Re: Seven new VMs, all in a row"
on Fri, 8 Apr 2005 13:57:13 +0900, Bill Guindon <agor...@gmail.com> writes:
|Yes, and my point is that there's a fine line between "I'm not sure if
|it's possible" and "I doubt it's possible". One challenges, the other
|discourages. Btw, I admire that you're not sure, and if you aren't
|sure, how can anyone else be sure until it's tried?
I take your point. I don't want to discourage anyone.
The world is before you, Peter, to be proved to be wrong.
matz.
Lothar Scholz
EH> On 07 Apr 2005, at 14:32, Lothar Scholz wrote:
EH> Wow, could you possibly be more negative?
Yes, but i don't think i was very negative. Its just the observation
that some of the most important parts of ruby are not written in ruby
and must be manually convert (Basic library, GUI Bindings, etc.)
The easy creation of binary extensions is a very important point for
me and the reason why i switched from TCL to Ruby.
EH> This kind of dismissive overlording behavior does not reflect the usual
EH> spirit of acceptance and togetherness found on this list, especially
EH> since so many projects of the same spirit are in progress at the same
EH> time (YARV, Rubidium, JRuby, MetaRuby and a few less-publicized ones).
The great number of projects which tries to do this and never come up with
anything useable yet (except JRuby which is an interpreter rewrite
in Java and does not belong to this list) should give anybody
a break to think about the problems a little bit more.
EH> This is a time for offering constructive criticism and helpful insight
EH> into the difficult problems ahead, not a time to say things are
EH> impossible!
I think it's extremely hard to do this task. And that's why i asked if
this is his Ph.D. to see if he has the huge amount of time to get this
up and running. And i did a google search and found only 5 postings from
Peter Suk since 2.Okt 2002 on this mailing list and some other
information - like that he did a port of the Smalltalk refactoring
browser. So it seems to be okay that i'm a little bit suspicious if
this is one of the many ruby newbie ideas that we see here every few
weeks or if we can start serious discussions.
I wish him good luck for this project and hope he is successful.
I'm sure i will try it out and offer constructive criticism in a later
stage of the project.
Lothar Scholz
>> > OTOH, if it is 30x faster and makes an alternate implementation of socket
>>
>> I'm very skeptical about the 30x faster claim.
>>
>> Phil
v> For what I've seen, reusing something like the Mono native backend
v> (available for major processor architectures) can yield even higher
v> performance gains.
v> A custom tailored VM in assembler can provide 100x and possibly more
v> depending on the architecture.
No. You simply forget that the Mono/.NET VM's are optimized for static
typed languages and they compile a frontend language into a classic
3 register instruction machine, not a stack based machine like JAVA
giving much more chance for optimizations.
There i don't see a problem for such high performance boosts.
But if we can't come up with a type inference machine, the theoretical
improvements (and thats what is done by YARV or Parot) are far from
that. If you are willing to spend a huge amount of memory you can go
the python way and keep multiple compiled versions (based on
differently use types) of small code fragments translated.
In 1995 i've written an emacs lisp compatible interpreter and it was
easy to get 8x more speed then the current one with program flow
optimizations (a FORTH like direct threaded interpreter) and together with
integration of the Boehm-Weisser GC and more optimizations i think it's easy
to get even a 20x improvement (especially on Ruby where the implementation
is not very sophisticated).
We can use techniques like Eiffel's system wide optimizations to get the
cost of a method call down to a "call 0x05346403" in most cases, so with
elimination of a VMT we can be even faster then C++.
But then we still need cascaded "if (argument = TYPE(Integer)) x += VALUE(argument)".
Lothar Scholz
YM> It's not impossible to make Ruby run 30x faster for some tasks, for
YM> example, YARV runs more than 100x faster in some small benchmarks.
YM> But I'm not sure if it's possible to make it 30x faster in average.
Since Smalltalk has only very few binary implemented functions i think
we should set the benchmark at something like the REXML parser which
does a lot of pure ruby processing.
And yes i think for tasks like this it is possible to get 30x faster,
but i think it might even be possible for YARV to get 20x faster.
Peter Suk
> This "20x to 30x speed increase" stuff is interesting. I would have
> thought it would be closer to something like 5x. Can anyone elaborate
> on
> how those increases are being acheived in the SmallTalk VMs? I'm
> wondering
> if it might also be worthwhile to incorporate some of these ideas into
> YARV -
> parhaps that would get us to high-speed Ruby even faster than trying to
> put Ruby on top of a SmallTalk VM?
Phil,
30X speed increase is for "Real World Applications" and is highly
dependent on Garbage Collection. I have seen many dozens of Smalltalk
business programs in my time as a consultant. One of the things they
do a lot of is create objects and garbage collect them. In fact,
memory management is probably the biggest single factor that comes up
in the performance of business apps after IO and DB access.
The Generational GC in VisualWorks is truly fantastic. I can sit
around all day with an infinite loop that allocates copies of some
random string like "Doo-Dah, Doo-Dah, All the live long day" running in
the Smalltalk image I am developing in, and there's no obvious effect.
(Yes, I did this earlier this week!) I can crank up this loop until it
is producing a billion new objects in several minutes, and my image is
fine, until I try to do something serious like have the Refactoring
Browser rename all implementors of the method #add: in the image, and
even then, it;s fine, it just does it slowly.
Another thing you should know in terms of anecdotes. A fellow
Smalltalker implemented a bunch of Encryption algorithms a few years
back in VisualWorks Smalltalk. (So we're talking lots of low-level bit
& byte slewing.) He benchmarked them against the reference DLLs from
RSA Data Security, which were written in C. The Smalltalk algorithms
ran at a comparable speed. One of them ran 3% faster! Again, this was
partly due to memory management. The DLLs naively malloc'd & free'd
everything immediately, while the Generational GC in Smalltalk provided
what amounted to a buffer cache optimization for free! (The other part
was he had our VM guys implement some bit & byte slewing primitive
ops.)
And lastly, please remember that VisualWorks and most of the commercial
Smalltalk implementations are actually JIT compilers. Most of the
time, we're running native machine language instructions.
If YARV out does Smalltalk VMs for speed of Ruby execution, then all
the better. The competitive pressure will ultimately be good for the
Ruby community. We'll just settle for providing all of the other great
benefits I've mentioned. (Like the fastest Ruby-In-Ruby!)
Peter Suk
Bill,
I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk
VM will be allocating Objects and JIT-ing bytecodes natively. There is
almost no Smalltalk in the Smalltalk VMs, and what semantics there are
are a very good fit for Ruby anyhow. All I'm doing is providing enough
meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby
compiler written in Ruby).
(But this language defined in itself is a weird concept to most
programmers, and I'll probably go blue in the face repeating this.)
Lothar Scholz
PS> I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk
PS> VM will be allocating Objects and JIT-ing bytecodes natively. There is
PS> almost no Smalltalk in the Smalltalk VMs, and what semantics there are
PS> are a very good fit for Ruby anyhow. All I'm doing is providing enough
PS> meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby
PS> compiler written in Ruby).
Is there a public available technical specification for the VM ?
why the lucky stiff
>I take your point. I don't want to discourage anyone.
>The world is before you, Peter, to be proved to be wrong.
>
> matz.
>
I feel impossibly good right now. Go now, Peter the young-breasted.
Defeat is an accomplishment not even the best of us could achieve.
_why
Jonas Hartmann
Slightly OT: Does Python ship with a JIT or JIT option?
Personally I always wondered why interpreted languages dont ship with any kind of JIT.
From a newbies point of view I understood this, before you mentioned the JIT, like the idea to take the ruby language and put it onto a very fast, general purpose VM, instead of using its own interpreter, which might be great but maybe not so fast.
If that's the point, then its a great thing - as long as security and features are kept more speed is always better.
Peter Suk wrote:
> [...]
> Ruby will be
> running on top of a Just-In-Time compiler, translating the bytecodes
> into native machine instructions. The VisualWorks JIT has been in
> development since 1989, so is one of the most mature JIT interpreters
> around. Its speed rivals Hotspot in real-world situations where GC is
> significant. Object allocation will be native -- Objects will just be
> Objects to the VM/garbage collector, with the Ruby/Smalltalk
> implementation language being moot.
> [...]
> No. I want to make the world safe for Pure-OO. A way I can help
> Pure-OO development is to vastly increase the power of the Ruby
> community by giving it access to the great technology developed for
> Smalltalk.
>
> The Smalltalk MT environment can produce native Windows DLLs that are
> indistinguishable from C++ DLLs. We would theoretically be able to do
> the same for Ruby when we can host Ruby on that VM.
> [...]
--
ionas
Yes, I am a newbie.
George
> VM will be allocating Objects and JIT-ing bytecodes natively. There
is
> almost no Smalltalk in the Smalltalk VMs, and what semantics there
are
> are a very good fit for Ruby anyhow. All I'm doing is providing
enough
> meta-Ruby to enable the Smalltalk VMs to run Ruby (including a Ruby
> compiler written in Ruby).
I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
not, how easily can they be emulated?
- blocks (with Ruby semantics in case they're different from
Smalltalk's)
- mixins
- instance variables that don't need to be predeclared with the class
- class variables & constants
- throw/catch
- exceptions
- continuations
- support for OS facilities like select, signals etc
Too great a mismatch between Ruby and Smalltalk semantics will mean
that an emulation layer will be needed, which will presumably reduce
the performance gains. Of course, I hope that's not true! Good luck
with the project.
-- George
gabriele renzi
>> OTOH, if it is 30x faster and makes an alternate implementation of socket
>
>
> I'm very skeptical about the 30x faster claim.
>
well, take a look at the OO Richards Benchmark
http://pws.prserv.net/dlissett/ben/
the VW vms is something like 70x faster than ruby, so if not too much is
wasted in the layering, there is still a fast vm.
IIRC there is also someone working on a python port on top of VW, which
should be quite fast.
(btw, remember the usual "lies, damn lies & benchmarks"
gabriele renzi
>>> OTOH, if it is 30x faster and makes an alternate implementation of socket
>>
>>I'm very skeptical about the 30x faster claim.
>>
>>Phil
>
>
> For what I've seen, reusing something like the Mono native backend
> (available for major processor architectures) can yield even higher
> performance gains.
I disagree. The IronPython implementation, on mono/x86 runs just 1.2
times faster than CPython.
On PPC IronPyrthon/mono runs even slower than the real CPython.
Surely there are still work to do, but I guess those "the clr is not
great for dynamic languages" claims may be true (I still think you may
found value in using it for interoperability)
Lothar Scholz
G> I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
G> Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
G> not, how easily can they be emulated?
I would like to add
- singleton methods
to this list as i think this does not exist in SmallTalk, and this is
a feature that could knock out the whole method compilation algorithm.
gabriele renzi
> Hello Peter, hello List,
>
>
> Slightly OT: Does Python ship with a JIT or JIT option?
no, Python is just a "normal" vm. There is a python JIT for x86, psyco,
sich works as a mofule for Python.
> Personally I always wondered why interpreted languages dont ship with
> any kind of JIT.
I guess... because it's hard :)
PS
please try to avoid top posting
http://www.caliburn.nl/topposting.html
Peter Suk
> Hello George,
>
> G> I'm curious - how good a fit are Ruby semantics for Smalltalk VMs?
> Do
> G> Smalltalk VMs/bytecode provide the following Ruby facilities, and,
> if
> G> not, how easily can they be emulated?
>
> I would like to add
>
> - singleton methods
>
> to this list as i think this does not exist in SmallTalk, and this is
> a feature that could knock out the whole method compilation algorithm.
Nope. (Rest is left as an excercise.)
Avi Bryant
George wrote:
> I'm curious - how good a fit are Ruby semantics for Smalltalk VMs?
They fit almost exactly, which is why 20-30x is not an outrageous speed
claim, and why comparisons with Mono, JVM, Parrot etc are entirely
missing the point: this is not equivalent to compiling Ruby to Java
classes, say, but to realising that Sun's Hotspot team had been writing
a VM *specifically to run Ruby* all along - which they effectively were
before Sun crippled it, but that's another story.
> Do
> Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
> not, how easily can they be emulated?
>
> - blocks (with Ruby semantics in case they're different from
> Smalltalk's)
Yes.
> - mixins
Yes. Lothar also asked about singleton methods, to which the answer is
also Yes.
> - instance variables that don't need to be predeclared with the class
Not exactly as Ruby does, but you can simply add new instance variables
to the class definition whenever you compile a method that uses a new
one.
> - class variables & constants
Yes.
> - throw/catch
Yes.
> - exceptions
Yes.
> - continuations
Yes (depending on the VM; Squeak, VisualWorks, and Dolphin all support
them).
> - support for OS facilities like select, signals etc
Not portably; the OS facilities and binary extension mechanisms are
highly specific to VMs.
One thing nobody's asked about is variable and optional arguments for
methods. Some VMs (like Squeak's) can handle this just fine, but some
may require that the number of args pushed onto the stack match the
selector. In any case, to do it optimally it would require generating
bytecode, not Smalltalk code, which makes targetting a whole range of
VMs difficult; to do it portably would mean creating an extra Array on
every message send.
Avi
Peter Suk
On Apr 8, 2005, at 3:39 AM, George wrote:
> I'm curious - how good a fit are Ruby semantics for Smalltalk VMs? Do
> Smalltalk VMs/bytecode provide the following Ruby facilities, and, if
> not, how easily can they be emulated?
>
> - blocks (with Ruby semantics in case they're different from
> Smalltalk's)
> - mixins
> - instance variables that don't need to be predeclared with the class
> - class variables & constants
> - throw/catch
> - exceptions
> - continuations
> - support for OS facilities like select, signals etc
I still have to look into continuations and OS facilities in detail.
(Pointers to where these are described in detail for Ruby?) However I
believe continuations are supported for Smalltalk exceptions. Mixins
will be supported by the Compiler manipulating Single Inheritance when
Classes are defined. But as for everything else:
> - blocks (with Ruby semantics in case they're different from
> Smalltalk's)
Almost the same as Smalltalk's.
> - instance variables that don't need to be predeclared with the class
Can be added dynamically in Smalltalk.
> - class variables & constants
Check.
> - throw/catch
Really the same as below.
> - exceptions
Check.
From now on, if you are completely ignorant of Smalltalk, I will cease
answering these questions here.
fl...@sanctacaris.net
And if so, what about incorporating this into Ruby 2.x one day? :-)
-- Ruediger Marcus
Am Freitag, 8. April 2005 04:14 schrieb Phil Tomson:
> In article <1112914652....@g14g2000cwa.googlegroups.com>,
>
> Avi Bryant <avi.b...@gmail.com> wrote:
> >Peter Suk wrote:
> >> Hello everyone,
> >>
> >> I thought I'd talk about my new project here, since there is a good
> >> chance that someone might be interested in it. I'm planning to put
> >> Ruby on top of Smalltalk VMs.
> >
> >Hi Peter,
> >
> >I'm definitely interested in this; please keep me posted. Actually, a
> >couple of people are aware that I was looking into the possibility of
> >doing this as a commercial product - since, from my benchmarking, we
> >can get about a 20x to 30x speed increase by hosting Ruby on a
> >commercial Smalltalk VM like VisualWorks, it seemed like it might be
>
> This "20x to 30x speed increase" stuff is interesting. I would have
> thought it would be closer to something like 5x. Can anyone elaborate on
> how those increases are being acheived in the SmallTalk VMs? I'm wondering
> if it might also be worthwhile to incorporate some of these ideas into YARV
> - parhaps that would get us to high-speed Ruby even faster than trying to
> put Ruby on top of a SmallTalk VM?
>
> Phil
--
Chevalier Dr Dr Ruediger Marcus Flaig
Institute for Immunology
University of Heidelberg
INF 305, D-69121 Heidelberg
"Drain you of your sanity,
Face the Thing That Should Not Be."
--
Diese E-Mail wurde mit http://www.mail-inspector.de verschickt
Mail Inspector ist ein kostenloser Service von http://www.is-fun.net
Der Absender dieser E-Mail hatte die IP: 129.206.124.135
Avi Bryant
Robert Feldt
> Hello everyone,
>
> I thought I'd talk about my new project here, since there is a good
> chance that someone might be interested in it. I'm planning to put
> under the covers, so Smalltalk VMs are a very good match for the
> language. This will give Ruby a much faster execution environment
> (perhaps 30X), VMs which are capable of incremental garbage collection,
> generational garbage collection that is so fast your progra still works
> even with an infinite loop allocating new objects (I do this as a lark
> sometimes), a wonderful debugger which will let programmers modify
> methods on the fly & continue execution, a "workspace" window where you
> can execute arbitrary code, a visual "inspect", a powerful "Refactoring
> Browser," an industrial strength OODB (Gemstone) with objects and
> methods you can define in Ruby, and a readily accessible meta-level
> which will allow Rubyists to readily modify their own language. (For
> example, you could then use Method wrappers to very quickly implement
> an Aspect-Oriented Ruby.)
>
> My strategy for doing this involves writing a Ruby parser (or, rather,
> translating the existing one in JRuby to Ruby) then writing a Smalltalk
> Parser object to request parsing of Ruby code into an AST from a Ruby
> program outside of Smalltalk. We then reify the AST inside the image
> and use it to compile Ruby methods into bytecodes. (Multiple syntaxes
> can coexist in one Smalltalk image.) Once this is done, we can then
> compile the external Ruby parser and bring it into Smalltalk.
> Afterwards, we can use the Refactoring Browser Smalltalk parser plus a
> little runtime type inferencing to incrementally transform the image
> into pure Ruby.
>
> We can do all of this without changing the structure of Ruby files &
> Modules or requiring Rubyists to do Smalltalk style image oriented
> development. And for those of you sharp enough to wonder: yes, we can
> handle Modules, Mixins, and fully qualified Method names without
> changing the Smalltalk VMs. (At least those that have Namespaces.)
>
> If anyone is interested, please drop me a line.
>
I'm interested; sounds like a great idea. However, I haven't been very
lucky turning up (free) info about the ("commercial") Smalltalk VM's
and their instruction sets etc (apart from the classic smalltalk
papers and I guess squeak where everything is open). Do you have good
pointers to such info?
Best,
Robert Feldt
robert...@gmail.com
Avi Bryant
VM spec. I still haven't got the hang of this new google groups UI...
Avi
Robert Feldt
> http://www.iam.unibe.ch/~ducasse/FreeBooks/CollectiveNBlueBook/oe-tour-sept19.pdf
>
Great, thanks Avi. Can I assume the commercial VMs are similar to this
or do you know of detailed descriptions of them?
BTW, wasn't Peter Deutsch himself onto a similar project for Python:
http://blog.amber.org/2004/08/12/python-on-smalltalks-vm/
Anyone seen something come out from that?
Best,
Robert
Peter Suk
> Yeah, I'm curious about that too... a couple of years ago I wrote a
> Python-2-native compiler but was very disappointed to find that it
> revved up things only 2x to 3x (to less than 1/10 the speed of C
> code), the matter obviously being that Python's way of object handling
> already consumed most of the CPU time.
Reference counting GC, isn't it? That automatically raises your
assignment overhead 2X.
> Obviously, the need for endless type checks, comparisons and
> conversions, not to mention memory allocation and deallocation, is a
> bottleneck, at least in Python -- and though I am not really familiar
> with the internals of the Ruby interpreter, I think that the problem
> will be pretty much the same. Also in Smalltalk.
Generational GC. (So you're not at the mercy of the system's malloc &
free) JIT. Implementing message sends as direct jumps for monomorphic
methods. (& polymorphic methods with a caching strategy.) Call chain
optimization. Those are the ones I know about. Not all Smalltalks use
them.
> So there must really be some fundamental stroke of genius involved....
> And if so, what about incorporating this into Ruby 2.x one day? :-)
As far as I am concerned, this project is a rehearsal for the
appearance of Ruby 2.x.
Lothar Scholz
AB> Sorry, my last post was meant to be in reply to Lothar's request for a
AB> VM spec. I still haven't got the hang of this new google groups UI...
Yes, nice but this is Squeak. It seems that this is just a bytecode
machine without a JIT. Is there any document available vor VisualWorks ?
Sergey Philippov
> In article <20050407220...@garnet.tc.umn.edu>,
> Thomas E Enebo <en...@acm.org> wrote:
>
>>On Fri, 08 Apr 2005, Lothar Scholz defenestrated me:
>>
>>>Yes maybe its possible to transfer part of the language (all?) into an
>>>smalltalk VM. But you can never get all the binary extensions to work,
>>>an emulation layer would kill all your speed benefits if possible
>>>at all. So maybe you should simply announce a ruby branch called SmallRuby,
>>>since this would not have so much todo with the current ruby anymore.
>>
>> OTOH, if it is 30x faster and makes an alternate implementation of socket
>
>
> I'm very skeptical about the 30x faster claim.
>
Don't be too sceptical:
fib(32): ruby - 7297ms, visualworks 7.2.1 - 79ms
~90x ;)
--
wbr, ps
sphilippov-at-gmail-com
Lothar Scholz
LS> Hello Avi,
AB>> Sorry, my last post was meant to be in reply to Lothar's request for a
AB>> VM spec. I still haven't got the hang of this new google groups UI...
LS> Yes, nice but this is Squeak. It seems that this is just a bytecode
LS> machine without a JIT. Is there any document available vor VisualWorks ?
just found this paragraph and then stopped reading the document:
---------------------------
In a typical system it often turns out that the same message is sent to instances of the
same class again and again; consider how often we use arrays of SmallInteger or
Character or String. To improve average performance, the VM can cache the
found method. If the same combination of the method and the receiver's class are
found in the cache, we avoid a repeat of the full search of the MethodDictionary
chain. See the method Interpreter > lookupInMethodCacheSel:class:
for the implementation.
VisualWorks and some other commercial Smalltalks use inline cacheing, whereby the
cached target and some checking information is included inline with the dynamically
translated methods. Although more efficient, it is more complex and has strong
interactions with the details of the cpu instruction and data caches.
---------------------------
So message dispatching with squeak is not much more efficent then what
i expect from YARV, a simple bytecode dispatcher with dynamic method
lookup tables and a small lookup cache. It now seems to be a simple question
of the GC and there i would again vote for the Boehm Weisser GC which
is a quite fast incremental GC working on all popular platforms and much easier
to integrate (even with typing hints) into "gc.c" then your project.
Did you ever thought about the legal problems of your problems when
using the visual works engine ? I did a short look at the cincom webpage and
i guess it is as expensive as ever, which means > 5000 US$ per
license. I hate it when the guys are not even publishing there prices
without a contact form.
Peter Suk
Check.
http://c2.com/cgi/wiki?CallWithCurrentContinuation
Avi Bryant
Robert Feldt wrote:
> On Apr 8, 2005 11:54 AM, Avi Bryant <avi.b...@gmail.com> wrote:
> >
http://www.iam.unibe.ch/~ducasse/FreeBooks/CollectiveNBlueBook/oe-tour-sept19.pdf
> >
> Great, thanks Avi. Can I assume the commercial VMs are similar to
this
> or do you know of detailed descriptions of them?
I don't know of any detailed descriptions. Apart from the obvious fact
that they have JITs and the stock Squeak engine doesn't, one difference
I know is that VisualWorks still uses an object table rather than
direct object references, which makes #become: much faster and probably
helps with garbage collection. However, for the purpose of knowing how
to model Ruby on top of the VM (the object format, classes and method
dictionaries, how context stacks work), they should all be very
similar, at least on the surface - never mind what optimizations are
going on below.
> BTW, wasn't Peter Deutsch himself onto a similar project for Python:
>
> http://blog.amber.org/2004/08/12/python-on-smalltalks-vm/
Yes, but that's a much harder problem, since Python has very different
semantics from Smalltalk, whereas Ruby's are for most purposes
identical.
Avi
Peter Suk
> Did you ever thought about the legal problems of your problems when
> using the visual works engine ?
Yes. I used to work for them.
> I did a short look at the cincom webpage and
> i guess it is as expensive as ever, which means > 5000 US$ per
> license. I hate it when the guys are not even publishing there prices
> without a contact form.
Yes. I wish they'd cut that out, but they can be quite "old-school"
software industry-wise.
The main purpose for the VisualWorks version is for commercial server
images that have to run fast. Certain organizations will prefer to
have a product with official support and will pay a premium for this
and speed. If/When this happens, I'll license their Object Engine as a
VAR, and pass on the licensing costs. (Also, they are looking for new
products to sell.) Otherwise, the Ruby image can just be a fast
platform for education and academic research in Ruby just as it is for
Smalltalk. Also, it is the platform that I know best, so it is a good
place for me to start. For purposes of just having the Refactoring,
Debugging, Browsing environment, people who want free as in speech will
probably opt for hosting on Squeak.
Matt Lawrence
> I'm not *building* a Ruby VM on top of a Smalltalk VM. The Smalltalk VM will
> be allocating Objects and JIT-ing bytecodes natively. There is almost no
> Smalltalk in the Smalltalk VMs, and what semantics there are are a very good
> fit for Ruby anyhow. All I'm doing is providing enough meta-Ruby to enable
> the Smalltalk VMs to run Ruby (including a Ruby compiler written in Ruby).
>
> (But this language defined in itself is a weird concept to most programmers,
> and I'll probably go blue in the face repeating this.)
Makes sense to me. All in all, this sounds like a really cool project and
I'm looking forward to trying it.
My Ruby skills are still a bit weak, but I know a lot about various
low-level architectures. Koichi's talk last year was very interesting.
-- Matt
Nothing great was ever accomplished without _passion_
Kujawa, Greg
from languages like C++ and Java the leanness and logic of Ruby is nice.
Smalltalk is something I haven't had the opportunity to work with. I have
downloaded the non-commercial Cincom Smalltalk ISO and am installing it now.
I have read that it comes with its own native GUI toolkit and other things.
Is Smalltalk as comprehensive and powerful as Ruby in terms of Ruby's many
additional packages and bindings (e.g. - DBI, winole32, win32api, Tk, GTK,
Qt, SMTP, POP, SSH, HTTP, etc.)?
-----Original Message-----
From: Peter Suk [mailto:peter.kwa...@mac.com]
Sent: Thursday, April 07, 2005 6:40 PM
To: ruby...@ruby-lang.org
Subject: Re: Seven new VMs, all in a row
On Apr 7, 2005, at 4:45 PM, Glenn Parker wrote:
> So, what are the potential platforms for Ruby on a Smalltalk VM?
Squeak alone has been ported to 32 platforms, running in all of them
bit-identically, including Acorn Risc, WinCE, Mac OSX, Mac System 9,
Windows, Linux, various Unixes. There is a Smalltalk VM for PalmOS.
(slow, though) The first target, VisualWorks, runs on 8 platforms,
including Various Windows, WinCE, Mac 8, Mac 9, Mac OSX, HPUX, AIX,
IRIX, Solaris-SPARC, Linux x86, Linux SPARC, Linux PPC.
Glenn Parker
>
>>- instance variables that don't need to be predeclared with the class
>
> Not exactly as Ruby does, but you can simply add new instance variables
> to the class definition whenever you compile a method that uses a new
> one.
Isn't that jumping the gun just a bit? An instance variable (in Ruby)
should not exist in an object until the line that assigns/creates it is
actually executed. It's a subtle point, but it could impact some types
of reflective programming. Maybe you can mask its existence somehow?
--
Glenn Parker | glenn.parker-AT-comcast.net | <http://www.tetrafoil.com/>
George
Ruby and Smalltalk do indeed sound like a good match. I remember Robert
Feldt saying a few years ago that he'd done some experiments with Ruby
on Smallscript's VM that looked promising.
Peter said:
> From now on, if you are completely ignorant of Smalltalk,
> I will cease answering these questions here.
Yikes! I _am_ almost completely ignorant of Smalltalk (I'm know the
basic concepts of the language, but I've never programmed in it), but
I've done some work on VMs and was genuinely interested in the answers.
Robert Feldt
> Avi / Peter -- thanks for providing the feedback I was looking for.
> Ruby and Smalltalk do indeed sound like a good match. I remember Robert
> Feldt saying a few years ago that he'd done some experiments with Ruby
> on Smallscript's VM that looked promising.
>
enough to build on at that time and haven't followed it since then.
Also I just did some ruby2smallscript (source to source) translation
experiments so it was a different approach.
Best,
Robert
Avi Bryant
Glenn Parker wrote:
> Avi Bryant wrote:
> >
> >>- instance variables that don't need to be predeclared with the
class
> >
> > Not exactly as Ruby does, but you can simply add new instance
variables
> > to the class definition whenever you compile a method that uses a
new
> > one.
>
> Isn't that jumping the gun just a bit? An instance variable (in
Ruby)
> should not exist in an object until the line that assigns/creates it
is
> actually executed. It's a subtle point, but it could impact some
types
> of reflective programming. Maybe you can mask its existence somehow?
Yes, pretty easily I'd think. For example: when you create a new
instance in Smalltalk, all the instance variables start out initialized
to Smalltalk's nil value (of class UndefinedObject). As soon as it was
referenced or assigned to, that would get replaced with some Ruby value
(possibly Ruby's nil, of class NilClass). So you could always tell for
a given instance which instance variables already "exist" inside the
Ruby semantics.
You could also just use a hashtable for each instance to hold all of
the variables, like Ruby does, but the fact that Smalltalk can do
direct instance variable access ends up being a nice speed and memory
gain, so I'd rather not give that up.
Avi
Patrick Down
definition at runtime from a database definition? ( I assume that's how
it works. )
Glenn Parker
The Smalltalk Ruby will still need to handle more dynamic methods of
instance variable creation:
class MyClass
def add_ivar(name)
instance_variable_set(name, nil)
end
end
What happens to instances that have already been created when a new
instance variable is seen by the compiler?
There are also issues with using more memory than necessary if the
interpreter creates every instance variable the moment it is observed by
the compiler.
I'm guessing Ruby instance variables will have to be created dynamically.
Avi Bryant
If you're talking about methods, that's no problem at all - like Ruby,
Smalltalk makes no distinction between compile-time and runtime (in
fact, Smalltalk environments don't even make a distinction between
edit-time and runtime), so adding (or removing, or modifying) methods
at "runtime" is the usual case.
If you're talking about instance variables: when you add a new instance
variable to a class, the system walks through all of the old instances
of that class and makes a new copy with space allocated for the new
inst var. The next step, which requires VM support, is to do an atomic
swap of all of the old instances for all of the new instances (some
Smalltalks can do this faster than others, but in the worst case it
requires a full garbage collection).
My guess is that for almost all applications, the occasional added cost
when adding inst vars would be more than made up for by the increased
speed when accessing them, and the reduced memory consumption (and thus
reduced load on the GC).
Avi
Avi Bryant
Glenn Parker wrote:
> The Smalltalk Ruby will still need to handle more dynamic methods of
> instance variable creation:
>
> class MyClass
> def add_ivar(name)
> instance_variable_set(name, nil)
> end
> end
>
> What happens to instances that have already been created when a new
> instance variable is seen by the compiler?
Just posted on that...
> There are also issues with using more memory than necessary if the
> interpreter creates every instance variable the moment it is observed
by
> the compiler.
Only in truly pathological cases - for normal numbers of instance
variables, the overhead of an external lookup table would be higher
than that of keeping a few unused slots in the body of the object. But
of course if the compiler saw that instances of a particular class
might have up to 50 instance variables, it could choose to implement
that class with a hashtable for variables. I'm not convinced that case
is likely enough to be worth checking and optimizing for, but who
knows.
Avi
Eric Hodel
You missed the second to last sentence, "... the cached target and some
checking information is included inline with the dynamically translated
methods."
This can be a big boost because you don't need to look up the method or
even dispatch to it anymore, its right there where it needs to be.
--
Eric Hodel - drb...@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04
Charles Mills
Peter Suk wrote:
> On Apr 8, 2005, at 4:41 AM, fl...@sanctacaris.net wrote:
>
> > Yeah, I'm curious about that too... a couple of years ago I wrote a
> > Python-2-native compiler but was very disappointed to find that it
> > revved up things only 2x to 3x (to less than 1/10 the speed of C
> > code), the matter obviously being that Python's way of object
handling
> > already consumed most of the CPU time.
>
> Reference counting GC, isn't it? That automatically raises your
> assignment overhead 2X.
and breaks all existing extensions, unless you have special compiler
support.
> > Obviously, the need for endless type checks, comparisons and
> > conversions, not to mention memory allocation and deallocation, is
a
> > bottleneck, at least in Python -- and though I am not really
familiar
> > with the internals of the Ruby interpreter, I think that the
problem
> > will be pretty much the same. Also in Smalltalk.
>
> Generational GC. (So you're not at the mercy of the system's malloc
&
> free)
A generational GC would be great. There has been talk of including one
in the next version of the Ruby interpreter. However C extensions also
pose a problem here as well - it seems creating a write barrier that
works with C libraries not written originally with Ruby in mind will be
very difficult/slow.
-Charlie
Peter Suk
> Avi Bryant wrote:
>>> - instance variables that don't need to be predeclared with the class
>> Not exactly as Ruby does, but you can simply add new instance
>> variables
>> to the class definition whenever you compile a method that uses a new
>> one.
>
> Isn't that jumping the gun just a bit? An instance variable (in Ruby)
> should not exist in an object until the line that assigns/creates it
> is actually executed. It's a subtle point, but it could impact some
> types of reflective programming. Maybe you can mask its existence
> somehow?
I've been giving this some thought. There's no reason why we can't
create ST instance vars at runtime. However, it will mean deferring
compilation. The suggestion to mask its creation is a good one from a
performance standpoint, and is certainly doable.
Peter Suk
Sorry, it was late at night, and that came off harsher than it should
have. It's just that I'm finding myself essentially repeating myself.
The feedback has informed my thinking and has been valuable, but I
should just tell people to go and find and read docs. I will get around
to posting them on the RubyForge site. This project is named Alumina.
Ruby is just one crystalline form of Alumina.
Peter Suk
> Peter Suk wrote:
>> Generational GC. (So you're not at the mercy of the system's malloc
> &
>> free)
>
> A generational GC would be great. There has been talk of including one
> in the next version of the Ruby interpreter. However C extensions also
> pose a problem here as well - it seems creating a write barrier that
> works with C libraries not written originally with Ruby in mind will be
> very difficult/slow.
VisualWorks gets around this and also solves problems the paradox with
not having "real" threads, wanting to do GC efficently, yet also
wanting to do synchronous calls to the OS. The dilemma is this: You
want real threads so your VM can utilize synchronous calls to the OS.
However, this complicates GC immensely. (ObjectStudio Smalltalk lost a
lot of performance because of having real threads. When it was time to
GC, the main VM thread would have to raise a flag and wait for all the
other threads to detect it.)
VisualWorks can use real threads, but only in the case of calling out
to a DLL, and these contexts are exempt from GC. This gives you the
ability to make synchronous calls without freezing the entire image,
yet also leaves your GC simple and fast.
Lothar Scholz
EH> On 08 Apr 2005, at 03:39, Lothar Scholz wrote:
EH> You missed the second to last sentence, "... the cached target and some
EH> checking information is included inline with the dynamically translated
EH> methods."
EH> This can be a big boost because you don't need to look up the method or
EH> even dispatch to it anymore, its right there where it needs to be.
No i did not miss it, it says that this is _NOT_ implemented in squeak,
only in the commercial VM's which i pointed out are still extrem
expensive for all of us who use it for some scripts in there
companies.
So i doubt that the Squeak Engine can do much better then YARV when
both use the same technologie (at least for message calling)
Lothar Scholz
CM> A generational GC would be great. There has been talk of including one
CM> in the next version of the Ruby interpreter. However C extensions also
CM> pose a problem here as well - it seems creating a write barrier that
CM> works with C libraries not written originally with Ruby in mind will be
CM> very difficult/slow.
This shouldn't be a problem. As long as you don't pass ruby values
into the c libraries as 'user data' the data space can be separated
very well. If you must pass some 'user data' into the lib simply use
a proxy pointer, an keep the ruby object into a gc known address space.
A hashtable with all ruby objects referenced from the c lib is a good
idea here.
A little bit more overhead and it breaks all larger current binary
extensions, but i think we must face the fact that Ruby 2.0 will give
as a lot of code breaks anyway.
Saynatkari
Le 8/4/2005, "Peter Suk" <peter.kwa...@mac.com> a écrit:
>On Apr 8, 2005, at 8:19 AM, George wrote:
>
>> Avi / Peter -- thanks for providing the feedback I was looking for.
>> Ruby and Smalltalk do indeed sound like a good match. I remember Robert
>> Feldt saying a few years ago that he'd done some experiments with Ruby
>> on Smallscript's VM that looked promising.
>>
>> Peter said:
>>> From now on, if you are completely ignorant of Smalltalk,
>>> I will cease answering these questions here.
>>
>> Yikes! I _am_ almost completely ignorant of Smalltalk (I'm know the
>> basic concepts of the language, but I've never programmed in it), but
>> I've done some work on VMs and was genuinely interested in the answers.
>
>Sorry, it was late at night, and that came off harsher than it should
>have. It's just that I'm finding myself essentially repeating myself.
>The feedback has informed my thinking and has been valuable, but I
>should just tell people to go and find and read docs. I will get around
>to posting them on the RubyForge site. This project is named Alumina.
>Ruby is just one crystalline form of Alumina.
Thank you for that apology! Please understand you are speaking to
a group of (mostly) interested _ruby_ developers, many of whom, I,
for instance, have little or no experience with Smalltalk let alone
its VM infrastructure. You announced the topic so I think it is fair
to expect you to answer any reasonable questions from your audience
considering our collective background.
The project certainly seems interesting to me; I am somewhat reserved
about it as I would rather see as much of the available talent to go
to YARV development rather than various different projects but, then
again, there seems to be a lot of talent to go around. Just produce
code and ideas that can be reused! :)
Good luck!
>--Peter
E
--
No-one expects the Solaris POSIX implementation!
Charles Mills
Lothar Scholz wrote:
> Hello Charles,
>
> CM> A generational GC would be great. There has been talk of
including one
> CM> in the next version of the Ruby interpreter. However C
extensions also
> CM> pose a problem here as well - it seems creating a write barrier
that
> CM> works with C libraries not written originally with Ruby in mind
will be
> CM> very difficult/slow.
>
> This shouldn't be a problem. As long as you don't pass ruby values
> into the c libraries as 'user data' the data space can be separated
> very well. If you must pass some 'user data' into the lib simply use
> a proxy pointer, an keep the ruby object into a gc known address
space.
> A hashtable with all ruby objects referenced from the c lib is a good
> idea here.
I am no expert on GC's, so maybe my questions are bogus, but how would
the GC 'know' which allocated regions to put behind the write barrier?
For example with the Array class how would the GC know to put
RARRAY(ary)->ptr behind the write barrier if ary made its way into an
older generation?
Would the memory allocation api (ALLOC_N,etc) have to be changed?
Could this be determined by when the allocation was made?
linus sellberg
> The project certainly seems interesting to me; I am somewhat reserved
> about it as I would rather see as much of the available talent to go
> to YARV development rather than various different projects but, then
> again, there seems to be a lot of talent to go around. Just produce
> code and ideas that can be reused! :)
I don't think it is harmful in any way. Consider web frameworks.
Everyone and their mothers seem to have written one for Ruby, yet still
there are no shortage of good ones (Rails etc).
Peter Suk
I am not a VM expert, but I have overheard the Smalltalk VM guys (4 of
them from different VMs in the same room!) talk about how write
barriers can be efficiently implemented by reserving special areas of
memory, choosing a certain region as your protected region, then using
pointer address comparisons.
Peter Suk
> Thank you for that apology! Please understand you are speaking to
> a group of (mostly) interested _ruby_ developers, many of whom, I,
> for instance, have little or no experience with Smalltalk let alone
> its VM infrastructure. You announced the topic so I think it is fair
> to expect you to answer any reasonable questions from your audience
> considering our collective background.
It's fair I answer them once. However, it is amusing that I'm getting
a lot of questions in the vein of "Does Smalltalk have X?" where the
answer is really: "Funny you should mention that, but arguably
Smalltalk was X's first widespread commercial implementation.
Smalltalk has had X for (over a decade | since the beginning in 1972.)"
> The project certainly seems interesting to me; I am somewhat reserved
> about it as I would rather see as much of the available talent to go
> to YARV development rather than various different projects but, then
> again, there seems to be a lot of talent to go around. Just produce
> code and ideas that can be reused! :)
Tell me that there will be a full-blown Object-image external to the
VM, and that all of the language-specific meta-stuff will be expressed
as first class Rite/Ruby Objects, to the point that *only* executing
bytecodes and allocating/GC-ing Objects is done by the VM, and I will
consider it! (This means that even the language itself exists only in
the image as 1st class Ruby/Rite Objects.)
Eric Hodel
> Hello Eric,
>
> EH> On 08 Apr 2005, at 03:39, Lothar Scholz wrote:
>
> EH> You missed the second to last sentence, "... the cached target and
> some
> EH> checking information is included inline with the dynamically
> translated
> EH> methods."
>
> EH> This can be a big boost because you don't need to look up the
> method or
> EH> even dispatch to it anymore, its right there where it needs to be.
>
> No i did not miss it, it says that this is _NOT_ implemented in squeak,
> only in the commercial VM's which i pointed out are still extrem
> expensive for all of us who use it for some scripts in there
> companies.
>
> So i doubt that the Squeak Engine can do much better then YARV when
> both use the same technologie (at least for message calling)
The above documentation is, in part, wrong.
ftp://st.cs.uiuc.edu/Smalltalk/Squeak/docs/OOPSLA.Squeak.html
See the last paragraph of Smalltalk to C Translation.
See also the last paragraph of Performance and Optimization.
itsme213
"Lothar Scholz" <mailin...@scriptolutions.com> wrote in message
> I would like to add
>
> - singleton methods
>
> to this list as i think this does not exist in SmallTalk, and this is
> a feature that could knock out the whole method compilation algorithm.
I doubt it will. Even in Ruby, singleton methods are implemented with
classes under the covers.
The fact that Ruby instance variables appear on demand, and so references to
instance-vars cannot be directly mapped to integer offsets from a known
class definition, will likely mean some performance trade-off.
Peter Suk
> On Apr 8, 2005, at 1:27 PM, Saynatkari wrote:
>
>> You announced the topic so I think it is fair
>> to expect you to answer any reasonable questions from your audience
>> considering our collective background.
>
> It's fair I answer them once. However, it is amusing that I'm getting
> a lot of questions in the vein of "Does Smalltalk have X?" where the
> answer is really: "Funny you should mention that, but arguably
> Smalltalk was X's first widespread commercial implementation.
> Smalltalk has had X for (over a decade | since the beginning in
> 1972.)"
Yikes, I walked back from the car when I realized...LISP! I don't want
to argue that Smalltalk was the first widespread commericial
implementation of X, because some Lisp-er will inevitably tell me that
it was done in 1970-something, and further come back to me with photos
from an archaeological dig where paleo-lispers were doing semi
Aspect-Oriented things with "around" methods and CDR was implemented by
bashing the end off a rock with a large teak club.
Lots of neat stuff was invented awhile back, and it's only just
*really* hitting the mainstream. I'm betting on Ruby to be that horse!
itsme213
"Peter Suk" <peter.kwa...@mac.com> wrote in message
> I thought I'd talk about my new project here
Sounds fantastic, I can't wait!
> And for those of you sharp enough to wonder: yes, we can
> handle Modules, Mixins, and fully qualified Method names without
> changing the Smalltalk VMs. (At least those that have Namespaces.)
Which ones do?
And what of Ruby 2.0 selector namesapces? Will they happen? And will that
map readily to a Smalltalk VM?
Glenn Parker
>
> VisualWorks can use real threads, but only in the case of calling out to
> a DLL, and these contexts are exempt from GC.
Well, that's a major hole, isn't it?
And I suspect today's Ruby has pretty much the same "feature" as
VisualWorks in this regard, that is to say I could create a Ruby DLL
that spawned real threads and let them do their own memory management.
Peter Suk
> "Lothar Scholz" <mailin...@scriptolutions.com> wrote in message
>
>> I would like to add
>>
>> - singleton methods
>>
>> to this list as i think this does not exist in SmallTalk, and this is
>> a feature that could knock out the whole method compilation algorithm.
>
> I doubt it will. Even in Ruby, singleton methods are implemented with
> classes under the covers.
You get the "no-prize"! Yes, this one is easy.
> The fact that Ruby instance variables appear on demand, and so
> references to
> instance-vars cannot be directly mapped to integer offsets from a known
> class definition, will likely mean some performance trade-off.
Already covered this.
Peter Suk
> "Peter Suk" <peter.kwa...@mac.com> wrote in message
>
>> I thought I'd talk about my new project here
>
> Sounds fantastic, I can't wait!
>
>> And for those of you sharp enough to wonder: yes, we can
>> handle Modules, Mixins, and fully qualified Method names without
>> changing the Smalltalk VMs. (At least those that have Namespaces.)
>
> Which ones do?
Avi pointed out that we don't really need them -- Smalltalk Namespaces
are really just data-structures to inform the Compiler, so we can have
Namespaces on top of any Smalltalk VM.
> And what of Ruby 2.0 selector namesapces? Will they happen? And will
> that
> map readily to a Smalltalk VM?
Selector namespaces are hard. We can do them with compiler tricks.
We'll just have to tackle that when it comes. (And who knows, maybe
Ruby2/Rite will make the Alumina (Ruby on a Smalltalk VM) superfluous?
I hope so. I hope Ruby2/Rite will be right and we'll have a meta-level
that is freely manipulable as 1st class Objects.
Peter Suk
> Peter Suk wrote:
>> VisualWorks can use real threads, but only in the case of calling out
>> to a DLL, and these contexts are exempt from GC.
>
> Well, that's a major hole, isn't it?
There is clearly some kind of disconnect here. Are you expecting that
the DLL be subject to Ruby GC & memory management? Are you saying it's
some kind of "hole" that the context which is doing the call-out is not
being GC'd for the duration of the call?
Huh?
> And I suspect today's Ruby has pretty much the same "feature" as
> VisualWorks in this regard, that is to say I could create a Ruby DLL
> that spawned real threads and let them do their own memory management.
Please re-read my message. Your response doesn't make any sense at
all, especially with the 2nd sentence -- unless I am missing out on
something.
And please, stop this stupid "our X is better than yours" nonsense.
Such pride is misplaced on a particular VM/Language/paradigm. I am
here to share what the Smalltalk world has been acruing for some time,
and am prepared to go through a lot of effort to do it.
Bill Kelly
>
> VisualWorks gets around this and also solves problems the paradox with
> not having "real" threads, wanting to do GC efficently, yet also
> wanting to do synchronous calls to the OS. The dilemma is this: You
> want real threads so your VM can utilize synchronous calls to the OS.
> However, this complicates GC immensely. (ObjectStudio Smalltalk lost a
> lot of performance because of having real threads. When it was time to
> GC, the main VM thread would have to raise a flag and wait for all the
> other threads to detect it.)
>
> VisualWorks can use real threads, but only in the case of calling out
> to a DLL, and these contexts are exempt from GC. This gives you the
> ability to make synchronous calls without freezing the entire image,
> yet also leaves your GC simple and fast.
Interesting. I'd been looking forward to the day
that Ruby supports "real" OS threads with great
anticipation. Sorry to hear there are such negative
performance trade-offs.
I'm unsure exactly what I'm trying to ask here...
But - did VisualWorks come up with some convenient
way to invoke a DLL call on a separate thread? From
the programmer's perspective, I mean. I'm imagining
something like making a DLL call and getting some
object back representing the thread, from which a
return value could eventually be obtained? ... Just
curious.
Most of my employer's Mac customers have dual processor
systems. Our app is currently C++ and sufficiently
multi-threaded to keep both CPUs busy. I've looked
forward to someday being able to write a multi-threaded
Ruby app that, using real OS threads, ran on both CPUs.
Anyway - not trying to sound negative... Just was
interested to know there are trade-offs...
Regards,
Bill
Peter Suk
On Apr 8, 2005, at 4:43 PM, Bill Kelly wrote:
> Interesting. I'd been looking forward to the day
> that Ruby supports "real" OS threads with great
> anticipation. Sorry to hear there are such negative
> performance trade-offs.
>
> I'm unsure exactly what I'm trying to ask here...
> But - did VisualWorks come up with some convenient
> way to invoke a DLL call on a separate thread?
Yes.
> From
> the programmer's perspective, I mean. I'm imagining
> something like making a DLL call and getting some
> object back representing the thread, from which a
> return value could eventually be obtained? ... Just
> curious.
There is something you have to specify when you hook up the DLL, but as
I understand it, from the caller's point of view, you just spawn your
own process and make a normal call.
> Most of my employer's Mac customers have dual processor
> systems. Our app is currently C++ and sufficiently
> multi-threaded to keep both CPUs busy. I've looked
> forward to someday being able to write a multi-threaded
> Ruby app that, using real OS threads, ran on both CPUs.
>
> Anyway - not trying to sound negative... Just was
> interested to know there are trade-offs...
Yes, the Java folks found this out the hard way. I would recommend
load-balancing. But it really depends on what your application is
doing and details of the architecture.
Glenn Parker
> On Apr 8, 2005, at 3:04 PM, Glenn Parker wrote:
>
>> Peter Suk wrote:
>>
>>> VisualWorks can use real threads, but only in the case of calling out
>>> to a DLL, and these contexts are exempt from GC.
>>
>> Well, that's a major hole, isn't it?
>
> There is clearly some kind of disconnect here. Are you expecting that
> the DLL be subject to Ruby GC & memory management? Are you saying it's
> some kind of "hole" that the context which is doing the call-out is not
> being GC'd for the duration of the call?
Perhaps there is a disconnect, but I didn't think I was that obtuse.
You said VW can use real threads, but only in the case of calling out to
a DLL, and no GC. To me, that strongly suggests that the code executing
in the real threads uses some traditional compiled language with manual
memory management, e.g. C. If that is the price for getting real
threads, it's not very attractive to me. If I have misunderstood, then
please explain further.
>> And I suspect today's Ruby has pretty much the same "feature" as
>> VisualWorks in this regard, that is to say I could create a Ruby DLL
>> that spawned real threads and let them do their own memory management.
>
> Please re-read my message. Your response doesn't make any sense at all,
> especially with the 2nd sentence -- unless I am missing out on something.
Well, maybe you're missing something, or maybe I need more help
understanding the VW technology. I've read all your messages and I
think I understand the general proposal, which is quite intriguing, or I
wouldn't waste my time on it.
> And please, stop this stupid "our X is better than yours" nonsense.
> Such pride is misplaced on a particular VM/Language/paradigm. I am here
> to share what the Smalltalk world has been acruing for some time, and am
> prepared to go through a lot of effort to do it.
Let's get one thing clear, I don't give a fig for language wars.
Smalltalk has obviously solved a lot of problems that Ruby still
struggles with, but it hasn't necessarily solved them in the way that
makes sense for Ruby. So, please pardon me if I "look a gift horse in
the mouth". Your willingness to invest your time and skills is
noteworthy, but it doesn't mean that a Smalltalk VM is the ultimate
answer for Ruby. Goodness knows, I'd love to see you deliver on the
promise of your proposal, because I think Ruby needs a radical
performance upgrade.
I don't know if I can convince you of this, but I'm trying to ask some
sincere and probing questions about your proposal. This kind of probing
is exactly what you invited by announcing your plans here. Why act so
surprised when you get a bunch of naive Smalltalk questions from a group
of Ruby folk, anyway?
Peter Suk
> Peter Suk wrote:
>> On Apr 8, 2005, at 3:04 PM, Glenn Parker wrote:
>>> Peter Suk wrote:
>>>
>>>> VisualWorks can use real threads, but only in the case of calling
>>>> out to a DLL, and these contexts are exempt from GC.
>>>
>>> Well, that's a major hole, isn't it?
>> There is clearly some kind of disconnect here. Are you expecting
>> that the DLL be subject to Ruby GC & memory management? Are you
>> saying it's some kind of "hole" that the context which is doing the
>> call-out is not being GC'd for the duration of the call?
>
> Perhaps there is a disconnect, but I didn't think I was that obtuse.
>
> You said VW can use real threads, but only in the case of calling out
> to a DLL, and no GC. To me, that strongly suggests that the code
> executing in the real threads uses some traditional compiled language
> with manual memory management, e.g. C. If that is the price for
> getting real threads, it's not very attractive to me. If I have
> misunderstood, then please explain further.
Why is it that you'd want real threads? What can real threads do that
Ruby VM threads can't do? The answer is synchronous calls out to OS
services. (As well as utilizing multiple processors, but that's
another issue.)
>>> And I suspect today's Ruby has pretty much the same "feature" as
>>> VisualWorks in this regard, that is to say I could create a Ruby DLL
>>> that spawned real threads and let them do their own memory
>>> management.
>> Please re-read my message. Your response doesn't make any sense at
>> all, especially with the 2nd sentence -- unless I am missing out on
>> something.
>
> Well, maybe you're missing something, or maybe I need more help
> understanding the VW technology. I've read all your messages and I
> think I understand the general proposal, which is quite intriguing, or
> I wouldn't waste my time on it.
In general, DLLs that you call out to are doing their own memory
management. At least I hope so, or they're going to be a source of
memory leaks.
>> And please, stop this stupid "our X is better than yours" nonsense.
>> Such pride is misplaced on a particular VM/Language/paradigm. I am
>> here to share what the Smalltalk world has been acruing for some
>> time, and am prepared to go through a lot of effort to do it.
>
> Let's get one thing clear, I don't give a fig for language wars.
> Smalltalk has obviously solved a lot of problems that Ruby still
> struggles with, but it hasn't necessarily solved them in the way that
> makes sense for Ruby. So, please pardon me if I "look a gift horse in
> the mouth". Your willingness to invest your time and skills is
> noteworthy, but it doesn't mean that a Smalltalk VM is the ultimate
> answer for Ruby. Goodness knows, I'd love to see you deliver on the
> promise of your proposal, because I think Ruby needs a radical
> performance upgrade.
>
> I don't know if I can convince you of this, but I'm trying to ask some
> sincere and probing questions about your proposal. This kind of
> probing is exactly what you invited by announcing your plans here.
> Why act so surprised when you get a bunch of naive Smalltalk questions
> from a group of Ruby folk, anyway?
I need to get used to the idea that not every implication of everything
I post here is going to be fully appreciated. Ruby has some great
attributes. But it is not the first to have these! If you look under
the covers, Smalltalk and Ruby are very alike. Startlingly alike.
Ruby makes some design decisions which sacrifice performance in
exchange for implementation & programmer convenience. But these
differences are really not that big a deal. At bottom, both
environments use the "everything is an object" principle. At bottom,
everything is just messages sent to objects. Everything is dynamic,
and there is no "compile-time vs. runtime" divide.
I appreciate the sincere and probing questions. I must admit to losing
patience with questions that are not well informed. Especially those
that are not "sincere." (i.e. really motivated by a "language wars"
mentality.) However in retrospect, you don't fall into this last
category.
Lothar Scholz
PS> Why is it that you'd want real threads? What can real threads do that
PS> Ruby VM threads can't do? The answer is synchronous calls out to OS
PS> services. (As well as utilizing multiple processors, but that's
PS> another issue.)
Having a database app where each "fetch_row" call into the database
driver have to do at least 2 sync operations and 2 thread schedules
really scares me when we are talking about high performance VM's.
In fact this is one the real weaknesses in the ruby implementation.
Does VM have a fork (or some other process like) model to use real
OS threads. TCL and Perl have this and it's part of there popularity.
Avi Bryant
Lothar Scholz wrote:
> Hello Peter,
>
> PS> Why is it that you'd want real threads? What can real threads do
that
> PS> Ruby VM threads can't do? The answer is synchronous calls out to
OS
> PS> services. (As well as utilizing multiple processors, but that's
> PS> another issue.)
>
> Having a database app where each "fetch_row" call into the database
> driver have to do at least 2 sync operations and 2 thread schedules
> really scares me when we are talking about high performance VM's.
You don't *have* to do external calls in a separate thread, it's just
that you can where it's useful (ie, blocking I/O calls like making
database queries). For something like fetch_row, you might just use
the normal, older callout mechanism, which does what Ruby does: blocks
the whole VM until you return.
Like Ruby, Smalltalk implementations generally use non-blocking I/O
(made to look like blocking I/O from the point of view of the
language's threads), so native threads are only needed when the DLL
you're calling into only provides a blocking API.
Anyway, unlike much of the VisualWorks VM, this part is actually quite
well documented:
http://www.cincomsmalltalk.com/CincomSmalltalkWiki/VisualWorks+THAPI
Avi
Peter Suk
> Glenn Parker wrote:
>
>> There are also issues with using more memory than necessary if the
>> interpreter creates every instance variable the moment it is observed
> by
>> the compiler.
>
> Only in truly pathological cases - for normal numbers of instance
> variables, the overhead of an external lookup table would be higher
> than that of keeping a few unused slots in the body of the object. But
> of course if the compiler saw that instances of a particular class
> might have up to 50 instance variables, it could choose to implement
> that class with a hashtable for variables. I'm not convinced that case
> is likely enough to be worth checking and optimizing for, but who
> knows.
Having your variables in a Hashtable is something I've heard of as a
means of giving Java objects more reflective & dynamic capabilities.
I've also done it as a quick and dirty way of storing Smalltalk objects
as serialized files, but still being able to read the old ones even in
after modifying their classes and adding instance variables. (Which is
not the right way to do it, but from an implementation standpoint is
slightly quicker than the right way.)
However, when you do this, you are taking a *big* performance hit. I
suspect that this is one of the reasons why the Ruby VMs and the
earlier Python VMs are so slow -- a trade-off has been made for
programmer/VM-implementor convenience. Alumina will maintain speed and
programmer convenience in exchange for VM-implementor inconvenience.
Nicholas Seckar
> Having your variables in a Hashtable is something I've heard of as a
> means of giving Java objects more reflective & dynamic capabilities.
It is possible for Ruby objects to have a dynamic set of attributes. How do
you propose to handle calling instance_variable_set with random names on
objects?
Would such an operation entail resizing every instance of that object's class?
Or would a hidden subclass be created, as can be done for singleton methods?
Another option is to use slots for the attributes we know about and provide
hash based fallback for those which surprise us.
Avi Bryant
All of these seem pretty reasonable; I expect you'd have to work really
hard to find any program in the wild that would cause a problem for any
of those approaches. (How many times does instance_variable_set get
used with a value that is *never* referenced directly as @ivar?)
It seems like these kind of details should probably be worked out in a
private mailing list; Peter, is there one set up for Alumina? Or would
you rather get further along before opening things up to kibitzing?
Avi
Glenn Parker
>
> Why is it that you'd want real threads? What can real threads do that
> Ruby VM threads can't do? The answer is synchronous calls out to OS
> services. (As well as utilizing multiple processors, but that's another
> issue.)
Yes, I want synchronous I/O and true multi-cpu execution. And if I
can't satisfy these needs without dropping into a very different
language, I'll be disappointed.
> In general, DLLs that you call out to are doing their own memory
> management. At least I hope so, or they're going to be a source of
> memory leaks.
Sure, but I am questioning the decision to rely exclusively on external
compiled DLLs to access real threads. I want to write as much code as
possible in the "primary" language, for all the obvious reasons.
> I need to get used to the idea that not every implication of everything
> I post here is going to be fully appreciated.
We can't read your mind. You might want to start compiling a FAQ.
> Ruby has some great
> attributes. But it is not the first to have these! If you look under
> the covers, Smalltalk and Ruby are very alike. Startlingly alike. Ruby
> makes some design decisions which sacrifice performance in exchange for
> implementation & programmer convenience. But these differences are
> really not that big a deal.
I appreciate the philosophical similarities, but I think you trivialize
the impact of implementation and (especially!) programmer convenience.
> At bottom, both environments use the
> "everything is an object" principle. At bottom, everything is just
> messages sent to objects. Everything is dynamic, and there is no
> "compile-time vs. runtime" divide.
I would be more interested in your list of things that are different,
since that is where the real work will have to be done.
itsme213
"Peter Suk" <peter.kwa...@mac.com> wrote in message
> > The fact that Ruby instance variables appear on demand, and so
> > references to
> > instance-vars cannot be directly mapped to integer offsets from a known
> > class definition, will likely mean some performance trade-off.
>
> Already covered this.
Replacing every existing object of a class with a new static layout has a
very different performance profile from updating the dynamic layout of a
single object. Silently creating a singleton class might result in subtle
changes in Ruby semantics. Going all-dynamic layout takes a big performance
hit.
Sounds like a performance trade-off area to me.
Peter Suk
If you want to influence the project, join it and contribute code.
Otherwise, it's pointless to discuss this now.
itsme213
"Avi Bryant" <avi.b...@gmail.com> wrote in message
> (How many times does instance_variable_set get
> used with a value that is *never* referenced directly as @ivar?)
I use it often to generate method definitions dynamically using closures
(without strings and eval).
class Class
def has_many coll, type=Object
coll = coll.to_s
iv = ('@' + coll).intern
getter = coll
setter = (coll + '=').intern
appender = ('append_to_' + coll).intern
define_method getter do
instance_variable_get iv
end
define_method setter do |c|
assert c.all? { |x| type === x }
instance_variable_set iv, c
end
define_method appender do |x|
assert type === x
(instance_variable_get iv) << x
end
end
end
class House
has_many :windows
end
This usage will increase in Ruby 1.9+ and 2.0 since closures can take &block
parameters, and closure param lists look just like method param lists.
Avi Bryant
Glenn Parker wrote:
> Peter Suk wrote:
> >
> > Why is it that you'd want real threads? What can real threads do
that
> > Ruby VM threads can't do? The answer is synchronous calls out to
OS
> > services. (As well as utilizing multiple processors, but that's
another
> > issue.)
>
> Yes, I want synchronous I/O and true multi-cpu execution.
And shared memory - right? Because with Ruby or Smalltalk, there's
nothing stopping you from using multiple CPUs by having multiple
instances of the VM, communicating through DRb or whatever.
The desire for native threads comes up quite a bit in the Smalltalk
world. It's interesting to see how the people at Cincom answer these
questions. Of the two Smalltalks they develop, ObjectStudio and
VisualWorks, one of them has native threads and the other has green
threads. The engineers that build their networking and database
systems always claim that the green threads scale much, much better -
porting such code from VisualWorks to ObjectStudio always involves a
lot of headaches in getting it to perform decently under the
disadvantage of native threads.
The best setup would probably be to have many green threads to each (of
multiple) native thread, but that's a huge engineering effort. I think
Matz made the right decision: keep things simple and under the direct
control of the VM.
> > At bottom, both environments use the
> > "everything is an object" principle. At bottom, everything is just
> > messages sent to objects. Everything is dynamic, and there is no
> > "compile-time vs. runtime" divide.
>
> I would be more interested in your list of things that are different,
> since that is where the real work will have to be done.
There really aren't many. Things that Ruby has that Smalltalk doesn't
(ie, would need to emulate somehow):
- Optional & variable argument methods
- Undeclared instance variables
Things any new implementation of Ruby will need to redo:
- Binary extension API
- Standard libraries
- Syntax
That's all I can think of offhand.
Avi