this morning it struck me that it would be nice to have Parrot not only run Perl 6 and similar byte code, but that any of the common interpreted languages be compiled to this same byte code.
Then no matter whether running a Perl 6, Python, Ruby, Tcl maybe even bash script the same interpreter library would be used. Then likely it would already be in memory speeding up start time.
And this would also ease cross language module/library inclusion. Imagine instantiating some Python class from Perl!
Apache would essentially have a mod_parrot. Maybe, if this can be tested very hard, we'd even have a Parrot kernel module for all Unices supporting that. Then exec() could perform compiled scripts right away, like machine code :-)
coralament / best Grötens / liebe Grüße / best regards / elkorajn salutojn
Daniel Pfeiffer
Availability of all the pieces your vision requires is at
least several years away, barring miracles. I don't par-
ticularly expect it to happen.
--
Cameron Laird <Cam...@Lairds.com>
Business: http://www.Phaseit.net
Personal: http://phaseit.net/claird/home.html
I think this was one of the early design goals of the people working
on Parrot. In fact, the name Parrot is taken from a April Fools hoax
merging of the worst parts of Perl and Python. But, having an
efficient common backend for several languages is a sound engineering
decision, so it is no longer a joke!
In fact I believe I read a "proof-of-concept" version of Python is
already running in Parrot.
.NET is already doing this for Windows. Not too popular because the
only way to do it is to buy Visual Studio.NET or download the massive
free .NET runtime. As Microsoft shoves .NET down more and more
throats, expect to see this at a computer near you.
There are some doubts about Parrot outside of the Perl community.
Larry Wall was adamant that Parrot would be a register machine, and
not a stack machine, but outside of the Perl community, people are
uncertain about Parrot's ability to do efficient register allocation.
Take this with a grain of salt, I avoid Perl whenever I can, and I get
all my Parrot news from Slashdot :-P
Particularly myself; in fact, it was named Parrot precisely to represent its
language-agnosticism.
The thought is also behind why I wrote the Python::Bytecode module, why I
learnt Ruby, and why I've been asking interesting questions about access to
the Ruby AST.
I don't think the reality is as far away as you imply.
--
The problem with big-fish-little-pond situations is that you
have to put up with all these fscking minnows everywhere.
-- Rich Lafferty
Simon certainly has worked with this far more than I.
As others have written, still others have tried, including Andrew
Kuchling for Python. Two of his comments discribing the problems in
doing so are
Andrew
da...@dalkscientific.com
> Then no matter whether running a Perl 6, Python, Ruby, Tcl maybe even
> bash script the same interpreter library would be used. Then likely it
> would already be in memory speeding up start time.
I for one have two thoughts about the continued success of Python. First
because I like Python and want to keep using it. Second is that the more
people that use Python, hopefully the less people that will use TCL
Take care,
Jay
Maybe it's not dead; it may have just ceased to be.
I had similar problems, since my next stop after the "Parrot tour",
where all this boring VM stuff was decided, happened to be New York
City, home of the extremely famous tic-tac-toe chicken.
(http://members.tripod.com/the-wire/wire0102/ny0102.html)
Since then, of course, the bird has been rescued, given a new home,
and is now much happier.
Whether we need to give a new home to our brand new Virtual Machine
performing bird is not clear, but I'm happily convinced that its soul
will not die off, and that people will make the most of its talents in
the most humane way possible.
--
"(Because if life's not shit, then you're not doing it right.)"
- Rebecca Luckraft
I'd think it quite plausible that if someone took the project on,
there could readily be a pretty functional Python compiler for Parrot
/before/ a Perl 6 one would be likely to emerge...
--
(reverse (concatenate 'string "ac.notelrac.teneerf@" "454aa"))
http://www3.sympatico.ca/cbbrowne/sap.html
Signs of a Klingon Programmer - 7. "Klingon function calls do not have
'parameters' - they have 'arguments' -- and they ALWAYS WIN THEM."
Only just this morning? I take it that parrot itself only came to your
attention recently, since that concept has been with the VM since it's
inception.
> Then no matter whether running a Perl 6, Python, Ruby, Tcl maybe even
> bash script the same interpreter library would be used. Then likely
> it would already be in memory speeding up start time.
The difficult task is compiling those languages into parrot bytecode.
This may not be horrible if those languages have their own bytecode,
which can then be translated into parrot bytecode. As an example, there
already exists a perl script which translates java .class files into
parrot bytecode.
For other languages, a compiler would actually be needed -- eg, for
bash.
> And this would also ease cross language module/library inclusion.
> Imagine instantiating some Python class from Perl!
Ya mean the same way that you can, using Jython, instantiate Python
classes from Java and vice-versa? :)
> Apache would essentially have a mod_parrot.
Well, obviously it would.
> Maybe, if this can be tested very hard, we'd even have a Parrot kernel
> module for all Unices supporting that. Then exec() could perform
> compiled scripts right away, like machine code :-)
Or, your whatever_2_parrot compiler insert an appropriate #! line at the
beginning, pointing to the parrot VM -- this is much simpler, as it
takes advantage of an existing mechanism.
--
my $n = 2; print +(split //, 'e,4c3H r ktulrnsJ2tPaeh'
."\n1oa! er")[map $n = ($n * 24 + 30) % 31, (42) x 26]
> Daniel Pfeiffer wrote:
> >
> > Hi,
> >
> > this morning it struck me that it would be nice to have Parrot not
> > only run Perl 6 and similar byte code, but that any of the common
> > interpreted languages be compiled to this same byte code.
Oh, yes. This is our answer to M$s lacking .NET technologies ;-)
> Only just this morning? I take it that parrot itself only came to your
> attention recently, since that concept has been with the VM since it's
> inception.
Sincerely yours, Roman A.Suzi
--
- Petrozavodsk - Karelia - Russia - mailto:r...@onego.ru -
Just my £0.02
--
FM
How cool would it be to just set up a computer with a listening parrotd
and DHCP client, plug it in to the Beowulf and have it automagically
recognised as a node and delegated tasks?
I have been thinking quite a bit about cross-platform Beowulfing
recently. My initial idea was to set up an rlogin or something on the
node machines, and have the Apple (that's what I call the conducting
machine: "Apple" because it sends Jobs away, does nothing for a while
then gets Jobs back :-) distribute the source code for the node service
and job processor, configure and build the code, then log out and use
the new node as normal. This would be faster (provided the building
didn't fall over) as Beowulfs go, but less secure and probably harder to
set up.
Comments [rational variety preferred]?
--
FM
Security is a *hard* problem.
Curses! You stole my idea :-) I see what you mean about security being
a problem...
--
FM
What advantage would this have over putting a #! line in the bytecode?
Most existing *nix kernels (except some very old ones) will look at the
#!, parse out the path to an executable program, and the run it.
> I may be wrong. This parrotd system would also help in projects like
> Beowulfs (although why you'd be using byte-compiled languages in
> parallel computing is beyond me), because you could use inetd or
> similar to spawn a parrot when data was received on a certain port.
I don't really see the point of having a parrotd, except if you want to
create something vaguely like PersistantPerl.
--
FM
Then why not just have a libparrot? Let the OS's caching sort it out --
if it's being used, libparrot will stay in memory. Assuming you aren't
going to replicate OS-level functionality, like processes and memory
protection.
Ian
But not quite more secure. Imagine userX running a script that does
something nasty - like replace some command and then wait for root to
call it. This was a very simple example, since probably parrot would
recreate interpreters per script, but I guess it still could be possible
since it would be the same process...
I guess it's a lot safer to just fork as it is now.
--
WK (written at Stardate 56849.3)
"Data typing is an illusion. Everything is a sequence of bytes."
-Todd Coram
It can't. Parrot can only understand parrot bytecode.
> Why not leave it running, so that any Perl/Python/Tcl/Ruby/Java/sh/
> BASIC/whatever code can get to it immediately?
Ignoring the problem of compiling those languages down to parrot
bytecode...
How should the kernel detect that a parrot-bytecode file is, in fact,
parrot bytecode?
For normal exe files, there's the execute bit and some standard headers.
For normal scripts, there's the execute bit and the #! line.
> A parrotd exec would consist of: "Run this." "OK".
> A #! consists of "Does this file exist?" "Yes." "Does it
> work?" "Yes." "OK, run this." "OK".
> I contend that a properly set up parrotd *should* be faster than the
> equivalent "each language set up separately" environment.
The speed hit of exec() is *not* the time it takes to check if the file
named on the #! line exists and has the executable bit set.... the speed
it is due to the time it takes to load that program into memory.
With that in mind, if you can special-case the kernel to pass scripts on
to special daemons, rather than loading up an interpreter for each
script, it will almost surely be faster, and not just for perl/parrot.
Some years back, RMS proposed a standard 'scripting' language called
Guile. See, various languages like Tcl, Perl, Python, etc. were
being developed and none of them were 1) developed under the FSF nor
2) a variation of Scheme. (A simplified and perhaps biased
interpretation of mine.)
A key part of his proposal was that Guile would support translators
from other languages, like Python, C, etc., to Guile.
It didn't happen. Writing those translators are hard because each
of the languages has different object models, which must be implemented
nearly perfectly. Guile is a full-fledge language built on years of
research into Lisp and scheme, so I would be surprised if it was any
easier to implement in Parrot's byte code. It didn't happen with
Guile, it's even less likely to happen with Parrot.
I can "imagine if parrot ..." But it isn't going to happen without
*huge* amounts of work; comparable to writing Jython. (A Python
implementation written on top of Java rather than C.)
On the other hand, consider the gcc suite of tools. These convert C,
C++, Pascal, Fortran, Objective C, ... into a more generic form. So
there is some known success in doing this. But I've looked at gcc
code and it isn't something I want to hack, while I've contributed code
to Python's C implementation.
Andrew
da...@dalkescientific.com
> Frodo Morris wrote:
> >
> > Benjamin Goldberg wrote:
> > > Frodo Morris wrote:
[...]
>
> > Why not leave it running, so that any Perl/Python/Tcl/Ruby/Java/sh/
> > BASIC/whatever code can get to it immediately?
>
> Ignoring the problem of compiling those languages down to parrot
> bytecode...
>
> How should the kernel detect that a parrot-bytecode file is, in fact,
> parrot bytecode?
>
> For normal exe files, there's the execute bit and some standard headers.
> For normal scripts, there's the execute bit and the #! line.
At least on *BSD, /sys/kern/imgact_*.c control which executable types
are allowed. There are imgact_aout.c, imgact_elf.c, imgact_gzip.c and
imgact_shell.c, and maybe one could add an imgact_parrot.c?
Regards,
Slaven
--
Slaven Rezic - slaven...@berlin.de
Berlin Perl Mongers - http://berliner.pm.org
Isn't it a commonplace that the n x m translation problem is best solved
by having a common intermediate? This seems more efficient when
m+n < n*m
it seems that only those with sufficient muscle can force this on the
world eg Sun/IBM/M$. They can clearly see the need to support only one
backend.
Funnily enough they kept telling me that as soon as I switched to
F99/C++ etc etc that everything would be portable. Now the buzz is
C#/Mono or Java, and perhaps Parrot. As long as the wheels keep turning
and the $'s flow they'll keep promising the next sliced bread.
--
Robin Becker
Yep, it is a commonplace, but that doesn't make it true:)
You see, for the common constructs this might be true, but each
language has it weird corners and for those you'll end up with
language pair specific translations inside the intermediate language,
i.e. if there *is* such a translation. There will always be some left
overs that can't be (easily) translated. e.g. suppose one of the
languages support continuations, but the others don't, bad luck.
Ofcourse for parrot this isn't a show stopper per see, as parrot
doesn't have to translate into each of the supported languages, it
just has to understand them. But still, parrot will have to support
all the dark corners of all supported languages, quite a mess me thinks.
--
groetjes, carel
> Some years back, RMS proposed a standard 'scripting' language called
> Guile. See, various languages like Tcl, Perl, Python, etc. were
> being developed and none of them were 1) developed under the FSF nor
> 2) a variation of Scheme. (A simplified and perhaps biased
> interpretation of mine.)
I've never seen any even perceived advantages of guile other than the
FSF thing. I don't get the impression that it is that great even as a
scheme implementation.
Cheers,
M.
--
We've had a lot of problems going from glibc 2.0 to glibc 2.1.
People claim binary compatibility. Except for functions they
don't like. -- Peter Van Eynde, comp.lang.lisp
...each. ;-)
Well, I suppose Sun and IBM are sort of sharing.
Paul
P.S. Projects like Parrot only really tend to succeed if they either
produce results quickly enough for casually interested parties or have
enough direction and motivation for the prolonged period of time that
they aren't producing highly usable output. Despite hearing about
Parrot occasionally, I'm not so sure that Parrot exhibits either of
these properties, but it would be nice to be proved wrong.
I would have to think for a bit to come up with a proof, but
it seems fairly likely to me that that would be impossible.
At the very least, for it to be possible, you would have to
program indefinite-length pointers and allow all available disk
space to be used as extended memory.
The problem that people keep forgetting is that Turing Equivilence
requires indefinite memory. Turing Machines are defined as having
infinite "tapes". Given any finite memory size, I can come up with
an interpreter the emulation of which would not fit in that finite
memory size.
--
Would you buy a used bit from this man??
Your confusion is in thinking that "portable" is a binary value. Fortran
was certainly more portable than assembler, as was C. Tcl is certainly
more portable than Fortran or C.
The other problem, of course, is that people keep improving their
capabilities, so what was portable is no longer adequate. Fortran is
quite portable, as long as you don't want to do 3D graphics driven by a
data glove.
As long as you don't want to do development in multiple languages or
dynamically load code safely over the net into some other OS process,
Tcl and Python are pretty good choices. Otherwise, you might want to
consider C# or Java.
See how it works? :-)
English was portable until everyone started using it ;)
--
Robin Becker
you can "dynamically load code safely over the net into some other OS process"
with python, too. Where do you see the problem?
with the other point (developing in multiple languages) i fully agree.
Although C/C++ and Java seem to be pretty well integratable with python.
I wonder if it would be possible to modify/extend the python-VM to allow for
"efficient" execution of C#. We would be set, then, wouldn't we?
regards,
holger
--
FM
There's also the problem of motivation: who's going to write those
translators?
Tcl and Python programmers? They have nicely usable
implementations already. Translation to Guile would buy them
nothing, except for having to rewrite every single C extension.
Guile programmers? If they're Guile programmers, they must like
programming in Scheme. Why would they work on translators for languages
they'll never use?
A Guile programmer might write a Python translator out of pure
bloody-minded evangelism, of course, but it would require supernatural
determination to continue *maintaining* it for the long-term, and
there would still be the risk that the outside community would look at
the translator and ignore it, saying (it's too slow | it doesn't run
extension X that I really need | Guile doesn't run on my platform).
Similar arguments can applied to translators to Parrot, of course.
--amk (www.amk.ca)
ABBESS: In food, in sport, and life-preserving rest
To be disturbed would mad man or beast.
-- _The Comedy of Errors_, V, i
I am not sure i understand this. Why does python need to enter
restricted mode?
And doesn't PSYCO also do JIT-alike compiling? That's at least how
i understood Armin. As he also suggests to rewrite the python-core
in python to make it faster (with psyco) maybe this could help
with an c#-interpreter?
wildly-guessing'ly yours,
holger
I don't think C# allows this, but am no expert. The implication is that
C# can be very much more agressive in its JIT compilation. When it sees
x*y for x and y known ints it can compile an integer multiply. Python
only sees x*y and without the type information it is forced to do type
lookup at run time.
As you point out psyco can do the compilation, but only into the generic
multiply (unless psyco can be hinted in some way).
The pyrex project goes further and allows C style typing which makes it
very fast, but it isn't JIT.
--
Robin Becker
XML, but it wasn't a programming language. And they all kicked to
death because XML parsing is now considered a MUST for any "sane"
language.
--
Giuseppe "Oblomov" Bilotta
"Da grande lotterò per la pace"
"A me me la compra il mio babbo"
(Altan)
("When I grow up, I will fight for peace"
"I'll have my daddy buy it for me")
Better put: python allows names to be dynamically rebound to
(reference to) object of any type regardless of previous binding.
> I don't think C# allows this, but am no expert. The implication is
that
> C# can be very much more agressive in its JIT compilation. When it
sees
> x*y for x and y known ints it can compile an integer multiply.
Python
> only sees x*y and without the type information it is forced to do
type
> lookup at run time.
>
> As you point out psyco can do the compilation, but only into the
generic
> multiply (unless psyco can be hinted in some way).
I believe that Psycho *does* compile to specific int multiply. Hence
up to 10x speedup. That's what Armin means by 'specializing'
compiler. The same 'x*y' could also be compiled to float multiply if
that also happens on another entry to same code. Correct version
would be selected as needed. Process has setup time and space
overhead, which is why programmer must specify (inner) functions to
psyco-ize.
Terry J. Reedy
There are some restrictions especially with new-style classes.
> I don't think C# allows this, but am no expert. The implication is that
> C# can be very much more agressive in its JIT compilation.
PSYCO is - in effect - extremly aggressive about optimization. It works
with the actual runtime types and defers compilation until type
information is available (when the execution path hits it). It allows
for multiple optimized code sets for one place. Depending on the
actually arriving types one code-fragment of the code set is choosen.
PSYCO emits machine code.
You might like to take a look at
if you didn't already.
> When it sees
> x*y for x and y known ints it can compile an integer multiply. Python
> only sees x*y and without the type information it is forced to do type
> lookup at run time.
Psyco will optimize this in common cases to a single machine
code instruction. Furthermore it might completly sum up compuations
with constants. But Psyco doesn't have any specializing optimizer.
This is all a result of how PSYCO works.
The more of the interpreter is written in python the more it
can optimize. That's because it can't really do much
about compiled machine code (C-extensions, C-core).
OM'ly
holger
Parrot will still have an NxM problem with datatypes. Parrot supports
datatypes through PMCs, which are basically arrays of C functions
equivalent to PyObject, so you'd have a PerlInt, PerlString,
PythonInt, PythonString, etc. Different PMCs would have different
results for various operations, so PerlInt 1 + PerlString "2" == 3,
while PythonInt 1 + PythonString "2" raises TypeError.
Bridging data types across languages allows the NxM specter to
reappear. What does PerlString "2" + PythonInt 1 do? Who decides?
When someone writes RubyInt and RubyString (assuming Ruby's strings
aren't identical to Perl's), do the PythonString and PerlString
maintainers need to add support for them?
Perl's strings are mutable and Python's aren't; do we need to copy all
the string parameters whenever Python code calls Perl code? Or will
it work as long as the Perl code doesn't trying modifying a string?
So much for seamless cross-language calls, then.
Even for the more limited world of Python extensions, it's hard to
ensure that data types interoperate as much as possible. For the
wider, weirder world of "all datatypes for all scripting languages",
arranging sensible conversions will be even more difficult.
--amk (www.amk.ca)
PROSPERO: A turn or two I'll walk,
To still my beating mind.
-- _The Tempest_, IV, i
>In message <2002110618...@prim.han.de>, holger krekel
><py...@devel.trillke.net> writes
>.....
>>I am not sure i understand this. Why does python need to enter
>>restricted mode?
>>
>>And doesn't PSYCO also do JIT-alike compiling? That's at least how
>>i understood Armin. As he also suggests to rewrite the python-core
>>in python to make it faster (with psyco) maybe this could help
>>with an c#-interpreter?
>>
>>wildly-guessing'ly yours,
>>
>> holger
>>
>I think that python allows any reference to change type dynamically.
>
>I don't think C# allows this, but am no expert. The implication is that
>C# can be very much more agressive in its JIT compilation. When it sees
>x*y for x and y known ints it can compile an integer multiply. Python
>only sees x*y and without the type information it is forced to do type
>lookup at run time.
>
>As you point out psyco can do the compilation, but only into the generic
>multiply (unless psyco can be hinted in some way).
>
I would think you could generate machine code speculatively as cached segments
based on the assumption that types for variables involved in the segment
would not change after the first binding, and then dynamically check a segment cache
validity flag and do what it currently does as fallback if the flag was set. The flag
would of course have to be set by code that does type-changing rebindings for the
particular set of variables used in a segment, so there is some overhead, but maybe
not high percentagewise. I imagine you'd have to play with tuning the segment
lengths and variable set sizes etc., but if the speculative assumption were right
sufficiently often, especially in hot loops with function calls to builtins which
would have type-signature-specific machine language variants available for common
situations, you'd get a speedup with no change in semantics. Of course, this implies
a pretty different VM implementation and support library, and multiple dynamically
coexisting representations of code at a different granularity level.
But once programmers knew that this was going on, they would watch how they
bound things to take advantage. And the nice thing is the language definition would
not change.
I guess I would want to get some statistics on dynamic rebinding type changes before
starting such a big project though. Has anyone done that already?
>The pyrex project goes further and allows C style typing which makes it
>very fast, but it isn't JIT.
Regards,
Bengt Richter
I don't think this is really correct -- I think it was mostly because
Tcl was a lame language, and was being pushed as an application
scripting language. Of course, Lisp is close to RMS's heart, which is
why the FSF proposed Scheme as an alternative over other languages.
Simplicity was always essential for an application scripting language,
which is probably why Perl wasn't a serious contender. It was also a
long time ago, when Python was young. And the translation dream.
> A key part of his proposal was that Guile would support translators
> from other languages, like Python, C, etc., to Guile.
>
> It didn't happen. Writing those translators are hard because each
> of the languages has different object models, which must be implemented
> nearly perfectly. Guile is a full-fledge language built on years of
> research into Lisp and scheme, so I would be surprised if it was any
> easier to implement in Parrot's byte code. It didn't happen with
> Guile, it's even less likely to happen with Parrot.
Part of the problem with Guile was that there wasn't any developer drive
-- RMS and some others had a vision of translators, but they weren't
involved in developing Guile -- it was mostly Scheme-o-philes, who
unsurprisingly weren't very interested in translators. The only truly
serious effort was to translate elisp, and even that wasn't really done
in a community manner.
Guile's implementation was also taken from another Scheme
implementation, which was not originally developed with any
consideration to translators.
But you're right, it is a really hard problem anyway. And there's lots
of details.
You have to deal with competing naming conventions -- Scheme allows
almost all characters in its functions, for instance. Lots of languages
have different namespaces for functions and variables. Something like
Smalltalk has an entirely different notion of method naming, even though
its semantics are close to these other languages. It seems small, but
it's a pain.
There's also issues with errors -- if you want a usable language, a
traceback like Python's should be a bare minimum in this day. But what
does that look like with mixed languages? What does that even look like
with one language, ontop of a VM that's meant to be generic? This is
often dealt with very poorly as VM developers mostly test with trivial
and correct programs.
And of course there's a huge number of problems with objects. Mutable
vs. immutable strings being a good example -- it's not just a n x m
problems, it's a straight-up no-good-solution problem. There's no
*right* way to deal with this. Each language means something different
by "string", and yet it's so important to each language that if you
expect them to interoperate *at all* you need to deal with that. Do you
annotate functions and methods so that some automatic interoperation
layer can be used? Do you force users to use awkward constructions,
exposing the fact that they're accessing another language? Do you
create wrappers? Do you just punt, and assume some simplistic behavior
will be good enough, even though it's not predictable to the programmer?
C#/CLR I believe has resolved these issues by forcing the languages to
adapt to a common set of semantics. I just can't imagine a common set
of semantics that would bring Perl and Python together -- Python and
Ruby, sure, but there's not enough common ground with Perl to do that.
> I can "imagine if parrot ..." But it isn't going to happen without
> *huge* amounts of work; comparable to writing Jython. (A Python
> implementation written on top of Java rather than C.)
>
> On the other hand, consider the gcc suite of tools. These convert C,
> C++, Pascal, Fortran, Objective C, ... into a more generic form. So
> there is some known success in doing this. But I've looked at gcc
> code and it isn't something I want to hack, while I've contributed code
> to Python's C implementation.
I don't know if GCC has a lot to offer in way of advise or experience.
It can compile many languages into various instructions, but it doesn't
do anything for interoperability AFAIK (except for deliberately similar
languages, like C and C++, which were built for interoperability).
Ian
(snip some)
> that's what I call the conducting
> machine: "Apple" because it sends Jobs away, does nothing for a while
> then gets Jobs back :-)
lol :-))))
(snip some more)
Laotseu
But you were around for this, weren't you, Darren?
--
Cameron Laird <Cam...@Lairds.com>
Business: http://www.Phaseit.net
Personal: http://phaseit.net/claird/home.html
Indeed. I was one of the first users, I suspect, compiling my own
safe-tcl interpreter. I have no idea what I was thinking when I included
Tcl in that list. Some double-negation turning into a single-negation
twixt brain and fingers, I think.