Python on parrot

6 views
Skip to first unread message

Kevin Tew

unread,
Apr 16, 2005, 12:37:50 AM4/16/05
to perl6-i...@perl.org
Sam,

Just wondering what the status is on python/parrot/pirate/pyrate.

These both look outdated.
http://www.intertwingly.net/stories/2004/10/05/pyrate.zip
http://pirate.versionhost.com/viewcvs.cgi/pirate/

Is there a up to date cvs repo?
Can we get this code checked into the parrot svn repo?

Kevin Tew

Sam Ruby

unread,
Apr 16, 2005, 9:05:12 PM4/16/05
to Kevin Tew, perl6-i...@perl.org
Kevin Tew wrote:
> Sam,
>
> Just wondering what the status is on python/parrot/pirate/pyrate.
>
> These both look outdated.
> http://www.intertwingly.net/stories/2004/10/05/pyrate.zip
> http://pirate.versionhost.com/viewcvs.cgi/pirate/

I haven't looked at it for a few months now. I do plan to revisit it
enough to get the Pie-thon tests completed by the time of OSCON (in August).

> Is there a up to date cvs repo?

http://pirate.tangentcode.com/

> Can we get this code checked into the parrot svn repo?

Unfortunately, no. Much of this code is copyright Michal Wallace.

The good news is that the "good stuff" is in the parrot repo already.
What is left - a simple translator - can and should, IMHO, be recoded
into Perl6 once enough of that is running.

- Sam Ruby

Sam Ruby

unread,
Apr 16, 2005, 9:50:32 PM4/16/05
to Kevin Tew, perl6-i...@perl.org
[I hope you don't mind me putting this back on the list - I would prefer
that everybody who is interested can follow along and/or participate]

Kevin Tew wrote:

> So why won't Michal Wallace sign over the copyright?

I talked to Michal briefly about this a while back. My impression was
that he wanted to sign over the copyright to the Python foundation.
Which makes a bit of sense - the goal of having everything run on a
single runtime does not necessarily imply that everything has to be
owned by a single organization and put into a single repository.

My own opinions is that Michal thinks too much. ;-)

My impression is that everybody here is reasonable, and if it made sense
for further development to be transferred to another organization, then
some reasonable arrangement would be made.

Also, I believe that much of the initial work is throwaway work anyway.
Build one to throw away, and all that.

> Cool I did notice all the python pmcs.
> By translator I assume you mean a interpreter/compiler to parrot byte
> code.

At the moment, it is to Python to IMC, but eventually going directly to
bytecode would be a good idea.

> Why would you do it in Perl6, why not self hosted in python?

Self hosted in Python is a good idea, once it can be bootstrapped.

Overview of the current process:

1) Leverage python's "compiler" class to convert source to AST
2) Pirate converts AST to IMC
3) Parrot converts IMC to bytecode

> I'm thinking of toying around with python and just want to leverage all
> the previous work.

Excellent. In the meantime, if you are interested in getting commit
access to the Pirate repository, I'm confident that that can be arranged.

My feeling, for what it is worth, the translator is known to be a
solvable problem. Determining the proper mapping of Python semantics to
Parrot is the research problem. The overwhelming majority of that work
is in getting the PMCs right. Not having to worry about the syntax of
python or the conversion to bytecodes allows one to focus on just that
aspect of the problem.

But, as with all open source projects, feel free to scratch your own itches.

> Looks like I'll start with a translator and some new test cases so they
> can be contributed copyright free.

There also are a fair amount of python test cases in the parrot
repository. parrot/languages/python/*/*.t and t/dynclass/python/*.t.

When I last looked, these were not complete. Undoubtably, there is
likely to be some minor regressions as I have not kept up with the
latest Parrot changes. If past history is any guide, this is not much
of a problem.

- Sam Ruby

Leopold Toetsch

unread,
Apr 17, 2005, 4:05:29 AM4/17/05
to Sam Ruby, perl6-i...@perl.org
Sam Ruby <ru...@intertwingly.net> wrote:

Hi Sam, long not hear ;)

> ... The overwhelming majority of that work


> is in getting the PMCs right.

I'm currently working on that. Some of the MMD functions are already
rewritten so that they can return new result PMCs. With the now working
MMD system I'v also changed Python (and Tcl) PMCs accordingly. E.g.
subtracting two PyInts gives a PyLong on overflow albeit the subtract is
inherited from the Integeer PMC.

See also:
Subject: [PROPOSAL] infix MMD operators
Subject: Again the infix ops

That's not yet finished and opcodes that create new detination PMCs
are still missing. I hope that it's done in a week or so.

> ... I have not kept up with the
> latest Parrot changes.

I'd appreciate if you can find the time to follow p6i. BTW we have
switched to SVN.

> - Sam Ruby

leo

Michal

unread,
May 15, 2005, 1:17:42 PM5/15/05
to Sam Ruby, Kevin Tew, perl6-i...@perl.org
On Sat, 16 Apr 2005, Sam Ruby wrote:

> [I hope you don't mind me putting this back on the list - I would prefer that
> everybody who is interested can follow along and/or participate]
>
> Kevin Tew wrote:
>> Sam Ruby wrote:

Hey guys,

I didn't see this until just now.

>>> Kevin Tew wrote:
>>>
>>>> Sam,
>>>>
>>>> Just wondering what the status is on python/parrot/pirate/pyrate.
>>>>
>>>> These both look outdated.
>>>> http://www.intertwingly.net/stories/2004/10/05/pyrate.zip
>>>> http://pirate.versionhost.com/viewcvs.cgi/pirate/
>>>
>>> I haven't looked at it for a few months now. I do plan to revisit it
>>> enough to get the Pie-thon tests completed by the time of OSCON (in
>>> August).

Sam, are you still interested in this?


>>>> Is there a up to date cvs repo?
>>>
>>> http://pirate.tangentcode.com/
>>>
>>>> Can we get this code checked into the parrot svn repo?
>>>
>>> Unfortunately, no. Much of this code is copyright Michal Wallace.
>>>
>>> The good news is that the "good stuff" is in the parrot repo already. What
>>> is left - a simple translator - can and should, IMHO, be recoded into
>>> Perl6 once enough of that is running.
>>>
>>> - Sam Ruby
>>>
>> So why won't Michal Wallace sign over the copyright?
>
> I talked to Michal briefly about this a while back. My impression was that
> he wanted to sign over the copyright to the Python foundation. Which makes a
> bit of sense -

Pirate is based on code created by Andrew Kuchling and the copyright
on that work already belongs to the Python foundation.

Mostly I think pirate is just a different project from parrot
and I don't want to hand over control of which patches to apply,
etc. So far, I've never rejected a patch or turned down anyone
for cvs access, but I'd like to reserve the right. (Also, Dan
and the parrot team had other ideas about how to implement python.)

> the goal of having everything run on a single runtime does
> not necessarily imply that everything has to be owned by a
> single organization and put into a single repository.

Yes. That was the other main idea. I was hoping that we could
leverage the objects that the python team had already created.

Do PMC's still have to be compiled inside the parrot source
tree? To me, the problem is that the PMC's are in the parrot
tree, not that the compiler isn't in there with it.


> My own opinions is that Michal thinks too much. ;-)

This is almost certainly true. :)


> My impression is that everybody here is reasonable, and if
> it made sense for further development to be transferred to
> another organization, then some reasonable arrangement
> would be made.

I hope this is true, too. :)


> Also, I believe that much of the initial work is throwaway
> work anyway. Build one to throw away, and all that.

I've never really bought that logic. :) It works, and (especially
after your last batch of refactorings) the code is pretty clean.
I would like to see the emitter code separated out into a builder
object though.

>> Cool I did notice all the python pmcs. By translator I
>> assume you mean a interpreter/compiler to parrot byte
>> code.
>
> At the moment, it is to Python to IMC, but eventually
> going directly to bytecode would be a good idea.

That's fine with me, as long as parrots imc compiler does the
work. :)


>> Why would you do it in Perl6, why not self hosted in python?
>
> Self hosted in Python is a good idea, once it can be bootstrapped.


I'm not sure what Kevin meant by self hosted but I've been
thinking about this lately, and what we can do today is
creating a back-end for the compiler that emits simplified
python code in a way that would give us things like
continuations (though with some loss of speed) without the
need to duplicate the python library.


> Overview of the current process:
>
> 1) Leverage python's "compiler" class to convert source to AST
> 2) Pirate converts AST to IMC
> 3) Parrot converts IMC to bytecode
>
>> I'm thinking of toying around with python and just want
>> to leverage all the previous work.
>
> Excellent. In the meantime, if you are interested in getting commit access
> to the Pirate repository, I'm confident that that can be arranged.

Sure. Kevin, if you're still interested let me know.


> My feeling, for what it is worth, the translator is known
> to be a solvable problem. Determining the proper mapping
> of Python semantics to Parrot is the research problem.
> The overwhelming majority of that work is in getting the
> PMCs right. Not having to worry about the syntax of
> python or the conversion to bytecodes allows one to focus
> on just that aspect of the problem.

You're absolutely right.

I finally had a chance to look into what you've done, both in the
compiler and on the PMC: I'm impressed.

To me the main concern is making the PMC's compatable (or at least
similar to) python's C API. This is why I originally suggested
wrapping the python library. I would like to be able to run things
like pygame, numeric, wxpython, and extensions written with pyrex.

I agree with your point about optimizing the objects for parrot. I
just still wish we had a generic catch-all wrapper for anything we
haven't hand-coded yet.

>> Looks like I'll start with a translator and some new test
>> cases so they can be contributed copyright free.

Kevin: did you actually do this? There would certainly be
a copyright on your contribution, it would just be owned by
the perl group instead of the python group. Is that your only
reason for wanting to redo all our work? If possible, I'd much
rather have you on the pirate team. Can we discuss this?


> There also are a fair amount of python test cases in the
> parrot repository. parrot/languages/python/*/*.t and
> t/dynclass/python/*.t.
>
> When I last looked, these were not complete. Undoubtably,
> there is likely to be some minor regressions as I have not
> kept up with the latest Parrot changes. If past history
> is any guide, this is not much of a problem.

I ran the tests off the latest parrot from subversion.
Only three tests are failing, and they all seem to involve
booleans:

set_bool() not implemented in class 'PyBoolean'


Everything else works great.


- Michal
http://withoutane.com/

Sam Ruby

unread,
May 15, 2005, 2:59:59 PM5/15/05
to Michal, Kevin Tew, perl6-i...@perl.org
Michal wrote:
> On Sat, 16 Apr 2005, Sam Ruby wrote:
>
>> [I hope you don't mind me putting this back on the list - I would
>> prefer that everybody who is interested can follow along and/or
>> participate]
>>
>> Kevin Tew wrote:
>>
>>> Sam Ruby wrote:
>
> Hey guys,
>
> I didn't see this until just now.
>
>>>> Kevin Tew wrote:
>>>>
>>>>> Sam,
>>>>>
>>>>> Just wondering what the status is on python/parrot/pirate/pyrate.
>>>>>
>>>>> These both look outdated.
>>>>> http://www.intertwingly.net/stories/2004/10/05/pyrate.zip
>>>>> http://pirate.versionhost.com/viewcvs.cgi/pirate/
>>>>
>>>> I haven't looked at it for a few months now. I do plan to revisit
>>>> it enough to get the Pie-thon tests completed by the time of OSCON
>>>> (in August).
>
> Sam, are you still interested in this?

I had better be:

http://conferences.oreillynet.com/cs/os2005/view/e_sess/6833

Parrot (including the build system) is a moving target. Long term, the
answer is no. Short term, the answer is yes.

That should be easy enough to fix.

> Everything else works great.

Cool!

- Sam Ruby

Michal Wallace

unread,
May 16, 2005, 10:49:55 AM5/16/05
to Kevin Tew, Sam Ruby, perl6-i...@perl.org
On Sun, 15 May 2005, Kevin Tew wrote:

> I've taken the code from
> http://www.intertwingly.net/stories/2004/10/05/pyrate.zip,
> which I assume to be Sam's code. It was basically stubs
> with a few AST nodes implemented when I started with it.

Er. Yes. I can't speak for Sam, but I interpreted his post
as a spike of how a pirate refactoring could go, not a call
to start a whole new incompatible project.

> And am working to generate a python to pir
> translator/compiler that uses Sam's python pmc's I've
> attached the source as it stands now. I'm just an
> aspiring parrot hacker. I'm sure that there are a lot of
> things I'm doing wrong or inefficiently. I'm looking for
> feedback and suggestions for improvement. Please comment.

Well, one thing that's innefficient is to write your own
compiler when there are already two mostly working examples
out there.

> I want to get my patches on top of Sam's initial work
> imported into the parrot SVN tree.
>
> I believe that having the python compiler code in parrots
> svn tree is imperative to build a python/parrot community.

I'm absolutely bewildered that you would take that route in order
to build a community. Between pie-thon and pirate there already
are two python compilers, and pie-thon already *is* in the parrot
tree... As far as I can tell, you didn't approach either existing
group and see if you could help. You just went off and did your
own thing. You're not building a community, you're creating yet
another faction.


> A python compiler in the parrot tree will lead to more
> widespread exposure to other parrot developers, increased
> peer review, testing by more people on a greater variety
> of platforms, etc.

This is a great goal, but I'm not sure it follows that
having it in the source tree would lead to more exposure.

To me, if you want exposure, a much better idea would be
to *talk* to people, either on mailing lists or blogs or
whatever. In other words, market the project. You need
things like documentation and a website. Where the code
lives is a fairly trivial concern. It sounds like you just
want parrot developers to stumble on to your code in the
tree and pitch in.

In my mind, the target audience for pirate is python
developers (like yourself). People who might have heard
about parrot, but don't necessarily know much about it
or how it works. But if you can tell all the nice things
we get:

- this buys us continuations without the genius-level
hacks required by stackless

- we'll be able to leverage CPAN and code written in
other languages...

- we'll have a native call interface

- (etc)

... then maybe python hackers will be interested. So they
should be able to download a python file and run it. The
python file should work like every other python file, and
leverage the distutils. It should have a setup.py install
script and if necessary, a windows installer and rpms. There
should be a link to download a binary version of parrot so
that the first step isn't to compile a virtual machine they
know nothing about - especially if they're developing on
windows without a compiler. To me, having the code burried
in the middle of the parrot tree is a huge barrier to entry
to new developers.

Of course if you're targeting parrot developers who may or may
not know python, that's another story... But from what I can
tell, the other compilers in the parrot svn tree just aren't
getting the kind of exposure you're talking about.

I don't know, maybe I'm just rambling, but I don't see how
putting the source in the parrot repository is either
necessary or sufficient for widespread exposure. :/


> Michal could you expound on your thoughts here, they sound interesting.

> I'm not sure what Kevin meant by self hosted but I've
> been thinking about this lately, and what we can do
> today is creating a back-end for the compiler that
> emits simplified python code in a way that would give us
> things like continuations (though with some loss of
> speed) without the need to duplicate the python library.

Sure. This is kind of off topic for the parrot list. I posted
a blog entry here:

http://sabren.com/index.php?p=62


> I meant a python to pir translator/compiler written in python.

Like this one: http://pirate.tangentcode.com/ ??

> I've looked at the PyPy project. They are doing cool
> stuff. I would eventually like to use their work to emit
> optimized pir for python, but they still have work to do.

So why don't you want to help them with that work? Right
now you're just reinventing the wheel. If you were writing a
backend for pypy, that would be one thing. That would
actually be useful. Better yet, not write up a little report
about what it would take to get pirate to work with pypy?
We could almost certainly leverage their type inference
engine. I'd love to see pirate become just another backend
for pypy. It seems to me that would do a lot more to build
community than the approach you're taking.


- Michal
http://withoutane.com/

Reply all
Reply to author
Forward
0 new messages