Cross-compiling Parrot

6 views
Skip to first unread message

Aldo Calpini

unread,
Feb 20, 2007, 6:55:08 AM2/20/07
to perl6-i...@perl.org
back in 2004, Dan Sugalski wrote:

http://www.nntp.perl.org/group/perl.perl6.internals/2004/09/msg25521.html

nowadays my effort of porting Parrot to the PocketPC platform, as you
may have suspected, force me to reopen the question.

there is, I know, a lot of work to be done. and this will surely require
a lot of time. and, very probably, much more knowledge than I currently
have. but I'm willing to learn, and to live long enough to complete the
task.

I intend to start submitting patches that do not break things that
already work. for example, everything that is pulled out from Perl's
%Config will need to be pulled out from somewhere else if, and _only
if_, we are currently cross-compiling. (well, this is the plan. I can't
exclude that something somewhere may break for some obscure reason, so
please bear with me in advance :-).

before I start dirting my hands with code, I'm here to ask you:

1) does anybody have objections to patching the current build system for
cross-compilation (even "yes, but not now because..." objections)?

2) does anybody already have a .plan or something in mind about it (so
that I may either learn from what others have thought, or avoiding
reinventing some wheel)?

should I be warnocked, I will silently assume "no" and "no" as answers
to the preceeding points :-)

cheers,
Aldo

Jerry Gay

unread,
Feb 20, 2007, 9:44:53 AM2/20/07
to Aldo Calpini, perl6-i...@perl.org
On 2/20/07, Aldo Calpini <da...@perl.it> wrote:
> back in 2004, Dan Sugalski wrote:
>
> http://www.nntp.perl.org/group/perl.perl6.internals/2004/09/msg25521.html
>
> nowadays my effort of porting Parrot to the PocketPC platform, as you
> may have suspected, force me to reopen the question.
>
> there is, I know, a lot of work to be done. and this will surely require
> a lot of time. and, very probably, much more knowledge than I currently
> have. but I'm willing to learn, and to live long enough to complete the
> task.
>
> I intend to start submitting patches that do not break things that
> already work. for example, everything that is pulled out from Perl's
> %Config will need to be pulled out from somewhere else if, and _only
> if_, we are currently cross-compiling. (well, this is the plan. I can't
> exclude that something somewhere may break for some obscure reason, so
> please bear with me in advance :-).
>
> before I start dirting my hands with code, I'm here to ask you:
>
> 1) does anybody have objections to patching the current build system for
> cross-compilation (even "yes, but not now because..." objections)?
>
no objection here! this is a long-desired feature, and is currently
unavailable. although i don't have a pocketpc to test on, i'll do my
best to help. for years, the dependence on perl 5's configure probes
has bothered me--just not enough to fix it. i'm glad to see this
porting effort will renew that effort.

> 2) does anybody already have a .plan or something in mind about it (so
> that I may either learn from what others have thought, or avoiding
> reinventing some wheel)?
>

parrot's configure system is *large*. and it's *working* (at least,
for the core platforms, most of the time.) that said, it's still
buggy, and we get reports of configure failures here and there. so,
changes to configure will affect everybody, and must be well tested.

therefore, i suggest a test-driven development style for configure
changes. first, we need higher test coverage on configure system. i'm
willing to organize a day or two to direct our efforts at testing the
configure system in order to get the work done quickly so your efforts
are not held up too long. once we have high-enough coverage, we can
begin refactoring with considerably more peace of mind than in the
current situation.

also, we need to smoke configure tests on multiple platforms. i
suggest initially you make your changes in a 'porting' branch, and we
merge the branch to trunk and smoke it in planned intervals. that way,
you get to make small, frequent commits to your branch, and rolling
back trunk changes is a simple command as well. also, we won't have to
modify our smokers to sometimes build a branch, they can keep running
against trunk.

since this style hasn't been practiced as a rule in the past, it'll
take some time to get used to--for all of us. but i believe it's the
safest way to proceed to modify the critical-yet-undertested subsystem
that is configure. are you willing to work in the way i described? how
does the project team feel about this approach?

speaking briefly about a technical plan--there have been discussions
of a way forward in the past. the general consensus is that miniparrot
be designed to be ANSI C-only, with as few platform-specific
extensions as possible, and zero dependence on perl's configure. after
miniparrot is built, it can be used to perform the remainder of the
build tasks in a portable manner. this gives us a lot of code reuse,
and should make future porting efforts easier. however, there's a lot
of this design left to flesh out, and none of it is implemented. i
think this is the right design, but we need to find one or more ways
to get there, and work towards that. however that discussion merits
one or more separate message threads.
~jerry

Joshua Isom

unread,
Feb 20, 2007, 1:57:15 PM2/20/07
to perl6-internals List, jri...@gmail.com

On Feb 20, 2007, at 8:44 AM, jerry gay wrote:

> On 2/20/07, Aldo Calpini <da...@perl.it> wrote:
> no objection here! this is a long-desired feature, and is currently
> unavailable. although i don't have a pocketpc to test on, i'll do my
> best to help. for years, the dependence on perl 5's configure probes
> has bothered me--just not enough to fix it. i'm glad to see this
> porting effort will renew that effort.

Using perl 5's configure probes also somewhat limits us to how the
vendor distributed perl. It should also be considered that we can't
rely on perl5 being available, especially since we intend to replace it
eventually, so rewriting all the perl to support cross compiling would
likely not be the best thing.

>> 2) does anybody already have a .plan or something in mind about it (so
>> that I may either learn from what others have thought, or avoiding
>> reinventing some wheel)?
>>
> parrot's configure system is *large*. and it's *working* (at least,
> for the core platforms, most of the time.) that said, it's still
> buggy, and we get reports of configure failures here and there. so,
> changes to configure will affect everybody, and must be well tested.
>
> therefore, i suggest a test-driven development style for configure
> changes. first, we need higher test coverage on configure system. i'm
> willing to organize a day or two to direct our efforts at testing the
> configure system in order to get the work done quickly so your efforts
> are not held up too long. once we have high-enough coverage, we can
> begin refactoring with considerably more peace of mind than in the
> current situation.

If we create a new configure system, we can essentially have two
configuration methods in the trunk, and have tests for the new one to
be sure it's working and portable before trying to get everyone to
start using it.

> also, we need to smoke configure tests on multiple platforms. i
> suggest initially you make your changes in a 'porting' branch, and we
> merge the branch to trunk and smoke it in planned intervals. that way,
> you get to make small, frequent commits to your branch, and rolling
> back trunk changes is a simple command as well. also, we won't have to
> modify our smokers to sometimes build a branch, they can keep running
> against trunk.
>
> since this style hasn't been practiced as a rule in the past, it'll
> take some time to get used to--for all of us. but i believe it's the
> safest way to proceed to modify the critical-yet-undertested subsystem
> that is configure. are you willing to work in the way i described? how
> does the project team feel about this approach?
>
> speaking briefly about a technical plan--there have been discussions
> of a way forward in the past. the general consensus is that miniparrot
> be designed to be ANSI C-only, with as few platform-specific
> extensions as possible, and zero dependence on perl's configure. after
> miniparrot is built, it can be used to perform the remainder of the
> build tasks in a portable manner. this gives us a lot of code reuse,
> and should make future porting efforts easier. however, there's a lot
> of this design left to flesh out, and none of it is implemented. i
> think this is the right design, but we need to find one or more ways
> to get there, and work towards that. however that discussion merits
> one or more separate message threads.
> ~jerry

I've noticed a lot that mentions miniparrot being used for building
parrot, but at the same time, we don't have anything to work with to
know just how limited miniparrot should be. Perhaps a step to work for
before rebuilding the configure subsystem(which theoretically should be
in pir to get rid of perl5 dependence), is to get an actual miniparrot
working? Currently miniparrot just seems to be a macroparrot with a
miniconfig. If we can't realistically get a miniparrot, maybe it
should be considered if building a miniparrot for configuration is a
good idea. Instead of rewriting everything now in perl5 to support
cross compiling, maybe we should dive in and try to see if we can get
it rewritten in pir. After all, we're getting more pir programmers
than perl5 programmers it seems.

Chromatic

unread,
Feb 20, 2007, 2:01:25 PM2/20/07
to perl6-i...@perl.org, Aldo Calpini
On Tuesday 20 February 2007 03:55, Aldo Calpini wrote:

> 2) does anybody already have a .plan or something in mind about it (so
> that I may either learn from what others have thought, or avoiding
> reinventing some wheel)?

It would be nice, though I don't know how feasable it is, to be able to plop
one or more data files where Configure can read them instead of running the
probes on the compiling system. That may make it much easier to generate
that data on the target.

-- c

Aldo Calpini

unread,
Feb 21, 2007, 4:39:59 AM2/21/07
to jerry gay, perl6-i...@perl.org
jerry gay ha scritto:

> On 2/20/07, Aldo Calpini <da...@perl.it> wrote:
>> 1) does anybody have objections to patching the current build system for
>> cross-compilation (even "yes, but not now because..." objections)?
>>
> no objection here! this is a long-desired feature, and is currently
> unavailable. although i don't have a pocketpc to test on, i'll do my
> best to help. for years, the dependence on perl 5's configure probes
> has bothered me--just not enough to fix it. i'm glad to see this
> porting effort will renew that effort.

that's all very good.

>> 2) does anybody already have a .plan or something in mind about it (so
>> that I may either learn from what others have thought, or avoiding
>> reinventing some wheel)?
>>

> [...]


> since this style hasn't been practiced as a rule in the past, it'll
> take some time to get used to--for all of us. but i believe it's the
> safest way to proceed to modify the critical-yet-undertested subsystem
> that is configure. are you willing to work in the way i described? how
> does the project team feel about this approach?

I have no problems with this approach. modulo the fact that I'm an
absolute beginner in the fine art of version control (some CVS, a little
bit of SVN, but just basic checkout-checkin), so I have no idea how to
go about branching and merging and all that stuff. but I'll find the
time to RTFM on my own :-)

> speaking briefly about a technical plan--there have been discussions
> of a way forward in the past. the general consensus is that miniparrot
> be designed to be ANSI C-only, with as few platform-specific
> extensions as possible, and zero dependence on perl's configure. after
> miniparrot is built, it can be used to perform the remainder of the
> build tasks in a portable manner. this gives us a lot of code reuse,
> and should make future porting efforts easier. however, there's a lot
> of this design left to flesh out, and none of it is implemented. i
> think this is the right design, but we need to find one or more ways
> to get there, and work towards that. however that discussion merits
> one or more separate message threads.

yuck. shortly after I sent my message I stumbled upon
http://rt.perl.org/rt3/Public/Bug/Display.html?id=31136. and my first
thought was "oh, no".

having the build system done by a miniparrot isn't going to be
convenient for cross-compiling. right now, I successfully built parrot
tweaking the source here and there, but as I said in another message,
the next steps in the Makefile (building the core Parrot library and
finally installing the product) are going to be a huge pain.

cross-compiling a simple, standalone, test executable to check for
features, headers etc and running it on the target platform is, well,
easy enough. on the other hand, running something (the miniparrot) which
probably expects to have the whole build directory at hand, this isn't
easy. copying files to and from the target device and generally
exchanging data is slow, unsafe, and ultimately just a hack. add to this
that we're talking about PDAs, which are usually much slower and
severely resource-constrained (both in RAM and storage) than a regular
desktop.

for PocketPC's sake, using a miniparrot to do most of the build is a
malus, not a bonus.

but this wouldn't affect, for example, the Nokia N800 port, which uses
scratchbox as an environment. this, however, comes at a cost: you can
build for the N800 _only_ on a *nix host platform. probably developing
something like scratchbox for CeGCC would be the Right Thing To Do. but
that's a completely different story (and a completely different effort,
alas).

all in all, this is something that will need to be considered very
carefully. and I guess I've ranted enough about it now :-)

cheers,
Aldo

Aldo Calpini

unread,
Feb 21, 2007, 6:42:52 AM2/21/07
to Joshua Isom, perl6-internals List
Joshua Isom ha scritto:

> Using perl 5's configure probes also somewhat limits us to how the
> vendor distributed perl. It should also be considered that we can't
> rely on perl5 being available, especially since we intend to replace
> it eventually, so rewriting all the perl to support cross compiling
> would likely not be the best thing.

I don't second that. I agree that relaying on Perl5's configure is not
the way to go, but I don't see any problem in using Perl5 as _the tool_
to build. while it's true that Parrot aims at replacing it, Perl5 is
going to be there for years to come. and what alternatives are there?
Perl5's own configuration/build system is based on bash and make, and
they are far more less available than Perl5 itself. IMO, using Perl to
configure is a much more powerful, maintainable, work-everywhere
solution than some autoconf-based stuff.

> If we create a new configure system, we can essentially have two
> configuration methods in the trunk, and have tests for the new one to
> be sure it's working and portable before trying to get everyone to
> start using it.

well, I didn't intend to build a whole new configure system from
scratch. I'm afraid this would be far more expensive than the current
amount of Free Time I have to spend on the project.

> I've noticed a lot that mentions miniparrot being used for building
> parrot, but at the same time, we don't have anything to work with to
> know just how limited miniparrot should be. Perhaps a step to work
> for before rebuilding the configure subsystem(which theoretically
> should be in pir to get rid of perl5 dependence), is to get an actual
> miniparrot working? Currently miniparrot just seems to be a
> macroparrot with a miniconfig. If we can't realistically get a
> miniparrot, maybe it should be considered if building a miniparrot for
> configuration is a good idea. Instead of rewriting everything now in
> perl5 to support cross compiling, maybe we should dive in and try to
> see if we can get it rewritten in pir. After all, we're getting more
> pir programmers than perl5 programmers it seems.

woot! I am _not_ going to become a pir programmer, sorry :-)
seriously, I'm not really sure that targeting "pir programmers" is a
desiderable thing. I mean, in the long term, how many people are going
to really learn pir and use it as a programming language? Parrot
maintainers, sure, and HLL compiler authors, but how many of them there
will be, as opposed to Perl(5..6) programmers?

cheers,
Aldo

Allison Randal

unread,
Feb 22, 2007, 2:42:57 AM2/22/07
to Aldo Calpini, perl6-i...@perl.org
Aldo Calpini wrote:
>
> 1) does anybody have objections to patching the current build system for
> cross-compilation (even "yes, but not now because..." objections)?

Not at all. Cross-compilation is definitely a core goal for Parrot.

> 2) does anybody already have a .plan or something in mind about it (so
> that I may either learn from what others have thought, or avoiding
> reinventing some wheel)?

We do. Unfortunately we can't rely on Perl 5 for the configure system.
It may seem like an easy way to gain cross-compilation in the short
term, but in the long term it will hurt us.

Miniparrot is the right way to go. It certainly needs work, though. As
you're thinking of solutions think of what miniparrot would need to
become cross-compiler aware.

> woot! I am _not_ going to become a pir programmer, sorry :-)

You'll be surprised at how easy it is to pick up, especially if you have
experience with dynamic languages. There are, of course, plenty of
Parrot tasks that don't require PIR skills, but don't give up before you
try it. You might like it.


> I have no problems with this approach. modulo the fact that I'm an
> absolute beginner in the fine art of version control (some CVS, a little
> bit of SVN, but just basic checkout-checkin), so I have no idea how to
> go about branching and merging and all that stuff. but I'll find the
> time to RTFM on my own :-)

That being the case, let's start you out submitting patches. It's where
everyone starts, and having a mentor to review patches at first is a
great way to pass along the knowledge of the experienced Parrot hackers.

For a change this size, we generally start with a proposal, review it as
a group, and then dive into implementation. In this case, given the
nature of the problem and your experience with the codebase, go ahead
and start experimenting with code, but write up your plans as you go.
We'll do an architecture review before too long to see how well the idea
fits with the overall plans for Parrot.

Patching the existing build system is a good way to start. It'll get you
more familiar with the codebase, and give us all a better sense of the
scope of the problem and the obstacles we're likely to encounter.

Thanks for volunteering,
Allison

Aldo Calpini

unread,
Feb 23, 2007, 12:45:26 PM2/23/07
to Allison Randal, perl6-i...@perl.org
Allison Randal ha scritto:

> We do. Unfortunately we can't rely on Perl 5 for the configure system.
> It may seem like an easy way to gain cross-compilation in the short
> term, but in the long term it will hurt us.
>
> Miniparrot is the right way to go. It certainly needs work, though. As
> you're thinking of solutions think of what miniparrot would need to
> become cross-compiler aware.

ok. I can't say that that I fully understand why, but I trust you.

> > woot! I am _not_ going to become a pir programmer, sorry :-)
>
> You'll be surprised at how easy it is to pick up, especially if you
> have experience with dynamic languages. There are, of course, plenty
> of Parrot tasks that don't require PIR skills, but don't give up
> before you try it. You might like it.

please, don't get me wrong. I have absolutely nothing against PIR, is
just that I'm so damn fluent in Perl5 :-)

I follow Parrot from the day it was born, I even wrote some - crude, I
admit it - code in PASM something like 4 years ago
(http://dada.perl.it/shootout/parrot_allsrc.html). but when the first
IMCC appeared, I felt it was just too much. maybe PIR nowadays isn't as
complicated as it was when I left it, I don't know. I will certainly
give it a closer look.

> That being the case, let's start you out submitting patches. It's
> where everyone starts, and having a mentor to review patches at first
> is a great way to pass along the knowledge of the experienced Parrot
> hackers.
>
> For a change this size, we generally start with a proposal, review it
> as a group, and then dive into implementation. In this case, given the
> nature of the problem and your experience with the codebase, go ahead
> and start experimenting with code, but write up your plans as you go.
> We'll do an architecture review before too long to see how well the
> idea fits with the overall plans for Parrot.

I just sent a first patch today (which is small and mostly^W harmless),
commenting what will be my next step(s). I have no "big plan" yet, it's
just a configure-fix-patch-reconfigure cycle, so I try to approach
obstacles as they pop out. when I come to the point that I will need a
"stronger" patch, something that may impact the overall architecture, I
intend to sit down and think about it for a while, write it down and
submit it for your approval.

thank you for your attention and wise words :-)

cheers,
Aldo

Allison Randal

unread,
Feb 27, 2007, 5:17:24 PM2/27/07
to Aldo Calpini, perl6-i...@perl.org
Aldo Calpini wrote:
> Allison Randal ha scritto:
>> We do. Unfortunately we can't rely on Perl 5 for the configure system.
>> It may seem like an easy way to gain cross-compilation in the short
>> term, but in the long term it will hurt us.
>>
>> Miniparrot is the right way to go. It certainly needs work, though. As
>> you're thinking of solutions think of what miniparrot would need to
>> become cross-compiler aware.
>
> ok. I can't say that that I fully understand why, but I trust you.

Well, we can't depend on Perl 5 because we're replacing it. And truly
cross-platform configuration and build tools are hard to find,
especially when you really want the same kind of dynamic features we're
used to in Perl. Miniparrot has whatever platform compatibility and
dynamic features we build into Parrot (or at least a functional subset
of them).

> I just sent a first patch today (which is small and mostly^W harmless),
> commenting what will be my next step(s).

I see the patch was applied. Looks good.

> I have no "big plan" yet, it's
> just a configure-fix-patch-reconfigure cycle, so I try to approach
> obstacles as they pop out. when I come to the point that I will need a
> "stronger" patch, something that may impact the overall architecture, I
> intend to sit down and think about it for a while, write it down and
> submit it for your approval.

Good plan.

Allison

Reply all
Reply to author
Forward
0 new messages