No Autoconf, dammit!
Dan Sugalski
people. The subject generates enough heat that I don't want to go
there again.
We are not using autoconf. Period.
Parrot's build process, when shipped will be:
*) Person building runs platform-specific script
*) Script builds miniparrot, which assumes ANSI C89 functionality only
*) Miniparrot runs provided configure program, which is in bytecode
*) Configure probes environment
*) Full parrot is built
*) Full parrot is then invoked to complete the build process, which
is driven by a program written in a parrot language and provided as
bytecode
--
Dan
--------------------------------------it's like this-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Robert Schwebel
sorry, although I'm a long term perl user I'm not that familiar with the
internals of the perl development process that I know all the old
stories ;)
The plan looks good, but some things are still unclear to me:
> *) Person building runs platform-specific script
Platform specific means also cross-platform specific? For example, I
would like to build Perl/Parrot on i686-unknown-linux-gnu and build for
arm-softfloat-linux-gnu. So the miniparrot you outlined is used during
the build process and must be compiled for i686, whereas the "real"
runtime engine has to be built for arm.
Autoconf handles this for example with --host and --build, using CC and
CC_FOR_BUILD in the Makefiles. I assume the variant you propose will
have similar functionality?
> *) Configure probes environment
How do you probe a cross environment? The build process will in this
case run on i686.
Robert
--
Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
Pengutronix - Linux Solutions for Science and Industry
Handelsregister: Amtsgericht Hildesheim, HRA 2686
Hornemannstraße 12, 31137 Hildesheim, Germany
Phone: +49-5121-28619-0 | Fax: +49-5121-28619-4
Josh Wilmes
While I am generally in favor of this idea (and I did get the first
miniparrots to work, pretty much as proof of concept), I do think it's
likely to be rather challenging (and interesting):
Remember, _pure_ C89 provides only these headers:
<assert.h> <complex.h> <ctype.h>
<errno.h> <fenv.h> <float.h>
<inttypes.h> <iso646.h> <limits.h>
<locale.h> <math.h> <setjmp.h>
<signal.h> <stdarg.h> <stdbool.h>
<stddef.h> <stdio.h> <stdlib.h>
<string.h> <tgmath.h> <time.h>
<wchar.h> <wctype.h>
This leaves out a number of things that one would really like to have
to do system probing, most notably:
- file info (stat, fstat)
- executing programs in any kind of sophisticated way (fork/exec, pipes)
My only real point is that, while I think pure c89 is a nice goal, in
practice we will need to relax this just a bit, to include at the very
least, things like unistd and bits of posix that are likely to be
everywhere. Should be interesting.
Probably isn't going to really gain momentum until we get some of the
build system written in something that compiles down to run on parrot..
How's that perl 6 compiler coming? ;-)
--Josh
Dan Sugalski
>While I am generally in favor of this idea (and I did get the first
>miniparrots to work, pretty much as proof of concept), I do think it's
>likely to be rather challenging (and interesting):
>
>Remember, _pure_ C89 provides only these headers:
>
> <assert.h> <complex.h> <ctype.h>
> <errno.h> <fenv.h> <float.h>
> <inttypes.h> <iso646.h> <limits.h>
> <locale.h> <math.h> <setjmp.h>
> <signal.h> <stdarg.h> <stdbool.h>
> <stddef.h> <stdio.h> <stdlib.h>
> <string.h> <tgmath.h> <time.h>
> <wchar.h> <wctype.h>
>
>This leaves out a number of things that one would really like to have
>to do system probing, most notably:
>
> - file info (stat, fstat)
Yep, these are a pain. Luckily the simple stuff (like "Does the file
even exist?") can be done with open and its return status.
> - executing programs in any kind of sophisticated way (fork/exec, pipes)
We do get system and popen, though.
>My only real point is that, while I think pure c89 is a nice goal, in
>practice we will need to relax this just a bit, to include at the very
>least, things like unistd and bits of posix that are likely to be
>everywhere. Should be interesting.
Yeah, there's no reason that the basic system can't have some #ifdefs
for the really common things.
>Probably isn't going to really gain momentum until we get some of the
>build system written in something that compiles down to run on parrot..
>How's that perl 6 compiler coming? ;-)
I'm more than happy to start writing the remaining environment probes
in pir. :)
Dan Sugalski
>Dan,
>
>sorry, although I'm a long term perl user I'm not that familiar with the
>internals of the perl development process that I know all the old
>stories ;)
>
>The plan looks good, but some things are still unclear to me:
>
>> *) Person building runs platform-specific script
>
>Platform specific means also cross-platform specific?
Not in this case, and I was being a bit broad. There are (or will be)
two different ways to build parrot.
The first is with a platform-specific script. That means shell
scripts for the various Unix (and Unix-ish, like Cygwin) platforms, a
.BAT file for windows, a .COM file for VMS, and so on. Whether
there's a per-platform shell script for the Unices or one generic one
that'll work well enough to bootstrap to the "Use parrot because it's
nicer" phase of the build's up in the air. This way assumes the user
has a command-line prompt of some sort and a C compiler. One of the
big reasons I want a non-make make tool is so we can teach it to
generate these scripts for us from the dependency lists.
The second is the 'developer' way, which will assume more tools -- a
working perl, make tool (maybe), possibly a working python or ruby
(or parrot with all three), possibly lex&yacc or the closest
equivalent.
The first method is for people who want to get the source tarball and
build it for installation. The second is for people who want to do
development work on parrot. Folks doing cross-platform stuff will
presumably use the second way. (I don't think this is too
unreasonable :)
> > *) Configure probes environment
>
>How do you probe a cross environment? The build process will in this
>case run on i686.
Since I've minimal (which is to say, absolutely no) cross-environment
build experience, this bit's a bit dodgy. For building parrot itself,
I'm assuming we can get away with a configuration file holding all
the info we need to build parrot -- basically everything parrot's
configure will probe for.
Building the ancillary stuff will be a bit problematic, since to do
so will need parrot, but the parrot you'll have won't run on the
platform you're on. So... I'm kinda at a loss there. I'm *more* than
happy to take suggestions or wholesale direction here.
Dan Sugalski
>At 9:23 on 09/08/2004 EDT, Dan Sugalski <d...@sidhe.org> wrote:
>
>> > - executing programs in any kind of sophisticated way
>>(fork/exec, pipes)
>>
>> We do get system and popen, though.
>
Bah. :( I was working on the assumption that it was, since popen
requires stdio.h.
>This URL is a fairly handy reference:
> http://www.unix.org/version3/inttables.pdf
Now *that* is a really useful table. Cool!
Robert Schwebel
> Whether there's a per-platform shell script for the Unices or one
> generic one that'll work well enough to bootstrap to the "Use parrot
> because it's nicer" phase of the build's up in the air. This way
> assumes the user has a command-line prompt of some sort and a C
> compiler. One of the big reasons I want a non-make make tool is so we
> can teach it to generate these scripts for us from the dependency
> lists.
Is my impression correct that nobody has ever tried crosscompiling perl,
and that nobody is really interested in doing it in the future?
I assume that, if you don't take this into account from the beginning it
is not very probable that it will ever work before Perl 7 :-)
> Folks doing cross-platform stuff will presumably use the second way.
> (I don't think this is too unreasonable :)
Ack.
> Since I've minimal (which is to say, absolutely no) cross-environment
> build experience, this bit's a bit dodgy. For building parrot itself,
> I'm assuming we can get away with a configuration file holding all the
> info we need to build parrot -- basically everything parrot's
> configure will probe for.
I'm still not sure what you mean with "probe". When doing cross builds
you cannot probe anything in the sense of "run some test code and look
at it's output", because (in my scenario) the build process is done on
i686, whereas the final code has to run on ARM. So you have to put all
the intelligence about which CPU core / plattform / libc / binary format
variant has which properties into some predefined magic: definitely
something one doesn't want to do without quite a lot of cross compiling
experience.
> Building the ancillary stuff will be a bit problematic, since to do so
> will need parrot, but the parrot you'll have won't run on the platform
> you're on. So... I'm kinda at a loss there. I'm *more* than happy to
> take suggestions or wholesale direction here.
Well, I still don't understand what the _technical_ arguments against
autotools are, besides not being written by LW ;)
Some pro arguments:
- runs on about all available platforms today, configure written in sh
- does proper cross compiler handling
- c89 handling is no problem
- everyone is used to configure/make/make install
Nicholas Clark
> On Wed, Sep 08, 2004 at 09:51:35AM -0400, Dan Sugalski wrote:
> > Whether there's a per-platform shell script for the Unices or one
> > generic one that'll work well enough to bootstrap to the "Use parrot
> > because it's nicer" phase of the build's up in the air. This way
> > assumes the user has a command-line prompt of some sort and a C
> > compiler. One of the big reasons I want a non-make make tool is so we
> > can teach it to generate these scripts for us from the dependency
> > lists.
>
> Is my impression correct that nobody has ever tried crosscompiling perl,
> and that nobody is really interested in doing it in the future?
No. The WinCE port of perl (in the Perl 5 source) is a cross compile on
Win32, as I understand it. The Zaurus packages are built as a cross compile
on another Linux, and should be repeatable based on the instructions in
the directory Cross/
> Well, I still don't understand what the _technical_ arguments against
> autotools are, besides not being written by LW ;)
>
> Some pro arguments:
>
> - runs on about all available platforms today, configure written in sh
sh doesn't run on all platforms that perl has done historically.
(platforms where perl is built natively, but not using the Configure script)
And sh is unlikely to get ported to them. While parrot might. Parrot doesn't
rely on a fork()/exec() process model and interprocess pipelines, whereas I'm
under the impression that sh does. And these are hard to emulate if they
are absent.
> - does proper cross compiler handling
Which is really useful.
> - c89 handling is no problem
Which is good.
> - everyone is used to configure/make/make install
On Unix. There is more to Perl than Unix.
Nicholas Clark
Gregory Keeney
> Is my impression correct that nobody has ever tried crosscompiling perl,
>
>and that nobody is really interested in doing it in the future?
>
>I assume that, if you don't take this into account from the beginning it
>is not very probable that it will ever work before Perl 7 :-)
>
>
Sounds like some of us with cross-compiling experience need to get our
hands dirty, once the basic build system is in place. I do think Dan's
plan to use a special configuration file that needs to perform no
probing will work – that is more or less what non-autoconf cross
compilation entails anyway. It is a pain – you have to know _everything_
about your target.
What category is this problem: easy or possible? Cross-compiling is
never easy. It should definitely be possible. Keeping enough information
around in a fancy cross-compiling perl script could be ... quiet nasty.
Frameworked properly, however, such a script could work both for unknown
and known targets. (Robert is likely screaming: "Your re-inventing
autoconf!!!")
Gregory Keeney
Robert Schwebel
> No. The WinCE port of perl (in the Perl 5 source) is a cross compile on
> Win32, as I understand it. The Zaurus packages are built as a cross compile
> on another Linux, and should be repeatable based on the instructions in
> the directory Cross/
I've tried to follow the instructions but they are done for a very
special Zaurus environment and not really generic. But I'll try again
with the recent versions, perhaps something has changed to the better
recently.
> sh doesn't run on all platforms that perl has done historically.
On which platforms shall perl run _today_ which is not able to run sh?
> And sh is unlikely to get ported to them. While parrot might. Parrot
> doesn't rely on a fork()/exec() process model and interprocess
> pipelines, whereas I'm under the impression that sh does. And these
> are hard to emulate if they are absent.
This is fine, especially for example for uCLinux systems. But hold on:
does somebody really _compile_ parrot on such a system? There are not
that much systems left today which have this needs and I doubt that
somebody will do a complete perl build on something like my 30 MHz ARM7
:-)
It seems to be a little bit strange to me that the ability to be
compiled on prehistoric systems seems to be more important than a
correct cross compiler environment.
Garrett Rooney
> On which platforms shall perl run _today_ which is not able to run sh?
VMS. Just because you don't use it doesn't mean that nobody uses it.
-garrett
Robert Schwebel
> Sounds like some of us with cross-compiling experience need to get our
> hands dirty, once the basic build system is in place.
I suppose I can do quite some testing in this case: with PTXdist I can
easily build complete Linux userland systems for quite a lot of strange
embedded processors (ARM, PPC, MIPS) and test if it works under these
circumstances.
> I do think Dan's plan to use a special configuration file that needs
> to perform no probing will work ??? that is more or less what
> non-autoconf cross compilation entails anyway. It is a pain ??? you
> have to know _everything_ about your target.
Hmm, can somebody specify which special parameters one needs to know?
Normally things work quite fine if C code ist only compiled with the
right cross compiler; some things like endianess have to be taken into
account, what else?
> What category is this problem: easy or possible? Cross-compiling is
> never easy.
Well, it depends. With autoconf (done right) it works right out of the
box, but let's end this here.
> It should definitely be possible. Keeping enough information around in
> a fancy cross-compiling perl script could be ... quiet nasty.
> Frameworked properly, however, such a script could work both for
> unknown and known targets. (Robert is likely screaming: "Your
> re-inventing autoconf!!!")
How did you know :-) It's just my experience that people normally start
with thinking that they can do it much easier than autoconf, and in the
end there come more and more requirements, scripts become more and more
compilcated and in the end you have something like autoconf but without
the huge ammount of experience which went into the system, so working in
less situations and being incompatible.
Dan Sugalski
> > sh doesn't run on all platforms that perl has done historically.
>
>On which platforms shall perl run _today_ which is not able to run sh?
No offense, but it *doesn't* *matter*. We're not using autoconf, as
the subject of this thread makes clear. That's not negotiable.
Robert Schwebel
> No offense, but it *doesn't* *matter*. We're not using autoconf, as
> the subject of this thread makes clear. That's not negotiable.
A really convincing argumentation.
Adam Herout
> It seems to be a little bit strange to me that the ability to be
> compiled on prehistoric systems seems to be more important than a
> correct cross compiler environment.
>
> On which platforms shall perl run _today_ which is not able to run sh?
For a particular project I am considering using Parrot on a custom
system based on Texas Instuments DSP processor - this class of systems
is described as weird rather than prehistoric.
I hope that Parrot might be the option for me in this project - and with
my minimal (understand "zero") cross-compiling experience, I still feel
that going without sh and the like would be easier for me and the idea
of just providing c89 functionality (and possibly a couple of more
functions) is very attractive.
Adam Herout
Herbert Snorrason
> On Wed, Sep 08, 2004 at 11:23:36AM -0400, Dan Sugalski wrote:
> > No offense, but it *doesn't* *matter*. We're not using autoconf, as
> > the subject of this thread makes clear. That's not negotiable.
>
> A really convincing argumentation.
I suggest we institute a "Rule One" for Dan. (And number two, too,
while we're at it.) It'd be easier that way.
The autoconf argument is contentious. That's why the thread was
started; to avoid the argument, as the decision has already been made.
Unfortunately, that seems impossible; it's a pandora's box.
--
Schwäche zeigen heißt verlieren;
härte heißt regieren.
- "Glas und Tränen", Megaherz
Gregory Keeney
>I suggest we institute a "Rule One" for Dan. (And number two, too,
>while we're at it.) It'd be easier that way.
>
>
Ooh, ooh, I know, I know!
Rule Number One:
• No one wants the ‽ [interrobang if your email client or font
doesn't like utf-8]
Rule Number Two:"
• Dan gets the ‽
Oh, ok. Sorry.
Herbert Snorrason
<gregory....@lmco.com> wrote:
> Rule Number One:
> • No one wants the ‽ [interrobang if your email client or font
> doesn't like utf-8]
> Rule Number Two:"
> • Dan gets the ‽
is right, even if he changed his mind", but sure. Those might work
too. In what context, though, I don't know...
Timm Murray
Hash: SHA1
<delurk>
I searched the list archives on groups.google.org to try to get more context
for this discussion, but didn't come up with much that seems relevent. Can
somebody point me to an old thread where Autoconf is discussed?
One other thing:
> *) Person building runs platform-specific script
If that script is going to be platform-specific anyway, why not use Autoconf
for the platforms that can handle it? You'd cover a rather large number of
platforms that way, and the ones you don't are going to need their own script
regardless.
</delurk>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFBPzOfbMnv87GOcv0RAgSLAJ9pH30rdHqR8mVxFuV9Q4uQJb40FwCcCS2z
I3QOuGrEcHdd+3WO83x4CMQ=
=Ltsg
-----END PGP SIGNATURE-----
Chip Salzenberg
> It seems to be a little bit strange to me that the ability to be
> compiled on prehistoric systems seems to be more important than a
> correct cross compiler environment.
Anyone doing cross-compilation should know enough about their target
environment to build a config file by hand. IMO. And yes, I *have*
done cross-compilation myself.
--
Chip Salzenberg - a.k.a. - <ch...@pobox.com>
"Persistence in one opinion has never been considered
a merit in political leaders." --Marcus Tullius Cicero
John Siracusa
> I suggest we institute a "Rule One" for Dan. (And number two, too,
> while we're at it.) It'd be easier that way.
That rule already exists, but I think Dan still feels insecure about
it ;) The Larry Way(tm) is to include the decision in the middle of a
mind-numbing, globe trotting spew of information. That way, there are
plenty of jucier tidbits to distract people. He also usually gives a
one or two sentence reason, which might help here too. An example:
"No Autoconf because Autoconf assumes the existence of more things
than Parrot can. Parrot has to build anywhere that has C89 support,
but Autoconf requires sh, among other things."
I don't even know if that's accurate, but in lieu of Larry's "Where's
Waldo Hidden Decree" technique, I think there needs to be something
more than "because I said so" to keep the natives from becoming
restless :)
-John
Dan Sugalski
>On Wed, Sep 08, 2004 at 11:23:36AM -0400, Dan Sugalski wrote:
>> No offense, but it *doesn't* *matter*. We're not using autoconf, as
>> the subject of this thread makes clear. That's not negotiable.
>
>A really convincing argumentation.
It wasn't an argument, though -- it was a statement of fact and
intent. Someone's got to decide this stuff, and in this case it's me.
(There are significant downsides to this, though, since I also get
the blame/crap/browbeatings when things don't work out, amongst other
things) More importantly, this is an old decision we're not going to
revisit.
Now, having said that, the other part of this, on cross-compilation,
*is* very interesting, and needs addressing, so I will. In a
different thread, so we can separate the flames. :)
Nicholas Clark
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> <delurk>
>
> I searched the list archives on groups.google.org to try to get more context
> for this discussion, but didn't come up with much that seems relevent. Can
> somebody point me to an old thread where Autoconf is discussed?
>
> One other thing:
>
> > *) Person building runs platform-specific script
>
> If that script is going to be platform-specific anyway, why not use Autoconf
> for the platforms that can handle it? You'd cover a rather large number of
> platforms that way, and the ones you don't are going to need their own script
> regardless.
Because the plan is to build a minimal parrot first, then use it to do
the probing. The platform specific script that gets run isn't going to do
any probing, so a straight generated bourne shell script will do the trick.
The probing is going to *have* to get written in something that compiles
down to parrot bytecode to work on the autoconf-deprived systems, so with
that as a given there's no need for autoconf ahead of that.
Nicholas Clark
Larry Wall
: For a particular project I am considering using Parrot on a custom
: is described as weird rather than prehistoric.
: I hope that Parrot might be the option for me in this project - and with
: my minimal (understand "zero") cross-compiling experience, I still feel
: that going without sh and the like would be easier for me and the idea
: of just providing c89 functionality (and possibly a couple of more
: functions) is very attractive.
In principle, cross-compile configuration is drop-dead easy. All you
need is a database of what the probe program *would* have answered
had you been able to run it on the other machine. (Getting someone
to write that database entry for you is the tricky part.) You also
have to be careful to separate architectural parameters from policy
parameters. An architectural parameter says your integers are 32 bits.
A policy parameter says you want to install the documentation in the
/foo/bar/baz directory. Cross compilation has to nail down the
architectural parameters while potentially deferring decisions on
policy to a later installation step.
An interesting question would be whether we can bootstrap a Parrot
cross-compile database using autoconf's *data* without buying into the
shellism of autoconf. Or give someone the tool to extract the data
from the autoconf database themselves, so we don't have to ship it.
Larry
Gregory Keeney
Larry Wall wrote:
>In principle, cross-compile configuration is drop-dead easy. All you
>need is a database of what the probe program *would* have answered
>had you been able to run it on the other machine. (Getting someone
>to write that database entry for you is the tricky part.) You also
>have to be careful to separate architectural parameters from policy
>parameters. An architectural parameter says your integers are 32 bits.
>A policy parameter says you want to install the documentation in the
>/foo/bar/baz directory. Cross compilation has to nail down the
>architectural parameters while potentially deferring decisions on
>policy to a later installation step.
>
>
Easy thing.
>An interesting question would be whether we can bootstrap a Parrot
>cross-compile database using autoconf's *data* without buying into the
>shellism of autoconf. Or give someone the tool to extract the data
>from the autoconf database themselves, so we don't have to ship it.
>
>
Possible thing.
>
>
Dan Sugalski
>On Wed, 08 Sep 2004 08:57:22 -0700, Gregory Keeney
><gregory....@lmco.com> wrote:
>> Rule Number One:
>> doesn't like utf-8]
>> Rule Number Two:"
>I was thinking more along the lines of "Dan is always right" and "Dan
>is right, even if he changed his mind", but sure. Those might work
>too. In what context, though, I don't know...
While "Dan is always right" has that nice ego-stroke effect, I don't
think too many people would or, really, should, stand for it. We'd be
better served with "The designer makes the final call, for better or
worse" as a rule one.
Herbert Snorrason
> While "Dan is always right" has that nice ego-stroke effect, I don't
> think too many people would or, really, should, stand for it. We'd be
> better served with "The designer makes the final call, for better or
> worse" as a rule one.
version as well. "For better or worse, someone eventually has to
decide. Let's just have that someone defined, so we don't run into too
much unneeded trouble."
Robert Schwebel
On Wed, Sep 08, 2004 at 09:40:44AM -0700, Larry Wall wrote:
> In principle, cross-compile configuration is drop-dead easy. All you
> need is a database of what the probe program *would* have answered
> had you been able to run it on the other machine. (Getting someone
> to write that database entry for you is the tricky part.) You also
> have to be careful to separate architectural parameters from policy
> parameters. An architectural parameter says your integers are 32 bits.
> A policy parameter says you want to install the documentation in the
> /foo/bar/baz directory. Cross compilation has to nail down the
> architectural parameters while potentially deferring decisions on
> policy to a later installation step.
Sounds perfectly fine. Is there a place in the current parrot code where
this information is already available? I could try to provide the
necessary bits for ARM to have a proof of concept.
Chip Salzenberg
> On Wed, Sep 08, 2004 at 11:23:36AM -0400, Dan Sugalski wrote:
> > No offense, but it *doesn't* *matter*. We're not using autoconf, as
> > the subject of this thread makes clear. That's not negotiable.
>
> A really convincing argumentation.
Robert, you seem not to understand that Dan -- or anyone else actually
running a project -- is not obligated to convince you. If you don't
like the decisions when they're made, you have the right and privilege
to take your ball and go home.
The captain can be right, and the captain can be wrong, but the
captain cannot be indecisive.
PS: Splunge.
Rhys Weatherley
> An interesting question would be whether we can bootstrap a Parrot
> cross-compile database using autoconf's *data* without buying into the
> shellism of autoconf. Or give someone the tool to extract the data
> from the autoconf database themselves, so we don't have to ship it.
What autoconf database? Autoconf uses probing for cross-compilation as well.
i.e. it runs the cross-compiler and sees what succeeds and what fails. Some
things are tricky, like detecting type sizes and endianness, because you
cannot run a program to printf the answer. But there are ways around that
described in the autoconf macro archive:
http://www.gnu.org/software/ac-archive/
As an example, the size of a type can be determined by cross-compiling several
test programs that contain this:
switch (0)
{
case 0: case (sizeof ($type) == $size):;
}
where "$size" is iterated over 1 2 4 8 16, etc. All compilations will fail
with a duplicate case error except the size you are looking for.
Essentially, you use the cross-compiler's knowledge of the platform to act as
the "database". You just need to be clever in how you format the query.
Just an FYI. It's possible that Parrot's probe system could use a similar
mechanism for cross-compilation to avoid the need for platform databases.
Cheers,
Rhys Weatherley.
Autoconf victim for 5 years now.
Gregory Keeney
> What autoconf database? Autoconf uses probing for cross-compilation as
> well.
>
>i.e. it runs the cross-compiler and sees what succeeds and what fails. Some
>things are tricky, like detecting type sizes and endianness, because you
>cannot run a program to printf the answer. But there are ways around that
>described in the autoconf macro archive:
>
> http://www.gnu.org/software/ac-archive/
>
>As an example, the size of a type can be determined by cross-compiling several
>test programs that contain this:
>
> switch (0)
> {
> case 0: case (sizeof ($type) == $size):;
> }
>
>where "$size" is iterated over 1 2 4 8 16, etc. All compilations will fail
>with a duplicate case error except the size you are looking for.
>Essentially, you use the cross-compiler's knowledge of the platform to act as
>the "database". You just need to be clever in how you format the query.
>
>Just an FYI. It's possible that Parrot's probe system could use a similar
>mechanism for cross-compilation to avoid the need for platform databases.
>
>
I don't think Parrot's probe system can help us here. Autoconf (as
described above) uses the target architecture compiler's knowledge of
the target system. We don't have anything equivalent, as we want to
bootstrap the cross compiler through Parrot, not the C compiler.
Maybe we can harvest some data from gcc's target database (though I am
not sure what licensing issues may be involved) for Parrot – that may be
more work than it is worth, however. I dug down in there once. It's kind
of scary.
I feel like I am missing something here, but I am not sure what…
Gregory Keeney
Rhys Weatherley
> I don't think Parrot's probe system can help us here. Autoconf (as
> described above) uses the target architecture compiler's knowledge of
> the target system. We don't have anything equivalent, as we want to
> bootstrap the cross compiler through Parrot, not the C compiler.
I suppose it depends upon what you mean by cross-compilation:
(a) compiling the C parts of parrot on platform A so it can run on platform B,
to JIT and run platform-independent bytecode.
(b) compiling the C parts of parrot to run on platform A, but when run the
parrot engine generates platform-dependent code for platform B.
The autoconf thing I described helps with (a), but maybe not (b). I had
assumed that you meant (a). Apologies if I was mistaken.
Cheers,
Rhys.
Dan Sugalski
Right, but we'll be using the C compiler. The only way to probe for
this sort of stuff is to emit little test programs and compile them.
(Which can be problematic when cross-compiling, thought the tricks in
the archive are pretty nifty)
Josh Wilmes
>
> > *) Person building runs platform-specific script
>
> If that script is going to be platform-specific anyway, why not use Autoconf
> for the platforms that can handle it? You'd cover a rather large number of
> platforms that way, and the ones you don't are going to need their own script
> regardless.
It's very likely that the unix version of the platform specific script
WILL include some very minimal probing to do things like find the
compiler. But that's really all it needs to do- remember, the goal here
(as proposed) is to build miniparrot, not to build parrot. That means
you don't *need* autoconf.
The latter part of the build process, where we actually build the real
parrot, is a different issue. Dan's ruled out autoconf, so we get to
reinvent that wheel. FWIW, i'm not sure that I would have gone this
route, but dan's made a choice, and the best thing to do (IMHO) is to
follow that road and see where it leads. If it turns out to be dumb, we
move on. If it works, great.
So far, parrot's environmental probing needs really aren't terribly
sophisticated- most of the work has already been done in the current
Configure system- the test programs exist, and the mechanism to compile
and run them is well understood.
The remaining work to build the parrotconf system is mostly tedious rather
than complex. Some of the harder bits (IMHO) will revolve around
figuring out how to compile and link things, dynamic loading, and
architecture detection ($Config{archname}, osname, osvers, etc). If you
look at autoconf or metaconfig, those tasks are inevitably full of wacky
system-specific stuff. But since this is perl, porting metaconfig's units
for seting those variables to something that can run on parrot is probably
the way things will go.
It's not TERRIBLY hard but it isn't very glamorous either :) I did start
looking into converting the archname code into perl, as a starting point,
a while back. I may still have that somewhere and could probably finish
it up- my reasoning was that if I could get it from shell code into perl
code, i could understand it better and then rewrite it from scratch again
in some form that we could compile to PIR. Some sort of OOish language
with a working compiler for parrot would be nice. I'm not too keen on the
idea of maintaining nasty little probing code in assembly. :)
--Josh
Thomas Seiler
> Rhys Weatherley wrote:
>
>> What autoconf database? Autoconf uses probing for cross-compilation as
>> well.
>
>> act as the "database". You just need to be clever in how you format
>> the query.
>
> described above) uses the target architecture compiler's knowledge of
> the target system. We don't have anything equivalent, as we want to
> bootstrap the cross compiler through Parrot, not the C compiler.
Couldn't we split the probing into two phases ?
Let's asume for a moment that it's easy to build a miniparrot for ethier
the host or the target.
The first phase would run on the host and prepare the tests and a
miniparrot for the target, but not run them.
The seconde phase would run on the target and run the actual test, while
populating a Configuration file.
Host Phase:
- Build host version of miniparrot
- Build target version of miniparrot (use cross compiler)
- Build all probing programms and test cases (use cross-compiler)
- Populate a sub-directory with target miniparrot and the remaining
test programms
Target Phase:
- Copy that directory to the target
- Run the configuration phase and get the Config file
The challange would be to get the miniparrot running on the target.
All the rest should be covered by the "normal" Parrot Configuration Probing.
This is about as general as it can get, or am I missing something ?
Thomas Seiler
Aaron Sherman
> have to be careful to separate architectural parameters from policy
> parameters. An architectural parameter says your integers are 32 bits.
> A policy parameter says you want to install the documentation in the
> /foo/bar/baz directory. Cross compilation has to nail down the
> architectural parameters while potentially deferring decisions on
> policy to a later installation step.
Actually, I'd say that they're all architectural parameters, and you
want to put them all in the database (e.g. you should define that for
Fedora Linux Core 2, the default documentation area is /usr/share/man,
but for SunOS 3, it's /usr/man). What the person compiling the program
OVERRIDES is their call (and in some contexts, the size of integers is a
POLICY decision, not architectural).
Aaron Sherman
> On Thursday 09 September 2004 02:40 am, Larry Wall wrote:
>
> > An interesting question would be whether we can bootstrap a Parrot
> > cross-compile database using autoconf's *data* without buying into the
> > shellism of autoconf. Or give someone the tool to extract the data
> > from the autoconf database themselves, so we don't have to ship it.
>
> What autoconf database? Autoconf uses probing for cross-compilation as well.
Well, that's one of the big problems with autoconf: it's NOT a database.
For example:
# AC_FUNC_GETMNTENT
# -----------------
AN_FUNCTION([getmntent], [AC_FUNC_GETMNTENT])
AC_DEFUN([AC_FUNC_GETMNTENT],
[# getmntent is in -lsun on Irix 4, -lseq on Dynix/PTX, -lgen on Unixware.
AC_CHECK_LIB(sun, getmntent, LIBS="-lsun $LIBS",
[AC_CHECK_LIB(seq, getmntent, LIBS="-lseq $LIBS",
[AC_CHECK_LIB(gen, getmntent, LIBS="-lgen $LIBS")])])
AC_CHECK_FUNCS(getmntent)
])
There's knowledge encoded in that, but it's not abstracted sufficiently.
Some assumptions could be made, and autoconf's knowledge could be
distilled a bit and then extracted into the [cross-]compiling database
that would be needed by Parrot.
Gregory Keeney
> Couldn't we split the probing into two phases ?
> Let's asume for a moment that it's easy to build a miniparrot for
> ethier the host or the target.
>
> The first phase would run on the host and prepare the tests and a
> miniparrot for the target, but not run them.
>
> The seconde phase would run on the target and run the actual test, while
> populating a Configuration file.
>
> Host Phase:
> - Build host version of miniparrot
> - Build target version of miniparrot (use cross compiler)
> - Build all probing programms and test cases (use cross-compiler)
> - Populate a sub-directory with target miniparrot and the remaining
> test programms
>
> Target Phase:
> - Copy that directory to the target
> - Run the configuration phase and get the Config file
>
> The challange would be to get the miniparrot running on the target.
> All the rest should be covered by the "normal" Parrot Configuration
> Probing.
>
> This is about as general as it can get, or am I missing something ?
The problem is that getting stuff on and off your target host is not
always trivial. Typically, if you can do what you describe, you don't
need to perform cross compilation: you can simply compile on your target
(not always, sometimes you may want to build a couple different
architectures at once, even if they are all unixy, or you may not have a
compiler on the target (think Windows)).
It is especially not true in the embedded world. Until I have parrot IO
libraries, I am not going to be getting a whole lot out of the target.
You could write the results in memory and then dump it, but ... well, it
seems simpler to hand supply the config information. Trying to get
answers from an embedded target is tricky and sometimes even potentially
dangerous.
Gregory Keeney
>
> Thomas Seiler
>
Brent 'Dax' Royal-Gordon
> > sh doesn't run on all platforms that perl has done historically.
>
> On which platforms shall perl run _today_ which is not able to run sh?
Windows, you insensitive clod. :^P
In all seriousness, this is an area where you have to be very careful
to remember that you aren't the world, and that not everybody runs
what you run. I run x86 Windows and Linux, and occasionally work with
FreeBSD; that doesn't mean my work on Parrot shouldn't support OS X
running on a G5 processor as a 64-bit program, or (heaven forbid) VMS
on Alpha.
--
Brent 'Dax' Royal-Gordon <br...@brentdax.com>
Perl and Parrot hacker
There is no cabal.
[I currently have a couple Gmail invites--contact me if you're interested.]
Brent 'Dax' Royal-Gordon
> > *) Person building runs platform-specific script
>
> If that script is going to be platform-specific anyway, why not use Autoconf
> for the platforms that can handle it?
By platform-specific, we mean that on Unix you'll have to run this command:
$ export CC=/usr/bin/gcc
$ buildmini/unix
While on Windows you'll need:
C:\parrot>set CC="c:\progra~1\micros~3\vc7\bin\cl.exe"
C:\parrot>set LINK="c:\progra~1\micros~3\vc7\bin\link.exe"
C:\parrot>buildmini\windows
And on VMS you'll need...er...I don't even know *what* incantation
you'd need, but I don't think it'd be pretty.
All of these platform-specific scripts would be some variant of this:
cp buildmini/miniconfig.h include/parrot/config.h
cp buildmini/miniplatform.h include/parrot/platform.h
cp buildmini/miniplatform.c src/platform.c
$CC -DMINIPARROT_UNIXISH -I./include foo.c
$CC -DMINIPARROT_UNIXISH -I./include bar.c
$CC -DMINIPARROT_UNIXISH -I./include baz.c
...
All of the heavy probing would be done in miniparrot, an environment
with consistent (if limited) semantics.
Thomas Seiler
> Thomas Seiler wrote:
>
>> Couldn't we split the probing into two phases ?
>>
> The problem is that getting stuff on and off your target host is not
> always trivial. [...]
>
> It is especially not true in the embedded world. Until I have parrot IO
> libraries, I am not going to be getting a whole lot out of the target.
Yeah, thats true. I haven't tought about that because all the embedded
systems we have touched in school had at least a serial port for
debugging purposes, and it was a simple library call away. But I suppose
that the serial port was part of the reason why they had chosen them...
> You could write the results in memory and then dump it, but ... well, it
> seems simpler to hand supply the config information. Trying to get
> answers from an embedded target is tricky and sometimes even potentially
> dangerous.
You have convinced me that this is not going to work, too bad...
Thanks,
Thomas Seiler
> Gregory Keeney
Dan Sugalski
>Timm Murray <tmu...@wumpus-cave.net> wrote:
> > > *) Person building runs platform-specific script
>>
>
>you'd need, but I don't think it'd be pretty.
@BUILD
or possibly
@MAKE
Horrible, I know. Oh, the pain, the aaaaagony! :-P
Spider Boardman
ajs> All of this depends on if Dan was saying "No autoconf RELIANCE,
ajs> dammit" or actually "No autoconf, dammit". The first is a reasonable
ajs> stance to take given the portability concerns. The second throws away
ajs> useful information.
From the perspective of someone who has long experience with autoconf, the
second does NOT throw away useful information. It's autoconf itself which
does that. [Yes, I do understand what you really meant, but this has
finally exceeded my "keep quiet" threshold.]
Putting it another way, autoconf has a long history of breaking on
less-favoured platforms. When it breaks, it's damnably hard for even an
otherwise-knowledgeable (but not autoconf/automake-savvy) end-user to fix
the results, or even to give any useful feedback to the development team
as to what went wrong. The problem is that, unlike metaconfig, autoconf
simply drops things from its generated config.status and config.h files.
[The relevant contrast with metaconfig is that it leaves explicit
commented-out #undef's of what could have been included, so that you can
repair its faulty findings if it gets them wrong.]
Thus, to this well-scarred developer, "No Autoconf, dammit!" means this:
"Use something that actually WORKS -- which autoconf does NOT!". It's not
just a problem when autoconf truly fails. In fact, that's a lot easier to
handle. The major pain comes when it thinks it worked, but it got things
wrong.
A recent case in point went as follows. Autoconf 'said', "You have no
usable pthreads interfaces, and no _r reentrant routines." What that
really meant was this: "I tried #include'ing <pthread.h> and didn't know
that I need -D_REENTRANT on your platform for that to work, so I couldn't
get my test programmes to compile because my probing is faulty."
Sure, I know enough to have been able to fix it. See above about
"otherwise-knowledgeable". It should not require a descent into the Black
Arts to get configuration probing to tell you what it couldn't figure out.
That is the source of my own opposition to the use of autoconf. I suspect
that similar experiences are also at least part of Dan's, but I don't
*know* that.
--s.
tho...@whatever.net.au
> The probing is going to *have* to get written in something that compiles
> down to parrot bytecode to work on the autoconf-deprived systems, so with
> that as a given there's no need for autoconf ahead of that.
How feasable would it be to write a sh -> parrot bytecode compiler?
:)
Cheers,
Tom
Jeff Clites
Ha, I'm sure it could probably be done, but of course "most" of what
the shell does it invoke other programs, so in the common case it still
wouldn't give you portability to non-Unix-like platforms.
JEff
Larry Wall
: Ha, I'm sure it could probably be done, but of course "most" of what
: wouldn't give you portability to non-Unix-like platforms.
Just translate it to a language that has most of Unix semantics built-in. :-)
Larry
Chip Salzenberg
Unless I've lost the thread of this discussion, that won't work.
Microparrot can't be built with e.g. a call to readlink(), because the
(non-)existence of readlink() is exactly the kind of thing that must
be probed for.
--
Chip Salzenberg - a.k.a. - <ch...@pobox.com>
"I don't really think it is a question of bright people and dumb
people, but rather people who can see the game they're playing and
those who can't." -- Joe Cosby
Larry Wall
: According to Larry Wall:
: > On Sat, Sep 18, 2004 at 12:27:36PM -0700, Jeff Clites wrote:
: > : Ha, I'm sure it could probably be done, but of course "most" of what
: > : the shell does it invoke other programs, so in the common case it still
: > : wouldn't give you portability to non-Unix-like platforms.
: >
: > Just translate it to a language that has most of Unix semantics built-in. :-)
:
: Unless I've lost the thread of this discussion, that won't work.
: Microparrot can't be built with e.g. a call to readlink(), because the
: (non-)existence of readlink() is exactly the kind of thing that must
: be probed for.
Sorry, I've obviously been overusing smileys lately. Tell you what, I'll
start putting two smileys when I'm really joking. :-) :-)
Larry