Building nci/dynclasses on HP-UX
Nick Glencross
I'm currently investigating the build process for nci and dynclasses on
HP-UX. As you may have seen from my previous posts, I'm using gcc and
the native bundled ld.
Although the build process isn't using the right flags to compile nci
and dynclasses, I've managed to compile things manually.
Certain files need special PIC treatment (at least on HP-UX) so that
they can be included in shared libraries:
src/extend.o
src/nci_test.o
dynclasses/*.o
more?
Hence, these files need:
GNU cc: -fpic
HP-UX cc: +z
and then for linking the library:
GNU ld: -shared
HP ld: -b
Also HP ld does not like -g, and should not be used. In fact, I'm not
aware that any flavours of ld should be using -g (it is ignored in GNU ld).
Most of the nci tests pass, except for tests 8 and 52 which both hang
(on the dlvar call with "nci_dlvar_int").
I'm still working on getting dynclasses working. Although compiled, the
libraries are failing to mmap in the backend of dlopen (probably
unresolved symbols, pic or something else).
Regards,
Nick
H.Merijn Brand
<ni...@glencros.demon.co.uk> wrote:
> Guys,
>
> I'm currently investigating the build process for nci and dynclasses on
> HP-UX. As you may have seen from my previous posts, I'm using gcc and
> the native bundled ld.
>
> Although the build process isn't using the right flags to compile nci
> and dynclasses, I've managed to compile things manually.
>
> Certain files need special PIC treatment (at least on HP-UX) so that
> they can be included in shared libraries:
>
> src/extend.o
> src/nci_test.o
> dynclasses/*.o
> more?
>
> Hence, these files need:
>
> GNU cc: -fpic
use -fPIC for more portability
> HP-UX cc: +z
use +Z -z for more portability and higher run-time safety
and never forget -Ae, but I can't see from this post if you already use it
further, if you use -O2 or up, add +Onolimit
> and then for linking the library:
>
> GNU ld: -shared
> HP ld: -b
>
> Also HP ld does not like -g, and should not be used. In fact, I'm not
> aware that any flavours of ld should be using -g (it is ignored in GNU ld).
>
> Most of the nci tests pass, except for tests 8 and 52 which both hang
> (on the dlvar call with "nci_dlvar_int").
>
> I'm still working on getting dynclasses working. Although compiled, the
> libraries are failing to mmap in the backend of dlopen (probably
> unresolved symbols, pic or something else).
--
H.Merijn Brand Amsterdam Perl Mongers (http://amsterdam.pm.org/)
using Perl 5.6.2, 5.8.0, 5.8.5, & 5.9.2 on HP-UX 10.20, 11.00 & 11.11,
AIX 4.3 & 5.2, SuSE 9.2 & 9.3, and Cygwin. http://www.cmve.net/~merijn
Smoking perl: http://www.test-smoke.org, perl QA: http://qa.perl.org
reports to: smokers...@perl.org, per...@perl.org
Nick Glencross
> On Fri, 03 Jun 2005 13:11:57 +0100, Nick Glencross
> <ni...@glencros.demon.co.uk> wrote:
>
>
>>Guys,
>>
>>I'm currently investigating the build process for nci and dynclasses on
>>HP-UX. As you may have seen from my previous posts, I'm using gcc and
>>the native bundled ld.
>
> use +Z -z for more portability and higher run-time safety
> and never forget -Ae, but I can't see from this post if you already use it
Good points about the flags. I only have the 'bundled' C compiler and am
stuck with gcc, so I've not actually been able to use the HP cc. You're
right, -Aa or c89 command-line, a very good point.
I've been trying to understand the rules that HP-UX's dlopen/dlsym play
by. I've still a few more experiments to try, but dlsym has only worked
for me if the the executable is also created PIC and the dynamic library
is linked against libparrot.a, resulting in huge dynclasses, but dlsym
returns NULL otherwise.... (mind you, it's worked pretty well for
nci_test.sl)
I fear that there may be some pointer alignment problems in hash because
I'm getting hangs which seem to be linked to finding strings in hash
tables. (That's pure conjecture)
Nick
H.Merijn Brand
<ni...@glencros.demon.co.uk> wrote:
> H.Merijn Brand wrote:
>
> > On Fri, 03 Jun 2005 13:11:57 +0100, Nick Glencross
> > <ni...@glencros.demon.co.uk> wrote:
> >
> >
> >>Guys,
> >>
> >>I'm currently investigating the build process for nci and dynclasses on
> >>HP-UX. As you may have seen from my previous posts, I'm using gcc and
> >>the native bundled ld.
> >
> > use +Z -z for more portability and higher run-time safety
> > and never forget -Ae, but I can't see from this post if you already use it
>
> Good points about the flags. I only have the 'bundled' C compiler and am
Get yourself a testdrive account on http://www.testdrive.hp.com/
Current available systems: http://www.testdrive.hp.com/current.shtml
All systems have native ANSI-C compilers installed
If you are serious in using that for HP-UX or OpenVMS, please contact
me off-list, since we have a special perl5 group there with just a
little bit more rights
> stuck with gcc, so I've not actually been able to use the HP cc. You're
> right, -Aa or c89 command-line, a very good point.
>
> I've been trying to understand the rules that HP-UX's dlopen/dlsym play
> by. I've still a few more experiments to try, but dlsym has only worked
> for me if the the executable is also created PIC and the dynamic library
> is linked against libparrot.a, resulting in huge dynclasses, but dlsym
> returns NULL otherwise.... (mind you, it's worked pretty well for
> nci_test.sl)
>
> I fear that there may be some pointer alignment problems in hash because
> I'm getting hangs which seem to be linked to finding strings in hash
> tables. (That's pure conjecture)
--
Leopold Toetsch
> I fear that there may be some pointer alignment problems in hash because
> I'm getting hangs which seem to be linked to finding strings in hash
> tables. (That's pure conjecture)
I'm not aware of any problems with alignment in hash.c. *But* we will
have troubles on that platform with doubles. Parrot PMCs are allocate
off a big piece of mem. A PMC currently happens to be 24 bytes (on
32-bit) but only with the debug-ish version field compiled in. With an
--optimze compiled parrot every second PMC would have unaligned doubles.
I'm not quite sure if there are other places in Parrot, where a double
could be only aligned at 4 bytes.
> Nick
leo
Andy Dougherty
> Nick Glencross wrote:
>
> > I fear that there may be some pointer alignment problems in hash because I'm
> > getting hangs which seem to be linked to finding strings in hash tables.
> > (That's pure conjecture)
>
> I'm not aware of any problems with alignment in hash.c. *But* we will have
> troubles on that platform with doubles. Parrot PMCs are allocate off a big
> piece of mem. A PMC currently happens to be 24 bytes (on 32-bit) but only with
> the debug-ish version field compiled in. With an --optimze compiled parrot
> every second PMC would have unaligned doubles.
That's likely to cause a problem. We've been recently hunting down odd
alignment-related oddities on p5p. As a first approximation, doubles need
to be aligned at least on the boundaries given by Perl5's
Config{'alignbytes'}. However, when those doubles are part of a structure,
it apparently can get more confusing. I don't think we sorted out all the
possibilities, but Nick C. was the last one fiddling with it.
In the end, there may be no recourse other than using unions.
--
Andy Dougherty doug...@lafayette.edu
Nick Glencross
> Folks,
>
> I hope that not too many of you are getting fed up with me going on
> about HP-UX. I'm nearly there with having it working with manually
> hacked Makefiles etc.
My hacked HP-UX build now passes most tests. Wohooo!
Failed Test Stat Wstat Total Fail Failed List of Failed
----------------------------------------------------------------------------
t/pmc/nci.t 2 512 56 2 3.57% 8 52
t/pmc/signal.t 2 512 3 2 66.67% 1-2
t/src/manifest.t 1 256 5 1 20.00% 3
3 tests and 69 subtests skipped.
Failed 3/151 test scripts, 97.35% okay. 5/2488 subtests failed, 99.76% okay.
nci tests 8 and 52 actually hang, so I've hacked them so that they just
fail.
Something bad happens with manifest which I haven't looked into (there
are lots of unitialised variable messages).
signal adds a 'Hangup' message from the shell.
Now for the hard part of integrating this back into the build process!
Cheers,
Nick
Nick Glencross
I hope that not too many of you are getting fed up with me going on
about HP-UX. I'm nearly there with having it working with manually
hacked Makefiles etc.
Some tweaks will be needed to the Configure tests (not too many), but
I'd just like to summarise what I believe to be the 'big picture' in a
single email.
I guess the main complexity comes from potentially having 3 C compilers
and 2 linkers. We have:
* The Bundled C compiler: which comes as standard with HP-UX,
is a K&R compiler and is pretty much just intended to generate
a new kernel. It won't be up to compiling parrot.
* A purchased HP C compiler. This is invoked with 'c89' or 'cc -Aa'.
Full C compiler.
* GNU C. Invoked with gcc or perhaps cc
Most of us know and love it.
* HP-UX linker, ld. Not sure if there is a bundled/commercial
version.
* GNU ld.
gcc may be configured to use the HP-UX or GNU linker backends.
HP-UX also has some quirks:
* Shared libraries *must* have execute permissions
* All C files which are to go into shared libraries *must* all be
compiled with 'Position Independent Code' flags
* The system ld doesn't export symbols in the main executable to be
visible to shared libraries by default
* The HP-UX linker does not like -g
* [Some strange alignment rules?]
I'll now try to clearly and concisely summarise the flags that are
required for compilation.
HP-UX C compiler:
cc: c89 or 'cc -Aa'
cc_shared: +z (should be Perl's cccdlflags variable)
ccdlflags(?): -Wl,-E if used with HP-UX ld
GCC compiler:
cc: cc or gcc
cc_shared: -fpic (Perl's cccdlflags variable)
ccdlflags(?): -Wl,-E if used with HP-UX ld
HP-UX ld:
ld: ld
ld_share_flags: -b
GNU ld:
ld: ld
ld_share_flags: -shared
The -Wl,-E (actually -E passed to ld) tells the linker to export symbols
from the resulting binary so that they are available to dynamically
loaded libraries.
-fPIC is possible instead of -fpic, and +Z -z is better than -z (thanks
H.Merijn), but we'd use whatever Perl supplies.
There basically seem to be a few assumptions in the configured system
which are a bit gcc/linux-centric.
For instance, the following files need to be compiled with $(cc_shared)
as they make their way into dynamic libraries:
src/extend.o
src/nci_test.o
dynclasses/*.o
The parrot executable need to be linked with (what I've called above)
$(ccdlflags) for dynclasses to work.
If no one sees any big misunderstands here, I'll press on with my tweaks
over the next few days,
Regards,
Nick
H.Merijn Brand
<ni...@glencros.demon.co.uk> wrote:
> Folks,
>
> I hope that not too many of you are getting fed up with me going on
> about HP-UX. I'm nearly there with having it working with manually
> hacked Makefiles etc.
>
> Some tweaks will be needed to the Configure tests (not too many), but
> I'd just like to summarise what I believe to be the 'big picture' in a
> single email.
>
> I guess the main complexity comes from potentially having 3 C compilers
> and 2 linkers. We have:
>
> * The Bundled C compiler: which comes as standard with HP-UX,
> is a K&R compiler and is pretty much just intended to generate
> a new kernel. It won't be up to compiling parrot.
Because it is not an ANSI compliant compiler, and does not pretend to be one
It's braindead and useless
> * A purchased HP C compiler. This is invoked with 'c89' or 'cc -Aa'.
> Full C compiler.
cc -Ae
> * GNU C. Invoked with gcc or perhaps cc
> Most of us know and love it.
On HP-UX, please use 3.0.4 or newer
For 64bit compiles on HP-UX 11i, do NOT use 4.0.0
> * HP-UX linker, ld. Not sure if there is a bundled/commercial
> version.
there is only one. Please advice HP-UX users to check if they have applied
the most recent patches, which makes a difference
> * GNU ld.
Not available for 32bit builds on HP-UX
> gcc may be configured to use the HP-UX or GNU linker backends.
>
> HP-UX also has some quirks:
>
> * Shared libraries *must* have execute permissions
>
> * All C files which are to go into shared libraries *must* all be
> compiled with 'Position Independent Code' flags
-z +Z for HP C
-fPIC for gcc
> * The system ld doesn't export symbols in the main executable to be
> visible to shared libraries by default
>
> * The HP-UX linker does not like -g
Yes, it does, but not in combination with -O2 or higher
HP C and GNU gcc with -O1 both work fine with -g
> * [Some strange alignment rules?]
>
> I'll now try to clearly and concisely summarise the flags that are
> required for compilation.
>
> HP-UX C compiler:
>
> cc: c89 or 'cc -Aa'
-Ae
> cc_shared: +z (should be Perl's cccdlflags variable)
-z +Z (capital Z)
> ccdlflags(?): -Wl,-E if used with HP-UX ld
-Ae will automatically pass this to ld
> GCC compiler:
>
> cc: cc or gcc
> cc_shared: -fpic (Perl's cccdlflags variable)
-fPIC
> ccdlflags(?): -Wl,-E if used with HP-UX ld
>
> HP-UX ld:
>
> ld: ld
> ld_share_flags: -b
>
> GNU ld:
>
> ld: ld
> ld_share_flags: -shared
>
> The -Wl,-E (actually -E passed to ld) tells the linker to export symbols
> from the resulting binary so that they are available to dynamically
> loaded libraries.
>
> -fPIC is possible instead of -fpic, and +Z -z is better than -z (thanks
> H.Merijn), but we'd use whatever Perl supplies.
The most recent perl5 hints for HP-UX already do so IIRC
> There basically seem to be a few assumptions in the configured system
> which are a bit gcc/linux-centric.
>
> For instance, the following files need to be compiled with $(cc_shared)
> as they make their way into dynamic libraries:
>
> src/extend.o
> src/nci_test.o
> dynclasses/*.o
>
> The parrot executable need to be linked with (what I've called above)
> $(ccdlflags) for dynclasses to work.
>
> If no one sees any big misunderstands here, I'll press on with my tweaks
> over the next few days,
--
Nick Glencross
> Folks,
>
> I hope that not too many of you are getting fed up with me going on
> about HP-UX. I'm nearly there with having it working with manually
> hacked Makefiles etc.
I've nearly finished making my updates. I need some help on one aspect
though...
extend.o and nci_test.o need to be built with $(CC_SHARED) in addition
to the normal flags. What's the most straightforward way of doing this?
I've had a couple of experiments with config/gen/makefiles/root.in, but
don't know what the 'right thing' to do is...
Any help would be appreciated...
Cheers,
Nick
Nick Glencross
> Nick Glencross wrote:
>
> extend.o and nci_test.o need to be built with $(CC_SHARED) in addition
> to the normal flags. What's the most straightforward way of doing this?
> I've had a couple of experiments with config/gen/makefiles/root.in, but
> don't know what the 'right thing' to do is...
Just to clarify, I've got the CC_SHARED variable set in the Makefile,
the help I need is on the actual line to do the compile... Should I just
list these two dependencies explicitly, or is there some clever magic?
Cheers,
Nick
Leopold Toetsch
Have a look at CFLAGS generated from config.../CFLAGS.in. A line:
src/extend.c +{${cc_shared}}
should to the trick.
> Cheers,
>
> Nick
leo
Andy Dougherty
Yup, you were right. A parrot compiled perl Configure.pl --optimize
core dumps with bus alignment errors almost instantly. As just one
example
$ cat t/op/number_2.pasm
set N0, 42.0
set N1, N0
print N1
print "\n"
end
$ /parrot -b -t -d ff -v t/op/number_2.pasm
debug = 0xff
Reading t/op/number_2.pasm
using optimization '0' (0)
Starting parse...
set %s, %s set
set %s, %s set
print %s print
print %s print
end end
emit set N0, 42.0
emit set N1, N0
emit print N1
emit print "\n"
emit end
6 lines compiled.
Running...
0 set N0, 42 - N0=0.000000,
Bus Error(coredump)
However, recompiling with -misalign (a special directive for Sun's compiler):
-misalign
(SPARC, PowerPC) Informs the compiler that data is not
properly aligned and thus very conservative loads and
stores must be used for data, that is, one byte at a
time. Using this option can cause significant perfor-
mance degradation when running the program. If you
compile and link in separate steps, this option must
appear on both the compile and link command lines.
allows that test to pass.
--
Andy Dougherty doug...@lafayette.edu
Nicholas Clark
> it apparently can get more confusing. I don't think we sorted out all the
> possibilities, but Nick C. was the last one fiddling with it.
I didn't actually try to work out all the possibilities. I managed to avoid
the problem by putting the double (or long double) first in the structure,
and all allocations are done in terms of sizeof() structures.
(Which I think is save, and we haven't had problem reports since the change)
Nicholas Clark