AIX support (test) patch

[Aleksander Czajczynski]

Hello,

This is a test version of patch adding AIX platform support to Harbour
compilation using GCC. Please do not include it until linker issues are
solved. When -lhbextern is passed to AIX ld utility (like in current
make scripts and hbmk2), even if the application isn't requesting it,
all the symbols are included in the resulting executable.

I've attached the patch itself and results of hbtest and speedtst.
Tested on AIX 6.1 virtual partition running under AIX VIOS hypervisor.
64-bit compilation is also possible and works, but using hbmk2 later is
tricky because -maix64 options have to be included all the time with GCC.

I'd like to hear comments, what else to include or test.

Some errors left:

first some: ../../../gzlib.c:518: warning: visibility attribute not
supported in this configuration; ignored
then a series of while linking:
ld: 0711-768 WARNING: Object
../../../../../src/3rd/zlib/obj/aix/gcc/gzread_dyn.o, section 1,
function .gz_error:
The branch at address 0x4fc is not followed by a recognized no-op
or TOC-reload instruction. The unrecognized instruction is
0x817F0058.

System requirements:

1. libm math library isn't installed on AIX by default, which is bad.
(via www.warmetal.nl/aix61)
Oracle has a few requirements when installed on an AIX box, and one of
them is not installed by default: bos.adt.libm. The package can be
installed using the NIM server:
bash-3.2# lslpp -l bos.adt.libm
lslpp: Fileset bos.adt.libm not installed.

bash-3.2# nimclient -l -L ms-soe6
...<cut>...
lpp_6100_04_03 lpp_source
...<cut>...
bash-3.2# nimclient -o allocate -a lpp_source=lpp_6100_04_03
bash-3.2# nimclient -l -c resources ms-soe6
lpp_6100_04_03 lpp_source
bash-3.2# nimclient -o cust -a lpp_source=lpp_6100_04_03 -a
filesets=bos.adt.libm
...<cut>...

bash-3.2# lslpp -l bos.adt.libm
Fileset Level State Description

----------------------------------------------------------------------------
Path: /usr/lib/objrepos
bos.adt.libm 6.1.4.0 APPLIED Base Application
Development
Math Library

2. when installing a GCC package install also a libgcc(_s) for Harbour
applications using
shared libraries

3. default gmake version distributed with AIX 6.1 is 3.80, upgrade to
3.81 is recommended

Regards, Aleksander Czajczyński

[Viktor Szakáts]

Hi Aleksander,

Great job.

I hope those missing bits will be cleared and we can commit this, 

AIX support was badly missing from Harbour since long.

Viktor

[Przemysław Czerpak]

On Wed, 28 Sep 2011, Aleksander Czajczynski wrote:

Hi Aleksander,

> This is a test version of patch adding AIX platform support to
> Harbour compilation using GCC. Please do not include it until linker
> issues are solved. When -lhbextern is passed to AIX ld utility (like
> in current make scripts and hbmk2), even if the application isn't
> requesting it, all the symbols are included in the resulting
> executable.

Your tests shows that it happens with all libraries, i.e. functions
from hbdebug are attached to final binaries and speedtst reports it
so it seems to be global problem.
Saml session with google shows that it's known problem also in native
xIC C compiler. It's exploited by code using startup initialization
and AIX linkers are not celver enough to exclude modules with CTOR
sections which are not referenced at all.
We have such code for registering PRG modules in global symbol table
and also for some optional subsystems like RDDs, GTs, ...
Maybe it was fixed in last years. I've found some messages about it
from 2004.

> I've attached the patch itself and results of hbtest and speedtst.
> Tested on AIX 6.1 virtual partition running under AIX VIOS
> hypervisor. 64-bit compilation is also possible and works, but using
> hbmk2 later is tricky because -maix64 options have to be included
> all the time with GCC.

I think we should try to add support for siwthing between 32 and 64 bit
mode in HBMK2. Many platforms has such funcitonality now.
Viktor what do you think about it?

> I'd like to hear comments, what else to include or test.

Very good job, thank you for your time.
Have you try to build Harbour using native xIC compiler?

> Some errors left:
> first some: ../../../gzlib.c:518: warning: visibility attribute not
> supported in this configuration; ignored

-DNO_VIZ disable it,
You can change in src/3rd/zlib/Makefile:
ifneq ($(filter $(HB_COMPILER),mingw mingw64 mingwarm cygwin djgpp),)
HB_CFLAGS += -DNO_VIZ
endif
to:
ifneq ($(filter $(HB_COMPILER),mingw mingw64 mingwarm cygwin djgpp),)
HB_CFLAGS += -DNO_VIZ
else
ifneq ($(filter $(HB_PLATFORM),aix),)
HB_CFLAGS += -DNO_VIZ
endif
endif

> then a series of while linking:
> ld: 0711-768 WARNING: Object
> ../../../../../src/3rd/zlib/obj/aix/gcc/gzread_dyn.o, section 1,
> function .gz_error:
> The branch at address 0x4fc is not followed by a recognized no-op
> or TOC-reload instruction. The unrecognized instruction is
> 0x817F0058.

This is also known problem. It happens when some functions inlined
by compiler are exported from shared librart explicitly or implicitly
through the -bexpall linker option. Look at:
https://www-304.ibm.com/support/docview.wss?uid=swg21443810
for details and possible workarounds.

best regards,
Przemek

[Viktor Szakáts]

Sent from my iPad

Hbmk2 already has some basic support for -cpu option
which could be nice generic way to solve this problem also.
OTOH -m32/-m64 is very strange solution so I'd avoid it
in hbmk2.

(if we want to think of darwin too, it will need some extra
tricks due to support for multi cpu targets (unibin), though
for me it's not personally important and can be solved with
custom flags nicely)

Viktor

[Aleksander Czajczynski]

Hello Przemysław!

On 2011-09-29 01:03, Przemysław Czerpak wrote:

Your tests shows that it happens with all libraries, i.e. functions
from hbdebug are attached to final binaries and speedtst reports it
so it seems to be global problem.
Saml session with google shows that it's known problem also in native
xIC C compiler. It's exploited by code using startup initialization
and AIX linkers are not celver enough to exclude modules with CTOR
sections which are not referenced at all.
We have such code for registering PRG modules in global symbol table
and also for some optional subsystems like RDDs, GTs, ...
Maybe it was fixed in last years. I've found some messages about it
from 2004.
  

I'm all lost with this linker, being so used to MSVC linker which optimizes out everything, this one is totally different story. For now i found out a workaround which look like this:
Typical "hbmk2 -st speedtst.prg" outputs: 6916722 Sep 29 01:53 speedtst
With my dirty workaround (see speedtstlink5.sh attachment) it's: 1925423 Sep 29 01:58 speedtst
Don't know about the newest AIX 7.1 release, all this was on 6.1 (dated November 9, 2007). Alternatively we could try gnu-ld, but i'm not sure about stability with this option.

Very good job, thank you for your time.
Have you try to build Harbour using native xIC compiler?
  

As it's proprietary and paid one, right now I don't have access to IBM XL C/C++ compiler on machines available to me. But i'll think about it.

Some errors left:
first some: ../../../gzlib.c:518: warning: visibility attribute not
supported in this configuration; ignored
    

-DNO_VIZ disable it,
You can change in src/3rd/zlib/Makefile:
   ifneq ($(filter $(HB_COMPILER),mingw mingw64 mingwarm cygwin djgpp),)
      HB_CFLAGS += -DNO_VIZ
   endif
to:
   ifneq ($(filter $(HB_COMPILER),mingw mingw64 mingwarm cygwin djgpp),)
      HB_CFLAGS += -DNO_VIZ
   else
      ifneq ($(filter $(HB_PLATFORM),aix),)
         HB_CFLAGS += -DNO_VIZ
      endif
   endif
  

I'll try that, thank you.

then a series of while linking:
ld: 0711-768 WARNING: Object
../../../../../src/3rd/zlib/obj/aix/gcc/gzread_dyn.o, section 1,
function .gz_error:
        The branch at address 0x4fc is not followed by a recognized no-op
        or TOC-reload instruction. The unrecognized instruction is
0x817F0058.
    

This is also known problem. It happens when some functions inlined
by compiler are exported from shared librart explicitly or implicitly
through the -bexpall linker option. Look at:
   https://www-304.ibm.com/support/docview.wss?uid=swg21443810
for details and possible workarounds.
  

First i'll try the -DNO_VIZ option for zlib, we'll see.

Best regards, Aleksander Czajczyński

[Aleksander Czajczynski]

Some errors left:
first some: ../../../gzlib.c:518: warning: visibility attribute not
supported in this configuration; ignored
    

-DNO_VIZ disable it,
You can change in src/3rd/zlib/Makefile:
   ifneq ($(filter $(HB_COMPILER),mingw mingw64 mingwarm cygwin djgpp),)
      HB_CFLAGS += -DNO_VIZ
   endif
to:
   ifneq ($(filter $(HB_COMPILER),mingw mingw64 mingwarm cygwin djgpp),)
      HB_CFLAGS += -DNO_VIZ
   else
      ifneq ($(filter $(HB_PLATFORM),aix),)
         HB_CFLAGS += -DNO_VIZ
      endif
   endif
  

I'll try that, thank you.

then a series of while linking:
ld: 0711-768 WARNING: Object
../../../../../src/3rd/zlib/obj/aix/gcc/gzread_dyn.o, section 1,
function .gz_error:
        The branch at address 0x4fc is not followed by a recognized no-op
        or TOC-reload instruction. The unrecognized instruction is
0x817F0058.
  

First i'll try the -DNO_VIZ option for zlib, we'll see.

With -DNO_VIZ no sign of these warnings.

I've done mentioned AIX linker workaround in makefiles and now Harbour compiles cleanly. If this (ugly as hell) solution is going to be accepted, diff file is attached. Something similar should be done inside hbmk2 sources also in such case, i could also do this change later in some spare time.

Regards, Aleksander Czajczyński