How to use libraries compiled for PPC

[abhishek bhat]

Hi
I want to be able to use C libraries that have been compiled for
PowerPC. I am using Cython on a Windows machine and have my .pyx
and .pyd.

The problem is, if I link these libraries that have been built for a
PowerPC, the Cython to C conversion will not work. What is the way to
get around this problem? Will I have to build all the libraries for
x86 first?

[Robert Bradshaw]

Yes, you have to build the libraries on the appropriate platform to be
able to link against/use them. However, I'm confused, you have Windows
on PowerPC?

- Robert

[abhishek bhat]

This is for an embedded setup where host & target are different
machines(arch). Development is done on a Win machine and code is
compiled using a cross-compiler for PPC.

Can I pass any flags to the Cython compiler to indicate that the
libraries being linked have been compiled for PPC?

Is it possible to compile Cython for PPC?

Re-compiling the libraries for x86 is not practical and I need to find
an alternate arrangement if this has to work. Are there any other
solutions ppl can think of?


On Sep 23, 11:44 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:

[Robert Bradshaw]

Cython itself produced platform-agnostic c. I don't know how much (if
any) support distutils has for cross-compiling; You'll probably need
to run Cython manually to produce the .c file and then run your c
compiler manually to do the cross-compile.

[Robert Bradshaw]

That all depends on what Python you're using to run this script.

Basically, there are three stages:

1) Compile the .pyx file to .c (Cython)
2) Compile the .c file to .o
3) Link the .o file and associated libraries to produce the .so (or
.pyd) extension module.

Distutils performs steps 2 and 3, using the compiler settings and
flags that were used to build the python executable you are using to
run the setup.py file. If this is a Windows x86 python, the produced
.o file will not link against the libxxx PPC library (nor would it be
usable in the PPC environment). Instead, run "cython queue.pyx" to
produce queue.c (step 1) and manually cross-compile to link against
the PPC libraries (including the PPC libpython2.x).

On Fri, Sep 23, 2011 at 1:24 PM, abhishek bhat <wimpy...@gmail.com> wrote:
> Robert
> I have not tried it but you are saying that if I use a disutils script
> like below and link my Cython code to a library called libxxx that has
> been previously compiled for PPC, I will have no issues generating the
> C code. Right?
>
> If that is correct then that should be sufficient for me to proceed
> since I can then cross-compile the generated C code for PPC as you
> have suggested.
>
> from distutils.core import setup
> from distutils.extension import Extension
> from Cython.Distutils import build_ext
>
> setup(
>    cmdclass = {'build_ext': build_ext},
>    ext_modules = [Extension("queue", ["queue.pyx"],
>                                libraries=["libxxx"])
>    ]
>
>
> )
>
> On Sep 23, 1:09 pm, Robert Bradshaw <rober...@math.washington.edu>

[Vitja Makarov]

2011/9/24 Robert Bradshaw <robe...@math.washington.edu>:

I'm cross-compiling Cython application for ARM in my project.
For cross-compilation I prefer to use Makefile (I didn't try to use
distutils, but I think it could have some problems with
cross-compilation).

My Makefile looks like this:

# Target arch GCC prefix, in your case should be something like ppc64-linux-
CROSS_COMPILE = arm-unknown-linux-uclibcgnueabi-

CC = $(CROSS_COMPILE)gcc
AR = $(CROSS_COMPILE)ar
RANLIB = $(CROSS_COMPILE)ranlib
STRIP = $(CROSS_COMPILE)strip

CYTHON = cython
CYTHON_FLAGS = -a -Wextra
CFLAGS += -g3 -O2 -I$(PWD)/../stage/usr/include/python2.7
LDFLAGS = -shared -fPIC

all: foo.so

foo.so: foo.c bar.c other <other dependencies>
$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.c,$^) -o $@

%.c: %.pyx
$(CYTHON) $(CYTHON_FLAGS) $< -o $@

--
vitja.

[abhishek bhat]

Robert
I have not tried it but you are saying that if I use a disutils script
like below and link my Cython code to a library called libxxx that has
been previously compiled for PPC, I will have no issues generating the
C code. Right?

If that is correct then that should be sufficient for me to proceed
since I can then cross-compile the generated C code for PPC as you
have suggested.

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("queue", ["queue.pyx"],
libraries=["libxxx"])
]


)

On Sep 23, 1:09 pm, Robert Bradshaw <rober...@math.washington.edu>
wrote:

[Chris.Barker]

On 9/23/11 1:09 PM, Robert Bradshaw wrote:
> I don't know how much (if
> any) support distutils has for cross-compiling;

Well, on OS-X distutils will cross compile for PPC and Intel (only one
is cross compile, of course).

That being said, the point of distutils is to build extensions that
"match" the python binary you're building with. In the OS-X case, it
compiles both platforms if the python binary you run it with was
compiled for both platforms.

Do you have a dual-platfrom Python (I doubt it, I think that's a pretty
unique OS-X concept)

so - you are probably better off building with some other method.

If you still want to use distutils, IO"mve gone in and hacked:

lib/python2.7/config/Makefile

to remove things to prevent cross-compilation, so you could probably
just as easily add stuff (on OS-X, it would be -arch PPC)

Curious though -- do you have Python running on your embedded PPC app?
You'd want to use the abive makefile from that build.

Maybe it would be easier to do all this with a VM that emulated PPC, if
there is one.

HTH,
-Chris

You'll probably need
> to run Cython manually to produce the .c file and then run your c
> compiler manually to do the cross-compile.
>
> On Fri, Sep 23, 2011 at 1:02 PM, abhishek bhat<wimpy...@gmail.com> wrote:

>> This is for an embedded setup where host& target are different

>> machines(arch). Development is done on a Win machine and code is
>> compiled using a cross-compiler for PPC.
>>
>> Can I pass any flags to the Cython compiler to indicate that the
>> libraries being linked have been compiled for PPC?
>>
>> Is it possible to compile Cython for PPC?
>>
>> Re-compiling the libraries for x86 is not practical and I need to find
>> an alternate arrangement if this has to work. Are there any other
>> solutions ppl can think of?
>>
>>
>> On Sep 23, 11:44 am, Robert Bradshaw<rober...@math.washington.edu>
>> wrote:
>>> On Fri, Sep 23, 2011 at 11:38 AM, abhishek bhat<wimpybob...@gmail.com> wrote:
>>>> Hi
>>>> I want to be able to use C libraries that have been compiled for
>>>> PowerPC. I am using Cython on a Windows machine and have my .pyx
>>>> and .pyd.
>>>
>>>> The problem is, if I link these libraries that have been built for a
>>>> PowerPC, the Cython to C conversion will not work. What is the way to
>>>> get around this problem? Will I have to build all the libraries for
>>>> x86 first?
>>>
>>> Yes, you have to build the libraries on the appropriate platform to be
>>> able to link against/use them. However, I'm confused, you have Windows
>>> on PowerPC?
>>>
>>> - Robert

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Robert Bradshaw]

On Mon, Sep 26, 2011 at 8:32 PM, Abhishek Bhat <wimpy...@gmail.com> wrote:
> Robert
> I think I figured part of my question. I built Python 3.2 on a Ubuntu box
> and saw that it generated the libpython32m.a and Python.exe so I have the
> DLL.
>
> Is there something in the C code generated by Cython that needs the Python
> DLL?

Yes, Cython generates an extension module that links against the
Python DLL and is meant to be loaded into a running Python session. It
does not translate Python into a standalone C program (though you can
link against the Python DLL statically to get the same effect).

> On Mon, Sep 26, 2011 at 7:05 PM, Abhishek Bhat <wimpy...@gmail.com>
> wrote:
>>
>> Robert
>> Thanks for your help.
>>
>> I did not quite understand why I need to link the libpython after I
>> converted the code to C.

>>
>> >> Instead, run "cython queue.pyx" to produce queue.c (step 1) and
>> >> manually cross-compile to link against the PPC libraries (including the PPC
>> >> >>libpython2.x).
>>

>> Also I could not figure how to generate the libpythonxx.dll (I am using
>> 3.2) for PPC. I know when py2exe is used on Windows it generates a DLL but
>> that can not be used for PPC obviously. How can I do it for PPC?
>>
>> I am trying to evaluate if ctypes would be sufficient for my purpose or I
>> need to Cythonize my code for better performance.
>>
>> Thanks
>> Abhishek
>>
>> On Fri, Sep 23, 2011 at 3:16 PM, Robert Bradshaw
>> <robe...@math.washington.edu> wrote:
>>>
>>> There's a .bat script in the bin directory, or you can do "python.exe
>>> cython.py [arguments]"
>>>
>>> On Fri, Sep 23, 2011 at 2:56 PM, abhishek bhat <wimpy...@gmail.com>
>>> wrote:
>>> > Robert
>>> > How to I access the Cython compiler on Windows? Where is the
>>> > cython.exe?
>>> >
>>> > On Sep 23, 1:33 pm, Robert Bradshaw <rober...@math.washington.edu>

>>> > wrote:
>>> >> That all depends on what Python you're using to run this script.
>>> >>
>>> >> Basically, there are three stages:
>>> >>
>>> >> 1) Compile the .pyx file to .c  (Cython)
>>> >> 2) Compile the .c file to .o
>>> >> 3) Link the .o file and associated libraries to produce the .so (or
>>> >> .pyd) extension module.
>>> >>
>>> >> Distutils performs steps 2 and 3, using the compiler settings and
>>> >> flags that were used to build the python executable you are using to
>>> >> run the setup.py file. If this is a Windows x86 python, the produced
>>> >> .o file will not link against the libxxx PPC library (nor would it be
>>> >> usable in the PPC environment). Instead, run "cython queue.pyx" to
>>> >> produce queue.c (step 1) and manually cross-compile to link against
>>> >> the PPC libraries (including the PPC libpython2.x).
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>

>>> >> On Fri, Sep 23, 2011 at 1:24 PM, abhishek bhat <wimpybob...@gmail.com>