guidelines for using intel compilers
Alex van Houten
Are there some general guidelines for using the Intel Compiler when compiling
cython modules?
Windows XP is my OS.
Found a link
http://groups.google.com/group/cython-users/browse_thread/thread/5a55b0dd9b7d01ab
that says one can set an environment variable CC to "icc".
But that does not seem to make any difference. It just compiles (coorectly)
using gcc.
Tried the different approach
cython mymodule.pyx-->mymodule.c
and then replace "gcc" in this
gcc -O3 -I some_path -I some_other_path_for_Numpy -o mymodule mymodule.c
-lpython26 -lm
expression by "icl", with the path environment variable set, of course.
got some errors:
icl: command line warning #10006: ignoring unknown option '/lpython26'
icl: command line warning #10006: ignoring unknown option '/lm'
....pyconfig.h(68): catastrophic error: could not open source file "io.h"
Any way to get it working with "python setup.py build_ext --inplace" ?
Thanks,
Alex
Lisandro Dalcin
> Hi,
>
> Are there some general guidelines for using the Intel Compiler when compiling
> cython modules?
>
> Windows XP is my OS.
>
That's explain it all... Python distutils supports MSVC and MinGW, and
no more...
> Found a link
> http://groups.google.com/group/cython-users/browse_thread/thread/5a55b0dd9b7d01ab
> that says one can set an environment variable CC to "icc".
> But that does not seem to make any difference. It just compiles (coorectly)
> using gcc.
>
These things are for Linux/OS X (actually, any POSIX system)
> Tried the different approach
> cython mymodule.pyx-->mymodule.c
> and then replace "gcc" in this
> gcc -O3 -I some_path -I some_other_path_for_Numpy -o mymodule mymodule.c
> -lpython26 -lm
>
> expression by "icl", with the path environment variable set, of course.
> got some errors:
> icl: command line warning #10006: ignoring unknown option '/lpython26'
> icl: command line warning #10006: ignoring unknown option '/lm'
>
These are just warnings...
> ....pyconfig.h(68): catastrophic error: could not open source file "io.h"
>
Ups!! This one looks bad... Is that an specific MS header? Perhaps
when using icc you have to add some extra include dirs...
BTW, the -lpython25 -lm lines are likely MinGW specific.
>
> Any way to get it working with "python setup.py build_ext --inplace" ?
>
You could try to drop a "intelcompiler.py" files in distutils,
implementing appropriate methods, and register it in ccompiler.py,
next you should be able to "python setup.py build --compiler=intel
build_ext --inplace".
--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
Alex van Houten
> That's explain it all... Python distutils supports MSVC and MinGW, and
> no more...
>
> >
>
> These things are for Linux/OS X (actually, any POSIX system)
Should I conclude that this hasn't been tried yet on a Windows OS?
Alex.
Lisandro Dalcin
No, what I say is that you can the redefinition of compiler commands
and compiler flags using environment variables only works for POSIX
systems, for Windows the things are implemented differently in Python
distutils, you cannot set CC=something and change the compiler.
Getting ICC to work should be very doable, but requires some coding,
some (very little) knol about distutils internals, and knol about how
to build DLL's with ICC. I know distutils, Do you know ICC? In such
case I could teach you how to hack distutils...
Alex van Houten
> > Should I conclude that this hasn't been tried yet on a Windows OS?
> >
>
> No, what I say is that you can the redefinition of compiler commands
> and compiler flags using environment variables only works for POSIX
> systems, for Windows the things are implemented differently in Python
> distutils, you cannot set CC=something and change the compiler.
>
> Getting ICC to work should be very doable, but requires some coding,
> some (very little) knol about distutils internals, and knol about how
> to build DLL's with ICC. I know distutils, Do you know ICC? In such
> case I could teach you how to hack distutils...
>
Thanks,
I don't actually know icc, but on the other hand I have just been using gcc in
the most basic
way until now (for compiling Cython modules) and I would have to look up its
manual before doing anything more fancy.
My goal is just to compare speed.
Allegedly icc (=icl?) should be able to speed up some code by a factor 2.
This is worth the effort.
What I will do is try to figure out how to replace the adverbs in
gcc -O3 -I some_path -I some_other_path_for_Numpy -o mymodule mymodule.c
-lpython26 -lm
for icc
Any teaching is most welcome!
Cheers,
Alex.
Alex van Houten
> > ....pyconfig.h(68): catastrophic error: could not open source file "io.h"
> >
>
> Ups!! This one looks bad... Is that an specific MS header? Perhaps
> when using icc you have to add some extra include dirs...
>
Ok, I got one step further in compiling cython generated C code with the Intel
compiler (icl) on Windows.
The path to the Python header files (the include dir) is of course added when
compiling. That directory contains "pyconfig.h" which requests "io.h", which is
a MinGW file. If one adds "io.h" to the Python header files include directory
(bit risky perhaps), it will subsequently crash
"...Python_dir\include\io.h(14): catastrophic error: could not open source file
"_mingw.h"
#include <_mingw.h>
.....compilation aborted "
If one adds the path to "...MinGW\include" it will still crash: "
C:\MinGW\include\sys/types.h(21): catastrophic error: could not open source file
"stddef.h"
#include <stddef.h>
.....compilation aborted "
Apparently "stddef.h" is missing in C:\MinGW\include\sys.
But the real problem is that "pyconfig.h" is so intimately connected to
MinGW while I am compiling with icl. Any workaround?
Alex.
mark florisson
Well of course it's tied, it wants to build everything with the same
compiler and flags to make it compatible. Wouldn't it work if you
compile python with icc and then use plain distutils? (I haven't
tried)
Alex van Houten
> Well of course it's tied, it wants to build everything with the same
> compiler and flags to make it compatible. Wouldn't it work if you
> compile python with icc and then use plain distutils? (I haven't
> tried)
>
>
Yes, that might work, but it seems quite an effort.
Would that also involve compiling Numpy and Scipy from source?
I don't think I want to go down that alley.
Why is it so much easier to compile Cython code with an Intel compiler
on Linux systems? I presume because gcc is the "native" Linux compiler.
Alex.
mark florisson
> mark florisson <markflorisson88 <at> gmail.com> writes:
>
>
>> Well of course it's tied, it wants to build everything with the same
>> compiler and flags to make it compatible. Wouldn't it work if you
>> compile python with icc and then use plain distutils? (I haven't
>> tried)
>>
>>
> Yes, that might work, but it seems quite an effort.
> Would that also involve compiling Numpy and Scipy from source?
> I don't think I want to go down that alley.
Well, just pip install should work if icc works.
Alex van Houten
> Well, just pip install should work if icc works.
Is it that easy? How would you install an Intel compiled Python?
just "pip install python26" or something?
But how do you tell pip to install the Intel compiled Python
instead of the gcc compiled Python?
Alex.
mark florisson
... no. You compile on a decent OS (i.e., not windows) by setting CC
to icc, and then you install setuptools, then pip, and then you use
pip install to install your favourite packages.
pip is for python packages, it needs to be installed for a certain
python environment first. You have permission to read documentation
for respective packages and google around for solutions, we don't
really do hand holding. A quick search found this for me:
http://software.intel.com/en-us/forums/showthread.php?t=72811
Alex van Houten
Thanks. That sounds like a nice solution!
We develop applications that will run on Windows, unfortunately.
Since our customers will use that OS.
But I can do all development on Linux, if I want to.
So I could use PyInstaller.
According to these release notes
http://www.pyinstaller.org/browser/tags/1.4/doc/CHANGES.txt
"
Add support for cross-compilation: PyInstaller is now able to
build Windows executables when running under Linux. See documentation
for more details.
"
Alex
mark florisson
pip should actually work just fine on windows. Note that pip is used
to build and install *other* (third party) packages, it cannot be used
to build standalone executables for your app.