Use Cython with OpenMP in Windows

[Yuxiang Wang]

Dear all,

I have a question: how do I use Cython with OpenMP in Windows?

According to this link https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows   I would be better off using MSVC. However, with VC++ 2010 Express Edition, OpenMP support is not included. I also tried to install Windows SDK, but the same error (no omp.h) pops up again.

I don't think I am the first one who ran into this... Could anyone give me a hint?

Thank you!

Shawn

[Yuxiang Wang]

I found this here:

https://mail.python.org/pipermail/cython-devel/2011-March/000227.html

"The free C/C++ compiler in Windows SDK supports OpenMP. This is the system C compiler on Windows. Visual C++ Express is an IDE for beginners and hobbyists. OpenMP on GCC is the same on Windows as on any other platform. Sturla"

However, as I mentioned in the OP, I installed Windows SDK, done `set DISTUTILS_USE_SDK=1`, and `setenv /x64 /release` , it still reports cannot find 'omp.h', because the file does not come with Windows SDK.

-Shawn

[Sturla Molden]

The Windows SDK compiler has OpenMP support, but you might need this patch:

http://www.microsoft.com/en-us/download/details.aspx?id=5687

You will need it unless you have Windows update set to update Visual
Studio.

Also remember to use the SDK version for .NET 3.5 for Python 2.7. Do not
use a newer Windows SDK.

A better solution is to use a recent MinGW (e.g. GCC 4.8.2) than a six year
old MSVC compiler. I recommend TDM-GCC.


Sturla


Yuxiang Wang <wangyux...@gmail.com> wrote:
> I found this here:
>
> <a
> href="https://mail.python.org/pipermail/cython-devel/2011-March/000227.html">https://mail.python.org/pipermail/cython-devel/2011-March/000227.html</a>

>
> "The free C/C++ compiler in Windows SDK supports OpenMP. This is the system
> C compiler on Windows. Visual C++ Express is an IDE for beginners and
> hobbyists. OpenMP on GCC is the same on Windows as on any other platform.
> Sturla"
>
> However, as I mentioned in the OP, I installed Windows SDK, done `set
> DISTUTILS_USE_SDK=1`, and `setenv /x64 /release` , it still reports cannot
> find 'omp.h', because the file does not come with Windows SDK.
>
> -Shawn
>
> On Thursday, May 29, 2014 12:45:36 PM UTC-4, Yuxiang Wang wrote:
>>
>> Dear all,
>>
>> I have a question: how do I use Cython with OpenMP in Windows?
>>
>> According to this link

>> <a
>> href="https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows">https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows</a>

>> I
>> would be better off using MSVC. However, with VC++ 2010 Express Edition,
>> OpenMP support is not included. I also tried to install Windows SDK, but
>> the same error (no omp.h) pops up again.
>>
>> I don't think I am the first one who ran into this... Could anyone give me
>> a hint?
>>
>> Thank you!
>>
>> Shawn
>>
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit <a
> href="https://groups.google.com/d/optout.">https://groups.google.com/d/optout.</a>
>
> ------=_Part_307_18900193.1401390867991
> Content-Type: text/html; charset=UTF-8
> Content-Transfer-Encoding: quoted-printable
>
> <div dir=3D"ltr">I found this here:<div><br></div><div>https://mail.python.=
> org/pipermail/cython-devel/2011-March/000227.html</div><div><br></div><div>=
> "The free C/C++ compiler in Windows SDK supports OpenMP. This is the syste=

> m C compiler on Windows.
>
> Visual C++ Express is an IDE for beginners and hobbyists.
>
> OpenMP on GCC is the same on Windows as on any other platform.
>

> Sturla"</div><div><br></div><div>However, as I mentioned in the OP, I insta=
> lled Windows SDK, done `set DISTUTILS_USE_SDK=3D1`, and `setenv /x64 /relea=
> se` , it still reports cannot find 'omp.h', because the file does not come =
> with Windows SDK.</div><div><br></div><div>-Shawn<br><br><br>On Thursday, M=
> ay 29, 2014 12:45:36 PM UTC-4, Yuxiang Wang wrote:<blockquote class=3D"gmai=
> l_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;=
> padding-left: 1ex;"><div dir=3D"ltr">Dear all,<div><br></div><div>I have a =
> question: how do I use Cython with OpenMP in Windows?</div><div><br></div><=
>> According to this link <a href=3D"https://github.com/cython/cython/wiki=
> /64BitCythonExtensionsOnWindows" target=3D"_blank" onmousedown=3D"this.href=
> =3D'https://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcython%2Fcyth=
> on%2Fwiki%2F64BitCythonExtensionsOnWindows\46sa\75D\46sntz\0751\46usg\75AFQ=
> jCNF8q2LqhQ56FHs_t9dmZ-oueAZpmg';return true;" onclick=3D"this.href=3D'http=
> s://www.google.com/url?q\75https%3A%2F%2Fgithub.com%2Fcython%2Fcython%2Fwik=
> i%2F64BitCythonExtensionsOnWindows\46sa\75D\46sntz\0751\46usg\75AFQjCNF8q2L=
> qhQ56FHs_t9dmZ-oueAZpmg';return true;">https://github.com/cython/<wbr>cytho=
> n/wiki/<wbr>64BitCythonExtensionsOnWindows</a> &amp;nbsp; I would be better off=
> using MSVC. However, with VC++ 2010 Express Edition, OpenMP support is not=
> included. I also tried to install Windows SDK, but the same error (no omp.=
> h) pops up again.</div><div><br></div><div>I don't think I am the first one=
> who ran into this... Could anyone give me a hint?</div><div><br></div><div=
>> Thank you!</div><div><br></div><div>Shawn</div></div></blockquote></div></=
>>
>
> <p></p>
>
> -- <br />
> <br />
> --- <br />
> You received this message because you are subscribed to the Google Groups &amp;=
> quot;cython-users&amp;quot; group.<br />
> To unsubscribe from this group and stop receiving emails from it, send an e=
> mail to <a href=3D"mailto:cython-users...@googlegroups.com">cython=
> -users+un...@googlegroups.com</a>.<br />
> For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
> s://groups.google.com/d/optout</a>.<br />
>
> ------=_Part_307_18900193.1401390867991--

[Yuxiang Wang]

Thank you Strula for your response!

1) Unfortunately the team is using Python 3.3 with 64 bits machines... Do you have any idea on which patch should I apply for MSC  v. 1600? Or is there an equivalent at all, or... Is that later SDKs does not come with OpenMP support?

2) I'd love to move to gcc, but again - mingw is 32 bits, and the link  https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows said not to use mingw-64, so... That's why I didn't.

Thanks again and look forward to your reply,

Shawn

> mail to <a href=3D"mailto:cython-users+unsub...@googlegroups.com">cython=

[Sturla Molden]

Yuxiang Wang <wangyux...@gmail.com> wrote:

> 1) Unfortunately the team is using Python 3.3 with 64 bits machines... Do
> you have any idea on which patch should I apply for MSC v. 1600? Or is
> there an equivalent at all, or... Is that later SDKs does not come with
> OpenMP support?

I have no experience with that.

> 2) I'd love to move to gcc, but again - mingw is 32 bits, and the link

> <a
> href="https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows">https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows</a>

> said
> not to use mingw-64, so... That's why I didn't.

That is outdated information.

MinGW-w64 is not in beta. They have released version 3.0 of the runtime.
The compiler itself is GCC 4.8.2.

http://mingw-w64.sourceforge.net

TDM-GCC, which is also 64-bit, is also not in beta:

http://tdm-gcc.tdragon.net

If the linker fails on missing libpython33.a, note that it is the same as
python33.lib.


Sturla

[Ian Bell]

How can we tell Python which gcc version to use on windows?  I have no less than 5 different versions of gcc for different reasons, if we install TDM - looks interesting - how do we tell python to use it?

I know you call the build command like "python setup.py build --compiler=mingw32 install" to force GCC on windows

Ian

 

Sturla

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Sturla Molden]

Ian Bell <ian.h...@gmail.com> wrote:

> How can we tell Python which gcc version to use on windows?

This is an MSVC problem and irrelevant for GCC. GCC can link any C runtime
on Windows, and Python (distutils) will pick the correct C runtime. It is
MSVC which is restricted to a specific C runtime, and requires that you use
the same version which was used to build Python. GCC does not have this
problem. But if you don't use distutils, you will have to manually select
the correct C runtime.

> I have no less
> than 5 different versions of gcc for different reasons, if we install TDM -
> looks interesting - how do we tell python to use it?

Whatever gcc binary is called in the terminal is used. Just make sure only
the GCC you want to use is in the PATH.

Sturla

[Ian Bell]

Great, thanks for clarifying something I had always wondered about.  So if you wanted to SET the gcc version to use, you would just munge the PATH prior to calling the setup.py script, right?  Sounds good to me.

[Sturla Molden]

Ian Bell <ian.h...@gmail.com> wrote:

> Great, thanks for clarifying something I had always wondered about. So if
> you wanted to SET the gcc version to use, you would just munge the PATH
> prior to calling the setup.py script, right? Sounds good to me.

Yes.

And if the GCC is a MinGW (including MinGW-w64) you should use

--compiler=mingw32

(Even 64-bit GCC is called mingw32 in distutils)



Sturla

[Yuxiang Wang]

Hi Strula,

First of all, oh my god thank you so much. Being able to use gcc will save me TONS of trouble. Thank you so much!!

Secondly, I strongly recommend to document what you have just said - update that wiki page, note that 1) GCC is usable too; 2) even mingw-w64 is called mingw32 in the python distutils; and 3) that python33.lib is essentially the same with libpython33.a. In that case people can use OpenMP and much more others issues with MSVC too!

Lastly, a quick question - how do I make libpython33.a? Do I just copy and paste and rename it? I tried pexports and it said "could not load PE image".

-Shawn

[Sturla Molden]

Yuxiang Wang <wangyux...@gmail.com> wrote:
> Hi Strula,
>
> First of all, oh my god thank you so much. Being able to use gcc will save
> me TONS of trouble. Thank you so much!!
>
> Secondly, I strongly recommend to document what you have just said - update
> that wiki page, note that 1) GCC is usable too; 2) even mingw-w64 is called
> mingw32 in the python distutils; and 3) that python33.lib is essentially
> the same with libpython33.a. In that case people can use OpenMP and much
> more others issues with MSVC too!
>
> Lastly, a quick question - how do I make libpython33.a? Do I just copy and
> paste and rename it? I tried pexports and it said "could not load PE image".

I think you should be able to copy and paste it, though I haven't tried (I
got libpython27.a from Enthought). MinGW is able to link .lib files from
MSVC, and the format of .lib and .a files is the same. But distutils will
not pass -python33.lib instead of -lpython33 to the gcc linker.

You could try to create libpython33.a manually:

Creeate a folder and cd to it.

> copy C:\Python33\libs\python33.dll

> gendef python33.dll

> dlltool --dllname python33.dll --def python33.def --output-lib
libpython33.a

> copy libpython33.a C:\Python33\libs\libpython33.a

And now:

> python setup.py build_ext --compiler=mingw32 -DMS_WIN64


Sturla

[Yuxiang Wang]

Thanks again Strula!

I did try copy and paste the python33.lib as libpython33.a and it did not work (still popping out those "undefined reference to '___imp__....'" error). I will try the gendef way... I sincerely hope that the day does come soon, when GCC is the standard way in Windows for Cython, and most importantly, well documented :)

Thanks again Strula!

-Shawn

[Sturla Molden]

Yuxiang Wang <wangyux...@gmail.com> wrote:

> sincerely hope that the day does
> come soon, when GCC is the standard way in Windows for Cython, and most
> importantly, well documented :)

Personally I'd love to see support for clang on Windows in distutils. clang
is a fantastic compiler, and has even better MSVC compatibility than MinGW.


I suspect the only change we need is to replace the call to cl.exe with
cl-clang.exe. It would just mean a very simple subclass of MSVCCompiler,
just changing self.cc:

http://hg.python.org/releasing/3.4.1/file/ea310ca42bb2/Lib/distutils/msvccompiler.py

Sturla

[Yuxiang Wang]

Strula,

That'll be really nice! I have been using Numba with LLVM and really liked it too. 

Also, an update in case someone in the future found this thread:

libpython33.a problem can be solved by using this installer: http://www.lfd.uci.edu/~gohlke/pythonlibs/#libpython

-Shawn

[Yuxiang Wang]

Sorry for forgetting to mention this...

Also, for those who would prefer to continue using MSVC, the VS 2010 Professional supports openmp. There is a chance that you can get it free on DreamSparks, as long as you can find an email address ending with .edu. Do remember to change the -fopenmp to /openmp for the compiler though!

-Shawn