building 64bit cython extensions on windows7 with mingw
Yoav Goldberg
I am not a windows user.
And I am having some trouble building my cython extension for a 64bit
windows7 machine (this may well be a python / mingw question).
It starts with many warning which I did not have on linux, and then
complains about " skipping incompatible c:\Python27\libs/libpython27.a
when searching for -lpython27" and hell brakes loose.
Any idea how to make this work?
Thanks,
Here's the output I get:
=======
E:\yoav\shared\ml>python setup.py build_ext -c mingw32
running build_ext
skipping 'ml.c' Cython extension (up-to-date)
building 'ml' extension
C:\MinGW64\bin\gcc.exe -mno-cygwin -mdll -O -Wall
-Ic:\Python27\include -Ic:\Python27\PC -c ml.c -o
build\temp.win-amd64-2.7\Release\ml.o
ml.c: In function '__pyx_f_2ml_31MultipleVectorsMulticlassParams_tick':
ml.c:11229:3: warning: statement with no effect
ml.c: In function '__Pyx_RaiseArgtupleInvalid':
ml.c:14964:9: warning: unknown conversion type character 'z' in format
ml.c:14964:9: warning: format '%s' expects type 'char *', but argument
5 has type 'Py_ssize_t'
ml.c:14964:9: warning: unknown conversion type character 'z' in format
ml.c:14964:9: warning: too many arguments for format
ml.c: In function '__Pyx_RaiseNeedMoreValuesError':
ml.c:15081:18: warning: unknown conversion type character 'z' in format
ml.c:15081:18: warning: format '%s' expects type 'char *', but
argument 3 has type 'Py_ssize_t'
ml.c:15081:18: warning: too many arguments for format
writing build\temp.win-amd64-2.7\Release\ml.def
creating build\lib.win-amd64-2.7
C:\MinGW64\bin\gcc.exe -mno-cygwin -shared -s
build\temp.win-amd64-2.7\Release\ml.o
build\temp.win-amd64-2.7\Release\ml.def -Lc:\Python27\libs
-Lc:\Python27\PCbuild\amd64 -lpython27 -lmsvcr90 -o
build\lib.win-amd64-2.7\ml.pyd
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.5.1/../../../../x86_64-w64-mingw32/bin/ld.exe:
skipping incompatible c:\Python27\libs/libpython27.a when searching
for -lpython27
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.5.1/../../../../x86_64-w64-mingw32/lib/libmingw32.a(lib64_libmingw32_a-mingw_helpers.o):
In function `_decode_pointer':
c:/crossdev/src/mingw-w64-svn/mingw-w64-crt/crt/mingw_helpers.c:20:
multiple definition of `_decode_pointer'
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.5.1/../../../../x86_64-w64-mingw32/lib/libmsvcr90.a(dkbgs00301.o):(.text+0x0):
first defined here
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.5.1/../../../../x86_64-w64-mingw32/lib/libmingw32.a(lib64_libmingw32_a-mingw_helpers.o):
In function `_encode_pointer':
c:/crossdev/src/mingw-w64-svn/mingw-w64-crt/crt/mingw_helpers.c:26:
multiple definition of `_encode_pointer'
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.5.1/../../../../x86_64-w64-mingw32/lib/libmsvcr90.a(dkbgs00311.o):(.text+0x0):
first defined here
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0xb): undefined
reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x22): undefined
reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x53): undefined
reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0xd0): undefined
reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0xfd): undefined
reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x18c): more
undefined references to `__imp__Py_NoneStruct' follow
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x58f): undefined
reference to `__imp__PyThreadState_Current'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x669): undefined
reference to `__imp_PyExc_TypeError'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x6a6): undefined
reference to `__imp_PyExc_SystemError'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x6c7): undefined
reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-2.7\Release\ml.o:ml.c:(.text+0x721): undefined
reference to `__imp_PyExc_TypeError'
[.... many more]
Lisandro Dalcin
> Hello,
Hi
> I am not a windows user.
>
Nor I am ...
>
> And I am having some trouble building my cython extension for a 64bit
> windows7 machine (this may well be a python / mingw question).
>
> It starts with many warning which I did not have on linux, and then
> complains about " skipping incompatible c:\Python27\libs/libpython27.a
> when searching for -lpython27" and hell brakes loose.
>
I bet that file is broken, it is a 32bits binary, then it doesn't work
on 64bits Windows.
> Any idea how to make this work?
>
1) Download this definition file:
http://wiki.cython.org/InstallingOnWindows?action=AttachFile&do=get&target=python27.def
[Note: I've generated that file some time ago from in 32 bits Vista,
but it is just a text file, so it should still work for you]
2) Make sure you have C:\MinGW\bin in your %PATH%, and then run this:
$ dlltool --dllname python27.dll --def python27.def --output-lib libpython27.a
After that, replace the broken libpython2.7.a in your Python
installation with the new file you just generated.
3) Cross fingers and try to build your extensions...
--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
Yoav Goldberg
On 3 March 2011 13:46, Yoav Goldberg <yoav.g...@gmail.com> wrote:
> Hello,
Lisandro Dalcin
> Thanks! Unfortunately (fortunately?) I am away from the windows machine by
> now, so I can not check your solution -- but I will try that next week.
>
> fwiw, I finally did manage to build the extension by:
> - installing Visual Studion 2008 express d
> - installing an SDK for the 64bit compiler (~1GB download for both)
Wow...
> - rebooting the machine (yes. this seem to be required but not mentioned
> anywhere. took me forever to figure it out),
For the future, never ever try to run newly installed software without
rebooting... Unless, of course, you software is well know,
multi-platform, open source (like Python or MinGW), these always work
;-)
> - running a file called
> "vcvars64.bat",
Mmm... interesting... are you sure you really need do run "vcvars64.bat" ?
> - and then finally "python setup.py build_ext --inplace".
> It still complained about not finding something called "mt.exe", but I do
> have a .pyd file, and I can import it in the interpreter, so, at least for
> now, I'm good.
"mt.exe" is the linker, perhaps you just got a warning...
>
> (would be nice to get it working on ming64 for the future,
> though).
>
Follow my instructions, it should work... BTW, I expect Python to fix
this issue in their binary installers for the next 2.7 micro release
(not sure the status of Python 3.2, could you give it a try?)...
Yoav Goldberg
On 3 March 2011 16:59, Yoav Goldberg <yoav.g...@gmail.com> wrote:> - running a file called
> "vcvars64.bat",Mmm... interesting... are you sure you really need do run "vcvars64.bat" ?
> (would be nice to get it working on ming64 for the future,
> though).
Follow my instructions, it should work... BTW, I expect Python to fixthis issue in their binary installers for the next 2.7 micro release
(not sure the status of Python 3.2, could you give it a try?)...
Sturla Molden
> Hello,
> I am not a windows user.
> And I am having some trouble building my cython extension for a 64bit
> windows7 machine (this may well be a python / mingw question).
>
> It starts with many warning which I did not have on linux, and then
> complains about " skipping incompatible c:\Python27\libs/libpython27.a
> when searching for -lpython27" and hell brakes loose.
>
> Any idea how to make this work?
Short answer: Don't.
There are issues with the mingw runtime conflicting with the MSVC
runtime on Windows 64. That is the reason the import libraries are omitted.
Use the free C/C++ platform compiler from Microsoft. Specifically, use
Windows 7 SDK for .NET 3.5, not later versions! Later or earlier
versions of the SDK will link with the wrong C runtime. (Feel free to
complain the Redmond for creating C runtime DLL hell on Windows, they
should be flogged.)
You do not need Visual Studio 2008 Express, as indicated in an earlier
post to this list.
Make sure you have an environment variable DISTUTILS_USE_SDK defined and
set to 1, so distutils will use the platform C/C++ compiler. Otherwise
it will use the compiler shipped with Microsoft Visual C++.
If you need Fortran (e.g. for f2py or fwrap), Intel, Absoft and Portland
have compilers compatible with Microsoft's C compiler and linker.
Regards,
Sturla
Sturla Molden
>
> There are issues with the mingw runtime conflicting with the MSVC
> runtime on Windows 64. That is the reason the import libraries are
> omitted.
>
Also observe that while you can get 64-bit mingw builds from e.g.
TDM-GCC, mingw-w64 is still in "beta" and there is no offcial 64-bit
mingw release. Unless you need the compiler's source code, this is not a
problem because Microsoft will give you a better C/C++ compiler for
free. The C/C++ compiler in the Windows platform SDK is the same
optimizing compiler shipped with Visual Studio professional and
enterprise, not the crippled compiler shipped with Visual Studio standard.
Sturla
Sturla Molden
>
> Windows 7 SDK for .NET 3.5, not later versions!
Which by the way is this download:
You want the ISO file GRMSDKX_EN_DVD.iso for AMD64. (It can build 32-bit
binaries for x86 as well.) Windows 7 can burn ISO images to DVD,
otherwise you will e.g. need Nero.
Before you run setup.py to build with distutils, do this in the SDK
command window:
C:\Program Files\Microsoft SDKs\Windows\v7.0>set DISTUTILS_USE_SDK=1
C:\Program Files\Microsoft SDKs\Windows\v7.0>setenv /x64 /release
The yellow text should now turn green, and you are ready to build AMD64
Python extension modules.
Sturla
Yoav Goldberg
In fact, I took the liberty of editing this a little and adding to the
cython wiki for future reference.
http://wiki.cython.org/64BitCythonExtensionsOnWindows
(right now this is linked from
http://wiki.cython.org/InstallingOnWindows, which is a bit hidden, but
I couldn't change the main page)
Yoav
Sturla Molden
I changed some minor details. Does it look ok to you?
Sturla
Yoav Goldberg
not using MingW-64, as it is the default choice for many people coming
from the Linux world.
Y
Sturla Molden
> Yes, this is much better. I would emphasize (in bold?) the part about
> not using MingW-64, as it is the default choice for many people coming
> from the Linux world.
>
I've added in some bold to emphazise the text.
I've added a comment regarding prefering MinGW when coming from Linux or
Mac. GCC is preferred on those platforms because it is the platform
C/C++ compiler. Thus it ensures binary compatability with installed
libraries. On Windows, Microsoft's compiler plays the same role.
Selecting MSVC on Windows is no different from selecting GCC on Linux.
MinGW is a free compiler, but not binary compatible with most components
on Windows (this is particularly bad for C++, COM, and structured
exception handling). Previously Microsoft did not distribute their
platform C/C++ compiler free of charge. They do now, however. Reasons
for using MinGW on Windows might be free compilers for C99 and Fortran.
This will otherwise require compilers from Intel.
Sturla