Re: cython in WinPython, and all of those ".bat" files in the scripts directory
1,730 views
Skip to first unread message
Pierre Raybaut
Sep 25, 2012, 2:42:22 PM9/25/12
to winp...@googlegroups.com
2012/9/25 Mark <markper...@gmail.com>:
> Sorry for asking what may be a very simple (or even technically incorrect)
> question , but I am trying to get a little more information.
>
> I was hoping that WinPython could possibly make my entry to using Cython a
> little easier. I have not had much luck with Cython to date, and I know that
> I am likely missing something simple. So I have downloaded WinPython to try
> some things out and see if I can get Cython running under a more controlled
> setting.
>
> Does the distribution also contain all of the Python header files and code
> required by Cython for compilation of cPython functions into modules? And is
Yes for the 32-bit version of WinPython (MinGW32 with gcc is
included), and no for the 64-bit version (you will have to install
your own C compiler).
> there a specific approach that is recommended for using Cython from
> WinPython.
No, WinPython is really close to a standard Windows Python
installation. When running one of the launchers (.exe files at the
root directory), the environment is exactly the same as the one you
would have after installing Python with the official installer. So to
run scripts installed with WinPython (like Cython) from a command
window, you would have to run cmd.exe. But you can also run everything
from Spyder: you write your .pyx script and build it in a terminal
inside Spyder.
> Also, when poking around in the python-2.7.3\Scripts folder, I see numerous
> "*.bat" files that I don't have in my normal Python distribution. Is there
> any information available regarding what these are used for?
As I tried to explain in the readme.txt in that directory, these batch
scripts are indeed specific to WinPython and they are only there to
help the user eventually writing its own launchers. For example, the
script "scripts\spyder.bat" does (almost) exactly the same things as
the launcher "Spyder.exe". But none of this script is required for
using WinPython, these are extras.
> Sorry for cluttering the list with such basic questions, but thanks in
> advance for any information that anyone can provide.
No, problem.
-Pierre
> -Mark
>
> --
> You received this message because you are subscribed to the Google Groups
> "WinPython" group.
> To unsubscribe from this group, send email to
> winpython+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> Sorry for asking what may be a very simple (or even technically incorrect)
> question , but I am trying to get a little more information.
>
> I was hoping that WinPython could possibly make my entry to using Cython a
> little easier. I have not had much luck with Cython to date, and I know that
> I am likely missing something simple. So I have downloaded WinPython to try
> some things out and see if I can get Cython running under a more controlled
> setting.
>
> Does the distribution also contain all of the Python header files and code
> required by Cython for compilation of cPython functions into modules? And is
Yes for the 32-bit version of WinPython (MinGW32 with gcc is
included), and no for the 64-bit version (you will have to install
your own C compiler).
> there a specific approach that is recommended for using Cython from
> WinPython.
No, WinPython is really close to a standard Windows Python
installation. When running one of the launchers (.exe files at the
root directory), the environment is exactly the same as the one you
would have after installing Python with the official installer. So to
run scripts installed with WinPython (like Cython) from a command
window, you would have to run cmd.exe. But you can also run everything
from Spyder: you write your .pyx script and build it in a terminal
inside Spyder.
> Also, when poking around in the python-2.7.3\Scripts folder, I see numerous
> "*.bat" files that I don't have in my normal Python distribution. Is there
> any information available regarding what these are used for?
As I tried to explain in the readme.txt in that directory, these batch
scripts are indeed specific to WinPython and they are only there to
help the user eventually writing its own launchers. For example, the
script "scripts\spyder.bat" does (almost) exactly the same things as
the launcher "Spyder.exe". But none of this script is required for
using WinPython, these are extras.
> Sorry for cluttering the list with such basic questions, but thanks in
> advance for any information that anyone can provide.
No, problem.
-Pierre
> -Mark
>
> --
> You received this message because you are subscribed to the Google Groups
> "WinPython" group.
> To unsubscribe from this group, send email to
> winpython+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
Mark
Sep 28, 2012, 2:15:26 PM9/28/12
to winp...@googlegroups.com
Thank you, Pierre. I again apologize for asking extremely simple questions.
My interest in your WinPython distributions stems from my lack of success using Cython in the past. I have always suspected that there is some sort of minor configuration issue occurring that is preventing me from making progress. Unfortunately I am still not having much success getting the basic Cython tutorial to work (a simple "Hello World!"; http://docs.cython.org/src/userguide/tutorial.html), even when using WinPython.
And I sincerely apologize if this question should be asked elsewhere, but this seemed like a reasonable place to start given the goals of WinPython.
1) I have installed WinPython in "C:\WinPython-32bit-2.7.3.0". This directory includes all of the links to Spyder, IPython, WPControlPanel, etc.
2) Using Spyder, I created the "helloworld.pyx" and "setup.py" files indicated in the Cython tutorial. These file sit in a directory called "C:\WinPython-32bit-2.7.3.0\python-2.7.3\myCython"
3) Still within Spyder, I used the "Tools" menu to open a command prompt (it opens to the locations where my .pyx and setup.py file are located).
4) I issue the command
running build_ext
cythoning hello.pyx to hello.c
building 'hello' extension
error: Unable to find vcvarsall.bat
I have searched my WinPython directory for the "vcvarsall.bat" file, and it doesn't exist.
Does anyone have any suggestions?
Thanks,
-Mark
My interest in your WinPython distributions stems from my lack of success using Cython in the past. I have always suspected that there is some sort of minor configuration issue occurring that is preventing me from making progress. Unfortunately I am still not having much success getting the basic Cython tutorial to work (a simple "Hello World!"; http://docs.cython.org/src/userguide/tutorial.html), even when using WinPython.
And I sincerely apologize if this question should be asked elsewhere, but this seemed like a reasonable place to start given the goals of WinPython.
1) I have installed WinPython in "C:\WinPython-32bit-2.7.3.0". This directory includes all of the links to Spyder, IPython, WPControlPanel, etc.
2) Using Spyder, I created the "helloworld.pyx" and "setup.py" files indicated in the Cython tutorial. These file sit in a directory called "C:\WinPython-32bit-2.7.3.0\python-2.7.3\myCython"
3) Still within Spyder, I used the "Tools" menu to open a command prompt (it opens to the locations where my .pyx and setup.py file are located).
4) I issue the command
"python setup.py build_ext --inplace" at the prompt (as listed in the tutorial) and then see the following output:
running build_ext
cythoning hello.pyx to hello.c
building 'hello' extension
error: Unable to find vcvarsall.bat
I have searched my WinPython directory for the "vcvarsall.bat" file, and it doesn't exist.
Does anyone have any suggestions?
Thanks,
-Mark
Mark
Sep 28, 2012, 2:15:48 PM9/28/12
to winp...@googlegroups.com
Thank you, Pierre. I again apologize for asking extremely simple questions.
My interest in your WinPython distributions stems from my lack of success using Cython in the past. I have always suspected that there is some sort of minor configuration issue occurring that is preventing me from making progress. Unfortunately I am still not having much success getting the basic Cython tutorial to work (a simple "Hello World!"; http://docs.cython.org/src/userguide/tutorial.html), even when using WinPython.
And I sincerely apologize if this question should be asked elsewhere, but this seemed like a reasonable place to start given the goals of WinPython.
1) I have installed WinPython in "C:\WinPython-32bit-2.7.3.0". This directory includes all of the links to Spyder, IPython, WPControlPanel, etc.
2) Using Spyder, I created the "helloworld.pyx" and "setup.py" files indicated in the Cython tutorial. These file sit in a directory called "C:\WinPython-32bit-2.7.3.0\python-2.7.3\myCython"
3) Still within Spyder, I used the "Tools" menu to open a command prompt (it opens to the locations where my .pyx and setup.py file are located).
4) I issue the command
running build_ext
cythoning hello.pyx to hello.c
building 'hello' extension
error: Unable to find vcvarsall.bat
I have searched my WinPython directory for the "vcvarsall.bat" file, and it doesn't exist.
Does anyone have any suggestions?
Thanks,
-Mark
On Tuesday, September 25, 2012 11:42:22 AM UTC-7, Pierre Raybaut wrote:
My interest in your WinPython distributions stems from my lack of success using Cython in the past. I have always suspected that there is some sort of minor configuration issue occurring that is preventing me from making progress. Unfortunately I am still not having much success getting the basic Cython tutorial to work (a simple "Hello World!"; http://docs.cython.org/src/userguide/tutorial.html), even when using WinPython.
And I sincerely apologize if this question should be asked elsewhere, but this seemed like a reasonable place to start given the goals of WinPython.
1) I have installed WinPython in "C:\WinPython-32bit-2.7.3.0". This directory includes all of the links to Spyder, IPython, WPControlPanel, etc.
2) Using Spyder, I created the "helloworld.pyx" and "setup.py" files indicated in the Cython tutorial. These file sit in a directory called "C:\WinPython-32bit-2.7.3.0\python-2.7.3\myCython"
3) Still within Spyder, I used the "Tools" menu to open a command prompt (it opens to the locations where my .pyx and setup.py file are located).
4) I issue the command
"python setup.py build_ext --inplace" at the prompt (as listed in the tutorial) and then see the following output:
running build_ext
cythoning hello.pyx to hello.c
building 'hello' extension
error: Unable to find vcvarsall.bat
I have searched my WinPython directory for the "vcvarsall.bat" file, and it doesn't exist.
Does anyone have any suggestions?
Thanks,
-Mark
On Tuesday, September 25, 2012 11:42:22 AM UTC-7, Pierre Raybaut wrote:
Pierre Raybaut
Sep 29, 2012, 4:55:54 AM9/29/12
to winp...@googlegroups.com
2012/9/28 Mark <markper...@gmail.com>:
compiler (MinGW32 with gcc).
> 2) Using Spyder, I created the "helloworld.pyx" and "setup.py" files
> indicated in the Cython tutorial. These file sit in a directory called
> "C:\WinPython-32bit-2.7.3.0\python-2.7.3\myCython"
Just a suggestion: avoid mixing installed libraries and your own.
Moving the 'myCython' folder to a 'C:\dev\myCython' folder would
probably be more cleaner.
> 3) Still within Spyder, I used the "Tools" menu to open a command prompt (it
> opens to the locations where my .pyx and setup.py file are located).
Ok.
> 4) I issue the command
>
> "python setup.py build_ext --inplace" at the prompt (as listed in the
> tutorial) and then see the following output:
>
>
> running build_ext
> cythoning hello.pyx to hello.c
> building 'hello' extension
> error: Unable to find vcvarsall.bat
>
> I have searched my WinPython directory for the "vcvarsall.bat" file, and it
> doesn't exist.
This is a classic error due to the fact that the default C compiler on
Windows for Python is Microsoft Visual C/C++ (that's what 'vc' stands
for in 'vcvarsall.bat'). So either you install the free version of
this compiler (Microsoft Visual C/C++ 2008 Express Edition -- why
2008? because Python 2.7 was built with this compiler version) or you
will have to tell Python to use MinGW32 instead of MSVC compiler with
the '-c' option as in the following command.
python setup.py build_ext -c mingw32 --inplace
That should work but it isn't with WinPython v2.7.3.0 32bit because I
have apparently forgot to include some essential files for MinGW32
(that's my fault: I did very limited testing of MinGW32 as I'm
replacing it with MSVC in all my projects due to an imminent migration
from 32-bit to 64-bit).
I'll fix this in WinPython v2.7.3.1:
http://code.google.com/p/winpython/issues/detail?id=9
In the meantime, you may install Microsoft Visual C/C++ 2008 Express
Edition, and everything will work as expected.
-Pierre
> Thank you, Pierre. I again apologize for asking extremely simple questions.
>
> My interest in your WinPython distributions stems from my lack of success
> using Cython in the past. I have always suspected that there is some sort of
> minor configuration issue occurring that is preventing me from making
> progress. Unfortunately I am still not having much success getting the
> basic Cython tutorial to work (a simple "Hello World!";
> http://docs.cython.org/src/userguide/tutorial.html), even when using
> WinPython.
>
> And I sincerely apologize if this question should be asked elsewhere, but
> this seemed like a reasonable place to start given the goals of WinPython.
>
> 1) I have installed WinPython in "C:\WinPython-32bit-2.7.3.0". This
> directory includes all of the links to Spyder, IPython, WPControlPanel, etc.
Ok, so this is the 32-bit version of WinPython including a free C
>
> My interest in your WinPython distributions stems from my lack of success
> using Cython in the past. I have always suspected that there is some sort of
> minor configuration issue occurring that is preventing me from making
> progress. Unfortunately I am still not having much success getting the
> basic Cython tutorial to work (a simple "Hello World!";
> http://docs.cython.org/src/userguide/tutorial.html), even when using
> WinPython.
>
> And I sincerely apologize if this question should be asked elsewhere, but
> this seemed like a reasonable place to start given the goals of WinPython.
>
> 1) I have installed WinPython in "C:\WinPython-32bit-2.7.3.0". This
> directory includes all of the links to Spyder, IPython, WPControlPanel, etc.
compiler (MinGW32 with gcc).
> 2) Using Spyder, I created the "helloworld.pyx" and "setup.py" files
> indicated in the Cython tutorial. These file sit in a directory called
> "C:\WinPython-32bit-2.7.3.0\python-2.7.3\myCython"
Moving the 'myCython' folder to a 'C:\dev\myCython' folder would
probably be more cleaner.
> 3) Still within Spyder, I used the "Tools" menu to open a command prompt (it
> opens to the locations where my .pyx and setup.py file are located).
> 4) I issue the command
>
> "python setup.py build_ext --inplace" at the prompt (as listed in the
> tutorial) and then see the following output:
>
>
> running build_ext
> cythoning hello.pyx to hello.c
> building 'hello' extension
> error: Unable to find vcvarsall.bat
>
> I have searched my WinPython directory for the "vcvarsall.bat" file, and it
> doesn't exist.
Windows for Python is Microsoft Visual C/C++ (that's what 'vc' stands
for in 'vcvarsall.bat'). So either you install the free version of
this compiler (Microsoft Visual C/C++ 2008 Express Edition -- why
2008? because Python 2.7 was built with this compiler version) or you
will have to tell Python to use MinGW32 instead of MSVC compiler with
the '-c' option as in the following command.
python setup.py build_ext -c mingw32 --inplace
That should work but it isn't with WinPython v2.7.3.0 32bit because I
have apparently forgot to include some essential files for MinGW32
(that's my fault: I did very limited testing of MinGW32 as I'm
replacing it with MSVC in all my projects due to an imminent migration
from 32-bit to 64-bit).
I'll fix this in WinPython v2.7.3.1:
http://code.google.com/p/winpython/issues/detail?id=9
In the meantime, you may install Microsoft Visual C/C++ 2008 Express
Edition, and everything will work as expected.
-Pierre
Mark Miller
Oct 1, 2012, 12:37:05 PM10/1/12
to winp...@googlegroups.com
Thanks Pierre: I look forward to seeing the next release. It will be
extremely to have something pre-configured that does not need to rely
on the microsoft compiler.
In the meantime, I did install the MS 2008 C++ compiler that you
suggested. Installation seems to be fine, but I am still getting
errors when trying to compile the 'hello world' example. Here's the
readout...I am just mentioning it in case it is indicative of some
other sort of problem that you might need to be aware of.
Thanks again, and I am definitely looking forward to the next release!
-Mark
C:\Marks\Python_code\Cython\hello>python setup.py build_ext --inplace
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo
/Ox /MD /W3 /GS- /DNDEBUG
-IC:\WinPython-32bit-2.7.3.0\python-2.7.3\include
-IC:\WinPython-32bit-2.7.3.0\python-2.7.3\PC /Tchello.c
/Fobuild\temp.win32-2.7\Release\hello.obj
hello.c
c:\winpython-32bit-2.7.3.0\python-2.7.3\include\pyconfig.h(227) :
fatal error C1083: Cannot open include file: 'basetsd.h': No such file
or directory
error: command '"C:\Program Files\Microsoft Visual Studio
9.0\VC\BIN\cl.exe"' failed with exit status 2
extremely to have something pre-configured that does not need to rely
on the microsoft compiler.
In the meantime, I did install the MS 2008 C++ compiler that you
suggested. Installation seems to be fine, but I am still getting
errors when trying to compile the 'hello world' example. Here's the
readout...I am just mentioning it in case it is indicative of some
other sort of problem that you might need to be aware of.
Thanks again, and I am definitely looking forward to the next release!
-Mark
C:\Marks\Python_code\Cython\hello>python setup.py build_ext --inplace
running build_ext
cythoning hello.pyx to hello.c
building 'hello' extension
creating build
cythoning hello.pyx to hello.c
building 'hello' extension
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
C:\Program Files\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo
/Ox /MD /W3 /GS- /DNDEBUG
-IC:\WinPython-32bit-2.7.3.0\python-2.7.3\include
-IC:\WinPython-32bit-2.7.3.0\python-2.7.3\PC /Tchello.c
/Fobuild\temp.win32-2.7\Release\hello.obj
hello.c
c:\winpython-32bit-2.7.3.0\python-2.7.3\include\pyconfig.h(227) :
fatal error C1083: Cannot open include file: 'basetsd.h': No such file
or directory
error: command '"C:\Program Files\Microsoft Visual Studio
9.0\VC\BIN\cl.exe"' failed with exit status 2
Mark
Oct 16, 2012, 5:59:46 PM10/16/12
to winp...@googlegroups.com
The newest release seems to be working for me now. Thanks for working on it, Pierre. Initially, after installing it, I was still getting some errors when compiling an extension with Cython:
cc1.exe: error: unrecognized command line option '-mno-cygwin'
error: command 'gcc' failed with exit status 1
I fiddled with this for a while, and was finally able to get everything to work after editing the file "cygwincompiler.py" in \Lib\distutils from the Python distribution. The changes were as follows (original commented-out code from line 322 followed by the edited version:
## self.set_executables(compiler='gcc -mno-cygwin -O -Wall',
## compiler_so='gcc -mno-cygwin -mdll -O -Wall',
## compiler_cxx='g++ -mno-cygwin -O -Wall',
## linker_exe='gcc -mno-cygwin',
## linker_so='%s -mno-cygwin %s %s'
## % (self.linker_dll, shared_option,
## entry_point))
self.set_executables(compiler='gcc -O -Wall',
compiler_so='gcc -mdll -O -Wall',
compiler_cxx='g++ -O -Wall',
linker_exe='gcc ',
linker_so='%s %s %s'
% (self.linker_dll, shared_option,
entry_point))
Just mentioning it in case it helps anyone else,
-Mark
cc1.exe: error: unrecognized command line option '-mno-cygwin'
error: command 'gcc' failed with exit status 1
I fiddled with this for a while, and was finally able to get everything to work after editing the file "cygwincompiler.py" in \Lib\distutils from the Python distribution. The changes were as follows (original commented-out code from line 322 followed by the edited version:
## self.set_executables(compiler='gcc -mno-cygwin -O -Wall',
## compiler_so='gcc -mno-cygwin -mdll -O -Wall',
## compiler_cxx='g++ -mno-cygwin -O -Wall',
## linker_exe='gcc -mno-cygwin',
## linker_so='%s -mno-cygwin %s %s'
## % (self.linker_dll, shared_option,
## entry_point))
self.set_executables(compiler='gcc -O -Wall',
compiler_so='gcc -mdll -O -Wall',
compiler_cxx='g++ -O -Wall',
linker_exe='gcc ',
linker_so='%s %s %s'
% (self.linker_dll, shared_option,
entry_point))
Just mentioning it in case it helps anyone else,
-Mark
Sol Simpson
Oct 16, 2012, 6:28:05 PM10/16/12
to winp...@googlegroups.com
I ran into this same issue today and after 2 hours gave up trying to fix it for the day. Your fix worked for me as well. Thank you for the tip.
I have also read else where that instead of having to type -c mingw32 when you compile, you can add a distutils.cfg file to your distutils folder in Python2.7.3\Lib with the contents:
I have also read else where that instead of having to type -c mingw32 when you compile, you can add a distutils.cfg file to your distutils folder in Python2.7.3\Lib with the contents:
[build]
compiler=mingw32
This worked from a command prompt inside Spyder for me (other than then giving the error you seem to have now resolved), but when I would try and have cython compile from source instead of using a setup.py file using the:
import pyximport; pyximport.install()
import _hello
syntax in my .py source file
format, where _hello is a _hello.pyx file, cython would still use vc to try and compile and give the old 'can not fine blabla.bat error'.
Does anyone know how to get around this one? ;)
Thanks!
Sol
stonebig
Jul 6, 2014, 5:03:36 AM7/6/14
to winp...@googlegroups.com
Hi Sol,
I would like my new experimental build to use the included wingw32.
Did you succeed in your experiments ?
What do you think should I do to have this working out of the box , with the IPython ?
I would like my new experimental build to use the included wingw32.
Did you succeed in your experiments ?
What do you think should I do to have this working out of the box , with the IPython ?
Sol Simpson
Jul 6, 2014, 12:21:01 PM7/6/14
to winp...@googlegroups.com
On Sun, Jul 6, 2014 at 5:03 AM, stonebig <stone...@gmail.com> wrote:
I would like my new experimental build to use the included wingw32.
Did you succeed in your experiments ?
What do you think should I do to have this working out of the box , with the IPython ?
I have not had time to even get close to as far as you have, sorry.
What error are you getting when you try and include wingw32 in the build?
stonebig
Jul 6, 2014, 12:56:32 PM7/6/14
to winp...@googlegroups.com
I don't see 'errors', in the build process itself.
Just I can't get cython work :
- 'winpython 3.3.5.0 official' works with cython (and I suspect, with VC2010 express I have on my pc),
- but winpython 3.3.5.1_build02 fails, with a :
D:\result_tests\WinPython-32bit-3.3.5.1_build02\python-3.3.5\lib\distutils\msvc9compiler.py in query_vcvarsall(version, arch) 285 286 if len(result) != len(interesting): --> 287 raise ValueError(str(list(result.keys()))) 288 289 return result ValueError: ['path']
... I may concentrate on publishing my build and documentation, so more people can see and solve remaining problems.
Sol Simpson
Jul 6, 2014, 1:46:09 PM7/6/14
to winp...@googlegroups.com
On Sun, Jul 6, 2014 at 12:56 PM, stonebig <stone...@gmail.com> wrote:
... I may concentrate on publishing my build and documentation, so more people can see and solve remaining problems.
If you can, this would probably be a very worthwhile use of your time.
Reply all
Reply to author
Forward
0 new messages