Cython compilation Problem "error: Unable to find vcvarsall.bat"

[Rolf]

I have installed pythonxy (26) on my WinXP computer which includes
Cython, but when compiding the code given on page (http://
docs.cython.org/src/userguide/tutorial.html) I get the followinng
error message:

running build_ext
skipping 'hello.c' Cython extension (up-to-date)
building 'hello' extension
error: Unable to find vcvarsall.bat

I cannot find this file on my computer. Acording to some google search
I gues this has something to do with a compiler configuration that has
to be set for MinGW instead of Visual C, so I put the MinGW bin
directory on m PATH, but it didn't help.

Next I read that some configuration is necessary to tell the Cython
compiler that MinGW has to be used instead of VC. Two options are
discussed

1. edit a file called 'distutils.cfg' located at C:\Python26\Lib
\distutils\distutils.cfg wich doesn't exist anywhere on my Computer

2. to run 'setup.py install build --compiler=mingw32' doesn't work out
either as setup.py is not on the PATH and there numerus setup.py
scripts in the Python26 directory.

Anyway, I don't think this is the straightforward way to solve the
problem as one expects Pythonxy will set the environemnt
automatically. These must be some solution more simple.

[Jon Olav Vik]

On May 30, 8:46 am, Rolf <kamha...@googlemail.com> wrote:
> I have installed pythonxy (26) on my WinXP computer which includes
> Cython, but when compiding the code given on page (http://
> docs.cython.org/src/userguide/tutorial.html)  I get the followinng
> error message:
>
> running build_ext
> skipping 'hello.c' Cython extension (up-to-date)
> building 'hello' extension
> error: Unable to find vcvarsall.bat
>
> I cannot find this file on my computer. Acording to some google search
> I gues this has something to do with a compiler configuration that has
> to be set for MinGW instead of Visual C, so I put the MinGW bin
> directory on m PATH, but it didn't help.
>
> Next I read that some configuration is necessary to tell the Cython
> compiler that MinGW has to be used instead of VC. Two options are
> discussed
>
> 1. edit a file called 'distutils.cfg' located at C:\Python26\Lib
> \distutils\distutils.cfg wich doesn't exist anywhere on my Computer

You can create it (a plain text file). To see where your distutils
package is located, try this in the Python interpreter:

>>> import distutils
>>> print distutils
<module 'distutils' from 'C:\Python27\lib\distutils\__init__.pyc'>

> 2. to run 'setup.py install build --compiler=mingw32' doesn't work out
> either as setup.py is not on the PATH and there numerus setup.py
> scripts in the Python26 directory.

That should be "python setup.py install build --compiler=mingw32",
executed from the command line in the directory containing the
setup.py script for your Cython file. Thus, only python needs to be on
your PATH, setup.py is the script to be executed, and "install build --
compiler=mingw32" are options that are interpreted by the "distutils"
package, which setup.py uses.

Hope this helps,
Jon Olav

[Rolf]

> That should be "python setup.py install build --compiler=mingw32",
> executed from the command line in the directory containing the
> setup.py script for your Cython file. Thus, only python needs to be on
> your PATH, setup.py is the script to be executed, and "install build --
> compiler=mingw32" are options that are interpreted by the "distutils"
> package, which setup.py uses.

I installed Python(xy), so there was no 'setup.py' in my Cython
directory. Anyway, I found a way to use this cmd. I downloaded and
unzipped a new Cathon distribution with a setup.py in it.
Unfortunately after successful installation, I realized this was in
vain. The error persited.

Finally, the compilation problem is solved.
By contrast to what is said in the documentation (http://
docs.cython.org/src/userguide/tutorial.html)

the command to build this module ist not
'python setup.py build_ext --inplace'

but 'python setup.py build_ext --inplace --compiler=mingw32'

(As I use the eclipse environment that comes with Python(xy), I had to
modify the Run configuration -> Arguments with "build_ext --inplace --
compiler=mingw32")
This is probably an installation bug of Python(xy) that by default
assumes the Visual C compiler although MinGW is installed along with
Python. I probably have to report them.

Besides, I didn't find any configuration file to change it.

Next problem in the row.

The above cited documentation says "... [compilation] will leave a
file in your local directory called helloworld.so in unix or
helloworld.dll in Windows."
There is an object file helloworld.o in my build directory but no dll
anywhere.

I wonder if there is an other argument necessary to link these objects
into a proper dll. Maybe "--linker=something"

[Rolf]

Problem solved and I will give a summary as it might be useful to
others. The problem rests with the documentation on both quite similar
pages.
* http://docs.cython.org/src/quickstart/build.html
* http://docs.cython.org/src/userguide/tutorial.html

There are two statements in these documentations that cause confusion
with windows users:

1. the command
$ python setup.py build_ext --inplace
should say
$ python setup.py build_ext --inplace --compiler=mingw32

2. the statement
'Which will leave a file in your local directory called helloworld.so
in unix or helloworld.dll in Windows.'
is wrong
at least on windows computers the extension is not .dll but .pyd

The import is possible only if the Python interpreter is launched from
that local directory. Which is not so much of a problem if you work in
an eclipse environment.

How can such a module be made available everywhere (through PYTHONPATH
probably), is not yet solved.

[Robert Bradshaw]

On Mon, May 30, 2011 at 8:23 AM, Rolf <kamh...@googlemail.com> wrote:
> Problem solved and I will give a summary as it might be useful to
> others. The problem rests with the documentation on both quite similar
> pages.
> * http://docs.cython.org/src/quickstart/build.html
> * http://docs.cython.org/src/userguide/tutorial.html
>
> There are two statements in these documentations that cause confusion
> with windows users:
>
> 1. the command
> $ python setup.py build_ext --inplace
> should say
> $ python setup.py build_ext --inplace --compiler=mingw32

Is this the only compiler that's supported?

> 2. the statement
> 'Which will leave a file in your local directory called helloworld.so
> in unix or helloworld.dll in Windows.'
> is wrong
> at least on windows computers the extension is not .dll but .pyd
>
> The import is possible only if the Python interpreter is launched from
> that local directory. Which is not so much of a problem if you work in
> an eclipse environment.

Calling all Windows users to fix our documentation, as I know nothing
about this environment.
https://github.com/cython/cython/tree/master/docs , just fork and
create a pull request. You can even fork and edit the file online.

- Robert

[Jon Olav Vik]

On May 30, 3:28 pm, Rolf <kamha...@googlemail.com> wrote:
> > That should be "python setup.py install build --compiler=mingw32",
> > executed from the command line in the directory containing the
> > setup.py script for your Cython file. Thus, only python needs to be on
> > your PATH, setup.py is the script to be executed, and "install build --
> > compiler=mingw32" are options that are interpreted by the "distutils"
> > package, which setup.py uses.
>
> I installed Python(xy), so there was no 'setup.py' in my Cython
> directory.

I'm sorry, I can see how this might be misunderstood. If you're
following the example on http://docs.cython.org/src/userguide/tutorial.html,
you will have written a file called helloworld.pyx. It is convenient
to put this in a separate directory, wherever you please. In that
directory you will also place a file called setup.py, which mentions
helloworld.pyx in the line
ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
Thus, each Cython file that you write will typically be in a separate
directory and accompanied by its own setup.py file (unless you're
using pyximport; I haven't been able to get that to work except in
trivial cases). Thus, if you've placed helloworld.pyx and setup.py in
c:\temp\helloworld, start a command-line session in Windows, and do as
follows:

cd c:\temp\helloworld

python setup.py build_ext --inplace --compiler=mingw32

(The words "setup.py install build" that I copy-pasted from your first
post is incorrect; there is no mention of "install build" on the
tutorial page. It should be "python setup.py build_ext --inplace",
possibly with a --compiler option at the end.) Here follows a full
transcript of how things work on my system ("copy con" is a DOS-ish
way of typing directly into a file; sane people would use a text
editor. The ^Z means Control-Z, the end-of-file character).

C:\Users\jonvi>cd c:\temp\
c:\temp>mkdir helloworld
c:\temp>cd helloworld
c:\temp\helloworld>copy con helloworld.pyx
print "Hello World"
^Z
1 fil(er) ble kopiert.

c:\temp\helloworld>copy con setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
)
^Z
1 fil(er) ble kopiert.

c:\temp\helloworld>python setup.py build_ext --inplace --
compiler=mingw32
running build_ext
cythoning helloworld.pyx to helloworld.c
building 'helloworld' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
c:\Rtools\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -IC:
\Python27\include -IC:\Python27\PC -c helloworld.c -o build
\temp.win32-2.7\Release\helloworld.o
writing build\temp.win32-2.7\Release\helloworld.def
c:\Rtools\MinGW\bin\gcc.exe -mno-cygwin -shared -s build
\temp.win32-2.7\Release\helloworld.o build\temp.win32-2.7\Release
\helloworld.def -LC:\Python27\libs -LC:\Python27\
PCbuild -lpython27 -lmsvcr90 -o c:\temp\helloworld\helloworld.pyd

c:\temp\helloworld>dir
2011-05-31 11:02 <DIR> build
2011-05-31 11:02 44 237 helloworld.c
2011-05-31 11:02 11 776 helloworld.pyd
2011-05-31 11:02 21 helloworld.pyx
2011-05-31 11:02 237 setup.py

c:\temp\helloworld>python -c "import helloworld"
Hello World

[Jon Olav Vik]

On May 31, 6:41 am, Robert Bradshaw <rober...@math.washington.edu>
wrote:

> On Mon, May 30, 2011 at 8:23 AM, Rolf <kamha...@googlemail.com> wrote:
> > There are two statements in these documentations that cause confusion
> > with windows users:
>
> > 1. the command
> > $ python setup.py build_ext --inplace
> > should say
> > $ python setup.py build_ext --inplace --compiler=mingw32
>
> Is this the only compiler that's supported?

Certainly not. Sturla Molden has repeatedly stated that Microsoft
Visual C++ is "the" system compiler on Windows.
http://article.gmane.org/gmane.comp.python.cython.user/2783

[Rolf]

On 31 Mai, 11:06, Jon Olav Vik <jono...@gmail.com> wrote:

> > I installed Python(xy), so there was no 'setup.py' in my Cython
> > directory.
>
> I'm sorry, I can see how this might be misunderstood. If you're
> following the example onhttp://docs.cython.org/src/userguide/tutorial.html,

Sorry, Jon for this confusion. These are in fact two issues that I
discussed at once.
1. Installation problem of cython into my Python(xy) environment
2. The example given in the tutorial

Unfortunately and most confusingly, both deal with 'setup.py'.

Forget about the first point, the installation of Cython. That is an
issue rather to be discussed with Python(xy) developers, but you may
help them by providing an interface to change the default Cython
compiler (see my suggestions below).

The secons point, the tutorial example, is to be discussed here.
In summary I'd like to suggest only two minor amandments to the
tutorial

1. For those like me who started Cython from Python(xy) on a windows
computer a MinGW compiler is installed instead of Visual C++. For us,
the examples given in these two tutorials don't work as by default
Cython expects the VC compiler. An error is issued "error: Unable to
find vcvarsall.bat". The solution to the problem is adding the
compiler directive that forces Cython to use MinGW. I really don't
know whether other compilers can be used or whether there is an
configuration file to change the default Visual C++ option. That would
be of interest to those who prepare the Python(xy) distributions.
Along with MinGW installation, the Python(xy) installation routine
should set the default compiler, but this is rather an issue of
Python(xy) developers better not to be discussed here to avaoid an
other confusion.

Thus the tutorial should explicitly state if there is an other
compiler nor VC in use you must set the compiler option "--
compiler=" and please provide a list of accepted compilers.

2. This is only a minor change to the tutorial. The compiled file in
windows is *.pyd (not *.dll) It may work like a dynamically linked
library, but the extension is still .pyd.

So I hope everything is clear now. it's really not a big thing, but it
caused me a lot of time to figure it out. May it help other.
Yours Sincerely
R.

[Dag Sverre Seljebotn]

The configuration setting in question can be found at almost at the end
of this document (bottom left of last page):

http://conference.scipy.org/proceedings/SciPy2009/paper_1/

Patches to documentation to improve this most welcome. It's rather
difficult for developers who are not on Windows to document such things.

Dag Sverre

[Rolf]

> http://conference.scipy.org/proceedings/SciPy2009/paper_1/

Thanks. this paper is quite useful. I tested this configuration and
the tutorial example worked without the compiler option set, so I made
the appropriate suggestions to Python(xy) developers there in their
news group.

This leaves us with only a minimal improvement to the tutorial
necessary.

<The compiled file in
<windows is *.pyd (not *.dll) It may work like a dynamically linked

<ibrary, but the extension is still .pyd.

I would like to make these changes as suggested above.

>Calling all Windows users to fix our documentation, as I know nothing
>about this environment.
>https://github.com/cython/cython/tree/master/docs , just fork and
>create a pull request. You can even fork and edit the file online.

... but I can't figure out how to log in.

Best Regards
R.

[Robert Bradshaw]

On Tue, May 31, 2011 at 11:08 PM, Rolf <kamh...@googlemail.com> wrote:
>
>> http://conference.scipy.org/proceedings/SciPy2009/paper_1/
>
> Thanks. this paper is quite useful. I tested this configuration and
> the tutorial example worked without the compiler option set, so I made
> the appropriate suggestions to Python(xy) developers there in their
> news group.
>
> This leaves us with only a minimal improvement to the tutorial
> necessary.

Excellent.

> <The compiled file in
> <windows is *.pyd (not *.dll) It may work like a dynamically linked
> <ibrary, but the extension is still .pyd.
>
> I would like to make these changes as suggested above.
>
>>Calling all Windows users to fix our documentation, as I know nothing
>>about this environment.
>>https://github.com/cython/cython/tree/master/docs , just fork and
>>create a pull request. You can even fork and edit the file online.
>
> ... but I can't figure out how to log in.

You have to create an account. You have to provide an email address
for confirmation, but that's about it, and I've never been spammed by
them (other than the subscriptions I explicitly added to follow what's
going on). Then you can browse to the file you want
(https://github.com/cython/cython/tree/master/docs/src ) and hit "edit
this file" (or "fork and edit" or something like that) in the upper
right hand corner.

- Robert