Newby's Cython doesn't compile

[laehc]

Hi, I'm new to all this and I can't get Cython to work. That's on
Windows Vista, with Python.2.6.1, and Iused Dave Cournapeau's 0.11.2
installers for Cython on Windows. I didn't get any error messages
while installing MinGW, gcc or Cython. Following the Cython manual, I
saved the "helloworld.pyx" script, then this as setup.py:

"
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
cmdclass = {’build_ext’: build_ext},
ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
)
"

and did this:

"To use this to build your Cython file use the commandline options:

$ python setup.py build_ext --inplace
"

I ran that in Cython\Distutils (where I'd put the setup and helloworld
files), and got "running build_ext" in the Command Prompt Window, but
it didn't produce any file. It's supposed to leave a file called
'helloworld.dll', in my local directory. I've tried Windows search,
google desktop search and looked in lots of folders.

This executes: import pyximport; pyximport.install()
This fails: import helloworld (module not found)

I don't know any C and I haven't tested gcc beyond running it with no
args and getting 'no input files'.
I made a DEF file called python26 using the python26.dll, because I
didn't want to risk installing python22.dll, I don't know if that
might be a problem.
Also, I've got two copies of every Compiled Python File in Cython but
no duplication of PY Files.

Can anyone help?

[Chris Colbert]

can you paste the output of that command?

[laehc]

I can't copy from the Command Window, so I've typed this:

C:\Python26\Lib\site-packages\Cython\distutils>python setup.py
build_ext --inplace
running build_ext
building 'helloworld' extension
error: None

C:\Python26\Lib\site-packages\Cython\distutils>


I've just realised that "error: None" is probably an error (I thought
it meant there were no errors).

[josef...@gmail.com]

On Tue, Nov 3, 2009 at 8:02 AM, laehc <niw...@googlemail.com> wrote:
>
>
> I can't copy from the Command Window, so I've typed this:

I'm on windowsXP, so it might not be exactly the same as on Vista
You can copy the content of the command window, by right clicking
on the header of the command window frame.
go to edit, select all, and then again with copy

(there are also shortcuts for fast edit)

> C:\Python26\Lib\site-packages\Cython\distutils>python setup.py
> build_ext --inplace
> running build_ext
> building 'helloworld' extension
> error: None
>
> C:\Python26\Lib\site-packages\Cython\distutils>

You shouldn't run your own scripts from any package directory.
It is possible that distutils gets confused by the local files.

Create a new folder outside of C:\Python26, preferably without space
in the folder path/name and move your .pyx and setup.py file there,
and try
setup.py build_ext --inplace

If it's successfull, you should see a call to gcc on the command
line. dll files in python have the extension .pyd

If it doesn't work automatically you could try to call cython directly.

>python C:\Python26\Scripts\cython.py -v helloworld.pyx

(I'm not using the latest cython, so there might be small differences)

Hope that helps,

Josef

[Lisandro Dalcin]

On Tue, Nov 3, 2009 at 11:02 AM, laehc <niw...@googlemail.com> wrote:
>
>
> I can't copy from the Command Window, so I've typed this:
>

Yes, you can... Just learn how, or use something no SO annoying as the
command windows, for example: http://sourceforge.net/projects/console/

>
> C:\Python26\Lib\site-packages\Cython\distutils>python setup.py
> build_ext --inplace
> running build_ext
> building 'helloworld' extension
> error: None
>

MinGW is not the default compiler, nor distutils try to employ it if
you have it installed... you have to additionally use the option
"--compiler=mingw32" when running "python setup.py ..."

In previous posts, I've explained how to configure distutils to use
MinGW by default. Note however that for this to work it may be
required to use cython-devel, as I've pushed some patches to make
'pyximport' work out of the box with MinGW.

>
> C:\Python26\Lib\site-packages\Cython\distutils>
>

Again, as Josef said, you do not have to run setup.py scripts inside
the install directories...

>
> I've just realised that  "error: None" is probably an error (I thought
> it meant there were no errors).
>

No, it is actually and error (means that the MSVC compiler was not
found), but distutils does not properly report it.

--
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

[laehc]

Thanks for saying how to copy from the Command Window, which works on
Vista.
I ran these as suggested and got no output in the window:

C:\MyPy>setup.py build_ext --inplace

C:\MyPy>setup.py build_ext --inplace

C:\MyPy>setup.py build_ext --inplace --compiler=mingw32

C:\MyPy>setup.py build_ext --compiler=mingw32 --inplace

C:\MyPy>python C:\Python26\Scripts\cython.py -v helloworld.pyx
Compiling C:\MyPy\helloworld.pyx
[Errno 2] No such file or directory: u'C:\\MyPy\\helloworld.pyx'

C:\MyPy>

The first four attempts opened setup.py in PyScripter (which I quit
from before each attempt, and BTW it's linked to Python with rpyc). I
tried running it there and got:

>>>
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help

error: no commands supplied
>>>


The fifth attempt got [Errno 2] as above.
I've got a file:

C:\Python\Lib\distutils\distutils.cfg

which contains:

[build]
compiler = mingw32

I might have to try cython-devel, depending on what answers I get to
this.
Thanks to all for the advice.

[Lisandro Dalcin]

I've just created the two files attached inside C:\MyPy, and things
definitely worked for me using this:

C:\MyPy>python setup.py build_ext --inplace --compiler=mingw32
[...lots of output...]
C:\MyPy>python -c "import helloworld"
Hello, World!

BTW, make sure you have your %PATH% properly setup...

--

[Christopher Barker]

laehc wrote:
>
> Thanks for saying how to copy from the Command Window, which works on
> Vista.
> I ran these as suggested and got no output in the window:
>
> C:\MyPy>setup.py build_ext --inplace

try:

C:\MyPy python setup.py .....

You may not have Windows set up to know that *.py is an executable -- so
you need to run python with the setup.py as your argument.

-Chris

--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[laehc]

Thanks for those two files which I downloaded, Lisandro. They work
with:

python setup.py build_ext --inplace

I've got files helloworld.pyd and helloworld.c, unfortunately I don't
have time to do anything with them right now.

My own files gave:

C:\MyPy>python setup.py build_ext --inplace

running build_ext
building 'helloworld' extension

error: unknown file type '.pyx' (from 'helloworld.pyx')

C:\MyPy>


The result is the same after making my files the same as the two I
downloaded, which only involved a ',' and a '!' in helloworld.pyx, and
indenting by three spaces instead of four in setup.py
I'm going to copy and paste and see if I can find exactly where my
files are wrong. They have the right extensions and Windows lists the
type as PYX File and PY File.

[Lisandro Dalcin]

OK, Then I'll give you a clue... When I wrote these two files, I've
actually first copy&paste from your email... Then when trying to run
setup.py, I got a failure... The problem what the quotes you used for
the 'build_ext' key in the cmdclass argument to setup() ... You have
to use single quotes like this: 'build_ext' or double quotes like
this: "build_ext" , but NOT like this ´build_ext´ (hope you can see
the difference in your mail reader)

[laehc]

Thanks for the clue, Lisandro. I had originally copied and pasted each
line from the installation

instructions but when it didn't work I keyed it in. After fixing that
I can get the .pyd and .c files and do:

>>> import helloworld
hello, world!
>>>

which looks good.

I can't get pyximport to work, but I'm guessing you might have had
enough for now, so I'll leave it

for a couple of weeks.

Thanks again for all the help.

[Lisandro Dalcin]

On Wed, Nov 4, 2009 at 11:51 AM, laehc <niw...@googlemail.com> wrote:
>
>
> I can't get pyximport to work, but I'm guessing you might have had
> enough for now, so I'll leave it
>

Mmm.. I've did all my testing with cython-devel... Could you give a
try? You do not need to intall Mercurial (or TortoiseHG), just point
your browser to this link http://hg.cython.org/cython/archive/tip.zip
to que a zip file you can unpack and install.

[josef...@gmail.com]

On Wed, Nov 4, 2009 at 9:01 AM, Lisandro Dalcin <dal...@gmail.com> wrote:
>
> On Wed, Nov 4, 2009 at 11:51 AM, laehc <niw...@googlemail.com> wrote:
>>
>>
>> I can't get pyximport to work, but I'm guessing you might have had
>> enough for now, so I'll leave it
>>
>
> Mmm.. I've did all my testing with cython-devel... Could you give a
> try? You do not need to intall Mercurial (or TortoiseHG), just point
> your browser to this link http://hg.cython.org/cython/archive/tip.zip
> to que a zip file you can unpack and install.
>

The only place I have compiler=mingw32 specified is in
Python25\Lib\distutils\distutils.cfg

I needed to add
[build_ext]
compiler=mingw32

to distutils.cfg, because the [build] option was not used for build_ext
and I got initially a Visual Studio not found error.

After that, the helloworld compiles and runs without problems.

A numpy example ends with a compiler error, I think, because it
doesn't find the numpy header files (numpy core include).
How do I tell pyximport where the numpy files are?

Thanks,

Josef

[Lisandro Dalcin]

On Wed, Nov 4, 2009 at 12:47 PM, <josef...@gmail.com> wrote:
>
> A numpy example ends with a compiler error, I think, because it
> doesn't find the numpy header files (numpy core include).
> How do I tell pyximport where the numpy files are?
>

Enter python, import numpy, print numpy.get_include(), and then add
that path under the [build_ext] section in distutils.cfg, like this:

[build_ext]
include_dirs= <NUMPY INCLUDE DIR>
compiler=mingw32

In short, if you run this:

$ python setup.py <command> --help

for each --<option-name> listed you can add a section with config
values in distutils.cfg, more or less like this:

[<command>]
options_name = value1 value2 value3

So you can add macros, library dirs, libraries, etc... Do you get the idea?

Some day I'll review these pyximport patches in the tracker... Passing
include dirs, and macros, and libraries, and any other stuff required
for building should have been a pyximport feature from the very
beginning...

[josef...@gmail.com]

On Wed, Nov 4, 2009 at 10:47 AM, Lisandro Dalcin <dal...@gmail.com> wrote:
>
> On Wed, Nov 4, 2009 at 12:47 PM,  <josef...@gmail.com> wrote:
>>
>> A numpy example ends with a compiler error, I think, because it
>> doesn't find the numpy header files (numpy core include).
>> How do I tell pyximport where the numpy files are?
>>
>
> Enter python, import numpy, print numpy.get_include(), and then add
> that path under the [build_ext] section in distutils.cfg, like this:
>
> [build_ext]
> include_dirs= <NUMPY INCLUDE DIR>
> compiler=mingw32

With this it works without problems, additionally build_dir can
be set as an option to pyximport.install.

So my basic pyximport usage works well
(and I can drop my hardcoded hacks)

>
> In short, if you run this:
>
> $ python setup.py <command> --help
>
> for each --<option-name> listed you can add a section with config
> values in distutils.cfg, more or less like this:
>
> [<command>]
> options_name = value1 value2 value3
>
> So you can add macros, library dirs, libraries, etc... Do you get the idea?

Yes, I will keep this in mind for next time, when I try to work around
some build problems.

Thanks,

Josef

[laehc]

On Nov 4, 2:01 pm, Lisandro Dalcin <dalc...@gmail.com> wrote:
...

> Mmm.. I've did all my testing with cython-devel... Could you give a
> try? You do not need to intall Mercurial (or TortoiseHG), just point

> your browser to this linkhttp://hg.cython.org/cython/archive/tip.zip

> to que a zip file you can unpack and install.

I've got the zip, but would I have to uninstall Cython first? I'm not
keen on that now I've got Cython more or less working. I've managed to
compile the fib and primes examples.
I hope that doesn't sound ungrateful after all the help I've had, I
would give cython-devel a try if I had a spare PC.
I'm not desparate for advice on pyximport but in case anyone wants to
see the output it's:

>>> import pyximport; pyximport.install()
>>> import helloworld
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "C:\Python26\lib\site-packages\pyximport\pyximport.py", line
288, in load_module
self.pyxbuild_dir)
File "C:\Python26\lib\site-packages\pyximport\pyximport.py", line
154, in load_module
raise ImportError("Building module failed: %s" % e)
ImportError: Building module failed: error: None
>>>

[Lisandro Dalcin]

On Wed, Nov 4, 2009 at 4:39 PM, laehc <niw...@googlemail.com> wrote:
>
> I'm not desparate for advice on pyximport but in case anyone wants to
> see the output it's:
>

Sorry, at this point I'm confused...

1) Are you using latest Cython 0.11.3, right? If you have a previous
version, things will not work.

2) Open the file
C:\Python26\Lib\site-packages\Cython\distutils\distutils.cfg and make
sure it has at least these two lines:

[build_ext]
compiler = mingw32

3) Do you have C:\MinGW\bin (depending where you installed MinGW)
listed in your %PATH% environment variable? If not distutils will not
be able to invoke 'gcc.exe' to compile stuff... But I bet this

Ah!! Now I see in your original post that you are using Cython
0.11.2... You really need 0.11.3 for (2) to work... Please, trust
me... you do not need a binary installer... So please do this:

1) Go to add/remove programs in control panel and remove your Cython
installation.

2) Download Cython-0.11.3 release,
http://www.cython.org/release/Cython-0.11.3.zip

3) Unpack the zip file and enter the Cython-0.11.3 directory in a
console windows

4) run this command:

python setup.py build -c mingw32 bdist_msi

Now you should have a shinny new MSI binary installed inside the
"dist\" folder... So you can see that creating a Python binary
installer is not black magic (at least on the user side), right?

[laehc]

I've installed Cython 0.11.3, using the link and the line:

python setup.py build -c mingw32 bdist_msi

but now Python can't find the Cython.Distutils module.
The installation produced screenfulls of output ending with:


running install_egg_info
Writing build\bdist.win32\msi\Lib\site-packages\Cython-0.11.3-
py2.6.egg-info
creating dist
removing 'build\bdist.win32\msi' (and everything under it)

C:\Python26\Cython-0.11.3>

I have a copy of all the output in case it's needed.
I changed this to take helloworld and didn't touch anything else:

setup.py

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension("helloworld", ["helloworld.pyx"])]
)

Then got this error on the console:

C:\MyPy>python setup.py build_ext --inplace

Traceback (most recent call last):

File "setup.py", line 3, in <module>
from Cython.Distutils import build_ext
ImportError: No module named Cython.Distutils

C:\MyPy>

After appending to the Python path by:

sys.path.append("C:\\PYTHON26\\Cython-0.11.3")

I can do this in PyScripter's interpreter window:

>>> from Cython.Distutils import build_ext
>>> build_ext
<class Cython.Distutils.build_ext.build_ext at 0x044A9A50>
>>>

but I checked to see if Python remembered the path if I close and
restart, and it doesn't. Could that be the cause of this problem?
Keeping Python open with the path set and the module imported does not
stop the module from being missing in the console.

Should I have installed Cython into site-packages? C:\Python looked as
if it might be the default so I left it alone. The install put nothing
into C:\Python26\Lib\site-packages, although the install produced
lines like this:

copying build\lib.win32-2.6\Cython\CodeWriter.py -> build
\bdist.win32\msi\Lib\site-packages\Cython
creating build\bdist.win32\msi\Lib\site-packages\Cython\Compiler
copying build\lib.win32-2.6\Cython\Compiler\Annotate.py -> build
\bdist.win32\msi
\Lib\site-packages\Cython\Compiler

[josef...@gmail.com]

These are general questions for installing a python package and
not cython specific.

python setup.py build -c mingw32 bdist_msi

only creates the installer but does not actually install the package
nor add it to the python path. The messages at the end just indicate
the paths in the build and installer not in your python site packages.

The build creates a dist folder which should contain the installer
that you can use to install the package into python's site-packages.

As an alternative, for the installation of packages, I also use
python setup.py install
or I use
easy_install -U zipfileofpackage

It is also possible to add a package to the python path without
installing in site-packages, but I think that's only relevant if you
want to edit the package.

(I'm not sure about the details of an msi installer, since I usually
install from a zipfile created with bdist)

Hope that helps,

Josef

[laehc]

It's OK, I've got it working. I set the path in an environment
variable. I'll post details in a second in case anyone else with a
similar problem gets this far. Anyway I can compile and import the
helloworld example. Thanks again for all the help, and you are right,
Lisandro, it isn't black magic (to the user) although I was having
some doubts for a while.

[laehc]

How to put the Cython path on the end of the PYTHONPATH environment
variable in Windows Vista.

(In case any newbies like me need to know.)

Click Start
right click Computer (in the list on the right)
click on Properties
click 'Advanced system settings' (it's on the left)
click 'Continue' on User Account Control
click Environment Variables (near bottom right)
Find the variable called PYTHONPATH in the list in the bottom half,
and select it.
click on Edit
A little window comes up.
Click in the text in the box labelled 'Variable value',
this has paths separated by semicolons.
Put a semicolon on the end, followed by the Cylon path,
for example like this:
;C:\PYTHON26\Cython-0.11.3
click OK as many times as necessary

[laehc]

I still can't get pyximport to work. I've added it to PYTHONPATH but
it still doesn't work.

>>> import pyximport; pyximport.install()
>>> import helloworld

Traceback (most recent call last):

File "<interactive input>", line 1, in <module>

File "C:\PYTHON26\Cython-0.11.3\pyximport\pyximport.py", line 288,
in load_module
self.pyxbuild_dir)
File "C:\PYTHON26\Cython-0.11.3\pyximport\pyximport.py", line 154,

[Lisandro Dalcin]

On Thu, Nov 5, 2009 at 4:13 PM, laehc <niw...@googlemail.com> wrote:
>
> It's OK, I've got it working. I set the path in an environment
> variable. I'll post details in a second in case anyone else with a
> similar problem gets this far. Anyway I can compile and import the
> helloworld example.
>

OK.

> Thanks again for all the help, and you are right,
> Lisandro, it isn't black magic (to the user) although I was having
> some doubts for a while.

Please, do not read me wrong... I was just trying to point that
getting a binary installer is REALLY easy... Of course, you have to
know how to do it ;-).

[Christopher Barker]

Lisandro Dalcin wrote:
> Some day I'll review these pyximport patches in the tracker... Passing
> include dirs, and macros, and libraries, and any other stuff required
> for building should have been a pyximport feature from the very
> beginning...

+1

This would be great if you can find the time.