Newbie trying to debug cython
Aviv Dorian
which is supposed to be similiar to make install, but it also let's
you uninstall stuff which is the reason I'm using it.
If I enter the python interactive shell, I'm able to import pyximport
and I succeded in
importing Cython code with it and using it.
And I also successfuly used a setup.py file to compile Cython code
into an .SO file
The problem:
I started following the tutorial on the website on how to Debug:
http://docs.cython.org/src/userguide/debugging.html#running-the-debugger
The main problem is that I do not have the command "cydbg", I'm not
sure why,
apparently it didn't get installed, I did a search on the entire file
system and a file named cydbg doesn't exist, there is "Cydbg.py" in
the /usr/local/lib/python2.7/dist-packages/Cython/Debugger.
I downloaded the tar.gz from the PyPI repository and added the "bin"
directory from the tar.gz to my path and debugging didn't work, the
cydbg shell comes up but all the commands, don't work or anything.
Please help me sort this out, Cython is great but I have to be able to
debug it.
Thank you very much!
mark florisson
> Hi I'm on Ubuntu and I installed Cyhton from PyPI using the tool pip,
> which is supposed to be similiar to make install, but it also let's
> you uninstall stuff which is the reason I'm using it.
>
> If I enter the python interactive shell, I'm able to import pyximport
> and I succeded in
> importing Cython code with it and using it.
> And I also successfuly used a setup.py file to compile Cython code
> into an .SO file
>
> The problem:
>
> I started following the tutorial on the website on how to Debug:
>
> http://docs.cython.org/src/userguide/debugging.html#running-the-debugger
>
> The main problem is that I do not have the command "cydbg", I'm not
> sure why,
> apparently it didn't get installed, I did a search on the entire file
> system and a file named cydbg doesn't exist, there is "Cydbg.py" in
> the /usr/local/lib/python2.7/dist-packages/Cython/Debugger.
It appears it's not included in the setuptools build, we'll fix that.
> I downloaded the tar.gz from the PyPI repository and added the "bin"
> directory from the tar.gz to my path and debugging didn't work, the
> cydbg shell comes up but all the commands, don't work or anything.
How do the commands not work? You should load python in the debugger
like 'file python' and then you can running your program, setting
breakpoints, etc (using 'cy run', 'cy break', etc). You do need Cython
debug information, so you need to compile your Cython source with
--gdb, or if you just want to run without debug info, you can start
the debugger with 'cygdb --' (and optionally load in debug info
later).
For now I suppose you could use the normal build from source and
uninstall with 'rm'.
Aviv Dorian
mark florisson
> Can you tell me how I know which files to remove if I want to uninstall?
>
> Also I don't really understand what you mean by "load python in the
> debugger"
>
> here's what I did exactly.
>
> I made a file called "fib.pyx" with code from the tutorial (code that
> calculates Fibonacci series).
> Next I compiled it using the command "python setup.py build_ext --pyrex-gdb
> --inplace"
>
> I also trited "python-dbg setup.py build_ext --pyrex-gdb --inplace",
> but it gave me an error (I have the python-dbg and python-dev packages
> installed):
>
> running build_ext
> failed to import Cython:
> /usr/local/lib/python2.7/dist-packages/Cython/Compiler/Code.so: undefined
> symbol: Py_InitModule4
> error: Cython does not appear to be installed
> [35053 refs]
The most important thing is that you have debug symbols, i.e. a
non-stripped python build.
You get that error because Cython was compiled for a different
installation, which you can get around by simply putting
/path/to/cython/dir in your PYTHONPATH and not building it.
The reason you get that error is that ubuntu and debian think they're
"smart" by modifying Python so that a debug build will pick up modules
named 'foo_d.so' with a fallback to 'foo.so', which means that in
reality you get cryptic errors instead of "ImportError: No module
named blah". The reason python-dbg is useful on ubuntu is that it is
not stripped in addition to being an actual python debug build.
> Next I executed "cygdb" - without any arguments
>
> GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2
>
> (gdb) cy run
> Python 2.7.1+ (r271:86832, Apr 11 2011, 18:05:24)
> [GCC 4.5.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> cy next
> File "<stdin>", line 1
> cy next
> ^
> SyntaxError: invalid syntax
>
> It gives an error, I don't really understand how gdb works, I always
> debugged with IDEs that have GUIs so I'm
> not too good at this. I tried writing "file python" like you said:
The problem is that you run python without arguments. You then end up
executing debugger commands in a python shell (the debugee). What you
want to do is:
cy run -c 'import my_cython_module'
to run a subprocess python that imports my_cython_module.
You can then return to the debugger in various ways, e.g. the process
may end, or it may segfault or abort, or it may run forever and you
may interrupt it with ^C. You may also want to set a breakpoint in
advance.
mark florisson
I personally wouldn't worry about it, but sys.prefix and
sys.exec_prefix will tell you where things are installed. It's
typically easiest to just import a package and remove that, e.g.
python -c 'import Cython; print Cython.__file__'. There will also be a
few scripts in <sys.exec_prefix>/bin.
I think for the debug version you'd be better off cloning your debug
build with virtualenv, and installing Cython in that. You can always
remove everything with rm -rf myvirtualenv if you're worried about
diskspace.
Aviv Dorian
mark florisson
> Thank you so much, It seems to be working.
>
> I added the source folder to the PYTHON path, compiled the cython code again
> using:
> python-dbg setup.py build_ext --pyrex-gdb --inplace
>
> Than I executed 'cygdb'
> when in the dgb shell I executed 'cy break fib.fib'
> than 'cy run -c 'import fib; fib.fib(200)'
> and it works!
>
> Thanks a lot, and one more question:
>
> Is there a chance I can make it work through an IDE suck as Eclipse or
> Netbeans that you know of?
Ironic misspelling of "such" :) I believe Eclipse talks to GDB over
the GDB machine interface, and that doesn't currently allow exposure
of Python functionality. I believe that at this point this would
require modifications to both GDB and Eclipse. An alternative would be
to implement the same functionality as cygdb on top of Eclipse's C/C++
debugging support. It's certainly all possible but you'd be in for
quite a bit of work.
If you really want to know more about it I'd refer you to this paper,
along with its references:
http://fmt.cs.utwente.nl/files/sprojects/72.pdf
Aviv Dorian
mark florisson
> I don't think I have enough knowledge right now to do something like that,
> I hardly even understand how debuggers work.
>
> I thought maybe it's possible to use cygdb directly through a graphical
> software,
> than have the software interact with it by sending commands like "cy run",
> "cy break" and so on.
>
> I went over the paper, didn't understand all of it but at the end there's a
> diagram that does make it
> look like cygdb was designed with the intention of allowing IDEs to interact
> with it.
That would have indeed been preferable. Some GUIs, I believe Insight
is one of them (http://sourceware.org/insight/), talk to GDB over a
pty. I believe it was abandoned because this is quite bothersome and
slight variations in the output may break things. In an ideal world
where GDB has a mature Python API, it would allow exposing things over
the machine interface.
However, one way to get around that would be to circumvent GDB's event
loop altogether, and start a simple server (e.g. twisted) inside GDB
to which you connect from the debugger's GUI. The server could then
delegate calls to the Cython debugger, which would return sane output
(this would require some modifications to the Cython debugger). The
other way to implement this currently (without modifying GDB) would be
to hook GDB to a pseudo terminal and run commands/parse output. I
sincerely believe that such an approach is inherently fragile.
If you want this for Eclipse you'd have to hook into the CDI, this
link has an introduction:
http://www.ibm.com/developerworks/opensource/library/os-eclipse-cdt-debug1/index.html
. This is all a lot to take in, but considering PyDev seems to have
Cython support (http://pydev.blogspot.com/2011/10/pydev-cython.html)
I'm sure many people would be happy to see this happen.
If at any point you want help or advice, feel free to contact me or
the cython-dev mailing list. The people in #gdb on irc.freenode.net
are also helpful and gdb also has a mailing list. If you want to know
more about GDB's Python API there is documentation here:
http://sourceware.org/gdb/onlinedocs/gdb/Python.html#Python and there
was a related talk at PyCon 2011 available here:
http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-using-python-to-debug-c-and-c-code-using-gdb-4895525