Launching without breaking

[Yang Zhang]

Is it possible to launch Python programs with the rpdb2 server
enabled, but without having to instrument all our code (with
rpdb2.start_embedded_debugger()) or interactively start programs?

The closest solution is rpdb2 -d, but that requires both an
interactive password prompt and a subsequent attach/go/detach. This
would be immensely useful for allowing arbitrary debugging of
long-running programs (especially if they're started
non-interactively), even in production.

Another nice feature, for additional security, is to use Unix sockets
instead of TCP sockets where available, since authentication is taken
care of by setting the permissions on the socket (no more passwords).
But for now, even localhost-only TCP listeners with empty passwords is
sufficient for our needs.

Thanks.
--
Yang Zhang
http://yz.mit.edu/

[Nir Aides]

Hi Yang,

I don't recommend having any debugger unconditionally active (silently) in your code. Python debuggers hook themselves into the interpreter using various techniques including monkey patching and may subtly affect the running program.

I recommend that you either:

1) write a few lines of code to respond to some event (e.g. configuration change, rpc call, etc...) by importing and starting the debugger. Do not import rpdb2 before that point.

2) Use rfoo.rconsole http://code.google.com/p/rfoo/

rconsole is a probe that can be used to inspect the state of long running Python programs including Linux daemons. It should not have any effect on the host program, but note that it currently accepts any local socket connection. You can use it to import rpdb2 and start it on demand.

Nir

--
You received this message because you are subscribed to the Google Groups "Winpdb" group.
To post to this group, send email to win...@googlegroups.com.
To unsubscribe from this group, send email to winpdb+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/winpdb?hl=en.

[Yang Zhang]

Thanks, rconsole looks interesting. I'm trying to build/install rfoo,
but I'm getting the following error - any hints?

Running setup.py egg_info for package rfoo
Unable to execute darcs -- if you are building a package with
'setup.py sdist', 'setup.py bdist_egg', or other package-building
commands, then the resulting package might be missing some files. If
you are not building a package then you can ignore this warning.
Installing collected packages: rfoo
Running setup.py install for rfoo
building 'rfoo.marsh' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c rfoo/marsh.c -o
build/temp.linux-x86_64-2.6/rfoo/marsh.o
gcc: rfoo/marsh.c: No such file or directory
gcc: no input files
error: command 'gcc' failed with exit status 1
Complete output from command /home/yang/work/pod/env/bin/python -c
"import setuptools;
__file__='/home/yang/work/pod/env/build/rfoo/setup.py';
execfile('/home/yang/work/pod/env/build/rfoo/setup.py')" install
--single-version-externally-managed --record
/tmp/pip-ABCj7V-record/install-record.txt --install-headers
/home/yang/work/pod/env/include/site/python2.6:
running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.6

creating build/lib.linux-x86_64-2.6/rfoo

copying rfoo/__init__.py -> build/lib.linux-x86_64-2.6/rfoo

copying rfoo/_rfoo.py -> build/lib.linux-x86_64-2.6/rfoo

creating build/lib.linux-x86_64-2.6/rfoo/utils

copying rfoo/utils/__init__.py -> build/lib.linux-x86_64-2.6/rfoo/utils

copying rfoo/utils/rconsole.py -> build/lib.linux-x86_64-2.6/rfoo/utils

running build_ext

building 'rfoo.marsh' extension

creating build/temp.linux-x86_64-2.6

creating build/temp.linux-x86_64-2.6/rfoo

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c rfoo/marsh.c -o
build/temp.linux-x86_64-2.6/rfoo/marsh.o

gcc: rfoo/marsh.c: No such file or directory

gcc: no input files

error: command 'gcc' failed with exit status 1

----------------------------------------
Command /home/yang/work/pod/env/bin/python -c "import setuptools;
__file__='/home/yang/work/pod/env/build/rfoo/setup.py';
execfile('/home/yang/work/pod/env/build/rfoo/setup.py')" install
--single-version-externally-managed --record
/tmp/pip-ABCj7V-record/install-record.txt --install-headers
/home/yang/work/pod/env/include/site/python2.6 failed with error code
1
Storing complete log in /home/yang/.pip/pip.log

[Nir Aides]

Do you have cython installed? it is a dependency.

[Yang Zhang]

Yeah, cython is installed (I was getting a different error before
cython was installed).

[Nir Aides]

What distribution of Linux are you using?

[Nir Aides]

I fired up a VM with vanilla ubuntu 10.04

The required packages are:

cython

python-support (installed on ubuntu as dependency cython)

python-dev

build-essentials

Once these packages are in place, build passes smoothly.

Let me know how it went and I will update the docs.

Nir

[Yang Zhang]

I'm also using Ubuntu 10.04, and all those packages are installed.

Strangely, running `python setup.py build` manually works, but not
`pip install rfoo`. Any ideas on why?

[Nir Aides]

yes, neither pip nor easy_install can install any package dependent on cython without considerable hackery. 

It is a known bug.

I will add a prominent warning to the google code project page. thanks.

Nir

[Nir Aides]

Hi Yang,

Did you eventually find rconsole useful?

Nir

[Yang Zhang]

Hi Nir,

I tried using it, and it works nicely as advertised, but currently
browsing things feels cumbersome. I have no global variables to
directly inspect from the rconsole thread, so I need to traverse frame
objects and look at their f_locals (at least with debuggers, I have
a smoother experience jumping around frames and whatnot).

Do you have any tips on how to effectively inspect/debug code from rconsole?

Thanks,

Yang

[Nir Aides]

Hi Yang, 

rconsole is not intended to be used as debugger and in particular I do not recommend to use it to traverse frame objects. The rconsole server should be started from the global scope of a module and may be used to inspect or modify its namespace or that of any other module imported into the namespace of that module.

I use it effectively to complement logging information by inspecting specific data structures directly. rconsole effectively helped me locate a hard to track down memory leak in a long running production daemon server by a combination of traversing global data structures and importing guppy.hpy on demand a few days after the server started running.

Nir