Is it possible to step through scheduler task code in debugger?

[Stephen Weiss]

I want to use a debugger (winpdb) to step through a scheduler module.

However, when I run the scheduler in the debugger (-K option) and I set a breakpoint in my module and submit a job to the scheduler_task table,
I get debug output (prints) in the console, but the breakpoint is never hit.  The row in the scheduler_task table changes to COMPLETED.

Thus, I know the scheduler is running my module in the debugger, but it just won't stop at the breakpoint I have set.

I believe the issue is with the multi-threaded nature of the scheduler and debugger, but I don't know how to resolve the problem.

Any help would be greatly appreciated.

Thanks!

[Niphlod]

it's a "general issue" with debugging remote processing http://pydev.org/manual_adv_remote_debugger.html

BTW: if you launch the function in the shell, you won't have access to the inner workings of the scheduler but you'll see what our function/module/etc is doing.... the env in which tasks are executed within the scheduler is pretty much the same of web2py's shell with models loaded (./web2py.py -M -S appname).

[Tim Richardson]

On Thursday, July 24, 2014 7:41:03 PM UTC+10, Niphlod wrote:

it's a "general issue" with debugging remote processing http://pydev.org/manual_adv_remote_debugger.html

BTW: if you launch the function in the shell, you won't have access to the inner workings of the scheduler but you'll see what our function/module/etc is doing.... the env in which tasks are executed within the scheduler is pretty much the same of web2py's shell with models loaded (./web2py.py -M -S appname).

I use a graphical IDE (Pycharm) and if I get stuck with a problem needing debugging, I run the scheduler job via the web server (in development, i.e. via the built-in Rocket server). That is, I make a controller which calls exactly the same thing as the scheduler would.  So you just debug it normally (which is remote debugging although It Just Works). As Niphlod points out, this is for practical purposes the same environment.  The scheduler process doesn't run the server code which allows remote debuggers to connect (I think).

[Stephen Weiss]

FWIW I did finally get it working, to an extent.

I'm working on a Windows 7 machine to do this.

In one command tool window, I'm running my web2py application.
In another command tool window, I'm running the scheduler for my web2py application.

In the model that contains the scheduled task that gets called I placed the following two lines along with the import of rpdb2 elsewhere.

   logger.debug('AWAITING DEBUGGER CONNECTION TO CONTINUE')
   rpdb2.start_embedded_debugger("whatever")

Note that I comment these lines out when not needing the debugger.

I submit a job to the scheduled_task table.

When I see the 'AWAITING DEBUGGER CONNECTION TO CONTINUE' message in the scheduler command tool window, I go to yet another command tool and start winpdb and go through the attach.

This connects me to the proper process and I can step through it.

NOTE: You should change the timeout to the queue task call to something like 600 seconds so that web2py doesn't think the process took too long to run and reaps it while you're debugging.

[Niphlod]

NB: shell and scheduler are pretty much the same.... scheduler and web not so much, although models are the same.