Questions on the scheduler

[blackthorne]

- Is it possible to call scheduled_workers functions asynchronously?
How?
- How should I adapt the behavior of a scheduled_worker as changing
its model and adding new functionality?

Thank you

[Massimo Di Pierro]

Not sure I understand. You do not call scheduled_workers functions.
You schedule tasks. Workers pick them up from queue an execute them
using the available power.

[Vinicius Assef]

Yet on schedules, if my task failed or timed out, will web2py try to
run it next time?

[blackthorne]

Massimo:
For instance, I created a method that returns the progress of the task
that I am running
I guess my questions could collapse into this example:

Consider,
in a db file

def plugin_run(plugin, **kw):
plugin_obj = load_plugin(plugin, **kw)
plugin_obj.start()
return plugin_obj.report() #this shall be a dictionary for
appearing in results properly

myscheduler = Scheduler(db_tasks, dict(plugin_run=plugin_run))

Eventually that plugin_obj will be picked up by a worker...
now, that it is running it is taking a lot of time and I wanted to do
something like..
from db_scheduler_worker where it is running task with
task_name="demo"
call "plugin_obj.progress()" defined inside





On Feb 10, 5:39 pm, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:

[blackthorne]

Vinicius:
I don't think so, it will be stored in db.scheduler_run with status =
'FAILED'

[Brian M]

Viniciusban - if a scheduler task fails or times out then no it will not be run again unless you change the status back to queued. Guess it is good because it makes you handle possible errors and prevents runaway tasks, but rather annoying during testing since you must reenable after errors or if things happen to take longer than initially expected.

[Massimo Di Pierro]

You can

db(db.scheduler_run.status=='FAILED').update(status='QUEUED')

and they will run again.

[blackthorne]

Sorry to rehash this topic, this is critical to me and eventually to you.

Scheduler in web2py works great. 

What I need is to access the workers asynchronously and invoke methods on it so that I can pause/resume/abort them and check on its level of progress.

Any ideas on this?

Thank you

[Wikus van de Merwe]

If all what you need is to report th task progress, you could periodically write the amount of work done to db from within the task. Then query db asynchronously with ajax to show it to the user. This could be done by extending the "scheduler_run" table.

I'm not sure if there is a way to stop a running task. I guess you would have to kill the worker process to do that. Pause/resume would be even more difficult as the state of a task have to be stored. Probably it would be easier to do using a task queue framework such as Celery [1] or implementing your own queue using e.g. RabbitMQ [2].

[1] http://celeryproject.org/
[2] http://www.rabbitmq.com/tutorials/tutorial-two-python.html

[Francisco G. T. Ribeiro]

Thank you for your answer. 

I am not very interested in importing more dependencies even though I know those projects you mention are great (I really like Erlang..).

Anyway you gave me an idea that could solve it. Implement a "messaging system" in web2py by keeping a table with those contents as "flags" that would be cyclically verified by the workers... that could work...

Thanks