web2py scheduler period and repeats result not expected

[黄祥]

hi,

i tested and learned from web2py documentation about web2py scheduler but ended with result not expected for period and repeats.
e.g.
models/scheduler.py
db.define_table('asdf',
    Field('asdf'),
    auth.signature    )

from gluon.scheduler import Scheduler

def demo1():
    db.asdf.insert(asdf = 'asdf')
    db.commit()

scheduler = Scheduler(db, tasks = dict(demo1 = demo1) )

scheduler.queue_task('demo1',
                     repeats = 0, period = 180)

running from windows command line :
cd C:\web2py
python web2py.py -K test -X

result not expected
1. db.asdf created on and modified on is not in period range. my expectation is created on is every 3 minutes (180 seconds) according to configuration above.
2. db.scheduler_task create a lot of records every something seconds.

my question is how can i handle those 2 problems above?

thanks and best regards,
stifan

[Tim Richardson]

Except that I pass the function, rather than a string, my usage is similar to yours:

scheduler.queue_task(demo1, 
                     repeats = 0, period = 180,prevent_drift=True)

[黄祥]

i've already tried, but got the same result + another strange output during start web2py. here is the detail step i took.
1. create new web2py app (copas the welcome folder and rename it into test)
2. copas the scheduler.py into models (test\models\scheduler.py)

test\models\scheduler.py

db.define_table('asdf',
    Field('asdf'),
    auth.signature )

from gluon.scheduler import Scheduler

def demo1():
    db.asdf.insert(asdf = 'asdf')
    db.commit()

scheduler = Scheduler(db, tasks = dict(demo1 = demo1) )

scheduler.queue_task('demo1', prevent_drift = True,
                     repeats = 0, period = 5)

3. running from windows command prompt :

cd C:\web2py
python web2py.py -K test -X

4. go to http://127.0.0.1:8000/test/appadmin/index and check the record of table asdf, scheduler_task, scheduler_run and scheduler_worker

no error traceback occured but the output is not expected for period of the record created on table 'asdf' and had a lot of scheduler_task records on the table.
another unexpected bonus is when running with the prevent_drift = True, the command prompt is hung for the first time, so that i must terminate the command line and then execute again, yet the prevent_drift = True result for period is still not expected.

[Niphlod]

@guys......if you queue a task inside models, you're queuing a new task at every request ! ;-D

[黄祥]

a, i c, so that's why scheduler_task table is have a lot of record. thank you so much simone. btw, what i want to learn in here is to have a scheduler to create a new record in database every day, how can i achieve it using testing example above?

what i have tried is :
models/db.py

db.define_table('asdf',
    Field('asdf'),
    auth.signature )

controllers/scheduler.py

from gluon.scheduler import Scheduler

def demo1():
    db.asdf.insert(asdf = 'asdf')
    db.commit()

scheduler = Scheduler(db, tasks = dict(demo1 = demo1) )

scheduler.queue_task('demo1', prevent_drift = True,
                     repeats = 0, period = 5)

it seems the scheduler is execute everytime i access http://127.0.0.1:8000/test/scheduler/demo1, and i think it's not a scheduler, it's manually execute.

any idea how to achieve it using web2py way?

[Niphlod]

you're missing a BIIIIG point. 

Either you need to queue a task as a result of some user action (i.e. send a reminder, calculate/aggregate/refresh some data, etc) --> usually it's a one-time-only activity, in which case, the following "pattern" applies

#models\scheduler.py

from gluon.scheduler import Scheduler

def demo1():
    db.asdf.insert(asdf = 'asdf')
    db.commit()

mysched = Scheduler(db, tasks = dict(demo1 = demo1) )

#controllers\a_controller.py
def an_action():
       ......
       #we need to schedule a background task
       mysched.queue_task('demo1')

Or you need a single task that repeats itself every once in a while (usually repeating tasks are "cron-like" and "maintenance-related"). 

In that case, you define the scheduler in models\scheduler.py, and you (as the admin-developer-god of the web application) queue a task in appadmin. 

Or you prepare a controller just for the occasion, that can only be accessed by you and run once (so you get only a SINGLE record in the scheduler_task table).

Now, back to your goal. If you want a single record to be added to the asdf table every day, you need a SINGLE task that has

repeats --> 0

period --> 60*60*24

prevent_drift --> True (if you care for "at the same time every day")

[黄祥]

yeah, i realize that something i'm lacked off, unfortunately, i don't know where is it. tried again using your code but still no luck
started from scratch

models\db_wizard_0.py

db.define_table('asdf',
    Field('asdf'),
    auth.signature )

models\scheduler.py
from gluon.scheduler import Scheduler

def demo1():
    db.asdf.insert(asdf = 'asdf')
    db.commit()

mysched = Scheduler(db, tasks = dict(demo1 = demo1) )

controlelrs\test.py
def queue_task():
    mysched.queue_task('demo1')
"""
    mysched.queue_task('demo1', prevent_drift = True,

                       repeats = 0, period = 5)
"""

command prompt
cd C:\web2py
python web2py.py -K test -X

checked the scheduler_task, scheduler_worker and asdf tables via appadmin.

count to 5 or 10 and then refresh the scheduler_task, scheduler_worker and asdf tables on appadmin.

nothing happen

i even created the scheduler_task manually from appadmin (http://127.0.0.1:8000/test/appadmin/insert/db/scheduler_task), but still no luck (no error occured, but the scheduler didn't run).
execute the test controller (http://127.0.0.1:8000/test/test/queue_task) got the same result.
for testing purpose, i set the period = 5, so that, i don't have to long waiting for the task execute by the scheduler.

i'm running on windows 7 64 bit, with web2py latest source code version, python 2.7.

any idea, where is my mistake on code above or step i took to run the scheduler?

[Niphlod]

ok. steps.

web2py.py -a yourpassword

go to /appname/test/queue_task

go to /appname/appadmin/select/db?query=db.scheduler_task.id>0

if nothing is there, then in controller test.py the function queue_task is not queuing the task.

if a single row is there, then, open a terminal and start the scheduler process with

web2py.py -K appname

wait 5 seconds, go to /appname/appadmin/select/db?query=db.scheduler_worker.id>0

if nothing is there, your worker process isn't actually working, or is unable to talk to the db.

if a single row is there, then it's working.

wait a few seconds, go to  /appname/appadmin/select/db?query=db.scheduler_task.id>0

check that the status turns to COMPLETED.

You can now go to  /appname/appadmin/select/db?query=db.asfd.id>0

to check for the record inserted by the task.

[黄祥]

yes, followed your step, i understood the logic flow of the web2py scheduler and made it work, thank you so much, simone for the pointers. just wondering, the precision for second is not exact on created_on, tested using period = 5 and period = 60
here is the steps i took :
cmd
cd C:\web2py\applications
xcopy welcome test /s /e /h
D

cmd
notepad C:\web2py\applications\test\models\db_wizard_0.py

db.define_table('asdf',
    Field('asdf'),
    auth.signature )

cmd
notepad C:\web2py\applications\test\models\scheduler.py

from gluon.scheduler import Scheduler

def demo1():
    db.asdf.insert(asdf = 'asdf')
    db.commit()

mysched = Scheduler(db, tasks = dict(demo1 = demo1) )

cmd
notepad C:\web2py\applications\test\controlelrs\test.py
def queue_task():
    #mysched.queue_task('demo1')

    mysched.queue_task('demo1', prevent_drift = True, repeats = 0, period = 5)

cmd
cd C:\web2py
python web2py.py -a a

web browser
http://127.0.0.1:8000/test/test/queue_task

http://127.0.0.1:8000/test/appadmin/select/db?query=db.scheduler_task.id%3E0

cmd

cd C:\web2py
python web2py.py -K test

web browser
http://127.0.0.1:8000/test/appadmin/select/db?query=db.scheduler_worker.id%3E0

http://127.0.0.1:8000/test/appadmin/select/db?query=db.scheduler_task.id%3E0

http://127.0.0.1:8000/test/appadmin/select/db?query=db.asdf.id%3E0

[Niphlod]

On Thursday, June 26, 2014 11:17:21 PM UTC+2, 黄祥 wrote:

yes, followed your step, i understood the logic flow of the web2py scheduler and made it work, thank you so much, simone for the pointers. just wondering, the precision for second is not exact on created_on, tested using period = 5 and period = 60

it'll never be. Scheduler is not precise to the second. When you queue a task, you specify a start_time that doesn't mean "launch the task at this time" but "from this time, you CAN launch this task".  

[黄祥]

a, i understood, the puzzle completed now. thank you so much, simone, for detail explaination about web2py scheduler.