Setting up the scheduler, and letting it run from a certain point of time

Lucas Schreiber

unread,

Feb 23, 2016, 6:11:47 PM2/23/16

to web2py-users

Hi there,

while trying to understand how to use the scheduler, i found this post:

https://groups.google.com/forum/#!topic/web2py/VCPZmSc0vLc

In the Post, this code is writen:

db.scheduler_task.insert(function_name='task1',

                         task_name='task1',

                         stop_time = now + timedelta(days=90000),

                         repeats=0,

                         period=10)

g

Where do i put this code? in the model? I think this was partly answered in the post, but i dont understand it. can someone explain this to me?

Also, is there a way to let a scheduler run at a time writen in a db once and then rest until the next time occurs? or is a permanent run every few seconds checking for certain criteria the better solution?

And just to see if I understood this so far correctly:

the scheduler function is basically a function I write into the model, offering everything a "normal" function offers, exectued at times i determine?

Kind regards 

And

Dave S

unread,

Feb 23, 2016, 7:10:42 PM2/23/16

to web...@googlegroups.com

On Tuesday, February 23, 2016 at 3:11:47 PM UTC-8, Lucas Schreiber wrote:

Hi there,
while trying to understand how to use the scheduler, i found this post:
https://groups.google.com/forum/#!topic/web2py/VCPZmSc0vLc

(I think there are some new posts, and reading the Web2Py book is very helpful:

<URL:http://web2py.com/books/default/chapter/29/04/the-core#web2py-Scheduler>

)

In the Post, this code is writen:
db.scheduler_task.insert(function_name='task1',
task_name='task1',
stop_time = now + timedelta(days=90000),
repeats=0,
period=10)
g
Where do i put this code? in the model? I think this was partly answered in the post, but i dont understand it. can someone explain this to me?

You put it where ever you need it. It can be executed in a Web2Py shell, it can be in a controller that responds to an input, etc. Ooops, I just re-read what you have there ... don't insert directly into the table, call queue_task() where ever you need to schedule a task.

(Note, there are a LOT of fields in the database table, this post will give you some idea:

<URL:https://groups.google.com/d/msg/web2py/V4FftTa0CfE/Yas8cNWUEAAJ>

)

I have my queue_task() in the model, but it is only executed when I don't already have a task queued. Since model files get loaded on each request, you don't want to be too careless about queuing a task there. Luckily, there is task_status().

Also, is there a way to let a scheduler run at a time writen in a db once and then rest until the next time occurs? or is a permanent run every few seconds checking for certain criteria the better solution?

Check out the other parameters of the queue_task() call (follow the book URL). There's a starttime as well as a period.

And just to see if I understood this so far correctly:
the scheduler function is basically a function I write into the model, offering everything a "normal" function offers, exectued at times i determine?

Essentially true. There is a request environment, but some fields that only apply to http requests aren't used, and some scheduler-specific fields are used. My code is mostly in a module file, so I have to use an instance of current to pass relevant information.

HTH

/dps

Lucas Schreiber

unread,

Feb 24, 2016, 1:10:54 PM2/24/16

to web2py-users

Alright, so far i am able to create the tables, but nothing runs.

So far i have this:

Model:

database:

dba = DAL(connection)

dba.define_table('test_table',

Field('test_table_id', 'integer'),

Field('test_table_value'))

scheduler:

from gluon.scheduler import Scheduler

scheduler = Scheduler(dba)

controller:

index:

def start():

import datetime

daytime_start = datetime.datetime(2017, 1, 1, 0, 0)

a = 'a'

dba.scheduler_task.insert(function_name='task_1',

task_name='task_1',

stop_time = daytime_start,

repeats=0,

period=10)

return dict(a=a)

def index():

text1 = 'test'

return dict (text1=text1)

module:

test.py:

def task_1():

c= 0

dba.test_table.insert(test_table_id = 1, test_table_value = 'a')

db.commit()

return dict(c=c)

How do i get it startet, e.g. filling the test_table with those values every n seconds?

Thank you very much for your help so far

Dave S

unread,

Feb 24, 2016, 1:38:52 PM2/24/16

to web2py-users

On Wednesday, February 24, 2016 at 10:10:54 AM UTC-8, Lucas Schreiber wrote:

Alright, so far i am able to create the tables, but nothing runs.

So far i have this:

Model:
database:
dba = DAL(connection)
dba.define_table('test_table',
Field('test_table_id', 'integer'),
Field('test_table_value'))

scheduler:

from gluon.scheduler import Scheduler
scheduler = Scheduler(dba)

controller:
index:

def start():
import datetime
daytime_start = datetime.datetime(2017, 1, 1, 0, 0)
a = 'a'
dba.scheduler_task.insert(function_name='task_1',
task_name='task_1',
stop_time = daytime_start,
repeats=0,
period=10)

DO NOT DO THIS! *********

Use the queue_task() call. There is more than one table involved, and rather than figuring out how they all fit together, use the code that already knows.

return dict(a=a)
def index():
text1 = 'test'
return dict (text1=text1)

module:

test.py:
def task_1():
c= 0
dba.test_table.insert(test_table_id = 1, test_table_value = 'a')
db.commit()
return dict(c=c)

How do i get it startet, e.g. filling the test_table with those values every n seconds?

Thank you very much for your help so far

If you haven't read the section in the book, I don't think I can be much help.

/dps

Lucas Schreiber

unread,

Feb 24, 2016, 7:57:43 PM2/24/16

to web2py-users

I have read the book. But it doesn't state where to actually write the code snipes given. Both, the book and google is very fancy about explaining how to start a scheduler by the console. But i literally do not see how i can start the scheduler.

If i write this into the controller,

scheduler.queue_task(
    task_1,
    pargs=[],
    pvars={},
    start_time=now, 		#datetime
    stop_time = None,		#datetime
    timeout = 60,               #seconds
    prevent_drift=False,
    period=60,                  #seconds
    immediate=False,
    repeats = 1
)

it atleast creates the tables and populates one of the tables with an set of records(scheduler_task) and does so whenever i call the function. But it only works as long as the function is in the same controller. if the function is in the modules, an error is returned. When i put this into the model, it creates the tabels but does not populate any of them.

some possible reasons why it does not run i have made up are:

-i'm queueing tasks but not creating a worker to work on them (but the book doesnt state anywhere to create a worker. I think therefore this cant be the solution)

-the function isn't defined proper and the worker does not know what to do.

Am Mittwoch, 24. Februar 2016 00:11:47 UTC+1 schrieb Lucas Schreiber:

Dave S

unread,

Feb 24, 2016, 8:30:33 PM2/24/16

to web2py-users

On Wednesday, February 24, 2016 at 4:57:43 PM UTC-8, Lucas Schreiber wrote:

I have read the book. But it doesn't state where to actually write the code snipes given. Both, the book and google is very fancy about explaining how to start a scheduler by the console. But i literally do not see how i can start the scheduler.

<URL:http://web2py.com/books/default/chapter/29/04/the-core#Scheduler-Deployment>

the -K option starts a process which runs the scheduler.

You can use the -X option with it to only have 1 instance of Web2Py running, with the Rocket server providing the internet access.

If you are using Apache or Nginx, I suppose you don't use -X, but I'm not quite graduated to that class yet.

(in the command-line options section,

<URL:http://web2py.com/books/default/chapter/29/04/the-core#Command-line-options>

the -K is illustrated as "-K SCHEDULER",

which might be confusing if you don't see that "SCHEDULER" is either the name of your app, a list of apps, or a list of app groups.)

If i write this into the controller,
scheduler.queue_task(
    task_1,
    pargs=[],
    pvars={},
    start_time=now, 		#datetime
    stop_time = None,		#datetime
    timeout = 60,               #seconds
    prevent_drift=False,
    period=60,                  #seconds
    immediate=False,
    repeats = 1
)
it atleast creates the tables and populates one of the tables with an set of records(scheduler_task) and does so whenever i call the function. But it only works as long as the function is in the same controller. if the function is in the modules, an error is returned. When i put this into the model, it creates the tabels but does not populate any of them.

some possible reasons why it does not run i have made up are:
-i'm queueing tasks but not creating a worker to work on them (but the book doesnt state anywhere to create a worker. I think therefore this cant be the solution)

This is the explanation. The -K option described at the above links is the key.

-the function isn't defined proper and the worker does not know what to do.

This would probably result in an error being logged in the scheduler_run table which you can look at to debug. (I am, of course, familiar with that!)

Good luck!

/dps

Niphlod

unread,

Feb 25, 2016, 3:11:04 AM2/25/16

to web2py-users

if anyone wants to familiarize with the scheduler, I always recommend https://github.com/niphlod/w2p_scheduler_tests

Lucas Schreiber

unread,

Feb 26, 2016, 10:23:46 AM2/26/16

to web2py-users

It finally worked. thank you very, very much for you patience and help.

It took a while but thanks to you i finally understood how the scheduler works.

Thankyou one more time :)

Am Mittwoch, 24. Februar 2016 00:11:47 UTC+1 schrieb Lucas Schreiber:

Dave S

unread,

Mar 28, 2016, 1:43:28 PM3/28/16

to web2py-users

On Thursday, February 25, 2016 at 12:11:04 AM UTC-8, Niphlod wrote:

if anyone wants to familiarize with the scheduler, I always recommend https://github.com/niphlod/w2p_scheduler_tests

+1 on this. This weekend, I picked up details on how to set args.

/dps

Mohit Jain

unread,

May 28, 2016, 6:53:53 AM5/28/16

to web...@googlegroups.com

I am stuck with a similar situation here.

I have the models/scheduler.py

from gluon.scheduler import Scheduler

def auto_insert():
    try:
        db.UserLogs.insert(user_name='temp',user_email='a...@as.com',activity='checking scheduler')
        return 'inserted'
    except:
        return 'failed'

scheduler = Scheduler(db, dict(auto_insert=auto_insert))

And I run the scheduler tasks as

[taship@taship web2py]$ python web2py.py -K taportaltest
web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2016
Version 2.9.11-stable+timestamp.2014.09.15.23.35.11
Database drivers available: SQLite(sqlite3), MySQL(pymysql), PostgreSQL(pg8000), MSSQL(pyodbc), DB2(pyodbc), Teradata(pyodbc), Ingres(pyodbc), IMAP(imaplib)
starting single-scheduler for "taportaltest"...

Then I use the appadmin to start the background tasks.

Application Name: 
Task Name: 
Group Name: 
Status: 
Function Name: 
Uuid: 
Args: 
Vars: 
Enabled: 
Start Time: 
Next Run Time: 
Stop Time: 
Repeats: 0=unlimited
Retry Failed: -1=unlimited
Period: seconds
Prevent Drift: Cron-like start_times between runs
Timeout: seconds
Sync Output: update output every n sec: 0=never
Times Run: 
Times Failed: 
Last Run Time: 
Assigned Worker Name:

and things seem to be fine...


scheduler_run.id scheduler_run.task_id scheduler_run.status scheduler_run.start_time scheduler_run.stop_time scheduler_run.run_output scheduler_run.run_result scheduler_run.traceback scheduler_run.worker_name
6 auto-insert COMPLETED 2016-05-28 16:18:47 2016-05-28 16:18:47 "inserted" None taship.iiit.a...
7 auto-insert COMPLETED 2016-05-28 16:19:06 2016-05-28 16:19:07 "inserted" None taship.iiit.a...
8 auto-insert COMPLETED 2016-05-28 16:19:22 2016-05-28 16:19:22 "inserted" None taship.iiit.a...
9 auto-insert COMPLETED 2016-05-28 16:19:38 2016-05-28 16:19:40 "inserted" None taship.iiit.a...
10 auto-insert COMPLETED 2016-05-28 16:19:55 2016-05-28 16:19:56 "inserted" None

scheduler_run.id	scheduler_run.task_id	scheduler_run.status	scheduler_run.start_time	scheduler_run.stop_time	scheduler_run.run_result	scheduler_run.traceback	scheduler_run.worker_name
6	auto-insert	COMPLETED	2016-05-28 16:18:47	2016-05-28 16:18:47	"inserted"	None	taship.iiit.a...
7	auto-insert	COMPLETED	2016-05-28 16:19:06	2016-05-28 16:19:07	"inserted"	None	taship.iiit.a...
8	auto-insert	COMPLETED	2016-05-28 16:19:22	2016-05-28 16:19:22	"inserted"	None	taship.iiit.a...
9	auto-insert	COMPLETED	2016-05-28 16:19:38	2016-05-28 16:19:40	"inserted"	None	taship.iiit.a...
10	auto-insert	COMPLETED	2016-05-28 16:19:55	2016-05-28 16:19:56	"inserted"	None

But the database (db.UserLogs) is not being updated :/

What am I missing here? Do I need to specifically import the database classes into scheduler.py?

Regards,

Mohit

Niphlod

unread,

May 29, 2016, 8:01:59 AM5/29/16

to web2py-users

you need to call db.commit() in your task: this is both on the book and probably on the 90% of the threads regarding the scheduler :-P

Reply all

Reply to author

Forward

Application Name:
Task Name:
Group Name:
Status:
Function Name:
Uuid:
Args:
Vars:
Enabled:
Start Time:
Next Run Time:
Stop Time:
Repeats:		0=unlimited
Retry Failed:		-1=unlimited
Period:		seconds
Prevent Drift:		Cron-like start_times between runs
Timeout:		seconds
Sync Output:		update output every n sec: 0=never
Times Run:
Times Failed:
Last Run Time:
Assigned Worker Name: