Scheduler-related database tables created on sqLite but NOT on MySQL

[Yassen D.]

Hello all,

I follow a simple video-tutorial (http://vimeo.com/27478796); so I create a model scheduler.py with this content:

def f():
    t = time.ctime()
    open('/tmp/tasks', 'w').write(t + '\n')
    return f

from gluon.scheduler import Scheduler
Scheduler(db, dict(our_func=f))


After saving it, I go to the database administration of that same application and can see db.scheduler_task, db.scheduler_run, db.scheduler_worker links; when I click on db.scheduler_task, the form is there and I can fill in the data, however, on saving, it spits this error:

Traceback (most recent call last):
  File "/home/www-data/web2py/gluon/restricted.py", line 217, in restricted
    exec ccode in environment
  File "/home/www-data/web2py-2.8.2/applications/socialjack/controllers/contacts.py", line 236, in <module>
  File "/home/www-data/web2py/gluon/globals.py", line 372, in <lambda>
    self._caller = lambda f: f()
  File "/home/www-data/web2py/gluon/tools.py", line 3239, in f
    return action(*a, **b)
  File "/home/www-data/web2py-2.8.2/applications/socialjack/controllers/contacts.py", line 54, in importcontacts
    scheduler.queue_task(testfunc, pargs=[request.vars.sna], kwargs=task_kwargs)
  File "/home/www-data/web2py/gluon/scheduler.py", line 983, in queue_task
    **kwargs)
  File "/home/www-data/web2py/gluon/dal.py", line 9114, in validate_and_insert
    value,error = self[key].validate(value)
  File "/home/www-data/web2py/gluon/dal.py", line 10036, in validate
    (value, error) = validator(value)
  File "/home/www-data/web2py/gluon/validators.py", line 668, in __call__
    row = subset.select(table._id, field, limitby=(0, 1), orderby_on_limitby=False).first()
  File "/home/www-data/web2py/gluon/dal.py", line 10450, in select
    return adapter.select(self.query,fields,attributes)
  File "/home/www-data/web2py/gluon/dal.py", line 1861, in select
    return self._select_aux(sql,fields,attributes)
  File "/home/www-data/web2py/gluon/dal.py", line 1826, in _select_aux
    self.execute(sql)
  File "/home/www-data/web2py/gluon/dal.py", line 1948, in execute
    return self.log_execute(*a, **b)
  File "/home/www-data/web2py/gluon/dal.py", line 1942, in log_execute
    ret = self.cursor.execute(command, *a[1:], **b)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
ProgrammingError: (1146, "Table 'socialjack.scheduler_task' doesn't exist")

If I do the same for the welcome application, it all works as expected.

Can anyone tell what is wrong here? The tables are indeed non-existent when inspecting the MySQL database. This is web2py 2.8.2., updated from mercurial repo, VERSION: 2.8.2-stable+timestamp.2013.12.04.19.34.45.

Thanks,
YD

[Massimo Di Pierro]

Are you sure you have not set DAL(...,migrate_enabled=False)?

[Yassen Damyanov]

Thanks, Massimo, that was it. Niphlod answered that in another thread
and I just got the problem solved.

Again, I would suggest showing the migration status somewhere in the
application header of the db administration page, of that sounds
reasonable and not hard to do. Would help to newcomers like me :)

Cheers,
Yassen

On Friday, December 6, 2013 7:08:05 AM UTC+2, Yassen D. wrote:Niphlod,
HUGE THANKS! I'll check that right now and post back.

On Thu, Dec 5, 2013 at 11:53 PM, Niphlod <nip...@gmail.com> wrote:
> check for
> db = DAL(....migrate_enabled=False)
>
> either this prevented the creation of the table, or some another
> migrate-related glitch.
> To create scheduler's tables, you just need to do (as you did)
>
> Scheduler(db, ...)
>
> NB: (just a naming-convention advice) use
> from gluon.scheduler import Scheduler
> mysched = Scheduler(db, ....)
> instead of
> scheduler = Scheduler(db, ...)
>
> to clearly state the difference between the module and the object that is
> used by your app ^_^
>
> If your tables are in the admin interface but not on the backend, check that
> your migrations are enabled and delete any pre-existing *scheduler*.table
> files in the databases/ folder. This will trigger the creation both of the
> new .table files and the tables on the backend.
>
> BTW: for tasks that needs **immediate** care from the scheduler, instead of
> lowering the heartbeat (and such having all workers to "hammer" the database
> asking for new tasks) just let the heartbeat as it is and use
> mysched.queue_task(....., immediate=True)
> for any user needing that task to run.
> In a few sec the first available worker will pick up the task
> Plan your workers number according on how many concurrent users will need in
> your app a task to be processed: a worker can only be processing one single
> task.
>
> for @all: if you have the need of 20 or 30 concurrent tasks (and such need
> 20 or 30 workers ) test the scheduler carefully: you may find a dedicated
> database more performant and you may also need to put workers in a sleeping
> state (DISABLED) to "alleviate" the db pressure when they are not needed.
> Then you'll need to set them to ACTIVE (or just delete all the records in
> the scheduler_workers table) before queueing a new task...they'll resume
> their (working) state in a heartbeat.
> If you need more than 30 workers..... use a different task processor:
> unfortunately the polling nature of the scheduler makes a "bad usecase" for
> such high demands.

> --
> Resources:
> - http://web2py.com
> - http://web2py.com/book (Documentation)
> - http://github.com/web2py/web2py (Source code)
> - https://code.google.com/p/web2py/issues/list (Report Issues)
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "web2py-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/web2py/kvPJBM6pojo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> web2py+un...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.