Starting scheduler worker from controller

[Gael Princivalle]

Hello.

Is it possible to run a scheduler worker from a controller?

Restarting a server kill all workers. I would like to run my workers automatically if the last heartbeat is old.

The standard way is going to ssh and running :

nohup python web2py.py -K application_name &

I would like to have in a controller:

def run_worker():
    something('nohup python web2py.py -K application_name &')

Thanks.

[Val K]

I start scheduler as Linux daemon,it works fine. Can you do that?

[Niphlod]

I'll say it once again: you need a scheduler to run out-of-band pieces of things (tasks) that ARE NOT embedded in the web-serving process(es).

At that point you may as well process out-of-band tasks with a simple ajax call, and save yourself setting up the scheduler at all.

It's strongly discouraged to start ANY process inside the webserver (stderr/stdout clumsiness with wsgi, enforced timeout on the webserver, DDoS attacks, etc). Starting a python process inside a controller (even if it's the scheduler) has the same exact cons.

The scheduler NEEDS (read: there is absolutely no valid reason not to) to be started as an external and independant process from the webserver: if you need it to be started when the server comes up,  you NEED to use your system's "service" architecture (init.d, systemd, supervisord, windows service, etc) .

Or, if you're using uwsgi, you can use its facility to spawn external processes (it's in the default installation script).

[Gael Princivalle]

Ok I've made the perfect system that start a new worker when it's necessary. My problem is that web2py don't run the tasks and I don't know why (so my system is not so perfect!!).

I've add this task in the scheduler. It only save a file "alert_file.txt":

def save_alert_file():
    file = open('%s/static/alert_file.txt' %request.folder, 'w')
    file.write('')
    file.close()

I run it every 300 second (5 minutes).

Webfaction servers use Centos 7.

I've made a bash script called hyd.sh that start the worker of my app called "hydrover_oleodinamica" if the file datetime is older than the current datetime minus 7 minutes:

file="$HOME/webapps/w2p_2_12_3/web2py/applications/hydrover_oleodinamica/static/alert_file.txt"
file_datetime_s=$(date -r $file +%s)
alert_datetime_s=$(date --date '- 7 minutes' +%s)
if [ $alert_datetime_s -gt $file_datetime_s ];

then
  nohup /usr/local/bin/python2.7 ~/webapps/w2p_2_12_3/web2py/web2py.py -K hydrover_oleodinamica & 2>&1 > $HOME/hyd.log &
fi

It works fine.

I've add it in the server crontab that run it every 10 minutes:

*/10 * * * * $HOME/hyd.sh

If I cancel all workers in the db and all related processes in the server, this system works like a sharm.

A worker is started, all tasks are running.

After 10 minutes no other worker is started, perfect.

If the Webfaction server get restarted, all the processes are killed include the workers.

In this case the worker is started, it's in the db but tasks don't run.

And so the crontab start a new worker every 10 minutes.

Someones knows why?

As I can't restart the Webfaction server by myself it's difficult to make test but I think that when a worker is started by my script after a server restart, the flag "Is ticker" is false in the worker db record.

What does it mean?

Thanks, regards.