How to know if the scheduler is running?

[Alfonso Serra]

In order to run a scheduler task, the scheduler must be running using
web2py.py -K myapp

Its all good but sometimes servers are stopped for mantainance or they kill processes if they are running for too long.

So if i have a controller that assigns a task, how do i detect if the scheduler is running or not?

Most hosting services doesnt let you run shell commands, so is there a way to run it within a controller? using the module subproccess maybe?

Thank you very much.

[Anthony]

You can create a cron job that starts the scheduler on restart of the OS.

Anthony

[Nico de Groot]

Or use something like Monit (on Linux) to check if scheduler is still running, restarting it if it's not.

Nico de Groot

[Alfonso Serra]

Thanks Nico ill give it a try.

[Niphlod]

BTW: exactly as a webapp is suppossed to be kept alive from the os, the same applies to the scheduler. That being said the controller that queue tasks can see if any scheduler is alive inspecting the scheduler_worker table.

[Niphlod]

there's a method for it, get_workers(). according to your heartbeat interval, you can inspect for any worker that has a last_heartbeat too much in the past to see if it's running or not.

[Alfonso Serra]

Thanks Niphlod.

So i can inspect for workers with a last_heartbeat less than 4 seconds ago (default interval is 3 i think). Is that right? It might work, thanks.

[Alfonso Serra]

This would be the funcion:

from gluon.scheduler import Scheduler
sched = Scheduler(db)

def sched_running():
    from datetime import datetime
    workers = sched.get_workers()
 
    for key, worker in workers.items():
        last = (datetime.now() - worker.last_heartbeat).seconds
        if last < 3:
            return True
    return False

[Niphlod]

I'd check for < 30 to be sure that the worker is not sleeping but the general idea is that.

BTW: don't ever ever do 

from datetime import datetime

given web2py's unique execution model, anything past that will have the import screwed . use plain

import datetime

if in need. 

BTW2: you don't need datetime at all sometimes ^_^ 

[Alfonso Serra]

Ok this should do it.

def sched_running():

   
    workers = sched.get_workers()
   
    for key, worker in workers.items():

        last = (request.now - worker.last_heartbeat).seconds
        if last < 4:
            return True
    return False

One last thing, any idea on how to start the scheduler from within a controller?

Im trying os.system("web2py -K myapp") but the scheduler runs on top of the server process and stops working.

Thanks
 

[Niphlod]

simply said, you can't reliably. 

The whole reason you offload tasks to a worker vs a really simpler ajax call is because it's a totally external process that is NOT managed by the webserver. webservers do kill long-running processes (usually after 60 seconds) spawned by themselves to avoid zombie stale resources.

[Alfonso Serra]

This looks like it works.

import subprocess
ret = subprocess.Popen(["python","D:/web2py/web2py.py", "-K", "myapp"], shell=True)
print ret.pid

I have the stdout in the same console as the server but both, the sched and the server, are working.

This whole scheduler thing looks very messy but it looks like the only way to run very long tasks.

Thanks again for the tricks to workaround these situations.

[Niphlod]

as soon as you acknowledge that:

- what you're trying to do is highly discouraged

- lots os safer/documented/supported/simpler ways exist  

feel free to experiment .

[Alfonso Serra]

Yes you are right, the connection resets often and even if it works its not worthy.

[Alfonso Serra]

Excuse me Niphlod, i can't find any docs about running the scheduler programatically.

Do you have any advise on how to do it?

Thanks.

[Dave S]

On Friday, March 11, 2016 at 6:20:09 AM UTC-8, Alfonso Serra wrote:

Excuse me Niphlod, i can't find any docs about running the scheduler programatically.

Do you have any advise on how to do it?

He does.

<URL:https://groups.google.com/d/msg/web2py/9WrqIxi75JY/UHxICIC-BQAJ>

/dps

 

[Niphlod]

that's to queue programmatically "once and forget". to start it, you NEED to look around your OS's facilities and start the scheduler process as a service or under a supervisor. you can't start the scheduler from within the web application for the limitations explained earlier.

http://web2py.com/books/default/chapter/29/13/deployment-recipes