APScheduler in a active/active cluster

[shrey....@invicto.in]

I was wondering, how apscheduler will behave in a cluster

Say I have an app which add jobs dynamically based on the requests, now I have an active active cluster, so various instances of the app are running.
In this situation, if they have their own job stores there should not be any problem I guess, as which ever node takes the request will perform the action at the scheduled time.
But lets assume, there is a database server which is shared amongst all app servers and job store is present in the database, so whenever request comes job gets added to the database server, now at the scheduled time, will all the apps try to perform that action or what exactly will happen in such a case? 

Is there a way to control, something like the scheduler which reads the jobs from job store and executes them runs only in one app, but request is taken from all the apps

[Alex Grönholm]

APScheduler does not support the sharing of job stores. That is a recipe for a disaster. The scheduler has to always know when the next job is due, and if something else goes modifying the job store without notifying the scheduler, it may sleep past the next due job.

--
You received this message because you are subscribed to the Google Groups "APScheduler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to apscheduler...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[shrey....@invicto.in]

So how do we achieve high availability for any app using scheduler, I understand active passive cluster will work if primary node comes down cluster will bring up the secondary node so scheduler will be up, but what about active active scenario where the requests are shared, in that scenario what will happen if the scheduler comes down.

[Alex Grönholm]

Sorry to say but APScheduler has not been designed for HA. If you have good ideas on how to properly implement that, I'm listening.

shrey....@invicto.in kirjoitti 15.08.2017 klo 23:31:

So how do we achieve high availability for any app using scheduler, I understand active passive cluster will work if primary node comes down cluster will bring up the secondary node so scheduler will be up, but what about active active scenario where the requests are shared, in that scenario what will happen if the scheduler comes down.

[shrey....@invicto.in]

I saw your rpc connection example, so I planned to run scheduler as a separate process and then use uwsgi legion/cluster(pacemaker) to keep the scheduler process up.

But here I am having some trouble running it as a separate process.
So i have a rest post method which will actually add the job to the scheduler, so I am able to connect through rpc but I am having issues when I try to do add_job with session scope

I am getting this error

ArgumentError: Session event listen on a scoped_session requires that its creation callable is associated with the Session class.

any idea?

[Alex Grönholm]

From "scoped_session" this sounds like your problem has to do with SQLAlchemy. But APScheduler's SQLAlchemy job store does not use sessions.

[shrey....@invicto.in]

Sorry forgot update here

Yea solved this issue, had to start the server with app context thats all.

with app.app_context():

        server = ThreadedServer(SchedulerService, port=12345, protocol_config=protocol_config)

        try:

            server.start()

        except (KeyboardInterrupt, SystemExit):

            pass

        finally:

            scheduler.shutdown()

Thanks man