Starting Multiple tasks at the same time.

Hemant Jhunjhunwala

unread,

May 19, 2016, 4:49:09 AM5/19/16

to APScheduler

Hi,

First of all I would like to thank everyone of you for building this wonderful python module.

I have written a Multithreaded python code using APScheduler.

The architecture of the same is such that APScheduler calls a function which in turn spawns a new thread.

It worked fine until I tried to start many jobs (around nine) at the same given time.

Could you please let me know if it's possible to spawn many tasks at the same time or if I am doing something wrong?

Thanks

Hemant

Alex Grönholm

unread,

May 19, 2016, 4:50:39 AM5/19/16

to apsch...@googlegroups.com

So what is going wrong? Have you enabled debug logging?

--
You received this message because you are subscribed to the Google Groups "APScheduler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to apscheduler...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hemant Jhunjhunwala

unread,

May 19, 2016, 4:59:06 AM5/19/16

to APScheduler

Hi,

Sorry I have not enabled debug logging, but I am maintaining a separate activity log.

Alex Grönholm

unread,

May 19, 2016, 5:00:13 AM5/19/16

to apsch...@googlegroups.com

So would you at least explain what goes wrong so I can start helping you? There's no problem spawning several threads at once.

Hemant Jhunjhunwala

unread,

May 19, 2016, 6:00:27 AM5/19/16

to APScheduler

Hi,

I am using BackgroundScheduler and the following function to populate tasks to it for the mentioned scenario.

CRONJOBS.add_job(function,'cron',args=JobId,id=job,minute=15,hour=16,day_of_week="0,1,2,3,4",end_date="2016-6-19")

Where JobId and job is different for all tasks, and using JobId as an argument to the 'function' which spawns a new thread.

There are lots of tasks scheduled at different times to run on different servers, and many task runs at the same time.

The problem which I am facing is that, this 'function' is not called when there are lots of tasks scheduled to run at the same time.

Thanks

Hemant

Alex Grönholm

unread,

May 19, 2016, 6:03:03 AM5/19/16

to apsch...@googlegroups.com

Perhaps you need to configure the executor to have more worker threads?

Hemant Jhunjhunwala

unread,

May 19, 2016, 6:04:36 AM5/19/16

to APScheduler

Hi,

A little edit.

The problem which I am facing is that, this 'function' is not called for all the tasks(it somehow misses to run a few of them), when there are lots of tasks scheduled to run at the same time.

Alex Grönholm

unread,

May 19, 2016, 6:05:23 AM5/19/16

to apsch...@googlegroups.com

So enable debug logging and see what happens in the scheduler.

Hemant Jhunjhunwala

unread,

May 20, 2016, 7:34:43 AM5/20/16

to APScheduler

Hi,

I enabled debug logging and waited for that event to happen again today.

Here is the logs of the same:

INFO:apscheduler.scheduler:Adding job tentatively -- it will be properly scheduled when the scheduler starts

...

#All the jobs were of the same name runCronJob

INFO:apscheduler.scheduler:Added job "runCronJob" to job store "default"

...

#Follwing warning was received at that time

WARNING:apscheduler.executors.default:Run time of job "runCronJob (trigger: cron[day_of_week='0,1,2,3,4', hour='16', minute='15'], next run at: 2016-05-23 16:15:00 IST)" was missed by 0:00:01.119165

Could you please have a look into this?

Alex Grönholm

unread,

May 20, 2016, 7:39:49 AM5/20/16

to apsch...@googlegroups.com

The warning was due to the designated run time being missed, either due to high system load or unavailability of the executor. Increase the misfire_grace_time value in your job to avoid this.

Hemant Jhunjhunwala

unread,

May 20, 2016, 8:22:03 AM5/20/16

to APScheduler

Hi,

Thank you so much.

I will implement this and let you know the results.

Hemant Jhunjhunwala

unread,

Aug 11, 2017, 1:44:09 AM8/11/17

to APScheduler

I just came across this thread again, and I found I forgot to reply here.

Well thanks a lot. It worked great.

Reply all

Reply to author

Forward