APScheduler-3.0.1 how to avoid miss jobs

2,740 views
Skip to first unread message

xjj2...@gmail.com

unread,
Jan 15, 2015, 6:18:05 AM1/15/15
to apsch...@googlegroups.com
hi all:
 i use APScheduler-3.0.1  as following, the function is the same. the are many job missed. 


if __name__ == '__main__':

    job_defaults = {
        'coalesce': False,
        'max_instances': 60
    }
    executors = {
    'default': ThreadPoolExecutor(48),
    'processpool': ProcessPoolExecutor(12)
    }
    pushScherJob = BlockingScheduler(executors=executors, job_defaults=job_defaults, timezone=utc)
    pushScherJob.add_job(get_data_from_redis,"interval",minutes=int(inter_minute),args=[0], misfire_grace_time=int(misfire_grace_time),executor='processpool')
    pushScherJob.add_job(get_data_from_redis,"interval",minutes=int(inter_minute),args=[1], misfire_grace_time=int(misfire_grace_time),executor='processpool')
    pushScherJob.add_job(get_data_from_db,"interval",hours=inter_hour,minutes=int(inter_hour_minute),args=[2], misfire_grace_time=int(misfire_grace_time), executor='processpool')
.......

    try:
        pushScherJob.start()
    except (KeyboardInterrupt, SystemExit):
        pushScherJob.shutdown()

in fact, there are some job missed.
There wrong log is :
[WARNING][2015-01-15 18:52:25,607][Process-9:19935][MainThread:140312580867808][base.py:103][run_job][Run time of job "get_data_from_redis(trigger: interval[0:08:00], next run at: 2015-01-15 10:59:24 UTC)" was missed by 0:01:00.839076]
[WARNING][2015-01-15 18:52:25,607][Process-9:19935][MainThread:140312580867808][base.py:103][run_job][Run time of job "get_data_from_redis(trigger: interval[0:20:00], next run at: 2015-01-15 11:11:24 UTC)" was missed by 0:01:00.839134]
[WARNING][2015-01-15 19:00:54,352][Process-5:19931][MainThread:140312580867808][base.py:103][run_job][Run time of job "get_data_from_db(trigger: interval[0:08:00], next run at: 2015-01-15 11:07:24 UTC)" was missed by 0:01:29.583880]

Is it the misfire_grace_time too small?  or  the function name must be different? Thanks very much!

xjj2...@gmail.com

unread,
Jan 15, 2015, 6:25:16 AM1/15/15
to apsch...@googlegroups.com
If i  use  bigger misfire_grace_time , then , it's or not lead the job more and more? The last job is not done, the new job coming?
Thanks 

在 2015年1月15日星期四 UTC+8下午7:18:05,xjj2...@gmail.com写道:

Alex Grönholm

unread,
Jan 15, 2015, 6:43:29 AM1/15/15
to apsch...@googlegroups.com
15.01.2015, 13:25, xjj2...@gmail.com kirjoitti:
If i  use  bigger misfire_grace_time , then , it's or not lead the job more and more? The last job is not done, the new job coming?
Thanks
Sorry but I can't understand what you're saying at all.
From what I gathered, you seem to be having trouble with missed jobs. Is raising misfire_grace_time an option for you?
Other than that, you can increase the logging level to DEBUG and then see what the scheduler is doing. That might help figure it out.
--
You received this message because you are subscribed to the Google Groups "APScheduler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to apscheduler...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

xjj2...@gmail.com

unread,
Jan 15, 2015, 8:50:54 PM1/15/15
to apsch...@googlegroups.com
@ Alex Grönholm,thanks for your help.

Yes,  i  use misfire_grace_time 60, 180,240,
but ,it still miss jobs.The log is following:
[MainThread:139812704102112][base.py:103][run_job][Run time of job "process_visit_ip_hour (trigger: interval[0:08:00], next run at: 2015-01-15 13:27:45 UTC)" was missed by 0:03:42.483209]
[Process-8:28352][MainThread:139812704102112][base.py:103][run_job][Run time of job "cache_visit_ip_day (trigger: interval[0:08:00], next run at: 2015-01-15 13:27:45 UTC)" was missed by 0:03:42.483435]

what's the worse, the is no job run at last.



,I use the method is wrong?

Thanks for your help again.

在 2015年1月15日星期四 UTC+8下午7:43:29,Alex Grönholm写道:
Reply all
Reply to author
Forward
0 new messages