OpenWFEru scheduler in Rails - jobs scheduled in environment.rb
tom...@o2.pl
I'm writing an auction site in RoR, where auctions have to be
scheduled for starting and for finishing. I'm using OpenWFEru and it
works like charm (John's great responses in the other thread I've
started previously contributed to that :)) with executing scheduled
"start auction" and "end auction" job.
My Scheduler instance is a global variable, to be usable amongst every
part of Rails application:
$scheduler = Scheduler.new
$scheduler.start
RAILS_DEFAULT_LOGGER.warn "started scheduler #{ Time.now }"
This works, this is good, this is fast.
However, when server process is restarted (this happens a lot during
development, as we all know, and can occur in production), all the
scheduled jobs are naturally lost. Thus I've added a static method to
Auction model which re-schedules everything.
When this "reschedule" method is invoked from some controller (i.e.
"manually" started by site admin), it works perfectly: jobs get
scheduled and are executed when they should be.
However, when I put the call to reschedule method in environment.rb,
the jobs get scheduled (the log-lines "job A scheduled for B at C" are
written, the job id's get assigned), but they don't survive afterwards
and are not executed.
BUT when "past" jobs get scheduled (i.e. to be executed immediately
after scheduling), they ARE executed.
I've replayed this behavior with even the simplest cases of jobs
scheduled from environment.rb - they seem to be scheduled, but are
forgotten and remain unexecuted. With the exception of jobs scheduled
to "a second ago".
The same jobs scheduled in controller work, of course, perfectly,
executing at given time.
This has something to do with Rails application lifecycle, but could
anyone point out how to prevent this "forgetfulness" and have jobs at
application start both scheduled and executed?
Tomek "Tomash" Stachewicz
John Mettraux
On 8/27/07, tom...@o2.pl <tom...@o2.pl> wrote:
>
> This has something to do with Rails application lifecycle, but could
> anyone point out how to prevent this "forgetfulness" and have jobs at
> application start both scheduled and executed?
Thanks a lot for the kind words.
So it seems this has nothing to do with the :discard_past param
(http://openwferu.rubyforge.org/rdoc/classes/OpenWFE/Scheduler.html#M000205)
Could you please paste more of your code here ? Seems like a very
interesting issue. You don't store anything about rescheduling in a
class variable of the Auction model don't you ?
Best regards, thanks again,
--
John Mettraux -///- http://jmettraux.openwfe.org
tom...@o2.pl
No, I don't store any info in a Model - that'd be one of the first
places I'd look for source of the bug.
As for the model's method called (from environment - working only with
"past" auctions, from controller - successfully scheduling
everything), it's pretty straightforward:
(Auction model static method)
#for application start - schedules auctions which were started
(scheduled) before server restart
def self.schedule_pending
#approved:
auctions = self.find(:all, :conditions => [ "status = ?",
'approved'])
for auc in auctions
#double "scheduling" - AuctionSchedule automatically detects
whether auction triggered
#should be started or finished
jid1 = $scheduler.schedule_at auc.start_real, :schedulable =>
AuctionSchedule.new, :id => auc.id
jid2 = $scheduler.schedule_at auc.end_real, :schedulable =>
AuctionSchedule.new, :id =>
end
#started:
auctions = self.find(:all, :conditions => [ "status = ?",
'started'])
for auc in
#auction already started - it needs only finishing
jid = $scheduler.schedule_at auc.end_real, :schedulable =>
AuctionSchedule.new, :id =>
end
end
AuctionSchedule model trigger method:
def trigger
@auction = Auction.find(params[:id])
if(@auction.status=='approved')
@auction.start
elsif(@auction.status=='started')
@auction.scheduled_finish
end
end
John Mettraux
On 8/28/07, tom...@o2.pl <tom...@o2.pl> wrote:
>
> Hello!
>
> No, I don't store any info in a Model - that'd be one of the first
> places I'd look for source of the bug.
Sorry for asking.
> As for the model's method called (from environment - working only with
> "past" auctions, from controller - successfully scheduling
> everything), it's pretty straightforward:
>
> (Auction model static method)
>
> #for application start - schedules auctions which were started
> (scheduled) before server restart
> def self.schedule_pending
> #approved:
> auctions = self.find(:all, :conditions => [ "status = ?",
> 'approved'])
> for auc in auctions
> #double "scheduling" - AuctionSchedule automatically detects
> whether auction triggered
> #should be started or finished
> jid1 = $scheduler.schedule_at auc.start_real, :schedulable =>
> AuctionSchedule.new, :id => auc.id
> jid2 = $scheduler.schedule_at auc.end_real, :schedulable =>
> AuctionSchedule.new, :id =>
> end
:id =>
points to nothing ?
tom...@o2.pl
> > No, I don't store any info in a Model - that'd be one of the first
> > places I'd look for source of the bug.
>
> Sorry for asking.
No problem, that'd be my first question too :)
> > jid2 = $scheduler.schedule_at auc.end_real, :schedulable =>
> > AuctionSchedule.new, :id =>
> > end
>
> :id =>
>
> points to nothing ?
>
Points to auc.id of course. It's there in the code, I just can't get
used to pasting code in this strange textarea.
I'd get a parse error if it would point to nothing, anyway :)
So, do You have any ideas? There's something strange going on with
lifecycle of jobs scheduled during execution of environment.rb.
John Mettraux
> lifecycle of jobs scheduled during execution of environment.rb.
Hi Tomek,
I would check that :
LOGGER.warn "scheduler.at_job_count is #{$scheduler.at_job_count}
(${scheduler.object_id})"
right after the scheduling and a bit later.
You could even put that check into the step() method of the scheduler
(4 times per second might get boring, but we would thus know).
You could also do
class << $scheduler
def lwarn (msg)
RAILS_DEFAULT_LOGGER.warn msg
end
end
Just to be sure that there no exception in the trigger (I'll have to
document that).
It would give us a better idea of what's going on.
Best regards,
tom...@o2.pl
$scheduler = Scheduler.new
class << $scheduler
def lwarn (msg)
RAILS_DEFAULT_LOGGER.warn msg
end
def step
super
RAILS_DEFAULT_LOGGER.debug "step: scheduler.at_job_count is
#{$scheduler.at_job_count} (#{$scheduler.object_id})"
end
end
$scheduler.start
RAILS_DEFAULT_LOGGER.warn "started scheduler #{ Time.now }"
Auction.schedule_pending
I've scheduled a job from the application level (to start an auction)
to execute at 13:40:30, then shut down the server, and then restarted
it (everything on standard development default Mongrel). This is a
piece from my log:
(snip: thousands lines like the one below)
2007-08-28 13:40:29 DEBUG logger (:) - step: scheduler.at_job_count is
23 (23456261299860)
2007-08-28 13:40:29 DEBUG logger (:) - step: scheduler.at_job_count is
23 (23456261299860)
2007-08-28 13:40:29 DEBUG logger (:) - step: scheduler.at_job_count is
23 (23456261299860)
2007-08-28 13:40:29 DEBUG logger (:) - step: scheduler.at_job_count is
23 (23456261299860)
2007-08-28 13:40:30 DEBUG logger (:) - step: scheduler.at_job_count is
22 (23456261299860)
2007-08-28 13:40:30 DEBUG logger (:) - step: scheduler.at_job_count is
22 (23456261299860)
2007-08-28 13:40:30 DEBUG logger (:) - step: scheduler.at_job_count is
22 (23456261299860)
2007-08-28 13:40:30 DEBUG logger (:) - step: scheduler.at_job_count is
22 (23456261299860)
(snip: thousands like the one above)
So, the job has been removed from queue (23->22) at proper time, but
is has not been executed (if it had been, there'd be an extra line in
the logs like.
!!!
Funny thing though - seems that calling Auction.schedule_pending in
environment.rb somehow "spoils" the scheduler and - WOW - makes other,
normally scheduled (at later time) jobs also unexecuted!
Calling it from within the application after it starts is completely
OK, though - schedules pending jobs, they get executed, any jobs
scheduled after the call are also successfully executed.
/!!!
Trying to redefine lwarn and step (what I actually did was defining a
class Scheduler2 < Scheduler and making $scheduler an instance of
Scheduler2) methods didn't give any results - step does what it should
(executes every 0.25s), lwarn doesn't give any errors.
I just don't know what to think about this.
John Mettraux
On 8/28/07, tom...@o2.pl <tom...@o2.pl> wrote:
>
> !!!
> Funny thing though - seems that calling Auction.schedule_pending in
> environment.rb somehow "spoils" the scheduler and - WOW - makes other,
> normally scheduled (at later time) jobs also unexecuted!
Does that mean that the scheduler becomes unusable ?
Maybe I will investigate the code just before the trigger but I was
careful in catching any trouble at that point, I will check once again
(tomorrow morning JST).
> Calling it from within the application after it starts is completely
> OK, though - schedules pending jobs, they get executed, any jobs
> scheduled after the call are also successfully executed.
> /!!!
>
> Trying to redefine lwarn and step (what I actually did was defining a
> class Scheduler2 < Scheduler and making $scheduler an instance of
> Scheduler2) methods didn't give any results - step does what it should
> (executes every 0.25s), lwarn doesn't give any errors.
>
> I just don't know what to think about this.
Me neither, I will try to reproduce that on my side with a mini rails
application.
Best regards, ttyl,
John Mettraux
Hi Tomek,
for now, I tested the scheduler as used by the OpenWFEru bpm engine.
The scheduler is used as a service inside of the bpm engine. The bpm
engine is created as a global var by environment.rb (well a script
required by environment.rb
(http://fisheye3.cenqua.com/browse/openwferu/trunk/densha/config)).
It works just fine, after restart and reschedule, jobs are executed on time.
I will try to set up a mini rails application to match more closely to
your case. Maybe it'd be easier for you to provide that stripped down
test case.
Best regards,
tom...@o2.pl
Thanks for Your help, John :)
I've solved the problem and other potential ones (like having multiple
mongrels, each with their own set of variables etc.) by writing a
simple DRb server which takes care of all the scheduling. Do You think
this qualifies for an article or gem/plugin/whatever? It seems that
many Rails apps could benefit from having one, central "scheduled jobs
server".
(Yes, I've been experimenting with BackgrounDrb when looking for a
scheduler and it's not good enough).
John Mettraux
On 8/30/07, tom...@o2.pl <tom...@o2.pl> wrote:
>
> Hello!
>
> Thanks for Your help, John :)
You're welcome !
I didn't find the time to write the mini rails app to double check, at
least I tested with the workflow engine wrapped around the scheduler.
> I've solved the problem and other potential ones (like having multiple
> mongrels, each with their own set of variables etc.) by writing a
> simple DRb server which takes care of all the scheduling.
Hey, well done !
> Do You think
> this qualifies for an article or gem/plugin/whatever? It seems that
> many Rails apps could benefit from having one, central "scheduled jobs
> server".
>
> (Yes, I've been experimenting with BackgrounDrb when looking for a
> scheduler and it's not good enough).
Yes, it's worth at lest a blog post or an article. I guess that
writing down the ideas for the post/article could help you in
determining if a gem/plugin is the way to go.
Nowadays it's very important for developers to show what they are
capable of doing.
I'd be glad to help.
Cheers,