scheduled/repeating jobs

[Brandon Keepers]

I need your advice.

I don't really have a need for scheduled or repeating jobs, but I get a lot of pull requests for hacks to make delayed_job support them. My personal opinion is that If something repeats on a consistent schedule, delayed_job is a poor tool for running them. Something like cron is much better suited. And if you do need repeating jobs, then it is as simple as defining a job that re-enqueues itself after it is done running.

I'm debating whether I should pull in a patch that supports repeating jobs, or just post some docs on the wiki that show how to do it. Does anyone have any strong feelings about this?

I was just looking through the forkqueue and terrbear has the best implementation that I've seen yet (least intrusive at least).

http://github.com/terrbear/delayed_job/commit/6df06985dc71a5f03526365f9bfb2270edf9e66f

Brandon

[Kelly Felkins]

Brandon,

I think repeating jobs are a good fit for delayed_job. Basically my app needs to do both: it needs to defer something based on a user action, and it needs to periodically do various functions. In my case there are immediate emails and daily digest emails to be sent. Both of these things are out of band, low priority tasks.

If I have 2 solutions - say, cron and delayed_job, then I have the chance that 2 rails stacks are loaded at once to perform low priority work. This seems wasteful and may require additional memory on my host.

I'm still working out a solution. One possibility is to use cron to insert a work task in the delayed job queue. I'm open to other suggestions too.

-Kelly

To unsubscribe from this group, send email to delayed_job+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.

--
Contact us now. We may be resting later.

[Daniel Morrison]

One way we've abused the current version is to have a job that then reschedules itself. For example, a sync job might run, then at the end, queue itself up again for a future time. While not quite the same as repeating jobs, it works in a pinch.

-Daniel

--

Daniel Morrison

[i] Collective Idea

http://collectiveidea.com

http://ideafoundry.info

349 Pine Avenue

Holland, MI 49423

616-990-0155

dan...@collectiveidea.com

[Menno van der Sman]

We run our repeating jobs with cron using "script/runner
'Delayed::Job.enqueue(RecurringJob.new)'", but this will load the
whole rails-environment just to queue a job. It is not a problem now,
but if there will be more recurring jobs or recur more frequently this
can generate quite some unwanted load.

Because of that I would be in favor of a recurring job feature.

Cheers,

Menno

[Karl Baum]

We have several repeating delayed jobs that enqueue themselves to run
on a regular basis. Few reasons we like this better than cron:

1) No extra code needed to guard against two instances of the same
job running at the same time
2) Manage status of jobs in one place
3) DJ's awesome retry logic

thx

[Fred Wu]

Hey guys,

As someone who had worked on a similar patch (
http://github.com/fredwu/delayed_job/commit/8bb23a3d3abb6d4090d7e87639f181c03c2869ab
), my vote would be +1. :)

For me personally, I'm not even after repeating jobs, but to run jobs
at a set interval.

Yes we could use cron, or some customised app code to do it. However,
for really simple jobs such as running them in a set interval, having
this ability built-in is really beneficial as there is no extra code
needed. This decreases the code complexity and makes delayed_job more
useful, it's a win win imo. :)

Cheers,
Fred

>        http://github.com/terrbear/delayed_job/commit/6df06985dc71a5f03526365...
>
> Brandon

[Michael Guterl]

I'm curious what the effect of having "extra" jobs just sitting in the
queue will have on performance. Maybe with the new inverted indexes
this won't be a problem.

Our situation may be different than others, but we have about 40
different recurring jobs in our system. We use cron and suffer the
startup cost penalty, but we've considered rufus-scheduler or
something similar to work as a long running process, reducing the high
startup cost of Rails.

Best,
Michael Guterl

> --
> To unsubscribe, reply using "remove me" as the subject.
>

[nitsujri]

Thanks Brandon for the great work that you do on this.

We personally would love to see delayed_job take up the cron ability.
It's not so much that cron is a bad solution, it'd be awesome to get
it all packaged in one. One system handling all things that aren't
going to happen immediately whether scheduled or requested.

We just created a fork that pulls in both your's and patches it with
Terry's.

Justin

On Apr 7, 6:51 am, Michael Guterl <mgut...@gmail.com> wrote:
> I'm curious what the effect of having "extra" jobs just sitting in the
> queue will have on performance.  Maybe with the new inverted indexes
> this won't be a problem.
>
> Our situation may be different than others, but we have about 40
> different recurring jobs in our system.  We use cron and suffer the
> startup cost penalty, but we've considered rufus-scheduler or
> something similar to work as a long running process, reducing the high
> startup cost of Rails.
>
> Best,
> Michael Guterl
>

> On Wed, Apr 7, 2010 at 1:30 AM, Fred Wu <ifre...@gmail.com> wrote:
> > Hey guys,
>
> > As someone who had worked on a similar patch (

> >http://github.com/fredwu/delayed_job/commit/8bb23a3d3abb6d4090d7e8763...