Background and scheduled jobs..

[Tim Uckun]

What are you guys using for background and scheduled tasks?

the rufus gems seems really cool for scheduled tasks and there are at
least a half a dozen projects for background tasks. Does anybody have
any experience with any of these?

[Aries Andrada]

Hi Tim,

What we do in some of our projects is we create rake tasks then run them as cron jobs. So far, it has served our needs well. Hope this helps.

Regards.

Aries

[nahum]

Don't use BackgroundRB, it's too much of a pain and each worker uses
the resources of a full rails app instance - ie mongrel.

You'd be surprised how far cron+rake can take you.

Nahum

On Nov 17, 12:54 pm, "Aries Andrada" <aries.andr...@gmail.com> wrote:
> Hi Tim,
> What we do in some of our projects is we create rake tasks then run them as
> cron jobs. So far, it has served our needs well. Hope this helps.
>
> Regards.
>
> Aries
>

[Cyrille]

We also use rake tasks, run from cron jobs: much simpler to manage.
You don't have to worry about if your BackgroundRb is running or
not...

Cyrille

[Tim Uckun]

On Mon, Nov 17, 2008 at 2:36 PM, Cyrille <cbon...@gmail.com> wrote:
>
> We also use rake tasks, run from cron jobs: much simpler to manage.
> You don't have to worry about if your BackgroundRb is running or
> not...
>

So how do you deal with background tasks? I have read that simply
firing off a thread or attempting to fork is not a good way to go.

This one looks pretty decent and lightweight.

http://codeforpeople.rubyforge.org/svn/bj/trunk/README

[Will Bryant]

On Mon, Nov 17, 2008 at 12:45 PM, Tim Uckun <timu...@gmail.com> wrote:

What are you guys using for background and scheduled tasks?

I currently recommend, depending on what semantics you need:

• Rake with cron for scheduled tasks.  
• Spawn for things that you want to split off from your apps on your appservers (eg. processing credit card payments against a remote gateway).  
• BackgroundRb for jobs you want to start remotely where you need to ensure only one runs at once (eg. running a slow reconciliation job for admins remotely).
• Workling for queues.
  

[barnacle]

Railscasts Episodes 127-130 go over various ways of running background
tasks. Check them out here - http://railscasts.com/

Glen

[Tim Uckun]

On Tue, Nov 18, 2008 at 6:54 AM, barnacle <barnacl...@gmail.com> wrote:
>
> Railscasts Episodes 127-130 go over various ways of running background
> tasks. Check them out here - http://railscasts.com/

Wow lots of cool stuff there.

I'll watch a bunch of them tonight.

[Gordon Anderson]

hi

Old thread, I know :)

Just came upon this, about how the github deal with the issue of
background tasks

http://github.com/blog/542-introducing-resque

Cheers

Gordon

[nahum]

It's a really good read and very interesting to find out that've
they've tried most of the main suspects. In the blog entry explain
what they liked and didn't about each.

nahum.

On Nov 4, 3:18 pm, Gordon Anderson <gordon.b.ander...@gmail.com>
wrote:

[Tim Uckun]

> Just came upon this, about how the github deal with the issue of
> background tasks
>

Our needs were much simpler and a little more specific so I rolled my
own. Basically it consists of a daemon you "poke" with a UDP packet
which contains a tiny bit of information like the type and the
database ID. The daemon pulls the record from the database and then
instantiates the object and calls the method specified in record with
the params in the record.

The thing that was different for me was the integration of
rufus-scheduler. I have a table called "scheduled_tasks" which
describe the job to be run and the schedule to run them. When the
model updates the record it poked the daemon which schedules the job.
The scheduled task simply creates a job record when the time is right
(related to the scheduled_)task) and pokes the daemon again which runs
the job.

This allows me to keep scheduled tasks on a per client basis and also
allows me to store the result every run of a scheduled task.

The daemon is written with eventmachine.

So far it's worked OK. No major problems for the volume of tasks I am running.

If I had a massive workload I would definitely look at gearman though.
http://gearman.org/

The nice thing about gearman is that you can run multiple gearman
instances so you have no single point of failure.

[Tim Uckun]

BTW here is an article about gearman and ruby. If you go to github
there is a ruby client there too with a few forks.

http://www.linux-mag.com/id/7330

If anybody uses it please let us know how it works out :)