Delayed Jobs

[James Carbine]

Howdy once again.

So I have a Create method I need to wrap into a delayed job. When I was at the ruby conf 2012 I was speaking with Ryan Shaw and he was saying how Canvas is using a heavy modified form of delayed job. So I guess my question is: Does Instructure have any documentation on how to use their modified version?

Thanks!

-James

[Brian Palmer]

James,

We don't yet, other than the tests at vendor/plugins/delayed_job/spec_canvas could be considered some form of documentation I guess. Right now our modified delayed jobs doesn't run outside of Canvas, but we'd love to eventually extract it out and offer it as an independent gem.

-- Brian

[James Carbine]

Okay, I'll take a look at the specs. What I want to do is inside of Canvas, so that works out for my case. I've got it so the user can key in a bunch of Class ID's and it when they submit that the system will parse those ID's and create a district, school, teacher, and a class and also the number of students that were specified and enroll them in that class. Well for one class, it's not so bad, takes a couple of seconds. But if I put in 30 of those Class ID's with 30 students in each class, that takes several minutes and the browser will most assuredly time out.

Any advise? :]

Thanks Brian!

-James

[James Carbine]

Okay, so I have it creating the job. I can see that it gets put into the delayed_job table in the database. But the workers aren't grabbing it and working it. I don't see the rake task to do rake jobs:work to get the jobs worked.

I think that delayed jobs is running but I'm unsure of how to check to see if it is running. Anyone know? Can anyone tell me how to start the workers doing their stuff?

Thanks!

-James

[James Carbine]

Does anyone know how to get Delayed::Job to get up off its duff and do some work? I have almost 2000 items in the queue waiting to be done on my localhost. On my production server, I have almost 3000 waiting to be done. On my staging server I have almost 9000 waiting. But here is the interesting thing, on my staging server, the last job in the queue has an id of 58377 which tells me that the queue is being worked, but not all the way. Is there a setting to tell it to start working? I don't see anything inside the delayed_jobs.yml file. I would presume it is started when the server is started. But for some reason mine is not working properly or at all. The last job on the staging server has a run_at time of a couple hours ago, so it isn't that it is waiting for the correct time to start.

Anyone with some idea? If it would be easier to see what I'm talking about I can screen share with whom ever can help me. :] In fact if anyone knows what to do and would be willing to pair with me to get on my feet that would be wonderful. :]

Thanks!

-James

On Monday, May 7, 2012 3:39:55 PM UTC-6, James Carbine wrote:

[Cody Cutrer]

script/server will run delayed jobs, but any other form of running a
web server will not. You need to run `script/delayed_job start` to
start a DJ daemon (or use script/canvas_init as an init.d script; it
calls delayed_job start after checking privileges and GEM_HOME).

Cody Cutrer
Software Engineer
Instructure, Inc.

[James Carbine]

Awesome, thanks Cody! I will do that!

-James

On Monday, May 7, 2012 3:39:55 PM UTC-6, James Carbine wrote:

[James Carbine]

okay, so it works. However, I'm not sure what is wrong, but it is acting incorrectly. Or maybe I'm calling it incorrectly. But send my function off to delayed jobs once, yet it the function is called 18 times... Any ideas where I can look to correct it from getting called 18 times when I only enqueue it once?

Thanks!

-James

On Monday, May 7, 2012 3:39:55 PM UTC-6, James Carbine wrote:

[Brian Palmer]

Does the job have any text in the last_error field? By default if a job fails with an exception raised, it'll re-try a set number of times. This is legacy behavior from the original delayed_jobs library -- we'd like to change it so that jobs by default only attempt once, but some other cleanup needs to happen before we can do that.

-- Brian

[James Carbine]

Not that I can see. But they are created so fast that it is hard to find the initial job that I created. I have resolved the issue by specifying in the delayed_jobs.yml file only one worker and he can only have one job before he dies. This stinks because things take a lot longer to happen, but for now until I can figure out why it is spawning so many jobs that will have to do.

One thing I have noticed, a bunch of the workers will pick up the one job, and they all will finish the job. it isn't left over with errors.

--James

[James Carbine]

Oh and one more note. This behavior is ONLY happening on my production/staging server. It does NOT happen on my localhost...