Automated Jobs

[Bin Corleone]

First of all, my thanks extend to all of the community members especially Graham Ballantyne and Christopher Bennell. 

I have setup up Canvas LMS on the following servers: 

1x 8GB/4vCPU/160GB SSD (Canvas/Cron Jobs)

2x 4GB/2vCPU/80GB SSD (Redis/PostgreSQL)

The application runs smoothly without any errors so far. 

As for automated jobs, I couldn't really understand how to separate it on a different node per Canvas installation documentation. 

I have found a post by Christopher Bennell on that point:

https://groups.google.com/forum/#!topic/canvas-lms-users/pKuHQZGh7wo

He mentions that it would be the same installation steps for Canvas LMS except for the webserver and the instance would need to have access to the database and other resources. 

I get it but I'm still confused, why do I have to replicate Canvas all over again? As I was searching, someone mentioned this "We don't yet, other than the tests at vendor/plugins/delayed_job/spec_canvas could be considered some form of documentation I guess. Right now our modified delayed jobs doesn't run outside of Canvas, but we'd love to eventually extract it out and offer it as an independent gem." 

Is it because everything is welded into Canvas? 

In addition, how would the main Canvas server execute its jobs on the job node? I mean how would it know? What are the connection or setup parameters? Other than the database, redis? 

[Graham Ballantyne]

On Jan 28, 2018, at 21:07 , Bin Corleone <mich4el...@gmail.com> wrote:

First of all, my thanks extend to all of the community members especially Graham Ballantyne and Christopher Bennell. 

Hey, no problem!

As for automated jobs, I couldn't really understand how to separate it on a different node per Canvas installation documentation. 

I have found a post by Christopher Bennell on that point:

https://groups.google.com/forum/#!topic/canvas-lms-users/pKuHQZGh7wo

He mentions that it would be the same installation steps for Canvas LMS except for the webserver and the instance would need to have access to the database and other resources. 

I get it but I'm still confused, why do I have to replicate Canvas all over again?

Strictly speaking, you don't; you can run delayed jobs on the same server(s) that host your web processes. Many of us choose to use a separate jobs server (two, in my case) for performance reasons.

All(most?) of the things that Canvas schedules as delayed jobs are things that Canvas could do synchronously. Canvas uses delayed jobs for things can either a) need to run later (e.g. cron-like jobs, like scheduling message delivery), and b) things that could take longer to execute than would be an acceptable amount of time to block the web process. That's a bit of an over-simplification, but accurate enough for this discussion. 

Your delayed jobs server needs a copy of the Canvas code because it's executing methods found in it, just… later. Instead of doing this because a user with a web browser told it to, it's doing it because a delayed job told it to.

Make sense?

In addition, how would the main Canvas server execute its jobs on the job node? I mean how would it know?

When the web app server needs to schedule something as a delayed job, it writes an entry to the delayed_jobs table in the database. The process that runs the jobs simply polls that table for jobs that need to be executed, and executes them. You can see this at http://yourcanvas/jobs. 

What are the connection or setup parameters? Other than the database, redis? 

To set up a separate server for jobs, you're just setting up another application server, minus actually running a web server. The setup is exactly the same as it is for an app server in terms of configuration; instead of running apache or nginx to serve the app over the web, you run the canvas_init process to start the job processors. You can run a web server on your jobs node if you wanted to – we do that at SFU; it isn't in our load balancer pool, so it isn't accessible to the outside world, but we use it for API requests for SIS imports and maintenance scripts that we want to run on our "offline" node instead of taking up resources on one of our web cluster nodes. My jobs servers, web servers, and file servers are all exactly the same; if I wanted to, I could "promote" any of the jobs servers into fully-fledged app servers by putting it in the load balancer pool, or turn an app server into another delayed jobs server by running canvas_init (I've done the latter before when we needed an extra jobs node).

Hope that helps explain it.

Graham.

– 
Graham Ballantyne 
Senior Software Engineer | IT Services 
Simon Fraser University | Strand Hall 1001 
8888 University Dr., Burnaby, B.C. V5A 1S6 

[Bin Corleone]

Thank you very much Graham!

It does make clear sense now. 

So to sum it up, the database is the one responsible for keep everything in sync in regard to automated jobs?

The Canvas I have now runs smoothly and I have installed canvas_init on the main app server. So now, I will have to create a job node, install canvas_init on it and then shut the one on the main app server, right? 

Thanks again.

[Graham Ballantyne]

Yup, you've got it right.

--

---
You received this message because you are subscribed to the Google Groups "Canvas LMS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to canvas-lms-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.