Canvas in Cluster (High-Availability) solution

[Iván García]

Hello,

I am trying to create a cluster of Canvas servers but Canvas refuses to work, I have setup the Cache and the Database outside the canvas server, leaving the Server only with Apache and what it needs to run, I placed the canvas folder in a shared file system but every time I access the URL is says Something went Wrong?

Is this kind of deployment even possible?

Thank you. 

[Graham Ballantyne]

Yes, it's possible, and how we run things at our institution; we have 20 application servers behind a load balancer.

Your apache and/or canvas logs should be giving you a hint as to the source of the error. If the error page you're seeing is coming from Passenger, you can set the PassengerFriendlyErrorPages setting in your apache config (https://www.phusionpassenger.com/library/config/apache/reference/#passengerfriendlyerrorpages) to get more detail.

Some things to keep in mind with a clustered setup:

- If you are using local storage (not S3), all your app servers need access to that store. We have ours on a NFS mount from our central filer mounted on all app servers.

- If you are using local storage, you will also need to have the same public/dist/brandable_css directory available on all servers. We generate brand configs once during our deploy process on one machine (the one where we run our delayed jobs), copy the resulting directory to our shared NFS space, and set up a symlink on all the servers.

– 
Graham Ballantyne 
Senior Systems Engineer —  IT Services 
Simon Fraser University — Strand Hall 1001 
8888 University Dr., Burnaby, B.C. V5A 1S6 

NOTA DE CONFIDENCIALIDAD: Este mensaje y sus adjuntos están intencionados únicamente para el uso del individuo o la organización a la cual está dirigido y puede contener información privilegiada y confidencial. Si el lector de este mensaje no es el receptor al que el mismo se dirige, por este medio se le notifica que cualquier revisión, distribución o copia de este mensaje o de la información contenida en el mismo, está prohibida. Si usted ha recibido por error este mensaje, por favor borre de su sistema el mensaje y sus adjuntos.

CONFIDENTIALITY NOTICE: This message and its attachments are intended only for the use of the individual or entity to which it is addressed and may contain information privileged and confidential. If the reader of this message is not the intended recipient, you are hereby notified that nay review, dissemination or copying of this message or the information contained herein is prohibited. If received in error, please delete the message and attachments from your system.

--

---
You received this message because you are subscribed to the Google Groups "Canvas LMS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to canvas-lms-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/canvas-lms-users/f2e82116-c4f1-4c1e-93ec-54563c4fbd0en%40googlegroups.com.

[Iván García]

Thanks for your reply.

In several tests I got different erros, I will try to recreate them with the PassengerFriendlyErrorPages.

One more question, I'm running this in Ubuntu and no matter what I do the Canvas application tries to start way before the shared storage is mounted, did you run with the same issue?

Kind Regards.

[Graham Ballantyne]

We have not run into that issue. Our NFS mount is defined in /etc/fstab so it mounts on boot. The stock systemd file for httpd (on RHEL, anyway) has After=network.target remote-fs.target, so it won’t try to start until remote file systems are mounted. 

– 
Graham Ballantyne 
Senior Systems Engineer | IT Services 
Simon Fraser University | Strand Hall 1001 
8888 University Dr., Burnaby, B.C. V5A 1S6

On Jun 25, 2021, at 13:47, Iván García <iga...@datum.com.gt> wrote:

Thanks for your reply.

To view this discussion on the web visit https://groups.google.com/d/msgid/canvas-lms-users/7e5d0a1a-99e3-40a3-a65a-6b4b3a3f37fan%40googlegroups.com.

[Iván García]

Thanks for your reply.

I ran a test again with the friendly Passenger pages and got this, if you can have a look:

Bad file descriptor @ rb_file_flock - /var/canvas/config/initializers/plugin_symlinks.rb (Errno::EBADF) /var/canvas/config/initializers/plugin_symlinks.rb:44:in `flock' /var/canvas/config/initializers/plugin_symlinks.rb:44:in `block in <top (required)>' /var/canvas/config/initializers/plugin_symlinks.rb:43:in `open' /var/canvas/config/initializers/plugin_symlinks.rb:43:in `<top (required)>' /var/canvas/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in `load' /var/canvas/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in `block in load' /var/canvas/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:291:in `load_dependency' /var/canvas/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.3.7/lib/active_support/dependencies.rb:318:in `load' /var/canvas/vendor/bundle/ruby/2.6.0/gems/railties-6.0.3.7/lib/rails/engine.rb:666:in `block in load_config_initializer'

The weird thing is that if a run the app from local disk and not shared disk it works.

Regards.

[Graham Ballantyne]

Wait, you put the entire app on shared storage? Don’t do that; you only need to share the file storage and brandable_css directories. 

– 
Graham Ballantyne 
Senior Systems Engineer | IT Services 
Simon Fraser University | Strand Hall 1001 
8888 University Dr., Burnaby, B.C. V5A 1S6

To view this discussion on the web visit https://groups.google.com/d/msgid/canvas-lms-users/d3c5f6b1-db13-4bc6-84d5-4d37617257f3n%40googlegroups.com.

[Iván García]

Hello,

Thanks for your reply. I'm new to Canvas and I could not find any documentation about creating a cluster, sorry, my bad.

I will put only the brandable_css on the shared storage, I'm using S3 so no local storage needs to be shared, am I right?

Do you have more path that needs to be shared?

Thanks again for your time.