Help with Canvas LMS production setup.

[Lakshmanan Sornanathan]

I'm setting up Canvas LMS on Ubuntu 14.04, following the production installation guide from the Production Start page. On the guide in the section "Canvas ownership" we need to run the following step:

sysadmin@appserver:/var/canvas$ RAILS_ENV=production bundle exec rake canvas:compile_assets

By running the that step, I get the following error:

rake aborted!
Error: running BrandableCSS CLI: ./node_modules/.bin/brandable_css

But instead if I run the same step without 'RAILS_ENV=production' than it works fine.

sysadmin@appserver:/var/canvas$ bundle exec rake canvas:compile_assets

Is it necessary to run that step with 'RAILS_ENV=production'?

If Yes:
How do I fix the installation steps so that I'm able to run the step with  'RAILS_ENV=production' successfully?

If No:
After completing the production installation,  when I proceed to the Canvas LMS login page I end up with a blank page. There are no errors in the Apache logs or web browser. Any pointers on where to start or how to solve this issue?

--

Kind Regards,

 

Lakshmanan Sornanathan

Education Technologist

Swinburne Online

[Lakshmanan Sornanathan]

Following is the terminal output showing the error that I get when I run that step with 'RAILS_ENV=production'.

ubuntu:/var/canvas$ RAILS_ENV=production bundle exec rake canvas:compile_assets
--> Starting: 'Making sure node_modules are up to date'
--> Finished: 'Making sure node_modules are up to date' in 7.686917473
--> Starting: 'compile css (including custom brands)'
compiled bundles/account_admin_tools.scss new_styles_high_contrast  in 314 ms
compiled bundles/account_admin_tools.scss legacy_high_contrast  in 866 ms
compiled bundles/account_admin_tools.scss new_styles_normal_contrast  in 866 ms
compiled bundles/account_admin_tools.scss legacy_normal_contrast  in 871 ms
compiled bundles/account_course_user_search.scss legacy_normal_contrast  in 866 ms
compiled bundles/account_course_user_search.scss legacy_high_contrast  in 866 ms
compiled bundles/account_course_user_search.scss new_styles_high_contrast  in 866 ms
compiled bundles/account_course_user_search.scss new_styles_normal_contrast  in 866 ms
compiled bundles/account_settings.scss legacy_high_contrast  in 866 ms
compiled bundles/account_settings.scss new_styles_normal_contrast  in 865 ms
compiled bundles/agenda_view.scss legacy_normal_contrast  in 864 ms
compiled bundles/account_settings.scss legacy_normal_contrast  in 866 ms
compiled bundles/account_settings.scss new_styles_high_contrast  in 866 ms
compiled bundles/agenda_view.scss new_styles_normal_contrast  in 865 ms
compiled bundles/alerts.scss legacy_high_contrast  in 864 ms
compiled bundles/alerts.scss new_styles_normal_contrast  in 864 ms
compiled bundles/alerts.scss legacy_normal_contrast  in 864 ms
compiled bundles/agenda_view.scss legacy_high_contrast  in 865 ms
compiled bundles/aligned_outcomes.scss legacy_normal_contrast  in 863 ms
compiled bundles/aligned_outcomes.scss legacy_high_contrast  in 864 ms
compiled bundles/aligned_outcomes.scss new_styles_normal_contrast  in 863 ms
compiled bundles/aligned_outcomes.scss new_styles_high_contrast  in 863 ms
compiled bundles/all_courses.scss legacy_normal_contrast  in 863 ms
compiled bundles/all_courses.scss new_styles_normal_contrast  in 863 ms
compiled bundles/all_courses.scss new_styles_high_contrast  in 862 ms
compiled bundles/all_courses.scss legacy_high_contrast  in 864 ms
compiled bundles/alerts.scss new_styles_high_contrast  in 865 ms
compiled bundles/agenda_view.scss new_styles_high_contrast  in 866 ms
compiled bundles/assignment_moderation.scss legacy_normal_contrast  in 863 ms
compiled bundles/assignment_moderation.scss new_styles_normal_contrast  in 863 ms
compiled bundles/assignment_moderation.scss legacy_high_contrast  in 863 ms
compiled bundles/assignment_moderation.scss new_styles_high_contrast  in 864 ms
compiled bundles/assignments_edit.scss legacy_normal_contrast  in 1351 ms
compiled bundles/assignments_edit.scss legacy_high_contrast  in 1351 ms
compiled bundles/assignments_edit.scss new_styles_normal_contrast  in 1351 ms
compiled bundles/assignments_edit.scss new_styles_high_contrast  in 1351 ms
compiled bundles/assignments.scss legacy_normal_contrast  in 1350 ms
compiled bundles/assignments.scss legacy_high_contrast  in 1350 ms
compiled bundles/attendance.scss legacy_high_contrast  in 1350 ms
compiled bundles/assignments.scss new_styles_normal_contrast  in 1351 ms
compiled bundles/attendance.scss legacy_normal_contrast  in 1350 ms
compiled bundles/attendance.scss new_styles_normal_contrast  in 1350 ms
compiled bundles/attendance.scss new_styles_high_contrast  in 1350 ms
compiled bundles/assignments.scss new_styles_high_contrast  in 1351 ms
compiled bundles/brand_config_index.scss legacy_normal_contrast  in 1349 ms
compiled bundles/brand_config_index.scss new_styles_normal_contrast  in 1350 ms
compiled bundles/brand_config_index.scss legacy_high_contrast  in 1351 ms
compiled bundles/brand_config_index.scss new_styles_high_contrast  in 1351 ms
compiled bundles/calendar2.scss legacy_high_contrast  in 1350 ms
compiled bundles/calendar2.scss legacy_normal_contrast  in 1351 ms
compiled bundles/calendar2.scss new_styles_normal_contrast  in 1350 ms
compiled bundles/calendar2.scss new_styles_high_contrast  in 1350 ms
compiled bundles/canvas_quizzes.scss legacy_normal_contrast  in 1350 ms
compiled bundles/canvas_quizzes.scss new_styles_high_contrast  in 1350 ms
compiled bundles/canvas_quizzes.scss legacy_high_contrast  in 1350 ms
compiled bundles/canvas_quizzes.scss new_styles_normal_contrast  in 1350 ms
brandable_css warning bundles/common.scss new_styles_normal_contrast contains a url() to: /images/inst_tree/node-drag.gif which doesn't exist on disk
brandable_css warning bundles/common.scss new_styles_normal_contrast contains a url() to: /images/mediaelement/skipback.png which doesn't exist on disk
compiled bundles/common.scss new_styles_normal_contrast  in 2698 ms
brandable_css warning bundles/common.scss new_styles_high_contrast contains a url() to: /images/inst_tree/node-drag.gif which doesn't exist on disk
brandable_css warning bundles/common.scss new_styles_high_contrast contains a url() to: /images/mediaelement/skipback.png which doesn't exist on disk
brandable_css warning bundles/common.scss legacy_normal_contrast contains a url() to: /images/inst_tree/node-drag.gif which doesn't exist on disk
brandable_css warning bundles/common.scss legacy_normal_contrast contains a url() to: /images/mediaelement/skipback.png which doesn't exist on disk
brandable_css warning bundles/common.scss legacy_high_contrast contains a url() to: /images/inst_tree/node-drag.gif which doesn't exist on disk
brandable_css warning bundles/common.scss legacy_high_contrast contains a url() to: /images/mediaelement/skipback.png which doesn't exist on disk
compiled bundles/common.scss new_styles_high_contrast  in 3523 ms
compiled bundles/common.scss legacy_normal_contrast  in 3525 ms
compiled bundles/common.scss legacy_high_contrast  in 3525 ms
error compiling sass { [Error: operation timed out after 4080 ms, 5 tries failure: Access Denied]
  failure:
   { [AccessDenied: Access Denied]
     cause:
      { [AccessDenied: Access Denied]
        message: 'Access Denied',
        code: 'AccessDenied',
        region: null,
        time: Thu Sep 08 2016 23:32:36 GMT+0000 (UTC),
        requestId: '3E3D814327F4F09C',
        extendedRequestId: '6GZwd0/BQN1C9ztWXVDOxUdWnWo7DkdnSIabLMlGEammp/3/2+sgNpkbEs4VHj4VwYYQ+BMIiio=',
        cfId: undefined,
        statusCode: 403,
        retryable: false,
        retryDelay: 32.41957230493426 },
     isOperational: true,
     code: 'AccessDenied',
     region: null,
     time: Thu Sep 08 2016 23:32:36 GMT+0000 (UTC),
     requestId: '3E3D814327F4F09C',
     extendedRequestId: '6GZwd0/BQN1C9ztWXVDOxUdWnWo7DkdnSIabLMlGEammp/3/2+sgNpkbEs4VHj4VwYYQ+BMIiio=',
     cfId: undefined,
     statusCode: 403,
     retryable: false,
     retryDelay: 32.41957230493426 },
  code: 'ETIMEDOUT' } AccessDenied: Access Denied
    at Request.extractError (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/services/s3.js:538:35)
    at Request.callListeners (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:668:14)
    at Request.transition (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:670:12)
    at Request.callListeners (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:668:14)
    at Request.transition (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/request.js:670:12)
    at Request.callListeners (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at callNextListener (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
    at IncomingMessage.onEnd (/var/canvas/node_modules/brandable_css/node_modules/aws-sdk/lib/event_listeners.js:211:11)
    at IncomingMessage.emit (events.js:129:20)
    at _stream_readable.js:908:16 operation timed out after 4080 ms, 5 tries failure: Access Denied

rake aborted!
Error running BrandableCSS CLI: ./node_modules/.bin/brandable_css

/var/canvas/lib/brandable_css.rb:266:in `block in run_cli!'
/var/canvas/lib/brandable_css.rb:252:in `run_cli!'
/var/canvas/lib/brandable_css.rb:232:in `compile_all!'
/var/canvas/lib/tasks/css.rake:12:in `block (3 levels) in <top (required)>'
/var/canvas/lib/tasks/css.rake:12:in `block (2 levels) in <top (required)>'
/var/canvas/lib/tasks/canvas.rake:125:in `block (2 levels) in <top (required)>'
Tasks: TOP => css:compile
(See full trace by running task with --trace)
ubuntu:/var/canvas$


--

[Graham Ballantyne]

Have you configured canvas_cdn.yml to use S3? That error appears to be coming from the process to upload assets to S3, and it's getting an Access Denied error. If so, double-check your config.

---

--

---
You received this message because you are subscribed to the Google Groups "Canvas LMS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to canvas-lms-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Graham Ballantyne
IT Services
Simon Fraser University

[Lakshmanan Sornanathan]

Hi Graham,

Thank you for your help. The config in the canvas_cdn.yml was correct. However, it was due to an access issue which was caused by the S3 bucket policy. Fixed that bucket policy and it went through well.

[Lakshmanan Sornanathan]

Hi Graham,

Brooke says hi.

I noticed that the step 'RAILS_ENV=production bundle exec rake canvas:compile_assets' only pushes up the brandable_css folder from the public/dist folder. When I went to the Canvas login page it trying to access files from the other folder (fonts, images, javascript and optimized) that was within the public/dist folder from S3 as well. I followed the steps from the production setup. Do you know if there are any other additional steps required in order to have all the folders under public/dist into S3?

[Graham Ballantyne]

Hi Graham,

Brooke says hi.

Right back at her!

I noticed that the step 'RAILS_ENV=production bundle exec rake canvas:compile_assets' only pushes up the brandable_css folder from the public/dist folder. When I went to the Canvas login page it trying to access files from the other folder (fonts, images, javascript and optimized) that was within the public/dist folder from S3 as well. I followed the steps from the production setup. Do you know if there are any other additional steps required in order to have all the folders under public/dist into S3?

The behaviour you're seeing is the expected behaviour. The bulk of the stuff generated by compile_assets is static and never changes. The stuff in brandable_css will change if you use the theme editor to customize the UI. Because those files need to be generated on the fly when you create a theme (and is done by the delayed jobs processor, which may or may not be running on the same server that is handling your web request), AND need to be available globally (e.g. if you have multiple servers in a cluster), they give you the option to save them to S3. In our case, we store those assets on local storage as well, on a NFS mount that is present on all servers in our cluster and symlinked into dist/public.

[Lakshmanan Sornanathan]

Hi Graham,

Thanks for the help, now I get why only the brandable_css is pushed up onto S3 during the production setup.


Kind Regards,
--

Lakshmanan Sornanathan

Education Technologist

Swinburne Online