RCE upload file issues with s3 Compatible Cloud Storage

[aluhe]

Hi there , I have deployed a self-hosted canvas web recently ,with canvas RCE API service installed on separate server. It functions well when I set to NFS storage , but if I change to s3 compatible cloud storage , it keeps showing 400 Bad Request error when user upload file inside RCE , The error is caused by the create_success url which has an extra “?” instead of “&” before the bucket parameter in the string list, see below：

https://institutiondomian/api/v1/files/61/create_success?include%5B%5D=preview_url&on_duplicate=rename&uuid=someuuid?bucket=bucketname-canvas_new_version&key=account_1%2Fattachments%2F61%2F3683.pdf&etag=%22eed43007721a768c96%22

 Another error is on the assignment submission re-upload scenario. Same 400 Bad Request error shows when uploading submission.zip.

 Any suggestions would be greatly appreciated.

[Nico López]

Hi, did you check this post? https://groups.google.com/g/canvas-lms-users/c/uiJsfa4A4ig/m/RWs_9LmNAgAJ

[Kirill Kopylov]

Hello, I also want to use S3 compatible storage (Minio). How can I specify the endpoint url? Could you please share your amazon_s3.yml config template.

Then, I will try to make a workaround for this RCE issue.

вторник, 28 июня 2022 г. в 10:13:01 UTC+3, aluhe:

[aluhe]

Hi, the endpoint url is something like "endpoint: 'https://s3.domain:443'  in amazon_s3.yml .

aluhe

[Kirill Kopylov]

Hello, I've investigated the problem.

First, i also need to provide this key in amazon_s3.yml to make Canvas somehow working with my MinIO.

force_path_style: true

Sorry, there is no simple solution for RCE problem.

This is a bug is in the S3-compatible API implementation.
As I found, it is not completely compatible (my MinIO also has this problem)

Canvas RCE highly relies on the correct processing of success_action_redirect POST parameter (see app/models/attachment.rb#L746 and packages/canvas-rce/src/rcs/api.js#L369 )

In Amazon S3, the succesful file upload returns 303 redirect to exact URL provided in success_action_redirect.
Canvas forms this parameter correctly.

For some reason, MinIO adds bucket, key and etag parameters as well as, for example, Huawei SAN S3 endpoint (I think this is embedded MinIO).

https://github.com/minio/minio/blob/e60b67d246ceb1a5c85a363685df2c4acbece3e7/cmd/bucket-handlers.go#L1120

https://info.support.huawei.com/storage/docs/en-us/oceanstor-100d-8.0.3/object-service-api-references/object_hwapi_00079.html



Best Regards,
Kirill

пятница, 1 июля 2022 г. в 16:31:23 UTC+3, aluhe:

[Kirill Kopylov]

Hello everyone, the problem was finally resolved in the MinIO version 2022-07-13!

It was fixed in this Pull request.

I've succesully connected Canvas with it (see my config here) and RCE file upload is now working!
We can now use our on-premise S3!

For MinIO Deployment I used Docker compose and NGINX reverse proxy with Let's Encrypt SSL Certs.

среда, 6 июля 2022 г. в 22:08:20 UTC+3, Kirill Kopylov: