cloud-content: jClouds and S3 storage.

[Miguel Pellicer]

Hi all,

I'm performing some tests with cloud storage, trying to connect my Sakai local instance with an Amazon S3 bucket, with the objective of using it as cloud storage using jClouds. The road so far...

1. I've configured the right properties + the sakai-configuration.xml.
2. I've uploaded all the existing content from my local instance to the bucket (Just Drag&Drop).
3. In my first testing, was unable to download any existing content, setting this property fixed the issue "useIdF...@org.sakaiproject.content.api.FileSystemHandler.blobstore=false"
   
4. In my second testing, files were downloaded with the internal resourceId as filename, setting this property fixed the issue "cloud.content.directurl=false"
5. After that, everything seems to work fine, I can upload files to S3 and download all the existing files, looks it's 100% functional as storage.
   

I have a doubt about performance and JVM memory management, looks like the JVM is involved in handling the files between Tomcat and S3, does this affect the performance? Does Sakai perform poorly like storing objects in the database?

I'm asking because I'm in a process of moving a file system from the Database storage to S3 storage, with the objective of gain performance. Is it a long-term viable storage solution?

Really appreciate if someone can throw some light here! I'd love to receive feedback around better configuration, this is the default I've tested:

prov...@org.sakaiproject.content.api.FileSystemHandler.blobstore=aws-s3 iden...@org.sakaiproject.content.api.FileSystemHandler.blobstore=XXXXXXXXXXXXX crede...@org.sakaiproject.content.api.FileSystemHandler.blobstore=YYYYYYYYYYYYYYYYYYY baseCo...@org.sakaiproject.content.api.FileSystemHandler.blobstore=ZZZZZZZZZZZZZZZ useIdF...@org.sakaiproject.content.api.FileSystemHandler.blobstore=false body...@org.sakaiproject.content.api.ContentHostingService=/data/ bodyPat...@org.sakaiproject.content.api.ContentHostingService=/deleted/ cloud.content.directurl=false

Thanks, M

-- 
-----------------------------------------------
Miguel Pellicer
CTO at EDF

Website: https://www.edf.global
LinkedIn: https://www.linkedin.com/in/mpellicer-edf
Office Phone: +34 - 96 381 35 75
Requesting a meeting: https://calendly.com/mpellicer
-----------------------------------------------

[Sam Ottenhoff]

I have a doubt about performance and JVM memory management, looks like the JVM is involved in handling the files between Tomcat and S3, does this affect the performance?

When a user uploads a file to Sakai, it should be an InputStream that gets sent to S3 via JClouds. If your Sakai instance is hosted inside AWS, the stream from Tomcat to S3 should be very fast. If you Sakai instance is hosted elsewhere, latency could be an important factor.

One performance gain is that you can offload object downloads to your front-end load balancer via the cloud.content.sendfile=true property. So instead of having Tomcat fetch the asset and send it to the user, your very efficient loadbalancer (e.g., Nginx) can send the asset to the user directly.

 

Does Sakai perform poorly like storing objects in the database?

S3 was built from the ground-up to store an unlimited number of file objects and has awesome features like versioning and cross-region replication. The database is the wrong place to store objects because of the difficulty with backups. I believe S3, or any object storage system, is a perfect place to store Sakai files.

[Miguel Pellicer]

Awesome, thanks! 

[Miguel Pellicer]

Hi again!

Today was the day, I've performed the migration from RDS to S3 (via EBS using CLI), everything looks good so far! Now all the contents are hosted in the S3 bucket and work like a charm.

For some reason, this configuration didn't work well, the content is not served well to the user:

cloud.content.directurl=false cloud.content.sendfile=true

I set both properties to false to make it work. Maybe both need to be "true"? I'm behind an Amazon ELB, maybe the headers are not supported by ELB...

Anyway...the cluster is working fine, despite it can be improved using these properties.

Thanks!

-- 
-----------------------------------------------
Miguel Pellicer
CTO at EDF

Website: https://www.edf.global

[adnan ashraf]

Miguel,

I will really appreciate if you can share the full configuration and changes needed to move content storage to s3?

Adnan

CTO at DevTek 

[Sam Ottenhoff]

First you need to activate blobstore in TOMCAT_HOME/sakai/sakai-configuration.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:util="http://www.springframework.org/schema/util"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
    http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.0.xsd">

<alias name="org.sakaiproject.content.api.FileSystemHandler.blobstore" alias="org.sakaiproject.content.api.FileSystemHandler" />

</beans>

Second you need to set the proper Sakai properties:

prov...@org.sakaiproject.content.api.FileSystemHandler.blobstore      = aws-s3
iden...@org.sakaiproject.content.api.FileSystemHandler.blobstore      = AZZZZZZZZZZZZZZ
crede...@org.sakaiproject.content.api.FileSystemHandler.blobstore    = UxTxkekcJZZZZZZZZZZZZ
baseCo...@org.sakaiproject.content.api.FileSystemHandler.blobstore = sakai-s3-bucket-name
useIdF...@org.sakaiproject.content.api.FileSystemHandler.blobstore  = false