Storing S3 and other credentials

[Vishal Joshi]

So far I have been able to avoid needing a server while using Firebase.  Now I am at a point where I need to upload blobs of data in S3.  My concurrent connections are not that high but since I am uploading pictures and videos in my app the disk storage is big.  

Firebase cost structure is not designed for huge file storage and uploads I therefore am intending to store these pictures and videos on S3 or Azure blob.  To avoid exposing my blob store keys I will have to create my own server side logic, create signed url for blobs and send them to my client side to then start the upload/download as required.

I would ideally want to avoid using my own hosted server code?  Is that possible?  On the team radar?  How has anyone else done this?

Thoughts?

Vishal

[Vince Stross]

Hello Vishal,

I'm not sure if this will fit your specific use case, but there are a lot of ways to achieve thick-client authentication using the Auth0 service. They also have Firebase integration, so I'm sure you can find a solution. Does your app currently require authentication and/or user accounts? 

If so, then you can definitely make this work. Here is a link to their docs: https://docs.auth0.com/

I'm currently working on a solution to use basic email/password authentication for an S3 hosted website, which is different than what you want to do. However, here is a link to a github project that uses S3 and Auth0 to make a Thick-client Dropbox clone: https://github.com/auth0/auth0-s3-sample

Would love to hear if you are able to get this working for your use case.

V

[Michael Lehenbauer]

Another possibility to look at might be filepicker.io (or an alternative).  They handle the uploading and storing in S3 part for you and you just get back a URL, ready for storing in Firebase.

-Michael

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Vince Stross]

Hey Michael,

I can see a lot of great uses for filepicker.io, thanks for the reference! I just quickly poked around their docs though, and noticed this comment on the Pick & Store section: "Note that the URLs that are returned will point to copies that are stored, not the versions that exist in Google Drive, Dropbox, etc."

I haven't dug any further yet, but that gives me some pause... have you used this before?

V

[Michael Lehenbauer]

I haven't actually, and am not 100% sure what that text is referring to.  Sorry!

-Michael

[Vince Stross]

Sorry, that was a bit hasty... I should really just go to bed. :)

This section of the docs outlines a Store method that allows direct write to S3... really cool!

V

[Michael Wulf]

What that message is telling you is that when a user "uploads" a file, it's not going to be a reference to the original file stored in the user's data, but a new "copy" stored on S3. Which is what you want, of course, since you don't want the users' changes to break your links.

[Seth Caldwell]

I had emailed support about implementing the single sign on standard amazon exposes (SAML) which would provide user federation: you would be able to access the Firebase user id during uploads and allow uploads only to buckets containing their user id. Netflix uses this feature of aws.

This really would be the best way to use aws directly from the client but we don't have a SAML implementation from Firebase yet.

Probably will soon though as it starts catching on.

For now you can create a different user in IAM that only has s3 privs and put that on the client.

Cheers, 

Seth

--

[Vishal Joshi]

Michael, Vince, Seth thanks for your replies.  

@Vince/Michael, I will try out your suggestions in a few days and will report back.

@Seth, completely agree with you that this needs to be built into firebase.  Although just S3 might be limiting.  I think having some kind of server side logic executer on firebase server would be nice.  That way we can obfuscate all sort of credentials.

Vishal Joshi | http://vishalrjoshi.com | @vishalrjoshi

--
You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/Z-V1HJJgLMI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

[Vince Stross]

At risk of sounding like an Auth0 fanboy, I realize, but they definitely support integration with Firebase and SAML. So a solution can be created to accomplish what you seek. :)