GAE app with images stored on S3

[Emmanuel Mayssat]

They are several discussion about storing images in database (blob) versus filesystem (s3).

Did anyone implement an app on GAE with an image interface on S3?

I am not even sure where to start....

Django has a 

http://django-storages.readthedocs.org/en/latest/index.html

but I use webapp2....

Thanks for any info,

[Alexander Trakhimenok]

We've implemented storing images to S3 from GAE for http://www.myclasses.org/ and pretty happy with results.

There is a 2 factors problem with serving images from GAE's blob-store:

1. Generation of get_serving_url takes a while. If you have many images to serve on a single page (e.g. list if classes and their avatars in our case) it can really slow your code and as result cost you money. So you don't want to do it on every page rendering.
2. We tried to cache the URL returned by get_serving_url() but it occurred it expires in a while (may be days or weeks) and there is no clear understanding when exactly and how to invalidate the URLs or HTML of pages we generate & cached in DB.

With S3 we can have static & stable URL what is perfect for us. And in conjunction with CloudFrount it works really fast - we see noticeable improvements in client pages rendering by browsers.

We used "boto" library to work with S3 that is not coupled with Django in any way (as far as I know) so you can try it as well.

Good luck,

Alex

Founder & Lead Dev @ Petamatic.com

[alex]

Have you guys ever considered Google Cloud Storage? We used to use S3
too, a couple years back, but eventually transfered all assets and
whatnot to GCS for various reasons. It's an awesome service.

https://developers.google.com/storage/

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/GCtIWJ99pIcJ.
>
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

[Stuart Langley]

> We tried to cache the URL returned by get_serving_url() but it occurred it expires in a while (may be days or weeks) and there is no clear understanding when exactly and how to invalidate the URLs > or HTML of pages we generate & cached in DB.

These's URLs never 'expire'. Did you file a bug for what you saw?

[Emmanuel Mayssat]

At this point, Google Cloud Storage integration with GAE is .... experimental...

Whatever that means! 

In other words, the interface is changing.

Not sure I want to deal with that.

(I will look at it more closely though)

Thanks,

[Emmanuel Mayssat]

Ok, I am considering the boto package/module/egg.

I can easily download it from github, but how can I install it with GAE?

It obviously should be in my app files uploaded to GAE.

How did you install boto in your app directory?

Also if you have sample code (upload + fetch) that would be awesome!

My understanding is that the uploaded file should go to GAE first and then AWS S3.

Not sure how the boto s3 code works at this point. But clarification would be welcome.

Thanks,

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.