Amazon EC2 and Thumbnails

[Brad Bode]

Stephen, Mezzanine rocks. There are definitely a few sticking points when deploying on Amazon EC2 / Elastic Beanstalk + S3, but most of them I have overcome. The biggest issue is the thumbnail template tag (Not for reasons that are your fault). For the sake of info, here is the issue:

Amazon EB and EC2 uses an interesting app deployment mechanism. Each time you update your app it increments a number in a directory, deploys your app into that directory, and starts the server. 

For instance, if you go with out of the box Mezzanine settings your static and media content goes here:

/opt/python/bundle/1/app/static/media

On subsequent deploys the new deploy will go here:

/opt/python/bundle/2/app/static/media

Note the incremental number change. 

So, the first issue is that the uploaded media disappears with each deploy because it is IN the app directory. The solution was to use django storages with S3 (and help from s3_folder_storage). That fixes the issue for uploading media and not have it be erased across deploys. 

If you didn't use S3, you could write your own script that runs on each deploy and copies over previous uploaded images. No fun and quite a patch. 

So that leads me to the final problem. Thumbnails does not work with S3 (you already know this). I understand the issue, but is there any other workaround? I suppose I could monkey patch the tag to do something completely different and upload the thumbnail to S3. 

Any thoughts on the issue?

PS> This question was briefly discuss here https://groups.google.com/forum/#!topic/mezzanine-users/WGVaNhD5vRs 

No clear solution was posted.

[Stephen McDonald]

Don't have an answer sorry - please keep the list posted if you have a good solution, it'd be quite welcome given the number of times this has come up already.

--
You received this message because you are subscribed to the Google Groups "Mezzanine Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Stephen McDonald
http://jupo.org

[Mario Gudelj]

Hey Brad,

I use EC2 and I don't get that behaviour. I use fabric and Mercurial to push to bitbucket, and then pull at the EC2 end and restart Gunicorn etc. How are you deploying your app so that the directory names are changed?

Cheers,

M

On Mon, Apr 21, 2014 at 2:41 AM, Brad Bode <for...@gmail.com> wrote:

--

You received this message because you are subscribed to the Google Groups "Mezzanine Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Mario Gudelj

M: 0415 193775

www.twoblokeswithapostie.com - Talk to us before you tell your clients: "No, Business Catalyst can't do that."

[Brad Bode]

I am using Elastic Beanstalk. That's probably the difference.  On an EC2 instance you have control over how the code is deployed, but on EB they deploy a clean instance from source every time. I am not sure why, but I assume this has to do with the load balancing aspect of EB. Load Balancing would imply no local storage since you could have multiple instances serving the same content. Amazons take is likely that you should be using S3 in this instance. 

One solution might be to create a new tag or monkey patch the existing thumbnail tag to store the thumbnails locally instead of using MEDIA_URL. All this means is that the thumbnails would be the only images served locally, rather than S3. The downside is that every time you deploy the thumbnails would need to be regenerated. 

Stephen, 

The main issue is that the thumbnails tag has no permissions to create a directory on S3. Why? When I upload a file via the Media section of Mezzanine they upload and create directories just fine in S3. However, thumbnails cannot. Any thoughts as to why?

I'll figure out a solution. The load balancing aspect of EB is important to me otherwise I would just switch to a single instance EC2. 

[Brad Bode]

Solved... for now. Although I wouldn't call it a bug I got the most help out of this bug listing : https://github.com/stephenmcd/mezzanine/issues/530

So the solution for me was two fold. First, I had to make sure all usages of the mezzanine thumbnail tag would work. Ideally this would be done without actually modifying mezzanine code. I chose and test Sorl-Thumbnail, which works with S3. 

I monkey patched the code to replace the implementation of the template tag first. You have to do this so that ALL usages of the template tag defer to the Sorl version (which works with S3). Remember, the app that registers this new tag, must come AFTER the mezzanine core app which registers the original.

    from django import template

    from sorl.thumbnail import get_thumbnail

    from mezzanine.core.templatetags import mezzanine_tags

    

    register = template.get_library("mezzanine_tags")

    

    @register.simple_tag

    def thumbnail(image_url, width, height, quality=95, left=0.5, top=0.5):

        im = get_thumbnail(image_url, "%sx%s" % (width, height), crop='center', quality=quality)

        return im.url

But in order to get the Mezzanine thumbnails to work in the Admin Interface you will need to replace the method directly (the admin uses a direct method call not the template tag): 

    mezzanine_tags.thumbnail = thumbnail

And finally, MAKE SURE that the module that registers this tag is register AFTER mezzanine otherwise it will not overwrite the mezzanine template tag. 

[Stephen McDonald]

That's actually real clean - thanks for sharing.

--

You received this message because you are subscribed to the Google Groups "Mezzanine Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Stephen McDonald
http://jupo.org