Amazon S3 Integration
John-Scott
I'm trying to integrate S3 support into my Django app using
http://code.djangoproject.com/wiki/AmazonSimpleStorageService for
now.
I would like to improve the code so that it might be officially
integrated into Django at some point. The first obvious question: Is
this a feature that others would like to see, enough so to warrant
having this feature 'baked-in'? Or should I just keep my hacks to
myself?
I want to rewrite the code so that it doesn't subclass FileField, as
it currently does, but instead adds or alters the parameters to
FileField which can then be inherited by ImageField. Currently, the
S3FileField subclasses FileField and then has to duplicate some of the
functionality of ImageField, so it's not very DRY. The options I want
to add to FileField would determine whether or not the file should be
stored on the filesystem (the default) or to S3, plus additional
options specific to S3 storage (bucket and key values, etc.). Obviousy
this would involve tinkering with Django's models code directly so I
want to solicit feedback from the Python/Django pros before I go down
this path.
Thoughts?
Cheers,
John-Scott
Jay Parlar
My guess would be that the core team would never accept changes to the
base fields which add provider (ie. Amazon) specific features. Things
like that are better left in contrib. I would personally rather have
an S3FileField.
Of course, if you changed FileFIeld such that you could use it with
*any* S3-style provider, by passing it the right data (maybe some
callbacks, or something), that'd be cool.
Jay P.
Fabien Schwob
> I'm trying to integrate S3 support into my Django app using
> http://code.djangoproject.com/wiki/AmazonSimpleStorageService for
> now.
>
> I would like to improve the code so that it might be officially
> integrated into Django at some point. The first obvious question: Is
> this a feature that others would like to see, enough so to warrant
> having this feature 'baked-in'? Or should I just keep my hacks to
> myself?
I think it's too specific for an inclusion in the official Django
distribution but it would be a great thing to create something like
django-amazon to handle all the Amazon Web Services in Django with :
* Your S3FileField to handle Amazon S3 [1]
* A generic view to handle Amazon FPS [2] (it's an idea, I can't think
of a particular use case right now)
* A context processor that pull an element from Amazon SQS [3] at put
it in the template.
* etc.
[1] http://aws.amazon.com/s3
[2] http://aws.amazon.com/fps
[3] http://aws.amazon.com/sqs
My 0.2 cents
--
Fabien SCHWOB
Robert Coup
> > I'm trying to integrate S3 support into my Django app using
> > http://code.djangoproject.com/wiki/AmazonSimpleStorageService for
> > now.
> >
> > I would like to improve the code so that it might be officially
> > integrated into Django at some point. The first obvious question: Is
> > this a feature that others would like to see, enough so to warrant
> > having this feature 'baked-in'? Or should I just keep my hacks to
> > myself?
>
> I think it's too specific for an inclusion in the official Django
> distribution but it would be a great thing to create something like
> django-amazon to handle all the Amazon Web Services in Django with :
Might be worth looking into boto[1] before going too far into
developing yet-another AWS client implementation, but integrating boto
into Django via fields, etc would be great!
[1] http://code.google.com/p/boto/
Rob :)
Fabien Schwob
Yes, but my idea was to use boto in django-amazon. The goal (in my
mind) is to provide a layer around boto to Django :
* A S3FileField around boto.s3
* A django_amazon.generics.views.process_payment(request, amount)
around a future boto.fps
--
Fabien SCHWOB
Robert Coup
> mind) is to provide a layer around boto to Django :
>
> * A S3FileField around boto.s3
> * A django_amazon.generics.views.process_payment(request, amount)
> around a future boto.fps
We're on the same wavelength. Cool :)
John-Scott
On Aug 8, 8:12 pm, "Jay Parlar" <par...@gmail.com> wrote:
Thanks for sharing your thoughts.
My initial thought was not so much to radically change how FileField
works (at least not for the end user), but to merely to pass in
arguments that would allow for other storage backends, to decouple the
actual storage mechanism from what is essentially just a varchar
pointer field in the model. My problem with S3FileField (which to be
clear, I did not author) is that it basically duplicates a lot of the
functionality of FileField and ImageField, since it has to play double
duty for image and all other file types.
Any changes to FileField would aim to be 'backwards compatible', so
file system storage would be the default in absence of a declared
backend, but maintain the single required argument 'upload_to'. That
way those using FileField for file system storage could continue doing
so without any modifications necessary. So it might be something as
simple as specifying FileField(upload_to='my-key-prefix-if-
any',storage='S3', bucket='mybucket') so that the pointer ultimately
stored in the database would be something like "my-bucket/my-key-
prefix-if-any/uploaded-file-name.extension" and then Django could
prepend the 'http://s3.amazonaws.com/' to generate the absolute url.
ImageField would just inherit this storage flexibility. Of course to
use S3 as the backend, the user would have to install whatever S3
connection library (whether the 'official' S3.py from Amazon, boto or
perhaps something customized for Django).
For now, I'm not so interested in working with any of Amazon's other
services...so I'm not envisioning an all encompassing Amazon library.
I just want to decouple the actual storage mechanism from the varchar
pointer field so that one can easily choose S3 or db BLOB or whatever
without too much hassle.
Once I've wrapped my head around Django's models code I'll try to give
some concrete code examples.
Thanks again for your thoughts, and the pointer to boto...I'll look
into it.
Cheers,
John-Scott