S3 Support with V2?

[michaelg]

I remember reading that s3 storage might be supported with v2 of
photologue. Can we get an example of how this works, or maybe include
it in the docs? Thank you!

[Justin]

I don't have an s3 account to test this out with right now but from my
current understanding it should just be a manner of setting your
DEFAULT_FILE_STORAGE class and AWS account info in settings.py:

http://code.larlet.fr/doc/django-s3-storage.html

If you want to use S3 for JUST Photologue, I can see making the
Photologue storage class configurable but it may not make the 2.0
release just because I would feel comfortable with the limited testing
I think it would get.

- Justin

[michaelg]

I already have S3storage working for another part of my project using
that same method, but when I try to save a Photo in Photologue, I get
an error saying that my storage backend (S3), does not support
absolute paths. I figured out that the admin_thumnail method is
causing the problem. When I remove it from the list_display, the error
goes away. Is there a way to edit admin_thumbnail so i can still view
photos from the admin?

Here is the complete error message:

Environment:

Request Method: GET
Request URL: http://localhost:8000/admin/photologue/photo/
Django Version: 1.0-final-SVN-8964
Python Version: 2.5.1
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.humanize',
...
'photologue']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.middleware.doc.XViewMiddleware')


Template error:
In template /Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/contrib/admin/templates/admin/
change_list.html, error at line 34
Caught an exception while rendering: This backend doesn't support
absolute paths.
24 : {% if cl.has_filters %}


25 : <div id="changelist-filter">


26 : <h2>{% trans 'Filter' %}</h2>


27 : {% for spec in cl.filter_specs %}


28 : {% admin_list_filter cl spec %}


29 : {% endfor %}


30 : </div>


31 : {% endif %}


32 : {% endblock %}


33 :


34 : {% block result_list %} {% result_list cl %} {% endblock %}


35 : {% block pagination %}{% pagination cl %}{% endblock %}


36 : </div>


37 : </div>


38 : {% endblock %}


39 :

Traceback:
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/core/handlers/base.py" in get_response
86. response = callback(request, *callback_args,
**callback_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/sites.py" in root
158. return self.model_page(request, *url.split('/',
2))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/views/decorators/cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/sites.py" in model_page
177. return admin_obj(request, rest_of_url)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/options.py" in __call__
189. return self.changelist_view(request)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/options.py" in changelist_view
662. ], context,
context_instance=template.RequestContext(request))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/shortcuts/__init__.py" in render_to_response
18. return HttpResponse(loader.render_to_string(*args,
**kwargs), **httpresponse_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/loader.py" in render_to_string
107. return t.render(context_instance)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
176. return self.nodelist.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
768. bits.append(self.render_node(node, context))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/debug.py" in render_node
71. result = node.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/loader_tags.py" in render
97. return compiled_parent.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
176. return self.nodelist.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
768. bits.append(self.render_node(node, context))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/debug.py" in render_node
71. result = node.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/loader_tags.py" in render
97. return compiled_parent.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
176. return self.nodelist.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
768. bits.append(self.render_node(node, context))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/debug.py" in render_node
71. result = node.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/loader_tags.py" in render
24. result = self.nodelist.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
768. bits.append(self.render_node(node, context))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/debug.py" in render_node
71. result = node.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/loader_tags.py" in render
24. result = self.nodelist.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/__init__.py" in render
768. bits.append(self.render_node(node, context))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/template/debug.py" in render_node
81. raise wrapped

Exception Type: TemplateSyntaxError at /admin/photologue/photo/
Exception Value: Caught an exception while rendering: This backend
doesn't support absolute paths.

Original Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/template/debug.py", line 71, in
render_node
result = node.render(context)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/template/__init__.py", line 915, in
render
dict = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/contrib/admin/templatetags/
admin_list.py", line 242, in result_list
'results': list(results(cl))}
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/contrib/admin/templatetags/
admin_list.py", line 237, in results
yield list(items_for_result(cl,res))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/contrib/admin/templatetags/
admin_list.py", line 161, in items_for_result
value = attr()
File "/Users/.../Sites/.../photologue/models.py", line 276, in
admin_thumbnail
(self.get_absolute_url(), func())
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/utils/functional.py", line 55, in
_curried
return _curried_func(*(args+moreargs), **dict(kwargs,
**morekwargs))
File "/Users/.../Sites/.../photologue/models.py", line 308, in
_get_SIZE_url
if not self.size_exists(photosize):
File "/Users/.../Sites/.../photologue/models.py", line 337, in
size_exists
if os.path.isfile(func()):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/utils/functional.py", line 55, in
_curried
return _curried_func(*(args+moreargs), **dict(kwargs,
**morekwargs))
File "/Users/.../Sites/.../photologue/models.py", line 316, in
_get_SIZE_filename
return os.path.join(self.cache_path(),
File "/Users/.../Sites/.../photologue/models.py", line 284, in
cache_path
return os.path.join(os.path.dirname(self.image.path), "cache")
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/db/models/fields/files.py", line 49, in
_get_path
return self.storage.path(self.name)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/site-packages/django/core/files/storage.py", line 81, in
path
raise NotImplementedError("This backend doesn't support absolute
paths.")
NotImplementedError: This backend doesn't support absolute paths.

[michaelg]

I also get a similar error when I try to delete the photo from the
admin:

Environment:

Request Method: POST
Request URL: http://localhost:8000/admin/profileimages/profileimage/1/delete/

Django Version: 1.0-final-SVN-8964
Python Version: 2.5.1
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.humanize',
...
'photologue']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.middleware.doc.XViewMiddleware')

Traceback:
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/core/handlers/base.py" in get_response
86. response = callback(request, *callback_args,
**callback_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/sites.py" in root
158. return self.model_page(request, *url.split('/',
2))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/views/decorators/cache.py" in _wrapped_view_func
44. response = view_func(request, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/sites.py" in model_page
177. return admin_obj(request, rest_of_url)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/options.py" in __call__

195. return self.delete_view(request, unquote(url[:-7]))
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/contrib/admin/options.py" in delete_view
693. obj.delete()
File "/Users/.../Sites/site/photologue/models.py" in delete
465. self.clear_cache()
File "/Users/.../Sites/site/photologue/models.py" in clear_cache
430. self.remove_size(photosize, False)
File "/Users/.../Sites/site/photologue/models.py" in remove_size
419. if not self.size_exists(photosize):
File "/Users/.../Sites/site/photologue/models.py" in size_exists
337. if os.path.isfile(func()):
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/utils/functional.py" in _curried
55. return _curried_func(*(args+moreargs), **dict(kwargs,
**morekwargs))
File "/Users/.../Sites/site/photologue/models.py" in
_get_SIZE_filename
316. return os.path.join(self.cache_path(),
File "/Users/.../Sites/site/photologue/models.py" in cache_path
284. return os.path.join(os.path.dirname(self.image.path),
"cache")
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/db/models/fields/files.py" in _get_path
49. return self.storage.path(self.name)
File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
site-packages/django/core/files/storage.py" in path
81. raise NotImplementedError("This backend doesn't support
absolute paths.")

Exception Type: NotImplementedError at /admin/profileimages/
profileimage/1/delete/
Exception Value: This backend doesn't support absolute paths.

> > On Aug 28, 1:36 pm, michaelg...
>
> read more »

[Justin Driscoll]

I really should have read up more before I said "it should work". Sorry about that. My bad?

Photologue does use absolute paths right now for managing the image cache. I'm not sure If that's the ONLY issue but I will look into it.

Let me know if you find anything out. If you or other users with S3 account are willing to help test and we can make some progress I'll start up a S3 support branch.

- Justin

[michaelg]

No worries, man. I guess that's what happens when you make a killer
photo app that people try to use for every possible scenario! I'm not
an expert django user, but I'd be happy to help in whatever way I
can. I'll let you know if I make any other observations as I work on
it on and off this next week.

On Sep 4, 11:21 am, "Justin Driscoll" <justin.drisc...@gmail.com>
wrote:

> I really should have read up more before I said "it should work". Sorry
> about that. My bad?
> Photologue does use absolute paths right now for managing the image cache.
> I'm not sure If that's the ONLY issue but I will look into it.
>
> Let me know if you find anything out. If you or other users with S3 account
> are willing to help test and we can make some progress I'll start up a S3
> support branch.
>
> - Justin
>

> > > File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/...
>
> read more »

[vinay]

I already have my photo album on S3 and wanted to integrate that into
Django with photologue. I have been playing around with it for the
past few days. I am seeing the same error as michaelg. Uploading using
gallery upload with zip files seem to be working okay.

How do make the error go away? I can play around with it and test it
during my spare time.

On Sep 4, 11:21 am, "Justin Driscoll" <justin.drisc...@gmail.com>
wrote:

> I really should have read up more before I said "it should work". Sorry
> about that. My bad?
> Photologue does use absolute paths right now for managing the image cache.
> I'm not sure If that's the ONLY issue but I will look into it.
>
> Let me know if you find anything out. If you or other users with S3 account
> are willing to help test and we can make some progress I'll start up a S3
> support branch.
>
> - Justin
>

> ...
>
> read more »

[Justin]

I think (think, being the key word here) that this problem is in
Photologues use of os.path.join to join path componants when working
with the image cache, etc. as it returns an absolute path. The first
thing you might want to try is replacing calls to os.path.join(path1,
path2) with '/'.join([path1, path2])

- Justin

> > > > File...
>
> read more »

[michaelg]

I think the root of the problem is self.image.path. Whenever I try to
get the path for anything on S3, I get an error saying:"Caught an

exception while rendering: This backend doesn't support absolute

paths." I don't think this is a specific problem for photologue, but
any app that tries to use the path component on S3. I might have to
figure out another storage solution if I can't figure this out soon.

> > > > >   768.  ...
>
> read more »

[Justin Driscoll]

So just to confirm, have you tried setting up a simple test model outside of Photologue and called field.path? Maybe I'll try to dig into the S3 storage class a little tonight to see if I can help at all. Have you talked with the S3 storage library author?

- Justin

[Claudio B.]

Thanks for this great package, still the best in its category after all this time.

It works perfectly in a local environment, but I stumbled upon similar problems when I deployed to the staging server, with assets stored on Rackspace CDN.

For this purpose, I am using a Django storage called cumulus.storage.CloudFilesStorage

I get two different problems (as highlighted in this group):

• self.image.path does not work properly because the image cannot be retrieved from the local machine. To solve this problem, I retrieve the image from their remote location with urllib and store them in a temporary local folder before processing them (e.g., when creating thumbnails)
  
• im.save is not sufficient to store a picture (e.g., a newly created thumbnail), as it also has to be uploaded to CDN. To solve this problem, I call the function to upload the local temporary picture to the CDN
  

With these changes, Django Photologue works perfectly also with a remote storage such as Rackspace (using cumulus).

You can see the changes here: https://github.com/ff0000/django-photologue/commit/6942bb

With this post, I would like to know if you want me to create a (better-written) patch to supply this behavior to Django Photologue so that you can integrate it in the code.

Or else, if you already working on a new version of Photologue that deals with remote storage.

Congratulations once again for your code.