Document URLs not using MEDIA_URL

[Christoph Lipp]

Hi there

We have setup a wagtail page and configured it using Amazon S3 for Media Storage.

The files we upload end up beeing stored on S3, but the download link  points to the local server.

I'm using a document chooser panel, the value gets stored in a property called "file". In the template I'm using {{ self.file.url }} to access the file url, but it seems wrong. Is there another tag which I have to use to get the correct document URL?

Cheers,
Christoph

[Matthew Westcott]

Hi Christoph,
In Wagtail, documents are always served through a Django view (wagtail.wagtaildocs.views.serve.serve) so that we can perform additional processing on document downloads, such as counting download statistics (and potentially applying access restrictions, in future). This means that they'll always be served under a local URL, even if the actual document is stored remotely.

I believe that storing documents on S3 should work (although I haven't tested it), but it'll end up using two lots of bandwidth as the data is transferred from S3 to your server and then to the user. I guess we could a setting to Wagtail to disable the 'document serve' view and expose the URL of the original document file instead (at the cost of losing the download stats / access restrictions), but we don't have that at the moment. (Patches welcome!)

Cheers,
- Matt

[Christoph Lipp]

Hi Matt

Thanks for your reply. We deployed wagtail to heroku, therefore we must use a storage service like S3.

I'm not quite sure what the problem is exactly I simply get a http 500 error code, have to enable the logs to be certain. I'll be back with more information as soon as I figure it out :)

Cheers,
Christoph

[Christoph Lipp]

Hi every

In the meantime I managed to figure out the error message.

   

Request Method:	GET
Request URL:
Django Version:	1.8.2
Exception Type:	NotImplementedError
Exception Value:	This backend doesn't support absolute paths.
Exception Location:	/app/.heroku/python/lib/python3.4/site-packages/django/core/files/storage.py in path, line 115
Python Executable:	/app/.heroku/python/bin/python
Python Version:	3.4.2
Python Path:	['/app',  '/app/.heroku/python/bin',  '/app/.heroku/python/lib/python3.4/site-packages/setuptools-16.0-py3.4.egg',  '/app/.heroku/python/lib/python3.4/site-packages/pip-7.0.1-py3.4.egg',  '/app',  '/app/.heroku/python/lib/python34.zip',  '/app/.heroku/python/lib/python3.4',  '/app/.heroku/python/lib/python3.4/plat-linux',  '/app/.heroku/python/lib/python3.4/lib-dynload',  '/app/.heroku/python/lib/python3.4/site-packages']
Server time:	Mon, 15 Jun 2015 14:35:42 +0200

Not quite sure what to do now...

Cheers,
Christoph

[Nar Chhantyal]

Actually it's already possible if you don't care about sendfile/access restrictions Wagtail provides.

Just use `document_link.file.url` instead of `document_link.url`. I had similar issue here https://github.com/torchbox/wagtail/issues/1407 That way, default django behavior works. Main issue of this is if you want to serve private files, you have to do something else.

On Friday, May 29, 2015 at 3:20:47 PM UTC+2, Matthew Westcott wrote:

[Christoph Lipp]

Thanks for the hint, unfortunately it doesn't work all the time. In the richtext editor the links are still rendered using the wagtail link.

[Caroline Simpson]

I just ran into the same issue.  As a work around, we've added a url override and created a redirect view that will serve the document files from s3 directly.  

Here is the code that we added, in case it helps.  It may not be the best solution, but it is working.

Added a views.py file (could really be any file) into our project with the following code:

from django.shortcuts import get_object_or_404
from django.views.generic import RedirectView

from wagtail.wagtaildocs.models import Document

class S3DocumentServe(RedirectView):
    def get_redirect_url(self, *args, **kwargs):
        document_id = kwargs['document_id']
        document = get_object_or_404(Document, id=document_id)
        return document.file.url

Then in urls.py, we added the url override before importing the wagtaildocs_urls:

...

from .views import S3DocumentServe

...

urlpatterns = patterns('',
   ...
   url(r'^documents/(?P<document_id>\d+)/(.*)$', S3DocumentServe.as_view(),
      name='wagtaildocs_serve'),
   url(r'^documents/', include(wagtaildocs_urls)),
   ...
)
...

Thanks, 

 Caroline

[Matthew Westcott]

I think this an important thing for us to fix for Wagtail 1.1, so I've put together a proposal for a comprehensive solution that (hopefully) covers all bases, including remote storage, storing docs locally and serving through django-sendfile, and the simple fallback option of serving directly through Django:
https://github.com/torchbox/wagtail/issues/1420

Cheers,
- Matt

> --
> You received this message because you are subscribed to the Google Groups "Wagtail support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wagtail+u...@googlegroups.com.
> To post to this group, send email to wag...@googlegroups.com.
> Visit this group at http://groups.google.com/group/wagtail.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/wagtail/ba0a69fc-bd81-4b97-8959-bb1a83dac946%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Christoph Lipp]

Hi Matt

Thank you very much, highly appreciated ! Already noticed the workaround for the 1.0 release, but your proposal is much cleaner ;-)

Cheers, Christoph

[Joss Ingram]

Hi,

i know this was a while ago but recently I've setup a Wagtail site on Azure, and we're using Azure Storage containers as a backend for images and docs. It's all working fine but obviously documents are served from the site not Azure storage. It seems to upload the documents to Azure and to the web server, but always serves the document from the web server. Am I right in thinking then that the document file on the remote storage is never actually used by Wagtail? Sorry if I'm not understanding this correctly.

I'm reluctant to implement any of the work arounds for reasons that have been mentioned above, and it's working well for images anyway.

Are there any plans to address this in the future?

Thanks

Joss

[Tobias McNulty]

I have what I believe to be a simple fix to this issue in a PR here, but have not been able to get any feedback on it, yet. It likely still needs a little cleanup, should the general approach be approved...

Regarding your question about how the document is served, I believe Django reads it byte by byte from the underlying storage, and then serves it to your users via the Python web process on your web server. So yes, the storage is used, but not in a particularly efficient way.

I would love to see this addressed, too. I don't use Azure, so it would be great to get the approach checked from your perspective.

Tobias McNulty
Chief Executive Officer

tob...@caktusgroup.com
www.caktusgroup.com

To unsubscribe from this group and stop receiving emails from it, send an email to wagtail+unsubscribe@googlegroups.com.

To post to this group, send email to wag...@googlegroups.com.

Visit this group at https://groups.google.com/group/wagtail.
To view this discussion on the web, visit https://groups.google.com/d/msgid/wagtail/63999df7-0f2c-4531-a0fb-615efabb3f4a%40googlegroups.com.

[Joss Ingram]

I think your suggested approach would work well for us actually.