Issues with django-storages and gunicorn processes

[Mike Herring]

I am using django-storages to store media files in Amazon S3.

Gunicorn spaws three worker processes, each running their own copy of S3BotoStorage. When I go into filebrowser and upload a file, the current worker process adds it to its entries list, and therefore knows that the file exists. When I go back to the Media Library, it switches to a different worker process and says the file doesn't exist.

Is there any way to get around this? Should django-storages share it's entries list across processes? Can it be run as a singleton? 

[Vijay Khemlani]

Media Library? I'm not aware of something like that in django-storages

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/5e1a0cf8-2ece-43a5-8199-78cbcbbfbdb3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mike Herring]

Sorry, I am using Mezzanine, which has a Media Library that uses filebrowser_safe. But this issue isn't specific to Mezzanine.

The problem exists in django-storages. The entries property is stored in local memory for each gunicorn worker process. So if you save or delete a file in one worker process, the others don't know about the change.

I've solved this issue by clearing the entries property and repopulating it from AWS at the beginning of each page request. I don't know if that's the ideal solution, but it works.