FileField - not stored to disk when using mod_wsgi?

[Joakim Hove]

I ave django file upload application. I have a model looking like this:

class BaseFile(Model):
    input_name = CharField( max_length = 132 )
    content = FileField( upload_to = "%Y/%m/%d" )
    upload_time = DateTimeField( auto_now_add=True )
    owner_group = CharField( max_length = 32 , default = None )

In the settings.py file I have set:

MEDIA_ROOT = "/tmp/storage"

When I use this with the development server things seem to work, I can upload files and if I go to the /tmp/storage directory I can see the %Y/%m/%d directories created and populated with my files.

When I try this using Apache and mod_wsgi things also seem to work:

1. The HTTP POST returns status 200.
2. A subsequent GET will return the files in question.

But - when I go the /tmp/storage directory there are no directories/files to be found and if I stop and start Apache the GET will fail with:

    IOError: No such file: /tmp/storage/2017/11/27/FILEX_Z1OZMf8 

So - it seems to me that Django keeps the files in memory when using mod_wsgi? I am using 100% default Django settings when it comes to storage backend. I see no errors in the Apache log.

[Jason]

I suspect this might be due to permissions, because on linux, /tmp is owned by root and the apache service runs as user apache.

Can you confirm that you're using mod_wsgi_py3 for python3 and not regular mod_wsgi, which is for python2 only.

[Joakim Hove]

I suspect this might be due to permissions, because on linux, /tmp is owned by root and the apache service runs as user apache.

I agree that this has a certain smell of permissions, but I have at least tried to rule that out. In particular I have:

1. I have created the directory /tmp/storage manually beforehand - and applied "chmod a+rwx /tmp/storage"
2. When I intentionally limit the permissions so that user 'apache' can not write in this directory I immediately get a "OSError . permssion denied" when uploading files.
   
3. I can not see anything in the apache error_log.
   

Can you confirm that you're using mod_wsgi_py3 for python3 and not regular mod_wsgi, which is for python2 only.

Ehhh - I am using Python2.7 and can confirm that I am using regular mod_wsgi.

[Jason]

Hmm, ok.  

What's your view that's responsible for handling that model instanciation?

[Joakim Hove]

What's your view that's responsible for handling that model instanciation?

 

This of course quickly gets slightly unwieldy - but thanks for the interest. The view/endpoint accepting the POST is this:

https://github.com/joakim-hove/fmu_storage/blob/master/ensemble/api/views.py#L56

The relevant function call is the `Simulation.upload( )` call on line 74; and that is implemeted here: https://github.com/joakim-hove/fmu_storage/blob/master/simulation/api/views.py#L107 (observe that there is a 'import as ' which enforces a class rename). The full model module is here: https://github.com/joakim-hove/fmu_storage/blob/master/simulation/models.py

Regards - Joakim

[Jason]

Alright... so what I suspect is happening is this is a direct result of your usage of /tmp as an upload location.

/tmp is for temporary files, and its very possible Apache is cleaning up those files created through its process when you terminate the server.  

I would create a different folder path and use that.

[Joakim Hove]

/tmp is for temporary files, and its very possible Apache is cleaning up those files created through its process when you terminate the server.  

I agree that could have been the case - but do observe that I can not see any files/directories when Apache is running either.

 

I would create a different folder path and use that.

OK - I have created a new directory `/data/storage` and set `MEDIA_ROOT` to point there. Results are the same.

I was wondering if there was some explicit way to force writing to files?