FileField and moving or updating files

[timc3]

Hi,

I have a fairly standard model:

class MediaObject(models.Model):
""" An instance of a MediaItem """
name = models.CharField(max_length=256)
path = models.FileField(_("File"), max_length=256,
upload_to='tempUpload/')

And thats all well and good, my uploads work and so on.
But what I currently do is I upload to a temporary directory (as shown
above), then the upload triggers some processing on the file.

Then I want to move the file. Before the process actually modified the
database directly, updating the path to a new location. But we don't
want to do that anymore as I want to provide a RESTapi to update the
location of the file.

I have read through: http://docs.djangoproject.com/en/dev/topics/files/#topics-files

But seeing as Django uses the python File type it seems that I can't
update the path?

Is there a method of moving the file or the path? I would prefer not
to read in the file into memory if possible (my files can be Gb's in
size).

The only way I thought of getting around this limitation is to provide
the RESTApi but just use raw SQL instead and update the location.
Though I think that should be the last resort.

Any help would be appreciated.

Thanks.

[timc3]

So I take it that there is no way of doing this?

[felix]

I was just learning/dealing with this now.

example:

class Mix(models.Model):
       mp3 = models.FileField(upload_to="mixes",blank=True)

# let's say the mix object already exists
mix = Mix.objects.get( .... etc )


# however you get a file

        file = request.FILES[u'Filedata']

# swfupload is awesome btw.

or by opening it from your post processed folder somewhere
or by creating a file directly in memory using content, or PIL etc.

then the correct way is this:

if mix.mp3.name: # if there is a previous file and we wish to delete it
   mix.mp3.storage.delete( mix.mp3.name )

get the field's defined storage to delete it

#name will be eg. "mixes/oldfile.mp3"

ideal_path = "mixes/newfile.mp3"

# save the new file into storage
saved_to = mix.mp3.storage.save( ideal_path, file )


the path may have _ added to it for non-clobber purposes and is returned with the real path

# assign the string directly to the file field input
mix.mp3 = saved_to

# and save the new path into the object
mix.save()


this is nice in that you can switch the storage later or use multiple storage systems across your site,
not just

from django.core.files.storage import default_storage
default_storage.save( path, file)


you can upload to "tempUploads/file.xxx" and then search for files whose names are still in tempUploads and thus not yet processed

felix :

crucial-systems.com

[azimix79]

Hi,
I have the same problem as above.
I'm sure your solution works Felix, but it is important for me to
avoid resaving the files (as they can be well over 500Mb-1Gb in size).

The ideal solution would be to just change the path in the model
without having to resave the file.

Any help is appreciated.
Thanks

[felix]

oh if its already where you want it (as long as its in the MEDIA ROOT)
then just set the file field with the string:

obj.filefield = "path/to/file.mov"
obj.save()


I might be wrong on this, but :
if you are working with a File that has a path, then you aren't really moving it.
say its in /tmp/87fac12
and you
obj.filefield.storage.save( newpath,file )

it moves the file on the filesystem (gives it a new path reference), but it doesn't have to actually move anything really.
its actually "rename"

the filesystem itself is a kind of database and you are just assigning a new path

look at django.core.files.move

felix :

crucial-systems.com

[felix]

and btw. if you have any wisdom regarding uploading really large files then please share !
I posted another thread on that.

[timc3]

Thanks for the information, its much appreciated.

Currently we are not doing anything that special. We use Nginx as our
webserver, with the configuration set to receive large files, then
django is then sent the file and we are just using the standard method
of multiple_chunks() .

Everything on Nginx and django seems to cope really well, but the big
pain comes from the web browsers themselves. Most seem to try and load
the file in to memory before uploading. This causes a lot of problems.