ImageField save issues (IOError: [Errno 37] No locks available)

[wnielson]

I've been running into an issue while trying to upgrade my code to use
Django's new file handling system. When I try to use the `save`
method of a `ImageField` instance, I get an `IOError: [Errno 37] No
locks available`. The file gets created but nothing is ever written
to it. Here is a basic example that illustrates the issue I keep
running into.

from django.db import models
from urllib import urlopen

class SomeModel(models.Model):
image = models.ImageField(upload_to='some_dir', blank=True,
null=True)


sm = SomeModel()
image = urlopen('http://ecx.images-amazon.com/images/I/
51pnPEb8STL._SS500_.jpg')
sm.image.save('my_filename.jpg', image, save=False)


Does anyone have any advice?

Thanks in advance,
Wes

[Malcolm Tredinnick]

On Mon, 2008-08-18 at 16:05 -0700, wnielson wrote:
> I've been running into an issue while trying to upgrade my code to use
> Django's new file handling system. When I try to use the `save`
> method of a `ImageField` instance, I get an `IOError: [Errno 37] No
> locks available`. The file gets created but nothing is ever written
> to it. Here is a basic example that illustrates the issue I keep
> running into.

We need some more details about your particular setup. Are you trying to
save to an NFS mounted filesystem? A bit of searching on Google for that
error message (which hopefully you also did before posting to the
mailing list, right?) shows that it's possible there.

If you're not using an NFS_mounted system as the destination, what sort
of OS and filesystem are you trying to save to? Do things change if you
target the file saves to somewhere else that is more or less guaranteed
to be locally mounted (e.g. /tmp on a Unix system)?

Regards,
Malcolm

[wnielson]

Thanks for the quick response. Yes, I did do some searching and ran
across the post about NFS (and yes I am using an NFS mount). However,
I can solve the problem by bypassing the `save` method like so:

from django.core.files import File

f = open('my_filename.jpg', 'w')
myfile = File(f)
myfile.write(image.read())
sm.image = myfile

I suppose this method works because it bypasses the `save` method's
use of the lock.

On Aug 19, 12:16 am, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

On Mon, 2008-08-18 at 16:42 -0700, wnielson wrote:
> Thanks for the quick response. Yes, I did do some searching and ran
> across the post about NFS (and yes I am using an NFS mount). However,
> I can solve the problem by bypassing the `save` method like so:
>
> from django.core.files import File
>
> f = open('my_filename.jpg', 'w')
> myfile = File(f)
> myfile.write(image.read())
> sm.image = myfile
>
> I suppose this method works because it bypasses the `save` method's
> use of the lock.

Precisely. Which makes it subject to race conditions and lots of
problems because of that.

You could also avoid the problem by not saving your file at all. Both
options are just symptom patching, rather than a solution to the
problem. :-)

Looks like we need to switch from flock-based locking to fcntl-based
locking where available. *sigh*. I'll take care of opening a ticket for
this.

Regards,
Malcolm

[wnielson]

For what its worth, I did manage a work around on the NFS side. In /
etc/export you need to have the `no_subtree_check` option enabled.
Apparently this has mild security implications (as stated in man
exports), but it does allow the locking to work properly.

Thanks for the insight Malcolm.

Cheers,
Wes

On Aug 19, 12:48 am, Malcolm Tredinnick <malc...@pointy-stick.com>
wrote:

[Malcolm Tredinnick]

On Mon, 2008-08-18 at 17:23 -0700, wnielson wrote:
> For what its worth, I did manage a work around on the NFS side. In /
> etc/export you need to have the `no_subtree_check` option enabled.
> Apparently this has mild security implications (as stated in man
> exports), but it does allow the locking to work properly.

I've opened #8043 anyway, since not everybody is going to have the
ability to reconfigure the NFS mount options. A big, hulking sysadmin
may be standing between them and the config file, for example. Which
makes this a bit of a showstopper in that case. Fortunately, it looks
like it's ultimately solvable.

Regards,
Malcolm