form file field upload error

[Josh Cartmell]

I'm getting the following error when I try to submit a Mezzanine form that includes two larger files (the files are about 1.3mb each):

Traceback (most recent call last):

  File ".../lib/python2.7/site-packages/django/core/handlers/base.py", line 105, in get_response

    response = middleware_method(request, callback, callback_args, callback_kwargs)

  File "...lib/python2.7/site-packages/Mezzanine-1.2.3-py2.7.egg/mezzanine/pages/middleware.py", line 80, in process_view
    processor_response = processor(request, page)

  File "...lib/python2.7/site-packages/Mezzanine-1.2.3-py2.7.egg/mezzanine/forms/page_processors.py", line 58, in form_processor
    f.seek(0)

ValueError: I/O operation on closed file

After submitting the serving produces a 500 error.  The form entries, and files end up in the database, but no emails are sent to people who should get copies.

Anyone have any tips or pointers?

Thanks,
Josh

[Stephen McDonald]

That seek code is from django-forms-builder and is very old but I remember it clearly - it was needed at the time in order to do the email attachments, Django would have read the files already, and we wanted to re-read them in order to attach them.

I'm not sure if it's still needed, I'd lean towards leaving it in there - but perhaps we could just catch the exception you get and if that solves the problem, we should have an approach that works.

--
You received this message because you are subscribed to the Google Groups "Mezzanine Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mezzanine-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Stephen McDonald
http://jupo.org

[Josh Cartmell]

I tried catching the exception around f.seek(0) and that just causes the exception to be thrown on the next line where f.read() is called.  Do you know of any way to get the absolute path to the file so that I could re open it if the exception is caught on f.seek(0)?

I've tried:
try:
    f.seek(0)
except ValueError:
    f = open(f.name)

and
try:
    f.seek(0)
except ValueError:
    f = open(os.path.abspath(f.name))

Both of which fail.

I still am a bit confused as to why the file is closed at all.

Thanks for the help.

[Josh Cartmell]

Bump =)

At the point in the page processor where f.seek is called, is there any way to access the files location on disk?  That way I could just re open it.  I spent a little while digging around and I can't even figure out where the forms store file uploads at all.

Thanks!

[Stephen McDonald]

Here's where the files get saved: 

https://github.com/stephenmcd/mezzanine/blob/master/mezzanine/forms/forms.py#L163

[Josh Cartmell]

Thanks for the pointer Steve, I'll see if I can figure out a fix, would you want me to submit it back in case anyone else has similar trouble?

[Josh Cartmell]

I found them, they are getting stored outside of the project directory in a folder called forms directly beneath the users' home directory.  Is that normal/expected behavior?

[Josh Cartmell]

Ok, I think I have a fix.  I change the attachment appending here:
https://github.com/stephenmcd/mezzanine/blob/master/mezzanine/forms/page_processors.py#L57

to this:

try:
    attachments = []
    for f in form.files.values():
        f.seek(0)
        attachments.append((f.name, f.read()))
except ValueError:
    attachments = []
    for field_entry in entry.fields.all():
        try:
            f = open(join(fs.location, field_entry.value))
        except IOError:
            pass
        else:
            f.seek(0)
            attachments.append((f.name.split('/')[-1], f.read()))

It tries to add the attachments the normal way, but if any of them produces a ValueError (in my case I believe this is due to the file no longer existing in /tmp) it looks through all the entries fields, checking if they are files and attaching them if so.

What do you think?

[Stephen McDonald]

Maybe we can just nuke the old way and only use the new one you've got? Also I guess the new way should use the storages API instead of assuming a local filesystem?

[Josh Cartmell]

That would work, I left the old way because I thought my new one might be a bit less efficient. You are probably right about using the storages API (I haven't really kept up with that bit of Django). I basically copied the way the admin opens the files when when you download them from viewing entries, so that may need updating too.

[Josh Cartmell]

Hey Steve sorry it's taken me a bit to get back to this.  I'm not really up on the storages API so it might be awhile before I get to this.  If you easily know how to take what I've done and make it use the storages API I don't mind at all.

I still also think it's a bit strange that the way things work right now uploaded files aren't even being saved in the project directory but are being stuck in a directory called "forms" which is a child of the home directory of the linux user running the site.

[Stephen McDonald]

On Thu, Feb 21, 2013 at 8:19 AM, Josh Cartmell <joshc...@gmail.com> wrote:

Hey Steve sorry it's taken me a bit to get back to this.  I'm not really up on the storages API so it might be awhile before I get to this.  If you easily know how to take what I've done and make it use the storages API I don't mind at all.

I still also think it's a bit strange that the way things work right now uploaded files aren't even being saved in the project directory but are being stuck in a directory called "forms" which is a child of the home directory of the linux user running the site.

The idea is that the files shouldn't be stored in the static directory or anywhere publicly accessible - there's a FORMS_UPLOAD_ROOT setting for configuring where they should go.

[Josh Cartmell]

That makes sense, I guess I wasn't thinking about it from a security standpoint, thanks!

[Alexander Kominek]

Hi guys,

I'm currently running into the same issue on my site - is going to have a permanent fix?

Thanks,

 - Alex

[Stephen McDonald]

This is finally fixed in https://github.com/stephenmcd/mezzanine/commit/9527662dc3c7d7dfaa6397e01be9ebc999f2646f

[Alexander Kominek]

Thanks, Stephen!

--

You received this message because you are subscribed to a topic in the Google Groups "Mezzanine Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mezzanine-users/eyaEfiCfuB8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mezzanine-use...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.