Error on File Upload

[Phyo Arkar]

Here with uWSGI setup i am getting this error:

 File "/home/v3ss/WorkSpace/FastTract/web2py/gluon/fileutils.py", line 376, in copystream


    data = src.read(chunk_size)
IOError: error waiting for wsgi.input data

It happens when file size > 300 MB.

I am running using uwsgi.
#!/bin/bash
./uwsgi/uwsgi --http :8000 --chdir ./web2py/ --python-path ./web2py/ --module uwsgihandler \
--socket /tmp/web2py.sock -p 6   \
--close-on-exec  # --socket-timeout 6000 --harakiri 6000 --http-timeout 6000  --harakiri-verbose --cpu-affinity 1

here is my controller :

# -*- coding: utf-8 -*-
import os
import urllib
import traceback
def upload():
    upfile = request.vars.file
    fname  = urllib.unquote(request.vars.file.filename)
    # cont_length = request.headers.CONTENT_LENGTH
    source_path= urllib.unquote(request.vars.source_path)

    file_path = os.path.join(source_path,fname)

    _chunk_writer(upfile,file_path)
    # print "results",type(upfile),upfile.file.read()
    return response.json("Done")


def _chunk_reader(FieldOBJ, chunk_size=1024):
    while True:
        chunk = FieldOBJ.file.read(chunk_size)
        # print "CHUNK",chunk
        if not chunk:
            FieldOBJ.file.close()
            break
        yield chunk


def _chunk_writer(file_obj,file_path):

    CHUNK_SIZE = 1024 * 1024

    try:
        with open(file_path, 'wb') as fp:
            for chunk in _chunk_reader(file_obj,CHUNK_SIZE):
                # print chunk
                if not chunk:
                    break
                fp.write(chunk)
        return True
    except:
        print "File Error Occoured",traceback.format_exc()
        return False

# def _shutil_chunkwriter(file_obj,file_path):

#     import shutil
#     CHUNK = 1024 * 1024
#     out_file=open(file_path, 'wb')
#     shutil.copyfileobj(file_obj,fp,CHUNK)
#     out_file.close()
#     file_obj.close

[Phyo Arkar]

Here is uwsgi log.

Its seriouly timing out ... what should i do.

Only 1 upload of large file (>300 MB) is saved , else is not written.

[pid: 24243|app: 0|req: 1/1] 127.0.0.1 () {36 vars in 663 bytes} [Fri Mar  9 03:25:27 2012] GET /admin => generated 3773 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 304 bytes (1 switches on core 0)

[pid: 24243|app: 0|req: 2/2] 127.0.0.1 () {36 vars in 680 bytes} [Fri Mar  9 03:25:29 2012] GET /favicon.ico => generated 573 bytes in 0 msecs (HTTP/1.1 400) 3 headers in 117 bytes (1 switches on core 0)

[pid: 24243|app: 0|req: 3/3] 127.0.0.1 () {46 vars in 936 bytes} [Fri Mar  9 03:25:34 2012] POST /admin/default/index => generated 63 bytes in 11649 msecs (HTTP/1.1 303) 3 headers in 183 bytes (1 switches on core 0)

[pid: 24243|app: 0|req: 4/4] 127.0.0.1 () {40 vars in 828 bytes} [Fri Mar  9 03:25:45 2012] GET /admin/default/site => generated 10599 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 304 bytes (1 switches on core 0)

[pid: 24243|app: 0|req: 5/5] 127.0.0.1 () {36 vars in 680 bytes} [Fri Mar  9 03:25:47 2012] GET /favicon.ico => generated 573 bytes in 0 msecs (HTTP/1.1 400) 3 headers in 117 bytes (1 switches on core 0)

[pid: 24244|app: 0|req: 1/6] 127.0.0.1 () {44 vars in 863 bytes} [Fri Mar  9 03:25:46 2012] POST /admin/default/check_version => generated 837 bytes in 4719 msecs (HTTP/1.1 500) 2 headers in 173 bytes (1 switches on core 0)

[pid: 24240|app: 0|req: 1/7] 127.0.0.1 () {40 vars in 814 bytes} [Fri Mar  9 03:25:46 2012] GET /admin/default/twitter => generated 16140 bytes in 5347 msecs (HTTP/1.1 200) 5 headers in 218 bytes (1 switches on core 0)

timeout !!!

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /FastTract/filemanager/upload (ip 127.0.0.1) !!!

Fri Mar  9 03:28:39 2012 - writev(): Broken pipe [proto/uwsgi.c line 120] during POST /FastTract/filemanager/upload (127.0.0.1)

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /FastTract/filemanager/upload (ip 127.0.0.1) !!!

Fri Mar  9 03:28:39 2012 - write(): Broken pipe [proto/uwsgi.c line 138] during POST /FastTract/filemanager/upload (127.0.0.1)

[pid: 24201|app: 0|req: 1/8] 127.0.0.1 () {48 vars in 931 bytes} [Fri Mar  9 03:25:10 2012] POST /FastTract/filemanager/upload => generated 0 bytes in 208654 msecs (HTTP/1.1 200) 6 headers in 0 bytes (1 switches on core 0)

timeout !!!

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /FastTract/filemanager/upload (ip 127.0.0.1) !!!

Fri Mar  9 03:32:56 2012 - writev(): Broken pipe [proto/uwsgi.c line 120] during POST /FastTract/filemanager/upload (127.0.0.1)

SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /FastTract/filemanager/upload (ip 127.0.0.1) !!!

Fri Mar  9 03:32:56 2012 - write(): Broken pipe [proto/uwsgi.c line 138] during POST /FastTract/filemanager/upload (127.0.0.1)

[pid: 24240|app: 0|req: 2/9] 127.0.0.1 () {48 vars in 996 bytes} [Fri Mar  9 03:28:40 2012] POST /FastTract/filemanager/upload => generated 0 bytes in 256338 msecs (HTTP/1.1 200) 6 headers in 0 bytes (1 switches on core 0)

timeout !!!