Upload a huge file

[Alex Dong]

For the website I'm working on, we need to allow the users to upload a
non-trivial file, normally 10-40MB each via our web.py pages. Here is
the code we use today:
>>> open(upload_folder + GUID().generate(), 'wb').write(web.input().file)

In the Cheetah template, I have a file upload input control like this
<input type="file" name="file" ... ...>

The code works perfect when I upload a small file say 300KB, but when
the file size grows up, the speed becomes very slow on a localhost
server.

So I'm wondering are there anyone on this list meet this problem?
What's your solution?

thanks,
Alex

[Tom Berger]

On 8/25/06, Alex Dong <alex...@gmail.com> wrote:

For the website I'm working on, we need to allow the users to upload a
non-trivial file, normally 10-40MB 

The code works perfect when I upload a small file say 300KB, but when
the file size grows up, the speed becomes very slow on a localhost
server.

The problem is (I guess) that you are reading the entire file into memory before saving it - what you want is to write directly from the request stream to disk.

Looking at the code, I see:

def data():
    """Returns the data sent with the request."""
    if 'data' not in ctx:
        cl = intget(ctx.env.get('CONTENT_LENGTH'), 0)
        ctx.data = ctx.env['wsgi.input'].read(cl)
    return ctx.data


So my guess is that ctx.env['wsgi.input'] is what you're looking for - that's just a guess, though - I have not tried this or even read the code to understand it.

tom



--
http://intellectronica.net/

[Aaron Swartz]

> In the Cheetah template, I have a file upload input control like this
> <input type="file" name="file" ... ...>

Do a web.input(file={}) and it won't load the file into RAM.