Large file upload with progress - cherrypy-users

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Large file upload with progress
Received-SPF: pass (google.com: domain of x.solarwin...@gmail.com designates 74.125.82.172 as permitted sender) client-ip=74.125.82.172;
MIME-Version: 1.0
In-Reply-To: <1347867967.2878.63.camel@keblar>
References: <7b2fbc35-dce3-4d0e-86e5-f7a3c7d2cb9c@googlegroups.com>
 <CAGSooMMFVMwEpxU-G1Q49hH33+-JxNoXTwMK4eGeHo5JXOz...@mail.gmail.com> <1347867967.2878.63.camel@keblar>
From: V G <x.solarwin...@gmail.com>
Date: Mon, 17 Sep 2012 16:11:20 -0400
Message-ID: <CAGSooMNphX=-RtJXoROV2BGjcMjxFy73b-zCrkTqQ2Q-6Sk...@mail.gmail.com>
Subject: Re: [cherrypy-users] Large file upload with progress
To: cherrypy-users@googlegroups.com
Content-Type: multipart/alternative; boundary=00504502e5dbbd01a704c9eb61c6

--00504502e5dbbd01a704c9eb61c6
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Sep 17, 2012 at 3:46 AM, Joel Rivera <riv...@joel.mx> wrote:

> You can use the idea of the fileUpload tool and then in the same tool
> add to the cherrypy.session the temporary file and external id (which
> can be created with javscript and should be unique from the client side)
> and the size of the request:
>
> e.g.
>
> POST /filemgm/_upload?fid=1234ab
>
> def bigfile(self, fid, **kwargs):
>     ...
>     ...
>     cherrypy.session[fid] = \
>           (TEMPORARY_FILEPATH,
>           cherrypy.request.headers['Content-length'])
>     cherrypy.session.save()
>     ...
>     ...
>
> Then periodically interrogate the server asking for the file
>
> GET /filemgm/_status?fid=1234ab
>
> the filemgm._status method adds the logics to return the upload
> percentage which you can know from the session, the fid parameter and
> the percentage from the relation of the header 'Content-length' and the
> current size of the file.
>
> e.g.
>
> import os
>
> def _status(self, fid):
>    try:
>        tempfilepath, length = cherrypy.session[fname]
>    except (TypeError, ValueError): # is None or unpack error
>        return 'done'
>    else:
>        currsize = os.stat(tempfilepath).st_size
>        return float(currsize) / length * 100 # float is just for python2
>
>
> For this to work you need to have some client-side code to keep updating
> the percentage to the user, you can easily make and async post with
> jquery to post the first request and then ask for the status of the fid.
>
> The fundamental problem is that you cannot respond to the client
> when the uploading of the request (the file) is being done, so I think
> that the solution should be in the form of a secondary request.
>
> The essence from the idea that I propose is
> 1.- Had an external id of the file that you are uploading.
> 2.- Make a secondary request to check the status of the id that the
>     first request make, which can be one with an streamed response
>     or many with precise responses.
>
> Also I'm overlooking that when the request is getting processed in the
> first request the file that has being created gets periodically flushed
> to have a trustworthy uploading percentage.
>
> I hope that help you to consider this as an alternative.
>
>

Excellent! That was exactly what I was looking for. Thanks so much!

--00504502e5dbbd01a704c9eb61c6
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">On Mon, Sep 17, 2012 at 3:46 AM, Joel Rivera <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:riv...@joel.mx" target=3D"_blank">rive=
r...@joel.mx</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

You can use the idea of the fileUpload tool and then in the same tool<br>
add to the cherrypy.session the temporary file and external id (which<br>
can be created with javscript and should be unique from the client side)<br=
>
and the size of the request:<br>
<br>
e.g.<br>
<br>
POST /filemgm/_upload?fid=3D1234ab<br>
<br>
def bigfile(self, fid, **kwargs):<br>
=A0 =A0 ...<br>
=A0 =A0 ...<br>
=A0 =A0 cherrypy.session[fid] =3D \<br>
=A0 =A0 =A0 =A0 =A0 (TEMPORARY_FILEPATH,<br>
=A0 =A0 =A0 =A0 =A0 cherrypy.request.headers[&#39;Content-length&#39;])<br>
=A0 =A0 cherrypy.session.save()<br>
=A0 =A0 ...<br>
=A0 =A0 ...<br>
<br>
Then periodically interrogate the server asking for the file<br>
<br>
GET /filemgm/_status?fid=3D1234ab<br>
<br>
the filemgm._status method adds the logics to return the upload<br>
percentage which you can know from the session, the fid parameter and<br>
the percentage from the relation of the header &#39;Content-length&#39; and=
 the<br>
current size of the file.<br>
<br>
e.g.<br>
<br>
import os<br>
<br>
def _status(self, fid):<br>
=A0 =A0try:<br>
=A0 =A0 =A0 =A0tempfilepath, length =3D cherrypy.session[fname]<br>
=A0 =A0except (TypeError, ValueError): # is None or unpack error<br>
=A0 =A0 =A0 =A0return &#39;done&#39;<br>
=A0 =A0else:<br>
=A0 =A0 =A0 =A0currsize =3D os.stat(tempfilepath).st_size<br>
=A0 =A0 =A0 =A0return float(currsize) / length * 100 # float is just for py=
thon2<br>
<br>
<br>
For this to work you need to have some client-side code to keep updating<br=
>
the percentage to the user, you can easily make and async post with<br>
jquery to post the first request and then ask for the status of the fid.<br=
>
<br>
The fundamental problem is that you cannot respond to the client<br>
when the uploading of the request (the file) is being done, so I think<br>
that the solution should be in the form of a secondary request.<br>
<br>
The essence from the idea that I propose is<br>
1.- Had an external id of the file that you are uploading.<br>
2.- Make a secondary request to check the status of the id that the<br>
=A0 =A0 first request make, which can be one with an streamed response<br>
=A0 =A0 or many with precise responses.<br>
<br>
Also I&#39;m overlooking that when the request is getting processed in the<=
br>
first request the file that has being created gets periodically flushed<br>
to have a trustworthy uploading percentage.<br>
<br>
I hope that help you to consider this as an alternative.<br>
<br></blockquote></div><br><br>Excellent! That was exactly what I was looki=
ng for. Thanks so much!<br>

--00504502e5dbbd01a704c9eb61c6--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
