How to perform Blobstore Asynchronous Requests for getting upload result.

[thstart]

I want to create an app which to perform upload of a CSV file to the cloud. Right now 
I am using upload_url = blobstore.create_upload_url('/upload') and get_uploads which are a synchronous.

Sometimes the app is getting an error or timeout.

I suppose timeout can be solved with asynchronous requests, but not the errors. For errors probably the best solution

is to generate another upload_url and automatically repeat the process, but it will reach the 30-60 secs limit.

Reading the documentation about the new Blobstore Asynchronous Requests I mention it is asynchronous during

generating upload URL. I don't think generating upload URL is slow. I would rather prefer the upload operation

itself to by asynchronous and with s call back my app the get the results.

How to accomplish that?

http://code.google.com/appengine/docs/python/blobstore/overview.html#Making_Asynchronous_Requests 

upload_url_rpc = blobstore.create_upload_url_async('/upload')
slow_operation()
upload_url = upload_url_rpc.get_result()
self.response.out.write("""<form action="%s" method="POST"
                           enctype="multipart/form-data">""" % upload_url)

In this example, the application carries out the slow_operation() code at the same time that Blobstore generates the upload URL.

[Matt Jibson]

Performing the upload itself is done by the user, not your application. Your app returns a URL that gets inserted into a form, a user selects a file, and it is uploaded to the blobstore. Only after the file is already in the blobstore is your application called again with the resulting blob key. create_upload_url() and get_uploads() should be called in separate handlers.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/UUVgGyapiPkJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[thstart]

>Performing the upload itself is done by the user, not your application. 

The app is in usage and I am aware how it works, the debugger show it very well.

But I get the issues described above:

1) errors.

2) timeouts.

[Robert Kluin]

Have you tried to catch and retry the timeouts / deadline errors, and
decrease the rpc deadline for the create call? The second will let
you try the first part of the solution more times if you hit errors.

If you're hitting these issues a lot, I'd file a production issue.
I've seen several other people reporting similar issues.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/5JxZuMeEtw8J.

[Stuart Langley]

What errors and what timeouts?

[thstart]

Occasionally I am getting error 500.  Occasionally a file of 10KB cannot be uploaded in the 30-60 Sec limit and I am getting timeout.

[Stuart Langley]

What 30-60 second limit? Are you trying to upload blobs from within your application?

[Constantine Vasil]

OK I have a form in appengine Python app. The 30 Secs used to be the time a process to finish in appengine. It was increased to 1 min.

On Mon, Feb 6, 2012 at 9:38 PM, Stuart Langley <slan...@google.com> wrote:

What 30-60 second limit? Are you trying to upload blobs from within your application?

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/AEAXnIvR29UJ.

[Matt Jibson]

That is the limit for a single request. The procedure to create a blobstore URL and then send a file is two requests. There is something fundamental that you either don't understand or are not communicating to us.

[thstart]

Do you never got 500 error?

[Andreas]

can u post your handler(s)?

i have upload forms and users upload even 500mb files without any problem.

the upload url has a life of 10min and the upload time does not count against request limit.

On Feb 7, 2012, at 2:35 AM, thstart wrote:

Do you never got 500 error?

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/eIW54xq_GywJ.

[thstart]

Here is an extracted source.

============================================================================ 

def upload():

        ###############################################################

        # BEGIN  Import

        ###############################################################

        if request.env.request_method == 'POST':                

            form_import_submit = True

            post_vars = request.post_vars

            

            file_csv_value = None

            delimiter = ','

            try:

                file_csv = post_vars.file_csv

                filename = file_csv.filename

        

                bufsize = file_csv.bufsize

                filename_size = bufsize

                """

                    http://code.google.com/appengine/docs/python/blobstore/blobreaderclass.html

                """

                blob_key = file_csv.type_options['blob-key']

                # Instantiate a BlobReader for a given Blobstore value.

                blob_reader = blobstore.BlobReader(blob_key)

                file_csv_value = blob_reader.read()

                filename_size = len(file_csv_value)

         except Exception, e:

                logging.info('%s:Error uploading=e=%s' % (sub, e))                                  

                msg = 'Error uploading.'

                result = 400

     http_host = 'My.appspot.com'

     network = 'TestNetwork'

     back_url = 'http://' + http_host + '/' + 'import' + '?network=' + network

     upload_url = blobstore.create_upload_url(back_url)

     upload_url = blobstore.create_upload_url(back_url)

============================================================================

     return dict(

                upload_url=upload_url,

                )

upload.html:

<form action="{{=upload_url}}" method="POST" enctype="multipart/form-data" data-ajax="false">

                    <fieldset>

                        <div class="fieldWrapper" data-role="fieldcontain">

                            <label for="file_csv">Upload CSV file:</label>

                            <input type="file" name="file_csv" id="file_csv" value=""  />

                        </div>                                  

                    </fieldset>              

                        <input type="submit" value="Upload" data-theme="e" data-ajax="false">

        </form>         

============================================================================ 

[Robert Kluin]

Hi,
Have you tried following the example in the docs of how to upload a blob?
http://code.google.com/appengine/docs/python/blobstore/overview.html#Uploading_a_Blob

Note that the handler is deriving from
blobstore_handlers.BlobstoreUploadHandler, and that they don't
directly get the uploaded data, rather they get a blob key.

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/T9dFYdwkmugJ.