413: Request entity too large

[Dmytro Titov]

Hi,

I've faced 413 error recently in my App Engine application. Googling showed that there's a limit of 32MB for inbound traffic per request and this can not be changed. 

People suggest using direct uploads to Google Cloud Storage to work it around. But the problem is that I'm not creating a client application - I'm only creating a back-end for existing client application. So I can't control how the upload is handled on the client side: this is something stable and also can not be changed. Technically it's a simple HTTP(S) upload with Basic authentication. 

What my back-end was supposed to do is to handle such uploads by creating some dynamic URLs protected with Basic authentication. It works excellent - but only for files < 32MB.

So now the question is: is it possible to achieve the same behaviour with Google Cloud Storage, e.i. can I programmatically create buckets with specific URLs and allow direct uploads there, protected by Basic authentication?

Or is there any other possible way to work-around such limitation?

I also wonder why does this 32MB limit even exists - it looks really ridiculous to me. But, I guess, no one will answer that question...

Anyway, thanks in advance.

[Stefano Ciccarelli]

Are you using Java?

https://cloud.google.com/appengine/docs/standard/java/blobstore/#Java_Using_the_Blobstore_API_with_Google_Cloud_Storage

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/92bdd787-0609-408c-9bc7-f4bca3c49004%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Stefano Ciccarelli 
GAE Application Division 
/ Director 
stefano.c...@mmbsoftware.it 

M.M.B. s.r.l. 
via Granarolo, 177/7 - 48018 Faenza (RA) - Italy 
tel. +39.0546.637711 - fax +39.0546.46077 
www.mmbsoftware.it - in...@mmbsoftware.it

Le informazioni contenute in questa comunicazione sono riservate e destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per errore, Vi preghiamo di rispondere a questa e-mail e successivamente cancellarla dal Vostro sistema.

[Dmytro Titov]

Thanks for the suggestion, but I'm afraid it won't help me. 

As far as I understand, the generated URL is temporary and I need a persistent URL for data uploading (and preferably URL with a human-readable name, that I can specify myself - not some kind of a hash code; but it's less important). 

Also, AFAIK, this generated upload URL can't be shielded with Basic HTTP authentication, which is a requirement for me. 

And the most crucial thing here: my client is not uploading data as "multipart/form-data" - he's sending a raw byte-stream. So I guess this upload URL will be unable to handle such input.

Correct me if I'm wrong, please.

Are there any other options?

[Dmytro Titov]

P.S. You might be confused by my original "dynamic URL" words. I meant that it's dynamic upon construction, but then this URL is preserved.

On Friday, January 26, 2018 at 3:46:31 PM UTC+1, Stefano Ciccarelli wrote:

[Stefano Ciccarelli]

As far as I know no.

GAE is designed for short lived requests, and is not the case for large uploads.

To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/fdf17232-0e96-4590-aad0-33c6f774fa01%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Attila-Mihaly Balazs]

Hello,

Some quick answers:

- yes, direct uploads are limited to 32MB and it's documented
- for large uploads the recommended way is to create a one-off upload URL which uploads the file to Google Cloud Storage. For Python I find the blobstore interface to be the easiest to use for this: https://cloud.google.com/appengine/docs/standard/python/blobstore/

Cheers,
Attila